G Narenthiran FEBNS FRCS(SN)
18 March 2024
g_narenthiran@hotmail.com
The ‘P’ in the P-value stands for probability. We will come to this shortly.
Generally, when we analyse data, we analyse the data for a sample and use statistics to infer the results for the population.
The type of statistics we commonly use in medicine is ‘frequentist statistics’. In frequentist statistics, we study the sample and infer for the population.
Example 1:
We measured the weight of our male and female patients. There were 20 male patients and 20 female patients. The mean weight for the male patients was 72 kg. The mean weight for the female patients was 55 kg. Here, we note a difference of 17kg between the mean weights of our male and female patients. This difference of 17kg is indeed an actual difference in the mean weights of our male and female patients.
Can we make an inference for all the male and female patients in the world based on our findings, i.e., from the findings from a sample? How can we generalise our findings for the population?
When we now use the t-test (unpaired student t-test) to analyse the difference in mean weight between our male and female patients, we are attempting to infer whether where this difference of 17kg would be actual if we measured the weights of all male and female patients in the universe i.e. we are trying to use the data we have gathered (from our sample) to infer for the population.
Let’s assume that when we undertook the t-test, the P-value was 0.04. What does this mean?
In statistics, we define the null hypothesis before we undertake an experiment.
The ‘null hypothesis’ (Ho) for our study is that:
“There is no difference in the mean weight of male and female patients.”
In statistics, we assume that the members of our study group (male and female patients in our ward) are samples from the whole population (all male and female patients in the universe) and the patients in our sample (those in our ward) were randomly chosen from the entire population of male and female patients.
The t-test assesses the probability that the difference that we noted between our groups, i.e., a difference of 17kg in the mean weight between our male and female patients, is by chance because of our random sampling from the population.
The p-value of 0.04 indicates that if we assume that the null hypothesis is true, i.e., there is no difference in the mean weight of all the male and female patients, then the 17kg difference in the mean weight that we observed might have been through chance, because of the random sampling, is 4%.
In medical statistics, if the probability of the difference between two groups occurring is equal or less than 5% (0.05), then we state the difference is statistically significant. However, we can only be sure that there is an actual difference if we study the whole population of male and female patients, which is impractical.
Conversely, if the p-value is greater than 0.05, then we infer that our null hypothesis is true and there is no statistical difference between the mean weights of our groups; the difference we noted (17kg) was through chance because of the random sampling of the population.
Ganealingam Narenthiran on October 19, 2024
Thank you very much Naren, very enlightening. If the P value were 0.048, could we say that there was a “tendency” toward significance? Thank you
Francesco
Dear Francesco,
Thank you for your question!
In most medical studies, the threshold for statistical significance is a P value of 0.05. This threshold is also stated as α, alpha. If the P value is equal to or less than 0.05, we then state there is a statistical difference.
In the example you provided, the P value in your study is 0.048, equal to or less than 0.05. Hence, there is statistical significance.
If the P value had been 0.057, you could state that it is ‘tending towards significance’, as it is just above 0.05. You can then check how well your study is ‘powered’ to notice a significant difference.
The ‘power’ of the study informs what chance the study has to detect a significant difference, if there indeed is.
A study with a power of 80% can detect significant results, 80% of the time. Conversely, there is a 20% chance that the test will not detect a significant difference, even when there is one. We would state that there is 20% chance of type II error or beta (β) error.
You could increase the power of your study by increasing the sample sizes or having more equal numbers in control and test groups or both.
If you are using a parametric test, e.g., t test, it would also be useful to check the distribution of data in each group indeed has a normal distribution.
Professional statistical software such as Stata, SPSS, and R would have tests to calculate the power or, the sample size needed for high power with t-tests. There will also be on-line calculators on the web, to undertake such calculations. If you need help with this, donot hesitate to contact me.
Thank you!
Naren