## Significance testing

In a clinical trial comparing an experimental treatment against a control treatment, typically a primary question of interest is: is the experimental treatment better than the control? To address this question using the data from a clinical trial we actually answer a different but related question - we test the null hypothesis of 'no difference.' In a randomized clinical trial where we are aiming to show that the experimental treatment is better than the control treatment, the null hypothesis is that there is no difference between the experimental and control treatments in their effect on the main outcome measure. To test this null hypothesis, often denoted by Ho, we compare the results for the group of patients treated with the experimental with the group treated with the control. The goal of doing this is to reject the null hypothesis of'no difference' in favour of an alternative hypothesis, that there is in fact a difference between experimental and control. A general approach to test this is to calculate the following 'Z' statistic for the main outcome measure

(observed difference between (difference anticipated in the

^ experimental and control) null hypothesis) (9 1)

standard error of the observed difference

If the null hypothesis is true, then on average the observed difference should be zero (i.e. no difference), so that large values of Z represent more significant results, that is, smaller p-values. The standard error (SE) of the observed difference represents the variability inherent in estimating the observed difference. Generally, the larger the trial, the smaller the SE and therefore the larger Z will be.

The difference anticipated in the null hypothesis is usually zero. In this instance the Z-statistic simplifies to:

observed difference

standard error of the observed difference

Once the value of Z is calculated, its value is referred to standard tables to obtain a p-value. This resulting p-value answers the question: what is the probability that we would have observed this difference or a more extreme difference, if the null hypothesis is true? If the p-value is small, say less than 0.001, then we may conclude that the null hypothesis is unlikely to be true. On the other hand, if the p-value is large, say greater than 0.5, then we cannot conclude with any certainty that the null hypothesis is not true. Issues around interpreting and presenting p-values are presented in Chapter 10, and also discussed below.

0 0