What size of type I error is acceptable

To help determine the size of type I error that is acceptable (i.e. the probability of wrongly concluding that there is a difference between treatments when in fact none exists) consider what strength of evidence against the null hypothesis of no treatment difference you would want to see before rejecting it. This quantity is often denoted by a and goes under several synonyms:

♦ the significance level,

♦ probability of a false-positive result.

At the end of the trial, one would reject the null hypothesis of no treatment difference, and conclude that there is likely to be a genuine difference between treatments, if the p-value from the formal test to compare treatments is less than a. This value is frequently set at 0.05 or 5 per cent but this is arbitrary and the consequences of wrongly concluding there is a difference should be considered for each trial being designed, bearing in mind its impact on sample size, and the significance level set accordingly. Fig. 5.1 shows the impact on sample size of changing the significance level.

As discussed in Chapter 9, p-values alone are an insufficient basis for drawing conclusions from a trial, and while levels must be specified in order to determine sample size, the conclusions should not alter radically if the p-value is very slightly above or below the pre-set significance level.

0 0

Post a comment