Clinical versus statistical significance

A slightly different type of false positive arises when a trial result for a given outcome measure is statistically significant, but not clinically important. This can occur if the study was over-powered for that outcome measure. Clearly it is unlikely that this would ever happen for a primary outcome measure if the sample size was estimated correctly, as it would be ethically dubious to randomize more patients than are strictly necessary to detect clinically relevant differences. Such a finding may though arise with respect to a secondary outcome measure, or through a meta-analysis of the primary outcome measure in several trials. Meta-analyses in particular provide the opportunity to detect even small differences reliably, and it is important not to let extreme p-values detract from the need to consider the clinical relevance of the size of the difference observed. An example of the former can be seen in the MRC/EORTC TE20 trial of three versus four courses of BEP chemotherapy for metastatic germ cell tumours described earlier [10]. In this, the primary outcome measure was progression-free survival, and quality of life was a secondary outcome measure. Based on the primary outcome, 800 patients were required. When the quality of life data was analysed [11], the subscales were transformed to a 1-100 scale, and a change of 10 points taken as a moderate difference which would be noticeable by the patient. In fact, the study was overpowered for such differences for many of the subscales and the study generated several statistically significant results when the actual changes were much less than 10 per cent. In this case, the authors themselves point this out when discussing the interpretation of the results; had no consideration been given to the size of difference which might be clinically relevant, the authors may well have come to different conclusions.

0 0

Post a comment