Comparing IPD and other approaches

Given the extra resource and commitment required, it is reasonable to question the need for the IPD approach. There is evidence that meta-analyses based only on data extracted from published papers may give estimates of treatment effects, and of their significance, that are not confirmed when all the randomized evidence is re-analysed in an IPD meta-analysis [20-22]. For example, a comparison of an IPD meta-analysis in advanced ovarian cancer (of platinum-based combination chemotherapy versus singleagent non-platinum drugs) [23] with a similar analysis using only data that could be extracted from the published papers (Table 11.1), found that the IPD analysis gave less encouraging results [20].

In this study the aim was to compare the overall differences in outcome between the approaches; comparing the policy of extracting data from published reports with the policy of collecting IPD. The analysis of published summary data, which included eight trials and 788 patients, favoured combination chemotherapy with an estimated improvement in survival at thirty months of 7.5 per cent. This was marginally significant at conventional levels (p = 0.027). The IPD meta-analysis which was based on eleven trials and 1329 patients suggested only a 2.5 per cent improvement and this did not reach conventional levels of significance (p = 0.30). Further investigation revealed that the

Table 11.1 Comparison of meta-analysis approaches using an example in advanced ovarian cancer

Individual patient data from trialists

Data extracted from publications







Odds ratio



Hazard ratio



95 per cent confidence interval






Absolute benefit at thirty months




Median follow up 6.5 years

Point estimate at 30 months

differences between the two approaches were attributable to a number of factors. These included:

♦ missing trials (unpublished plus those which did not publish the required information),

♦ excluded patients,

♦ the point in time at which analyses using data from publications were based,

♦ additional long-term follow-up which was available with the IPD approach.

None of these factors were found to be very much more important than the others. Rather they each contributed cumulatively to the difference. The analysis based on the summary data available from published reports not only provided a conventionally statistically significant result but gave an estimated benefit three times larger than that suggested by the IPD. Given the poor prognosis for advanced ovarian cancer, when balanced against other factors such as toxicity, ease of administration and cost, the clinical interpretation of the results from the two approaches could well be different.

A similar investigation, comparing the results of an IPD review with analyses using data from published papers of trials comparing ovarian ablation with control in the treatment of breast cancer also found differences in the results obtained by the two approaches [24]. However, in this case, it was the IPD that gave the strongest evidence in favour of the intervention. The meta-analysis of published data available in 1990 was inconclusive about the effect of ovarian ablation. However, the meta-analysis using IPD, largely because of prolonged follow up, showed a significant reduction in the annual odds of death with an absolute improvement in overall survival of 10.2 per cent at fifteen years for women randomized to ovarian ablation.

In contrast, an investigation comparing the results of different types of meta-analysis of trials investigating selective decontamination of the digestive tract (a form of antibiotic

Table 11.2 Comparison of meta-analysis approaches using an example in selective decontamination of the digestive tract [25]

Type of meta-analysis


Odds ratio

95% CI

Data extracted from publications




Summary data obtained from trialists

975/3 564



IPD obtained from trialists 829/3357 0.89 0.76-1.05

IPD obtained from trialists 829/3357 0.89 0.76-1.05

prophylaxis for patients in intensive care units) found that there was little difference between three types of approach (Table 11.2) [25]. This comparison used data from seventeen trials for which all three methods of systematic review (data extracted from publications, tabular summary data provided by trialists or IPD provided by trialists) could be used, out of a total of twenty-five trials including 4310 patients identified as relevant to the therapeutic question.

In this case the authors concluded that despite the advantages of IPD, the approach is costly and that analysis of summary data provided by study investigators is a valid alternative to IPD, when categorical data are used and censoring is not relevant. Here the authors aimed to compare the differences attributable only to the data type used and exactly the same trials were used in each analysis. It is not clear how the additional eight trials (where data were not available in all three formats) would have affected the outcome in terms of the estimates achieved with the most comprehensive and inclusive analysis possible with each ofthe three approaches.

These examples illustrate the difficulties of trying to generalize and extrapolate from literature-based analyses. In most cases, it will not be possible to estimate whether the results of such analyses will differ from analyses based on the full IPD, and if so in which direction and to what extent they will be changed, until the IPD has been collected and analysed.

10 Ways To Fight Off Cancer

10 Ways To Fight Off Cancer

Learning About 10 Ways Fight Off Cancer Can Have Amazing Benefits For Your Life The Best Tips On How To Keep This Killer At Bay Discovering that you or a loved one has cancer can be utterly terrifying. All the same, once you comprehend the causes of cancer and learn how to reverse those causes, you or your loved one may have more than a fighting chance of beating out cancer.

Get My Free Ebook

Post a comment