In an attempt to avoid some of the problems associated with time-related changes, some non-randomized comparisons may make use of concurrent controls. Here, all patients are treated and assessed prospectively, but receive different treatments by a non-random mechanism. Typical examples include comparing the results of a treatment given in one hospital with those of a different treatment given in a different hospital. Clearly though, there may well be other systematic differences between hospitals; for example, in the type of supportive or ancillary care they provide or in the type of patient being referred. As another example, it has been suggested that treatments can be compared more fairly if patients are allowed to choose which treatment they wish to receive. The assumption is that patients will not harbour the biases or pre-conceptions of their doctors. However, even if these biases have not been expressed to the patient, the 'volunteer' effect is well known - patients feeling fitter or more confident opting perhaps for the new treatment whilst those feeling less robust prefer to receive the established standard therapy for their condition. The groups of patients selecting specific treatments may also differ in more subtle ways and again these may be very important with respect to treatment outcome, but not necessarily quantifiable.
In recent years there has been a great deal of interest in recording common datasets on all newly diagnosed cancer patients. The suggestion that these might be used to make treatment comparisons crops up with a degree of regularity with proponents arguing that because registration is complete, treatment effects can be estimated using statistical models to adjust for the imbalances in important patient characteristics. Unfortunately,
such comparisons may still suffer from problems associated with both historical and concurrent controls:
♦ The treatments patients have received are the result of clinical judgement as to what was appropriate. The factors determining that choice may not be quantifiable, and even if quantifiable, may not be recorded.
♦ As with historical controls, patients will not necessarily have been identified or evaluated in a consistent manner.
♦ Adjusted (or matched) analyses can be used (see Section 9.4.3) but the important prognostic factors may not be known or agreed, or even if agreed, recorded in an appropriate manner. There are many possible adjusted analyses which could be carried out, each potentially resulting in a different treatment effect. Finally, the assumptions underlying such multivariate models may not always be met, adding to the difficulty of determining which is the 'best' estimate of treatment effect.
Thus the difficulties in interpreting treatment comparisons from non-randomized studies are considerable. They may generate hypotheses, or provide useful supplementary data to non-randomized studies in order to determine if a randomized trial might be worthwhile (e.g. ), but they cannot provide an accurate estimate of treatment effect. If they are truly the only feasible form of comparison, patient characteristics must be taken into account in the analysis and careful consideration given to all the possible biases relevant to the particular comparison, including the direction in which they are most likely to act.
Was this article helpful?