Expected difference questionnaire

What difference between treatments do you think a trial such as this, with several thousand patients, would show? Here clinicians are being asked not to predict the results of the planned trial, but those of a very large and impeccably conducted trial addressing the same question. When asked this question in a structured way, individual clinicians will rarely give a point estimate, it is much more natural to give different weights to various differences. The questionnaire asks the clinician to distribute 100 per cent across various differences as shown in Fig. 5.5.

In the hypothetical example, the clinician is somewhat cautious, giving most weight to there being a modest benefit to the experimental treatment, for example a 25 per cent chance of there being a 0-5 per cent absolute improvement and a 30 per cent chance of there being a 5-10 per cent absolute improvement. She also considers that there is only a 15 per cent chance that the experimental treatment will produce an absolute improvement of more than 10 per cent. The clinician also has some concerns (5 per cent probability) that it could actually be as much as 10-15 per cent worse than control treatment.

Control treatment (C) better

Experimental treatment (E) better

Absolute difference between arms (per cent)

20+

15-20

10-15

5-10

0-5

0-5

5-10

10-15

15-20

20+

Total

Hypothetical example

0

0

5

10

15

25

30

10

5

0

100

Your entry

100

Fig. 5.5 Example of 'expected difference' questionnaire.

Ask as many relevant individuals as possible to complete these questionnaires to provide a representative spread of clinical opinion. An overall distribution of anticipated results can be obtained by averaging the percentages allocated to each of the effect-size categories; often this will form a symmetrical distribution of which the mean or median is a reasonable guess at the typical likely difference. Figure 5.6 shows the summary distribution of responses to the question posed with respect to the CHART bronchus trial; here the median difference was approximately 10 per cent.

Fig. 5.6 Overall distribution of expected differences - CHART bronchus trial. B. Worthwhile difference questionnaire

Here a different question is posed; ignoring what differences may be likely, we aim to elucidate the types of differences that, if observed, would be likely to change that clinician's practice. Using the CHART bronchus trial as an example again, we ask the following questions:

Suppose the result of the trial is known with absolute certainty. What is the largest difference in 2-year survival that would lead you to continue with conventional radiotherapy (CRT) as standard treatment? What is the smallest difference that would lead you to adopt CHART?

The clinician is then asked to mark these two points on a diagram, as shown in Fig. 5.7, beginning with a hypothetical example.

"I- ,i,.i.ï,.i.ii.i1iiïl,i1,1,,1,ï,.,.i,.i.i.fi1ii1i1,i..|ni.i1.i1.1,i|..1..1.ii.i.|-1

—15 to -10-10 to -5 -5 to 0 0 to 5 5 to 10 10 to 15 15 to 20 20 to 25 25 to 30 30 to 35 <-- benefit to conventional benefit to CHART--> Range of likely differences

Fig. 5.6 Overall distribution of expected differences - CHART bronchus trial. B. Worthwhile difference questionnaire

Here a different question is posed; ignoring what differences may be likely, we aim to elucidate the types of differences that, if observed, would be likely to change that clinician's practice. Using the CHART bronchus trial as an example again, we ask the following questions:

Suppose the result of the trial is known with absolute certainty. What is the largest difference in 2-year survival that would lead you to continue with conventional radiotherapy (CRT) as standard treatment? What is the smallest difference that would lead you to adopt CHART?

The clinician is then asked to mark these two points on a diagram, as shown in Fig. 5.7, beginning with a hypothetical example.

If a clinician would treat patients with CHART if the 2-year survival rate was more than 10 per cent higher than with CRT, and would use CRT if the survival rate was better, or no more than 5 per cent worse with CRT, they would answer as below:

Absolute difference in 2-year survival

20% 15% 10% 5% Advantage to conventional radiotherapy

Your answer:

Advantage to conventional radiotherapy

What factors (toxicity, cost etc) affected your answer?

Fig. 5.7 'Worthwhile difference' questionnaire.

While some clinicians will give a single decision point, most will indicate two. Between these points is what is known as the 'range of equivalence.' Outside this range the clinician is reasonably certain that s/he would consider one or other treatment as standard. Within the range the decision is not so clear-cut and would probably depend on balancing a number of factors.

'Worthwhile' differences can be summarized by taking the average left-hand side end, and the average right-hand side end of the ranges of equivalence to produce a single range. In the CHART bronchus trial, the average range was 11-14 per cent. However, it can also be useful (see example 1 below) to view the ranges individually.

0 0

Post a comment