OSU logo Oklahoma State University
Center for Health Sciences
Medical Physiology - Evidence-Based Medicine
Printer Friendly

Introduction

What is EBM?

Patient Care Model
Life-long Learning
Why is EBM Important?
Available Evidence?
EBM Issues

The Well-built Question

The EBM Process
Anatomy of a Question

Finding Evidence

Selecting a Resource
Searching the Resource
Reviewing Search Results
Returning to the Patient

Evaluating Evidence

Evaluating the Validity
Validity Questions

 

Knowledge Test

Multiple Sclerosis
Case #2
Case #3
Case #4

 

Reference/
Glossary
Feedback

 

Are the results of this therapy study valid?

1. Was the assignment of patients to treatment randomized?

The assignment of patients to either group (treatment or control) must be done by a random allocation. This might include a coin toss (heads to treatment/tails to control) or use of randomization tables, often computer generated.

Research has shown that random allocation comes closest to insuring the creation of groups of patients who will be similar in their risk of the events you hope to prevent. Randomization balances the groups for prognostic factors (ie. disease severity) which eliminates over-representation of any one characteristic within the study groups. Randomization should also be concealed from the clinicians and researchers of the study to help eliminate conscious or unconscious bias.

Article: This information is found under the METHODS section. The patients in this study were randomly assigned to receive digoxin or placebo. The randomization was stratified by left ventricular ejection fraction as well as by study center. It appears that randomization was concealed and done by telephone through a coordination center. The researchers enrolling the patients did not have any information about the randomization scheme. (Additional details about the study methodology were reported in a previous article.)

2. Were all the patients who entered the trial properly accounted for at its conclusion? Was follow-up complete?

All patients who started the trial should be accounted for at the end of the trial. If patients are not accounted for, the validity of the study may be jeopardized. A good study will have better than 80% follow-up for their patients. Patients may drop out of a study for various reasons, including because of adverse events related to the treatment. If these patients are not included in the results, they can make the treatment look better than it really is (and vice versa). To be sure of a study's conclusions, lost patients should be assigned to the "worst-case" outcomes and the results recalculated. The results are still valid if the recalculations do not change the end results.

Article: Yes, the final status of 98.6% of patients is known at the end of the trial. In addition, the investigators comment on a sensitivity analysis (how sensitive the results would be to the "worst" case scenario if all the lost patients had the worst outcome). In this case, a sensitivity analysis showed that the "lost" patients would not effect the overall mortality results, even if all had died. They do not state how a "worst" case scenario would effect hospitalizations

Were patients analyzed in the groups to which they were (originally) randomized?

Anything that happens after randomization can affect the chances that a patient in a study has an event. Patients who forget or refuse their treatment should not be eliminated from the study analysis. Excluding noncompliant patients leaves behind those that may be more likely to have a positive outcome, thus compromising the unbiased comparison that we got from the process of randomization. Therefore patients should be analysed within their assigned groups. This is called "intention to treat" analysis.

Yes, analysis was done by intention to treat. This information is in the METHODS section, under Statistical Analysis.

3. Were patients, clinicians, and study personnel "blind" to treatment allocation?

Blinding means that the people involved in the study do not know which treatments are given to which patients. This eliminates bias and any preconceived notions as to how the treatments should be working. When it is difficult or unethical to blind patients to a treatment, such as a surgical treatment, then a "blinded" researcher is needed to interpret the results.

Article: The researchers and data analysts who looked at clinical outcomes were blinded to the treatment groups. It is not clear if the patients and clinicians were blinded to the study arms. In some patients, if clinical need arose, a switch was made to open label treatment. Measures were taken to minimize unblinding, however, this was not always possible.

4. Were the groups similar at the start of the trial?

The treatment and the control group must be similar for all prognostic characteristics except one: whether or not they received the experimental treatment. This information is usually displayed in tables that outline the baseline characteristics of both groups.

Article: This information is found in Table 1, which presents the baseline characteristics of the patients in the 2 study arms. Characteristics appear to be evenly distributed within the 2 groups.

5. Aside from the experimental intervention, were the groups treated equally?

Both groups must be treated the same except for administration of the experimental treatment. If "cointerventions" (interventions other than the study treatment which are applied differently to both groups) exist, they must be described in the methods section of the study.

Article: It appears that both groups were treated the same. There are no reported differences in co-interventions, follow-up or outcome measures.

6. Are the results of this study valid?

Article: This study methodology appears to be sound and the results are valid.

7. What are the results of the study?

Main results: Analysis was by intention to treat. Mortality did not differ between groups. 1181 deaths occurred in the digoxin group compared with 1194 deaths in the placebo group. (34.8% vs. 35.1%, P=0.8) The digoxin group however, had lower rates of hospitalizations overall compared with the placebo group (64.3% vs. 67.1%, (P=0.006).

Conclusions: Digoxin did not affect mortality but reduced hospitalizations in patients with heart failure and normal sinus rhythm.

Outcomes:

Outcome

Digoxin

Placebo

RRR

ARR

NNT

Mortality

34.8%

35.1%

nonsignificant p=0.08

Total hospitalization

64.3%

67.1%

4.1%

2.8%

36

hospitalization for CHF

27%

35%

23%

8%

13

hospitalization for CV causes

49.9%

54.4%

8.3%

4.5%

22

Reprinted with permission from the American College of Physicians. (ACP Journal Club 1997 Sep/Oct;127(2):34)

Key issues for studies of Therapy:

  • randomization 

  • follow-up (80% or better) 

  • blinding (the more blinding the better) 

  • baseline similarities (established at the start of the trial)

 

Key terminology for estimating the size of the treatment effect

 

Outcome 

Risk of outcome

+ -

Treated (Y)

a

b

Y = a/(a + b)

Control (X)

c

d

X = c/(c + d)

  • Relative Risk (RR) is the risk of the outcome in the treated group (Y) compared to the risk in the control group.
    RR = Y / X

  • Relative Risk Reduction (RRR) is the percent reduction in risk in the treated group (Y) compared to the control group (X).
    RRR = 1 - Y / X x 100%

  • Absolute Risk Reduction (ARR) is the difference in risk between the control group (X) and the treatment group (Y).
    ARR = X - Y

  • Number Needed to Treat (NNT) is the number of patients that must be treated over a given period of time to prevent one adverse outcome.
    NNT = 1 / (X - Y)


Source:
Guyatt GH ; Sackett DL ; Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA 1993 Dec 1;270(21):2598-601.

Note: For criteria for other types of studies, see the following
supplements(Medical Physiology--PE-8618 students may skip these supplements):  Diagnosis | Prognosis | Etiology/ Harm

We now go to the next section, Knowledge Test: Case Nos. 1, 2, 3, and 4.