AAOS Comprehensive Orthopaedic Review

Section 2 - General Knowledge

Chapter 24. Evidence-Based Medicine

I. Basics of Evidence-Based Medicine

A. Definition—Evidence-based medicine is the practice of integrating individual clinical expertise with the best available clinical evidence from systematic research to maximize the quality and quantity of life for individual patients.

 

B. Goal—To achieve the best possible patient management and patient outcomes through the combination of empirical evidence, clinical expertise, and patient values.

 

C. Steps in evidence-based medicine

 

1. Formulate an answerable question

 

2. Track down the best evidence of outcomes available

 

3. Appraise the evidence

 

4. Apply the evidence (integrate with clinical expertise)

 

5. Evaluate the effectiveness and efficiency of the process

 

D. Assessing evidence

 

1. Not restricted to randomized trials and meta-analyses

 

2. Cluster evidence of similar validity and provide indications of similarity across the different sorts of questions (prognosis, diagnosis, etc)

 

3. Grade is matched directly to the level of evidence and quality of study



II. Types of Studies

A. Therapeutic studies—Investigate the results of treatment.

 

   *Khaled J. Saleh, MD, or the department with which he is affiliated has received research or institutional support from Stryker and Smith & Nephew, has received royalties from Smith & Nephew, and is a consultant for or an employee of Stryker and Smith & Nephew.

 

1. Level I

 

a. High-quality randomized controlled trial with statistically significant difference or no statistically significant difference but narrow confidence interval (CI)

 

b. Systematic review of level I randomized controlled trials (in which study results are homogeneous)

 

2. Level II

 

a. Lesser-quality randomized controlled trial (eg, <80% follow-up, no blinding, or improper randomization)

 

b. Prospective comparative study

 

c. Systematic review of level II studies or level I studies with inconsistent results

 

3. Level III

 

a. Case-control study

 

b. Retrospective comparative study

 

c. Systematic review of level III studies

 

4. Level IV—Case series or poor-quality cohort and case-control studies.

 

5. Level V—Expert opinion.

 

B. Prognostic studies—Investigate the effect of a patient characteristic on the outcome of disease.

 

1. Level I

 

a. High-quality prospective study (eg, all patients enrolled at the same point in their disease with ≥80% follow-up)

 

b. Systematic review of level I studies

 

2. Level II

 

a. Retrospective study

 

b. Untreated controls from a randomized controlled trial

 

c. Lesser quality prospective study (eg, patients enrolled at different points in their disease or < 80% follow-up)

 

d. Systematic review of level II studies

 

3. Level III—Case-control study.

 

4. Level IV—Case series.

 

5. Level V—Expert opinion.

 

C. Diagnostic studies—investigating a diagnostic test

 

1. Level I

 

a. Testing of previously developed diagnostic criteria in series of consecutive patients (with universally applied reference gold standard)

 

b. Systematic review of level I studies

 

2. Level II

 

a. Development of diagnostic criteria on basis of consecutive patients (with universally applied reference gold standard)

 

b. Systematic review of level II studies

 

3. Level III

 

a. Study of nonconsecutive patients (without consistently applied reference "gold" standard)

 

b. Systematic review of level III studies

 

4. Level IV

 

a. Case-control study

 

b. Poor reference standard

 

5. Level V—Expert opinion.

 

D. Economic and decision analyses—Developing an economic or decision model.

 

1. Level I

 

a. Sensible costs and alternatives; values obtained from many studies; multi-way sensitivity analyses

 

b. Systematic review of level I studies

 

2. Level II

 

a. Sensible costs and alternatives; values obtained from limited studies; audits and chart reviews

 

b. Systematic review of level II studies

 

3. Level III

 

a. Analyses based on limited alternatives and costs; poor estimates

 

b. Systematic review of level III studies

 

4. Level IV—No sensitivity analyses.

 

5. Level V—Expert opinion.



Examples of Study Types in Orthopaedic Surgery

Therapeutic study, evidence level I

Moseley JB, O'Malley K, Peterson NJ, et al: A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002;347:81-88.

Therapeutic study, evidence level II

Jaeger M, Maier D, Kern WV, Sudkamp NP: Antibiotics in trauma and orthopedic surgery—A primer of evidence-based recommendations. Injury 2006;37 (Suppl 1):S74-S80.

Prognostic study, evidence level II

Leadbetter WB, Ragland PS, Mont MA: The appropriate use of patellofemoral arthroplasty: An analysis of reported indications, contraindications, and failures. Clin Orthop Relat Res 2005;436:91-99.

Shervin N, Rubash HE, Katz JN: Orthopaedic procedure volume and patient outcomes: A systematic literature review. Clin Orthop Relat Res 2007;457:35-41.

Diagnostic study, evidence level II

Harris MB, Sethi RK: The initial assessment and management of the multiple-trauma patient with an associated spine injury. Spine 2006;31 (Suppl 11):S9-S15.

Diagnostic study, evidence level III

Vaccaro AR, Kreidel KO, Pan W, et al: Usefulness of MRI in isolated upper cervical spine fractures in adults. J Spinal Disord 1998;11:289-294.

Economic and decision analysis study, evidence level II

Brauer CA, Neumann PJ, Rosen AB: Trends in cost effectiveness analysis in orthopaedic surgery. Clin Orthop Relat Res 2007;457:42-48.

Economic and decision analysis study, evidence level IV

Hurwitz SR, Tornetta P III, Wright JG: An AOA critical issue: How to read the literature to change your practice: An evidence-based medicine approach. J Bone Joint Surg Am 2006;88:1873-1879.

Glossary of Evidence-Based Medicine Terms

Absolute risk reduction (ARR) Difference in the event rate between control group (control event rate, CER) and treated group (experimental event rate, EER): ARR = CER - EER.

Blinded Study in which any or all of the clinicians, patients, participants, outcome assessors, or statisticians are unaware of who received which study intervention. A double-blind study usually means the patient and clinician are blinded but is ambiguous, so it is better to state who is blinded.

Case-control study Identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looking back to see if they had the exposure of interest.

Case series Report on a series of patients with an outcome of interest. No control group is involved.

Clinical practice guideline Systematically developed statement designed to assist practitioner and patient make decisions about appropriate health care for specific clinical circumstances.

Cohort study Identification of two groups (cohorts) of patients, one that received the exposure of interest and one that did not, and following these cohorts forward for the outcome of interest.

Confidence interval (CI) Quantification of the uncertainty in measurement; usually reported as 95% CI, which is the range of values within which we can be 95% sure that the true value for the whole population lies.

Control event rate (CER) see Event rate.

Cost-benefit analysis Converts effects into the same monetary terms as the costs and compares them.

Cost-effectiveness analysis Converts effects into health terms and describes the costs for some additional health gain (eg, cost per additional event prevented).

Cost-utility analysis Converts effects into personal preferences (or utilities) and describes how much it costs for some additional quality gain (eg, cost per additional quality-adjusted life-year [QALY]).

Crossover study design Administration of two or more experimental therapies one after the other in a specified or random order to the same group of patients.

Cross-sectional study Observation of a defined population at a single point in time or time interval. Exposure and outcome are determined simultaneously.

Decision analysis Application of explicit, quantitative methods to analyze decisions under conditions of uncertainty.

Ecological survey Based on aggregated data for some population as it exists at some point or points in time; to investigate the relationship of an exposure to a known or presumed risk factor for a specified outcome.

Experimental event rate (EER) see Event rate.

Event rate Proportion of patients in a group in whom the event is observed. Thus, if out of 100 patients, the event is observed in 27, the event rate is 0.27. Control event rate (CER) and experimental event rate (EER) are used to refer to this in control and experimental groups of patients, respectively.

Evidence-based health care Extends the application of the principles of evidence-based medicine to all professions associated with health care, including purchasing and management.

Evidence-based medicine The conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.

Likelihood ratio Likelihood of a given test result in a patient with the target disorder compared with the likelihood of the same result in a patient without that disorder.

Meta-analysis Systematic review or overview that uses quantitative methods to summarize the results.

N-of-1 trials Patient undergoes pairs of treatment periods organized so that one period involves the use of the experimental treatment and one period involves the use of an alternate or placebo therapy. The patients and physician are blinded, if possible, and outcomes are monitored. Treatment periods are replicated until the clinician and patient are convinced that the treatments are definitely different or definitely not different.

Negative predictive value (-PV) Proportion of people with a negative test who are free of disease.

Number needed to treat (NNT) Number of patients who need to be treated to prevent one bad outcome; the inverse of the absolute risk reduction.

Odds Ratio of events to nonevents. If the event rate for a disease is 0.2 (20%), its nonevent rate is 0.8 (80%), then its odds are 0.2/0.8 = 0.25. See also Odds ratio.

Odds ratio Odds of an experimental patient suffering an event relative to the odds of a control patient.

Overview Systematic review and summary of the medical literature.

P value Probability of obtaining the same or more extreme data assuming the null hypothesis of no effect; P values are generally (but arbitrarily) considered significant if P < 0.05.

Positive predictive value (+PV) Proportion of people with a positive test who have disease. Also called the posttest probability of disease after a positive test.

Posttest probability Proportion of patients with that particular test result who have the target disorder (posttest odds/[1 + posttest odds]).

[

Figure 1. Basic design of a randomized clinical trial. IRB = Institutional Review Board; DSMB = Data and Safety Monitoring Board.]

Power Probability that the sample mean will be sufficiently different from the mean under null hypothesis to allow rejection of the null hypothesis.

Randomized controlled clinical trial Group of patients is randomized into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest (Figure 1).

Relative risk reduction (RRR) Percent reduction in events in the treated group event rate (EER) compared to the control group event rate (CER).

Sample size Calculations performed before the inception of a study to determine the number of subjects needed to participate to avert a type I error or type II error. See

Table 1.

Type I error Probability of rejecting the null hypothesis when it is really true. The probability of making a type I error is denoted by the Greek letter α.

Type II error Probability of failing to reject a null hypothesis that is really false. The probability of making a type II error is denoted by the Greek letter β.

[Table 1. Common Formulas for the Determination of Sample Size]

Top Testing Facts

1. Evidence-based medicine is the practice of integrating individual clinical expertise with the best available clinical evidence from systematic research.

 

2. Evidence-based practice guidelines serve to assist the practicing orthopaedic surgeon in his or her quest to improve patient care by consolidating the relevant evidence and indicating the strength of the recommendation for treatment options.

 

3. Grades I through V of studies are matched directly to the level of evidence.

 

4. Therapeutic studies investigate the results of treatment.

 

5. A study's power is the probability that the sample mean will be sufficiently different from the mean under null hypothesis to allow one to reject the null hypothesis. Sample size plays a critical role in power analysis. When trying to detect small differences between groups, larger sample sizes are necessary.

 

6. A type II error is the probability of determining that there is no difference between treatment groups when there actually is a difference. Type II errors often occur when the sample size of the treatment group is too small and leads to a false-negative result.

 

7. A type I error is a false-positive conclusion that occurs when one rejects a null hypothesis that is actually true.

 

8. The P value is the probability of obtaining the same or more extreme data, assuming the null hypothesis is of no effect (P < 0.05).

 

9. The confidence interval (CI) is a quantification of the uncertainty of measurement. Typically, a 95% CI reports the range of values within which one can be 95% certain that the true value for the whole population lies.



Bibliography

Centre for Evidence-Based Medicine (www.cebm.net).

The Cochrane Collaboration (www.cochrane.org).

Online EBM: www.ebm.bmj.com.

Evidence-Based Medicine Working Group: Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420-2425.

Evidence-Based Medicine Working Group. Users' guides to the medical literature I-XXV. JAMA 1993-2000.

Davidoff F, Haynes B, Sackett D, Smith R: Evidence-based medicine: A new journal to help doctors identify the information they need. BMJ 1995;310:1085-1086.

Greenhalgh T: How to Read a Paper: The Basics of Evidence-Based Medicine, ed 3. Malden, MA, Blackwell Publishing, 2006.

Sackett DL, Richardson WS, Rosenberg WS, Haynes RB (eds): Evidence-Based Medicine. London, England, Churchill-Livingstone, 1996.

Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: What it is and what it isn't. BMJ 1996;312:71-72.

Thoma A, Farrokhyar F, Bhandari M, Tandan V: Users' guide to the surgical literature: How to assess a randomized controlled trial in surgery. Can J Surg 2004;47:200-208.