INTRODUCTION
The evidence-based clinical practice and patient-derived outcomes assessment movements burst onto the scene of clinical medicine in the 1980's and 1990's as a result of contemporaneous technical, societal, and economic influences. Work by Wennberg and colleagues revealed large small-area variations in clinical practice, with some patients thirty times more likely to undergo an operative procedure than other patients with identical symptoms merely because of geographic location. 4-9 Further critical research suggested that up to 40% of some surgical procedures might be inappropriate and that up to 85%of common medical treatments were not rigorously validated. 1-3 Meanwhile, the costs of health care were rapidly rising to over two billion dollars per day, increasing from 5. 2% of the gross domestic product in 1960 to 16. 2%in 1997. As a result of the lack of evidence-based practice and the escalating costs of care, there has been an increased focus on the clinical effectiveness of care.

Clinical epidemiology provides the methodology to assess the clinical effectiveness of care. Epidemiology is the study of the causation and distribution of disease, using statistical and experimental methods. Epidemiologic methods focus on study design, data interpretation, and quantitative methods.

	Figure 1: Children's Hospital Orthopaedic Clinical Effectiveness Research Group

CHOCERG
We are currently developing the Children's Hospital Orthopaedic Clinical Effectiveness Research Group (CHOCERG) . The research group's mission is to improve the effectiveness of management of musculoskeletal conditions in children and adolescents via applied epidemiologic research. The research group's goals are to provide epidemiologic and biostatistical support for pediatric orthopaedic clinical research, to apply advanced epidemiologic methods to pediatric orthopaedic clinical research, and to develop independent fields of inquiry into musculoskeletal conditions affecting public health. Specifically, we are developing study sections in trauma, spinal disorders, hip disorders, upper extremity injuries, and sports medicine. We are developing a clinical research team (Figure 1) and resources to apply advanced

	Figure 2: Diagnostic Performance

epidemiologic methodology including performing randomized clinical trials, prospective cohort studies, and quantitative synthesis methods such as meta-analysis, decision-analysis, and cost-effectiveness analysis. In addition, we intend to pursue fields of inquiry into musculoskeletal conditions affecting public health, including racial variations in the utilization of orthopaedic care, the volume-outcome relationship in orthopaedic care, and the development and validation of patient-derived outcome assessment instruments in pediatric orthopaedics.

EXAMPLES
DIAGNOSTIC PERFORMANCE
The diagnostic performance of an ancillary test involves establishing the sensitivity, specificity, and predictive value of that test (Figure 2) . In a clinical study to evaluate the diagnostic performance of MRI of the knee in children and adolescents, we studied a consecutive series (118 knees in 113 patients) of pediatric patients (<16 years old) treated for intraarticular knee disorders who had an initial clinical diagnosis, followed by a MRI diagnosis, followed by arthroscopic findings. 10 There were 139 clinical lesions, 128 MRI lesions, and 135 arthroscopic lesions. There were no significant differences between clinical exam and MRI with respect to agreement with arthroscopic findings (clinical exam:70%;MRI:74%) , overall sensitivity (clinical exam:71%; MRI:72%) and overall specificity (clinical exam:92%;MRI:94%) . Stratified analysis by diagnosis revealed significant differences only for sensitivity of lateral discoid meniscus and specificity of medial meniscal tears (Figure 3) . MRI had significantly lower sensitivity for younger children (<12 years old:62%;12-16 years old: 78%;p =0. 05) . MRI also demonstrated lower specificity for younger children (<12 years old:90%;12-16 years old: 96%;p =0. 02) . We concluded that selective MRI does not provide enhanced diagnostic utility over clinical exam, particularly in children, and should be used judiciously in cases where the clinical diagnosis is uncertain and MRI input will alter the treatment plan.

Figure 3:Diagnostic Performance of MRI of the Knee in Children and Adolescents Reproduced with permission from the American Journal of Sports Medicine

MULTIVARIABLE METHODS:REGRESSION
Regression analysis is a method of mathematical modeling used to determine independent associations of multiple independent variables with a binary categorical dependent variable (logistic regression) , a continuous dependent variable (linear regression) , an ordinal dependent variable (ordinal logistic regression) , censored time-to-event data (Cox regression) , or count data (Poisson regression) . Regression is commonly used to predict outcomes, or to establish independent associations, controlling for confounding and colinearity.

We have used logistic regression to develop a clinical prediction rule to differentiate between septic arthritis and transient synovitis of the hip in children. 11 We reviewed children who presented to Children's Hospital from 1979 to 1996 with an acutely irritable hip. Diagnoses of true septic arthritis, presumed septic arthritis, and transient synovitis were explicitly defined based on joint fluid white blood cell count, joint fluid and blood cultures, and clinical course. We found that septic arthritis patients differed significantly (p<0. 05) from transient synovitis patients with regard to ESR, serum WBC count and differential, weight-bearing status, history of fever, temperature, effusion on radiograph, history of chills, history of recent antibiotic use, hematocrit, and gender. Four independent multivariate clinical predictors were identified to differentiate between septic arthritis and transient synovitis: history of fever, non weight-bearing, ESR >40mm/hr, and serum WBC count >12, 000/mm 3 . The predicted probability of septic arthritis was determined for all sixteen combinations of these four predictors (Figure 4) and can be summarized as:<0. 2% for 0 predictors, 3% for 1 predictor, 40% for 2 predictors, 93%for 3 predictors, and 99. 8% for 4 predictors. The chi-square test for trend and the area under the receiver operating characteristic (ROC) curve indicated excellent diagnostic performance of this group of multivariate predictors in identifying septic arthritis.

Figure 5: Partial Tears of the ACL in Children and Adolescents		Figure 6: Decision Tree. Operative versus Nonoperative Management of Acute Achilles Tendon Ruptures:Expected Value Decision Analysis.

We have also used logistic regression to determine independent associations, controlling for confounding. In a prospective cohort study of 9,410 professional skiers who underwent preseason knee screening from 1992-1997, we studied the effect of functional knee bracing on subsequent knee injury in ACL deficient (ACLd) skiers. 12 ACL deficiency was operationally defined as an abnormal Lachman or pivotshift examination with >5mm KT-1000 manual maximum side-to-side difference in a skier without prior ACL surgery in either knee. ACLd skiers self-selected the use of functional knee brace during skiing. Controlling for knee laxity, gender, age, ski occupation, Lachman and pivot-shift grades, we found that absence of bracing remained an independent, multivariate risk factor for knee injury. ACLd skiers without bracing were estimated to be over six times more likely to sustain a subsequent knee injury (OR =6. 6;95%CI:1. 5-31. 7) . Because of the increased risk of subsequent knee injury in nonbraced skiers, we recommend functional bracing for ACL deficient skiers.

	Figure 7: One-Way Sensitivity Analysis:Operative versus Nonoperative Management of Acute Achilles Tendon Ruptures:Expected Value Decision Analysis.

SURVIVORSHIP ANALYSIS
Survivorship analysis is used to analyze time-censored data, which poses unique challenges to univariate methods. We have used survivorship analysis to study the functional outcome of partial ACL injuries in children and adolescents treated without reconstruction. 13 A cohort of 45 skeletally mature and immature patients <17 years old who had an acute hemarthrosis, MRI signal changes within the ACL, an IKDC grade A or B Lachman and pivot-shift examination under anesthesia, an arthroscopically documented partial ACL injury, and minimum 2-year follow-up was studied prospectively. All patients were treated without ACL reconstruction and underwent a structured rehabilitation program. Fourteen patients (31%) underwent subsequent reconstruction. Using survivorship analysis, we found that >50%tear (p=0. 01) (Figure 5) , predominantly posterolateral tear (p=0. 01) , and older chronological age (p=0. 01) were independent, timedependent risk factors for subsequent reconstruction.

DECISION ANALYSIS
Expected-value decision analysis is a methodological tool, based in gaming theory, that allows for the quantitative analysis of decision-making under conditions of uncertainty. The process of expected-value decision analysis involves the creation of a decision tree to structure the decision problem, roll-back analysis to calculate expected-value and determine the optimal decision-making strategy, and sensitivity analysis to determine the effect on decision-making of varying outcome probabilities and utilities. We used expected-value decision analysis to assess to determine the optimal management strategy, operative versus nonoperative, for acute Achilles tendon rupture. 14 Outcome probabilities were determined from a previously published quantitative synthesis of 83 published studies of acute Achilles tendon rupture from 1954-97. Patientderived utility values were obtained from a visual analog scale questionnaire of 76 active males, age 30-50 years old without a history of Achilles tendon injury. Roll-back analysis revealed operative treatment to be the optimal management strategy. (EV:6. 52 vs 6. 28;marginal value:0. 24) (Figure 6) . Threshold values were determined for the probability of rerupture from nonoperative treatment (0. 06) , the probability of a moderate complication from operative treatment (0. 16) , the utility of rerupture (3. 3) (Figure 7) , and the utility of a moderate complication (1. 3) . We concluded that operative management was the optimal management strategy for acute Achilles tendon rupture given the outcome probabilities and patient utilities that we studied. Lowering rates of rerupture from nonoperative management or increasing rates of moderate complications from operative management would favor nonoperative management. Patients with a low utility for rerupture favor operative management, whereas patients with a low utility for complications favor nonoperative management. We advocate a model of doctor-patient shared decision-making in which both outcome probabilities and patient preferences are considered in order to optimize the decision process.

COST-EFFECTIVENESS ANALYSIS
Economic analysis of medical procedures includes costidentification studies which determine costs and cost-evaluation studies which assess both costs and benefits. Cost evaluative study designs include cost-benefit analysis, costeffectiveness analysis, and cost-utility analysis. In cost-benefit analysis, both costs and benefits are analyzed in terms of monetary units. In cost-effectiveness analysis, a ratio is determined of cost per health effect. In cost-utility analysis, this health effect is measured as quality-adjusted life years (QALY's) . We studied the cost and effectiveness of routine pathological examination of surgical specimens from primary total hip and knee replacement for osteoarthritis by comparing clinical and pathological diagnoses in 1,234 consecutive cases of primary THR (n=471) and TKR (n=763) performed between 1992 and 1995 at the Brigham and Women's Hospital for the clinical diagnosis of osteoarthritis. 15 We found a low prevalence of discrepant (2. 3%) and discordant (0. 1%) diagnoses. The cost per discrepant diagnosis was $4, 383 and the cost per discordant diagnosis was $122, 728. We concluded that routine pathological examination of surgical specimens from primary THR and TKR for the clinical diagnosis of osteoarthritis had limited cost-effectiveness at our hospital due to the low prevalence of findings that altered patient management.

	Figure 8: Randomized Clinical Trial study schema

OUTCOMES ASSESSMENT
Health care outcomes assessment measures include generic measures, condition-specific measures, and measures of patient satisfaction. The legitimacy and importance of patient satisfaction as an outcome measure has burgeoned over the past decade because of its use for a variety of purposes such as indicating quality of care, assessing health care delivery, developing patient care models, impacting general health status, and allowing for continuous quality improvement. Furthermore, as the paradigm for health care has shifted toward a market model, patient satisfaction has become an outcome measure with great clinical and economic implications. We studied the determinants of patient satisfaction with outcome after ACL reconstruction in a cohort of 202 patients with minimum two year follow-up and complete demographic, surgical, subjective, and objective data. 16 We found that some specific objective variables were associated with satisfaction, however subjective variables of symptoms and function had the most robust associations with patient satisfaction. The seven independent multivariate determinants (adjusted R 2 = 0. 83, p<0. 001) of patient satisfaction included Lysholm score, overall subjective knee function, IKDC range-of-motion subscale, patella tenderness, full giving-way, flexion contracture, and swelling.

STUDY DESIGN
Associations between exposures and outcomes can be spurious due to chance, confounding, and bias. In hypothesis testing, the probability of a finding an association in a study by chance when, in reality, there is no association (type I error) is estimated by alpha, which is the p-value. The probability of finding no association in a study when, in reality, there is an association (type II error) is estimated by beta, which is 1 -power. Bias is nonrandom systematic error in design or conduct of a study. There are many forms of bias including selection bias, nonresponder bias, state of health bias, detection bias, performance bias, recall bias, acceptability bias, and publishing bias. A confounding variable has associations with both the exposure and the outcome and thus may distort their relationship. Frequent confounders include age, gender, socioeconomic status, and comorbidities.

Studies may be performed prospectively or retrospectively. Study designs may be observational, such as case reports, retrospective case series, case-control studies, prospective cohort studies, and cross-sectional studies, or they may be experimental, such as randomized clinical trials. Bias and confounding can be minimized through study design, with methods such as randomization, blinding, stratification, and striving for similar follow-up and assessment of groups.

The randomized clinical trial (RCT) is the highest level of clinical evidence because it minimizes bias and confounding. All variables between the groups are theoretically distributed equivalently, except the allocated study variable, because of randomization. Disadvantages of RCT's include difficulty, expense, length of time for study, ethical considerations, lack of acceptance by patients and clinicians, and lack of external generalizability. The design of a randomized clinical trial involves establishing explicit inclusion criteria, pre-hoc sample size calculation, patient accrual, blinded outcome assessment, interim analysis with stopping rules, and intent to treat analysis.

Mininder S. Kocher, MD, MPH is an Instructor in Orthopaedic Surgery at Harvard Medical School, Program in Clinical Effectiveness at Harvard School of Public Health, and Director of Children's Hospital Orthopaedic Clinical Effectiveness Research Group.

Address correspondence to:

Mininder S. Kocher, MD, MPH
Department of Orthopaedic Surgery
The Children's Hospital
300 Longwood Avenue
Boston, MA 02115

Print Manuscript   •  View References  •  Download PDF version   •  Close window