High-Yield Orthopaedics: A New Orthopaedic In-Training Examination (OITE) Review Handbook for Residents
Hai V. Le, MD, Patrick K. Cronin, MD, Stella J. Lee, MD, Shaina A. Lipa, MD, Bonnie Y. Chien, MD Nishant Suneja, MD, George S.M. Dyer, MD, FACS
©2019 by The Orthopaedic Journal at Harvard Medical School
BACKGROUND The Miller’s Review of Orthopaedics (MRO) and the AAOS Comprehensive Orthopaedic Review (COR) are two commonly used review books for the Orthopaedic In-Training Examination (OITE). However, both resources are quite extensive in terms of book length. The authors recently published a concise OITE review handbook titled High-Yield Orthopaedics (HYO). This study aimed to validate HYO by determining the prevalence of 2018 OITE answers found in the handbook.
METHODS The 2018 OITE questions were categorized into one of three question types by the first author. Taxonomy 1 (T1) questions tested recall or recognition; taxonomy 2 (T2) questions tested interpretation; and taxonomy 3 (T3) questions tested problem-solving. Two authors (HVL and PKC) then independently reviewed HYO to determine if answers to the 2018 OITE questions were available in the handbook. Interobserver reliability was assessed via percent agreement and Cohen’s kappa (κ) coefficient. Discrepancies were resolved after joint review.
RESULTS Of the 270 OITE questions available for review, 97 (35.9%) were classified as T1 questions, 85 (31.5%) as T2 questions, and 88 (32.6%) as T3 questions. The answers to 198 OITE questions (73.3%) were detected in HYO—58 answers to T1 questions (59.8%), 63 answers to T2 questions (74.1%), and 77 answers to T3 questions (87.5%). The concordance between the two reviewers was 258 out of 270 questions (95.6%) with κ coefficient of 0.88.
CONCLUSION Compared to MRO and COR, High-Yield Orthopaedics is a more concise resource that still comprehensively reviews all 11 knowledge domains tested on the OITE. Answers to 198 of the 270 questions (73.3%) on the 2018 OITE were found in the handbook. HYO is a valid and high-yield review material for OITE preparation.
LEVEL OF EVIDENCE Level V Expert Opinion
KEYWORDSOITE, ABOS, AAOS, orthopaedic, orthopaedic surgery, in-training exam, review book, handbook, high-yield
The Orthopaedic In-Training Examination (OITE) is a standardized test administered by the American Academy of Orthopaedic Surgeons (AAOS) annually to orthopaedic residents across 20 countries including the United States and Canada.1 The 275-question exam covers 11 domains of orthopaedic knowledge: basic science, foot and ankle, hand, hip and knee, oncology, pediatrics, shoulder and elbow, spine, sports medicine, trauma, and practice management. The OITE has been validated2 and is considered predictive of success in orthopaedic surgery residency.3
Currently, there are several review books available for OITE preparation. The two most commonly utilized print resources are the Miller’s Review of Orthopaedics (MRO) and the AAOS Comprehensive Orthopaedic Review (COR).4 Although MRO and COR are valuable and effective study materials, both resources are quite extensive in terms of book length. The 7th Edition of MRO comprises 904 pages, and the 2nd Edition of COR (3 volume set) comprises 1300 pages (Table 1). Therefore, the authors aimed to design an OITE review resource that is comprehensive yet succinct.
In 2019, High-Yield Orthopaedics (HYO) was published as a 148-page OITE review handbook for orthopaedic providers.5 To test the validity of HYO as a review resource for the OITE, the authors examined the prevalence of answers to 2018 OITE questions found in the handbook.
The first edition of High-Yield Orthopaedics: OITE & ABOS Review for Orthopaedic Providers was published in 2019 by Kindle Direct Publishing (Figure 1). At a length of 148 pages, the included content was primarily based on practice exam questions from AAOS ResStudy6 (including questions from the 2012-2017 OITEs) and Orthobullets7. Consistent with the formal OITE subsections, HYO is divided into 11 domains of orthopaedic surgery knowledge.
In this validation study, questions on the 2018 OITE were categorized by the first author according to the taxonomy originally described by the AAOS during development of the OITE.8 Under this classification:
- Taxonomy 1 (T1) questions test recall or recognition
- Taxonomy 2 (T2) questions test interpretation
- Taxonomy 3 (T3) questions test problem-solving
Next, HYO was reviewed to determine whether answers to the 2018 OITE questions were available in the handbook. This was completed by two independent reviewers (HVL and PKC), and any discrepancy was resolved following joint review. Of note, while HVL and PKC both authored this study, only HVL authored HYO. Interobserver reliability was assessed via calculation of percent agreement and Cohen’s kappa (κ) coefficient. The prevalence of answers to 2018 OITE questions in HYO was calculated as a percentage.
The 2018 OITE consisted of 275 multiple-choice questions. Of those, 5 questions were omitted by the AAOS, and the remaining 270 questions were included in this study. Of the 270 questions, 97 (35.9%) were classified as T1 questions, 85 (31.5%) as T2 questions, and 88 (32.6%) as T3 questions. HVL and PKC detected answers to 198 and 202 OITE questions in HYO, respectively (Figure 2). After joint review, the authors determined that the answers to 198 OITE questions (73.3 %) were found in HYO. This comprised answers to 58 T1 questions (59.8%), 63 T2 questions (74.1%), and 77 T3 questions (87.5%) (Table 2). Percent agreement between the two reviewers for all questions was 95.6% (258 out of 270 questions) with κ coefficient of 0.88 (95% CI 0.82 – 0.95).
Yes and No indicate whether or not answers were found in HYO
The OITE was first administered by the AAOS in 1963 and was designed to evaluate orthopaedic residents’ ability to “recall information, apply knowledge, and interpret clinical data to solve a problem”.9 The OITE also serves as a metric for orthopaedic residency programs to assess their residents' Accreditation Council for Graduate Medical Education (ACGME) core competency of medical knowledge.1 A major predictor of the American Board of Orthopaedic Surgery (ABOS) Part I performance is performance on the OITE.10,11 Dougherty et al. found that residents who averaged in the 27th percentile or lower on the OITE had a 57% likelihood of failing ABOS Part I.11 In 2018, 9% (75 of 835) of all examinees failed ABOS Part I.12 For these reasons, a stronger emphasis on OITE performance is needed across all residency programs.
Intuitively, residents who place a greater emphasis on the OITE and devote more time for exam preparation perform better on the OITE.13,14 For preparation, there are currently many available online and print resources. As previously noted, the commonly utilized MRO and CRO textbooks are comprehensive but also lengthy.4 MRO is 904 pages, while COR is 1300 pages. HYO, on the other hand, is 148 pages (Table 1). The authors were able to condense content without sacrificing comprehensiveness by structuring the handbook in bullet-point format. Additionally, HYO achieved brevity by focusing primarily on topics that were tested on the recent 2012-2017 OITEs.
To validate the new resource, this study sought to compare HYO with MRO and COR in terms of review content for the OITE. Since the leading author of this study (HVL) authored HYO, an independent reviewer (PKC) was asked to analyze the handbook in order to avoid conflicts of interest. Overall, the answers to 198 of the 270 OITE questions (73.3%) were detected in the handbook. The percent agreement between the two reviewers was high at 95.6% of the OITE questions examined with κ coefficient of 0.88 supporting the strength of agreement between the two reviewers.15 HYO provided answers to a higher proportion of T3 questions (87.5%) compared to T1 or T2 questions (59.8% and 74.1%, respectively). In a similar study examining OITEs from 2006 to 2010, Krueger et al. found that MRO and COR contained answers to 60% and 62% of the questions, respectively.4
The OITE is an important test that evaluates orthopaedic residents’ ACGME core competency of medical knowledge and is a predictor of ABOS Part I performance. While MRO and COR are valuable OITE review textbooks, the authors recently published a handbook that is more concise while still comprehensively reviewing the knowledge domains tested on the OITE. HYO is a valid and high-yield review material for OITE preparation.