Scientific Article

Volume 20 • June 2019

High-Yield Orthopaedics: A New Orthopaedic In-Training Examination (OITE) Review Handbook for Residents

Hai V. Le, MD, Patrick K. Cronin, MD, Stella J. Lee, MD, Shaina A. Lipa, MD, Bonnie Y. Chien, MD Nishant Suneja, MD, George S.M. Dyer, MD, FACS

Disclosure Information

©2019 by The Orthopaedic Journal at Harvard Medical School

OITE Review PDF

ABSTRACT

BACKGROUND The Miller’s Review of Orthopaedics (MRO) and the AAOS Comprehensive Orthopaedic Review (COR) are two commonly used review books for the Orthopaedic In-Training Examination (OITE). However, both resources are quite extensive in terms of book length. The authors recently published a concise OITE review handbook titled High-Yield Orthopaedics (HYO). This study aimed to validate HYO by determining the prevalence of 2018 OITE answers found in the handbook.

METHODS The 2018 OITE questions were categorized into one of three question types by the first author. Taxonomy 1 (T1) questions tested recall or recognition; taxonomy 2 (T2) questions tested interpretation; and taxonomy 3 (T3) questions tested problem-solving. Two authors (HVL and PKC) then independently reviewed HYO to determine if answers to the 2018 OITE questions were available in the handbook. Interobserver reliability was assessed via percent agreement and Cohen’s kappa (κ) coefficient. Discrepancies were resolved after joint review.

RESULTS Of the 270 OITE questions available for review, 97 (35.9%) were classified as T1 questions, 85 (31.5%) as T2 questions, and 88 (32.6%) as T3 questions. The answers to 198 OITE questions (73.3%) were detected in HYO—58 answers to T1 questions (59.8%), 63 answers to T2 questions (74.1%), and 77 answers to T3 questions (87.5%). The concordance between the two reviewers was 258 out of 270 questions (95.6%) with κ coefficient of 0.88.

CONCLUSION Compared to MRO and COR, High-Yield Orthopaedics is a more concise resource that still comprehensively reviews all 11 knowledge domains tested on the OITE. Answers to 198 of the 270 questions (73.3%) on the 2018 OITE were found in the handbook. HYO is a valid and high-yield review material for OITE preparation.

LEVEL OF EVIDENCE Level V Expert Opinion

KEYWORDSOITE, ABOS, AAOS, orthopaedic, orthopaedic surgery, in-training exam, review book, handbook, high-yield

 

The Orthopaedic In-Training Examination (OITE) is a standardized test administered by the American Academy of Orthopaedic Surgeons (AAOS) annually to orthopaedic residents across 20 countries including the United States and Canada.1 The 275-question exam covers 11 domains of orthopaedic knowledge: basic science, foot and ankle, hand, hip and knee, oncology, pediatrics, shoulder and elbow, spine, sports medicine, trauma, and practice management. The OITE has been validated2 and is considered predictive of success in orthopaedic surgery residency.3

Currently, there are several review books available for OITE preparation. The two most commonly utilized print resources are the Miller’s Review of Orthopaedics (MRO) and the AAOS Comprehensive Orthopaedic Review (COR).4 Although MRO and COR are valuable and effective study materials, both resources are quite extensive in terms of book length. The 7th Edition of MRO comprises 904 pages, and the 2nd Edition of COR (3 volume set) comprises 1300 pages (Table 1). Therefore, the authors aimed to design an OITE review resource that is comprehensive yet succinct.

TABLE 1
Comparison of page length and book dimensions between the most recent edition of HYO, MRO, and COR
OITE Review Table 1
All Rights Reserved. Permission For Use Required.

 

In 2019, High-Yield Orthopaedics (HYO) was published as a 148-page OITE review handbook for orthopaedic providers.5 To test the validity of HYO as a review resource for the OITE, the authors examined the prevalence of answers to 2018 OITE questions found in the handbook.

MATERIALS AND METHODS

The first edition of High-Yield Orthopaedics: OITE & ABOS Review for Orthopaedic Providers was published in 2019 by Kindle Direct Publishing (Figure 1). At a length of 148 pages, the included content was primarily based on practice exam questions from AAOS ResStudy6 (including questions from the 2012-2017 OITEs) and Orthobullets7. Consistent with the formal OITE subsections, HYO is divided into 11 domains of orthopaedic surgery knowledge.

In this validation study, questions on the 2018 OITE were categorized by the first author according to the taxonomy originally described by the AAOS during development of the OITE.8 Under this classification:

  • Taxonomy 1 (T1) questions test recall or recognition
  • Taxonomy 2 (T2) questions test interpretation
  • Taxonomy 3 (T3) questions test problem-solving
FIGURE 1
Cover page for High-Yield Orthopaedics: OITE & ABOS Review for Orthopaedic Providers
OITE Review Figure 1
All Rights Reserved. Permission For Use Required.

 

Next, HYO was reviewed to determine whether answers to the 2018 OITE questions were available in the handbook. This was completed by two independent reviewers (HVL and PKC), and any discrepancy was resolved following joint review. Of note, while HVL and PKC both authored this study, only HVL authored HYO. Interobserver reliability was assessed via calculation of percent agreement and Cohen’s kappa (κ) coefficient. The prevalence of answers to 2018 OITE questions in HYO was calculated as a percentage.

RESULTS

The 2018 OITE consisted of 275 multiple-choice questions. Of those, 5 questions were omitted by the AAOS, and the remaining 270 questions were included in this study. Of the 270 questions, 97 (35.9%) were classified as T1 questions, 85 (31.5%) as T2 questions, and 88 (32.6%) as T3 questions. HVL and PKC detected answers to 198 and 202 OITE questions in HYO, respectively (Figure 2). After joint review, the authors determined that the answers to 198 OITE questions (73.3 %) were found in HYO. This comprised answers to 58 T1 questions (59.8%), 63 T2 questions (74.1%), and 77 T3 questions (87.5%) (Table 2). Percent agreement between the two reviewers for all questions was 95.6% (258 out of 270 questions) with κ coefficient of 0.88 (95% CI 0.82 – 0.95).

FIGURE 2
Prevalence of answers to 2018 OITE questions in HYO
OITE Review Figure 2

Yes and No indicate whether or not answers were found in HYO

All Rights Reserved. Permission For Use Required.

 

TABLE 2
Table 2 summarizes the variables of interest and results of this study.
OITE Review Table 2
All Rights Reserved. Permission For Use Required.

 

DISCUSSION

The OITE was first administered by the AAOS in 1963 and was designed to evaluate orthopaedic residents’ ability to “recall information, apply knowledge, and interpret clinical data to solve a problem”.9 The OITE also serves as a metric for orthopaedic residency programs to assess their residents' Accreditation Council for Graduate Medical Education (ACGME) core competency of medical knowledge.1 A major predictor of the American Board of Orthopaedic Surgery (ABOS) Part I performance is performance on the OITE.10,11 Dougherty et al. found that residents who averaged in the 27th percentile or lower on the OITE had a 57% likelihood of failing ABOS Part I.11 In 2018, 9% (75 of 835) of all examinees failed ABOS Part I.12 For these reasons, a stronger emphasis on OITE performance is needed across all residency programs.

Intuitively, residents who place a greater emphasis on the OITE and devote more time for exam preparation perform better on the OITE.13,14 For preparation, there are currently many available online and print resources. As previously noted, the commonly utilized MRO and CRO textbooks are comprehensive but also lengthy.4 MRO is 904 pages, while COR is 1300 pages. HYO, on the other hand, is 148 pages (Table 1). The authors were able to condense content without sacrificing comprehensiveness by structuring the handbook in bullet-point format. Additionally, HYO achieved brevity by focusing primarily on topics that were tested on the recent 2012-2017 OITEs.

To validate the new resource, this study sought to compare HYO with MRO and COR in terms of review content for the OITE. Since the leading author of this study (HVL) authored HYO, an independent reviewer (PKC) was asked to analyze the handbook in order to avoid conflicts of interest. Overall, the answers to 198 of the 270 OITE questions (73.3%) were detected in the handbook. The percent agreement between the two reviewers was high at 95.6% of the OITE questions examined with κ coefficient of 0.88 supporting the strength of agreement between the two reviewers.15 HYO provided answers to a higher proportion of T3 questions (87.5%) compared to T1 or T2 questions (59.8% and 74.1%, respectively). In a similar study examining OITEs from 2006 to 2010, Krueger et al. found that MRO and COR contained answers to 60% and 62% of the questions, respectively.4

CONCLUSION

The OITE is an important test that evaluates orthopaedic residents’ ACGME core competency of medical knowledge and is a predictor of ABOS Part I performance. While MRO and COR are valuable OITE review textbooks, the authors recently published a handbook that is more concise while still comprehensively reviewing the knowledge domains tested on the OITE. HYO is a valid and high-yield review material for OITE preparation.

REFERENCES
1. Marsh JL, Hruska L, Mevis H. An electronic orthopaedic in-training examination. J Am Acad Orthop Surg. 2010 Oct;18(10):589-96.
2. Buckwalter JA, Schumacher R, Albright JP, Cooper RR. The validity of orthopaedic in-training examination scores. J Bone Joint Surg Am. 1981 Jul;63(6):1001-6.
3. Spitzer AB, Gage MJ, Looze CA, Walsh M, Zuckerman JD, Egol KA. Factors associated with successful performance in an orthopaedic surgery residency. J Bone Joint Surg Am. 2009 Nov;91(11): 2750-5.
4. Krueger CA, Shakir I, Fuller BC. Prevalence of answers to orthopaedic in-training examination questions in 3 commonly used orthopedic review sources. Orthopedics. 2012 Sep;35(9):e1420-6.
5. Le HV, Tsai C, Lee SJ, Chien BY, Lipa SA, Qudsi RA, Suneja N, Dyer GSM. High-Yield Orthopaedics: OITE & ABOS Review for Orthopaedic Providers [Kindle]. Seattle, WA: Kindle Direct Publishing; 2019. 148 p.
6. American Academy of Orthopaedic Surgeons. ResStudy [Internet]. Rosemont (IL): American Academy of Orthopaedic Surgeons; 2019 [cited 30 Mar 2019] Available from: http://learn.aaos.org.
7. Lineage Medical, Inc. Orthobullets [Internet]. Santa Barbara (CA): Lineage Medical, Inc; 2019. [cited 30 Mar 2019] Available from: https://www.orthobullets.com.
8. Buckwalter JA, Schumacher R, Albright JP, Cooper RR. Use of an educational taxonomy for evaluation of cognitive performance. J Med Educ. 1981 Feb;56(2):115-21.
9. American Academy of Orthopaedic Surgeons. Orthopaedic In-Training Scored and Recorded Examination 2018 (OITE) [Internet]. Rosemont (IL): American Academy of Orthopaedic Surgeons; 2019 [cited 19 Mar 2019] Available from: https://www.aaos.org/store/product/?productId=10785326.
10. Herndon JH, Allan BJ, Dyer G, Jawa A, Zurakowski D. Predictors of success on the American Board of Orthopaedic Surgery examination. Clin Orthop Relat Res. 2009 Sep;467(9):2436-45.
11. Dougherty PJ, Walter N, Schilling P, Najibi S, Herkowitz H. Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: a multicenter study. Clin Orthop Relat Res. 2010 Oct;468(10):2797-802.
12. The American Board of Orthopaedic Surgery. Part I Examination Statistics [Internet]. Chapel Hill (NC): The American Board of Orthopaedic Surgery; 2019 [cited 30 Mar 2019] Available from: https://www.abos.org/certification-exams/part-i/exam-statistics.aspx.
13. Miyamoto RG Jr, Klein GR, Walsh M, Zuckerman JD. Orthopedic surgery residents' study habits and performance on the orthopedic in-training examination. Am J Orthop (Belle Mead NJ). 2007 Dec;36(12):E185-8.
14. Evaniew N, Holt G, Kreuger S, Farrokhyar F, Petrisor B, Dore K, Bhandari M, Ghert M. The orthopaedic in-training examination: perspectives of program directors and residents from the United States and Canada. J Surg Educ. 2013 Jul-Aug;70(4):528-36.
15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977 Mar;33(1):159-74.