Impact of a Postbaccalaureate Research Program on Medical Career Advancement
Evan T. Zheng, BA, David N. Williams, MA, PhD, Peter M. Waters, MD, MSSc
The authors report no conflict of interest related to this work.
©2020 by The Orthopaedic Journal at Harvard Medical School
PDFPURPOSE The Orthopedic Clinical Effectiveness Research Center (CERC) at Boston Children’s Hospital provides research coordinators the opportunity to gain clinical research experience following their undergraduate education. The purpose of this study was to evaluate the ability of this structured research program to achieve participant goals and facilitate career development.
METHOD The authors conducted a survey of coordinators who have worked at CERC since its inception in 2005. Information regarding coordinator employment objectives, long-term career goals, and demographics was collected. Coordinator career paths and CERC’s perceived effectiveness at facilitating identified goals were also assessed.
RESULTS 59 of 73 (80.8%) coordinators for whom contact information was available completed surveys. Objectives that were particularly important to respondents at the beginning of employment included gaining clinical exposure, gaining research skills, and confirming long term career aspirations. Participants rated their time at CERC as being especially helpful with gaining clinical exposure, gaining research skills, learning about working in an academic hospital, and gaining exposure to the field of orthopedic surgery. Twenty-five of 29 former coordinators (86.2%) who reported an initial goal of attending medical school matriculated following their employment. On a 1-10 scale, the median rating of CERC’s overall influence on facilitating a coordinator’s career goals was 8 (IQR 7 – 9).
CONCLUSION A structured research program designed for recent college graduates interested in a career in medicine was effective at facilitating short-term and long-term participant goals. Academic institutions can use such programs to both aid research and support participant career development.
LEVEL OF EVIDENCE Level lV Case Series
KEYWORDS Medical education, career development, clinical research, medical school admissions, mentorship
The number of annual applicants to U.S. medical schools has increased by nearly 25% since 2010.1 Meanwhile, the number of matriculating students has risen by only about 17%.1 In this context of rising selectivity, more medical students are taking time off between completion of their undergraduate education and enrollment in medical school, with 65% of matriculating medical students in 2019 reporting having taken at least 1 “gap” year.2 While students may take gap years for a variety of reasons and be involved in a range of activities, 45% of students reported being involved in research during this time.2
Although a significant portion of medical students participate in research between their undergraduate education and medical school, little quantitative study has investigated the effectiveness of such efforts.3 In contrast, postbaccalaureate premedical programs, in which around 15% of matriculating medical students participate, have been the subject of numerous prior reports assessing program characteristics and efficacy.4-9 As participation in gap year research endeavors increases, similar analyses of research programs is warranted.
In 2005, the Department of Orthopedic Surgery at Boston Children’s Hospital established an Orthopedic Clinical Effectiveness Research Center (CERC) to facilitate effective collaboration on clinical research studies. Like the department’s clinical activities, CERC is organized into teams including: upper extremity, sports, spine, hip, trauma, lower extremity, tumor, and cerebral palsy (Figure 1). Teams work closely with the department’s pediatric orthopedic surgeons and are led by the CERC manager with support from biostatisticians, trainee associated personnel, and research coordinators. Coordinators come from a variety of backgrounds, but most commonly have completed their undergraduate education and plan to attend medical school.
As part of the original design of CERC through needs assessment analysis, entry level research positions were established for college graduates interested in a health professional career. These positions were intended to give CERC research coordinators exposure and experience in clinical effectiveness research, while providing mentoring on future career options. Coordinators are employed for 1-3 years during their gap year(s) as they decide on their future plans.
CERC research coordinators are hired following a review of application materials including a resume, cover letter, and brief supplemental questions. Candidates also participate in interviews with other research staff and departmental attending physicians. Coordinator responsibilities upon employment include recruiting and consenting patients for research studies, collecting data for ongoing projects, and working closely with other team members to submit and review study protocols. Ultimately, the goal for research coordinators is to present and publish academic work during their time at CERC.
The stated goals of our program include providing mentorship and career advancement to research coordinators via clinical exposure, acquisition of research skills, and academic development through publication. The purpose of this study was to assess our program’s effectiveness at achieving its stated goals, to identify other research coordinator values and objectives, and to assess the program’s impact on coordinator career progression.
The primary approach for our study was to survey all coordinators who have been employed at CERC since its creation in 2005. A survey tool was designed to evaluate coordinator objectives prior to employment, how effective CERC was at facilitating these initial goals, and how employment impacted career trajectories. Institutional review board approval was not required for this quality improvement project per institutional policy.
Survey design began with pre-interviews with a small group of representative research coordinators to better understand both long term career aspirations and short term CERC employment goals.10,11 Through this process, we generated a list of objectives that coordinators described as being of importance to individuals pursuing research positions.
We then constructed a survey with several sections; the first section was designed to assess coordinator objectives at the start/ prior to their employment at CERC. Stated CERC objectives and others identified during pre-interviewing were synthesized into the list presented in Table 1. We asked coordinators to rate how important each of these objectives was to him/ her at the start of employment at CERC on a 1 – 10 scale, with 1 representing “Not important” and 10 representing “Very important”.12 Respondents then had the opportunity to identify and rate any additional objectives that were important to them at the start of their employment. Next we asked participants to identify their desired career position following CERC employment and the degree of certainty in wanting to pursue this goal. A list of possible positions was provided which included Doctor of Medicine (MD), Doctor of Osteopathic Medicine (DO), post-graduate medical education residency, physician assistant, nursing, masters/ other degree, or other work. Survey respondents could then identify and describe any hesitations they had toward pursuing their stated goal.
Fifty-nine of the 73 coordinators (80.8%) for whom contact information was available completed the survey. 39 (66.1%) respondents were female. Most identified as white (n=50, 84.7%), 7 (11.9%) identified as Asian, 3 (5.1%) as Hispanic or Latino, 3 (5.1%) as Black or African American, and 1 (1.7%) as American Indian or Alaska Native.
Survey respondents attended a broad range of undergraduate institutions with 40 different universities reported. Mean undergraduate GPA was 3.62 (sd 0.23) and mean science GPA was 3.56 (sd 0.30). 23 (39.0%) respondents reported enrolling in some form of post-graduate study prior to employment at CERC, and 27 (45.8%) held another position between their undergraduate education and CERC employment. 36 (61.0%) participants reported having taken the MCAT, with a mean reported score percentile of 86.6% (sd 11.2%).
At the start of their time at CERC, most respondents (n=38, 64.4%) aspired to attend medical school immediately following employment. Thirty-seven (62.7%) hoped to attend an MD granting school while 1 (1.7%) had the goal of obtaining a DO degree. Four participants already with MD degrees (6.8%) desired to match to residency following CERC employment. Six respondents (10.2%) wanted to pursue nursing, 3 (5.1%) a career as a physician assistant, 3 (5.1%) a master’s or other degree, and 4 (6.8%) wanted to pursue another job. The median level of certainty in pursuing these stated goals was 9.5/10 (IQR 7.3 – 10). Twenty-one respondents (35.6%) reported having some degree of hesitation toward pursuing their goal, with common themes including the financial and time burden of the education/ training process, an incomplete understanding of different provider types and roles, ability to gain admission to schools or programs, and work life balance.
Analysis of coordinator objectives at the start of employment revealed that gaining clinical exposure, gaining research skills, and confirming or solidifying long term career aspirations were particularly important to respondents. Each of these objectives received a median rating of 10 in importance. Ratings for each of the 14 objectives we assessed are displayed in Table 1.
Median duration of coordinator employment at the time of survey completion was 19.6 months (IQR 12.0mo – 31.6mo). Assessment of how helpful CERC was at aiding respondents with the previously described 14 objectives revealed CERC was particularly helpful with providing their desired clinical exposure, gaining research skills, learning about working in an academic hospital, and gaining exposure to the field of orthopedic surgery in particular. CERC’s helpfulness for each of the 14 objectives assessed are reported in Table 2.
For each of these objectives, we assessed the relationship between how important each was at the start of employment with how helpful CERC was during employment. For this analysis, objectives were classified as being “highly important” if a coordinator rated it as an 8/10 or higher in importance. Similarly, CERC was judged to have been “highly helpful” with an objective if a coordinator rated CERC’s helpfulness an 8/10 or higher. A plot of the proportion of respondents who rated CERC as being “highly helpful” for each objective vs the proportion of respondents who rated the objective as “highly important” is shown in Figure 2.
Twelve (20.3%) respondents stated that their career goals changed or shifted in some way during their time at CERC. Commonly expressed themes included gaining an improved understanding of personal fit among different provider types, the field of orthopedic surgery, research, or academic medicine.
Respondents who had either completed their time at CERC or had identified their positions after employment were asked to describe their career trajectories. Twenty-eight were accepted to/ attended medical school with 24 at MD schools and 3 at DO schools. Median satisfaction with being accepted to these programs was 10 (IQR 9 – 10). Alpha Omega Alpha (AOA) honors were attained by 3 of the 12 (25%) respondents who responded to this question.
Fifteen survey participants matched to residencies since their time at CERC, all of whom reported matching to their desired specialty. A variety of specialties were reported including general surgery (n=3, 20.0%), orthopedic surgery (n=2, 13.3%), pediatrics (n=2, 13.3%), internal medicine – pediatrics (n=2, 13.3%), internal medicine (n=1, 6.7%), family medicine (n=1, 6.7%), emergency medicine (n=1, 6.7%), psychiatry (n=1, 6.7%), anesthesiology (n=1, 6.7%), and obstetrics and gynecology (n=1, 6.7%). Median satisfaction with matching to these programs was 10 (IQR 9 – 10). Four former coordinators had matched to a post-residency fellowship, all in their reported field of choice. Median satisfaction with matching to their respective programs was 10 (IQR 9.5 – 10). Six former coordinators were now attending physicians; median satisfaction with receiving these positions was 10 (IQR 10 – 10).
A proportion of former coordinators did not pursue careers as physicians. Three were physician assistants, with median satisfaction of 10 (IQR 9.5 – 10). Five were employed in nursing, with median satisfaction of 10 (10 – 10). Six pursued a masters or other degree with median satisfaction of 9 (IQR 8 – 10). Eight were employed in other positions with median satisfaction of 10 (IQR 9.5 – 10).
Coordinator Medical School Success
We were particularly interested in the careers of research coordinators who had the initial goal of attending medical school following CERC employment. Twenty-nine survey respondents were former coordinators who had this initial goal. 18 (62.1%) were female; 24 (82.8%) identified as White, 2 (6.9%) as Asian, 2 (6.9%) as Black or African American, 1 (3.4%) as Hispanic or Latino, and 1 (3.4%) as American Indian or Alaska Native.
Academic backgrounds revealed a wide range of undergraduate institutions represented. Mean GPA was 3.61 (sd 0.21) with mean science GPA of 3.60 (sd 0.29). Mean reported MCAT percentile was 89.0% (sd 10.1%), corresponding to a score between 513 and 514 according to AAMC data.20 The median level of participant certainty of wanting to attend medical school was 10 (IQR 8.75 – 10). Seven participants (24%) expressed some hesitations toward this goal, citing common themes that were previously described.
Objectives that were of particular importance to these coordinators at the start of their employment at CERC included gaining clinical exposure, confirming or solidifying long term career aspirations, and producing publications. Ratings for all 14 assessed objectives are reported in Table 1. The median length of time coordinators were employed at CERC was 21.7 months (IQR 13.4mo – 30.4mo). A search of PubMed revealed a median of 4 publications (IQR 2 – 6) per respondent produced from research efforts at CERC.
Survey participants rated CERC as being especially helpful with objectives including gaining clinical exposure, gaining research skills, and collecting letters of recommendation. Complete ratings of the 14 objectives are reported in Table 2. A graph illustrating the relationship between the proportion of coordinators rating CERC as “highly helpful” for each objective and the proportion of coordinators rating each objective as “highly important” is presented in Figure 2. This subset of respondents rated CERC’s overall influence in facilitating their career goals a median score of 9/10 (IQR 7 – 10).
Twenty-five of these 29 (86.2%) coordinators who had the initial goal of attending medical school went on to matriculate. AAMC data of all applicants to U.S. medical schools from 2017 – 2018 through 2019 – 2020 indicate that the acceptance rate for the cohort of applicants with corresponding GPA and MCAT scores (3.60 – 3.79, 514 – 517) was 73.6% (P=0.58).20 Of the 4 coordinators who did not enter medical school, 3 pursued other advanced degrees including a juris doctorate, and a master’s in biomedical engineering. These coordinators who did not attend medical school rated CERC’s overall influence in facilitating their career goals a median of 8.5/10 (IQR 7.75 – 9).
The goals of the CERC program include facilitating research coordinator career progression via clinical exposure, development of research skills, and academic advancement through publication. Our study confirmed the importance of these objectives to coordinators themselves, and also identified other objectives important to this population of largely gap year students. In addition to gaining clinical exposure and research skills, confirming long term career aspirations was of great importance to survey respondents. These three objectives were rated as being “highly important” by around 80% of participants (Figure 2). While coordinators were interested in objectives that have been previously shown to be assessed by medical school admissions committees such as obtaining letters of recommendation and gaining medical/ clinical work experience,21 our data supports the notion that students also pursue gap year positions to gain critical experience to solidify long term career plans.
Our survey results suggest that CERC was able to help coordinators with the particular objectives that were most important to them. Figure 2 demonstrates a strong positive correlation between the proportion of coordinators who viewed an objective as being “highly important” and the proportion of coordinators who viewed CERC as being “highly helpful” with the objective. The objectives “gaining clinical exposure”, “gaining research skills”, and “confirming or solidifying long term career aspirations” located in the upper right quadrant of Figure 2A were both very important and greatly facilitated by employment at CERC. These objectives represent a relative strength of our program.
The ability to help coordinators with the objectives they identified as being important corresponded with a high rating for CERC’s overall influence in facilitating coordinator career goals. Coordinator career paths after CERC employment may also reflect the program’s ability to aid in career progression. Over 86% of former coordinators who had the initial goal of attending medical school matriculated following employment. Despite mean coordinator GPA and MCAT values at the lower end of the corresponding AAMC ranges, our matriculation rate was over 10% higher than the national statistic. Research coordinators also reported continued high levels of achievement following admission to medical school with 25% of respondents being selected to AOA, and all respondents matching to their desired residency and fellowship specialties. While many additional factors including medical school grades, pre and post CERC experiences are likely of equal, if not greater significance in specialty choice, it is notable that coordinators pursued a wide range of specialties following CERC, signifying the broad impact and transferable attributes of even specialized research programs. Additionally, coordinators who pursued careers other than becoming a physician also expressed high levels of satisfaction with their careers after CERC.
While admissions processes are multi-faceted and constantly evolving,21-23 applicants must demonstrate the academic competence, personal qualities, and experiences necessary for admission to medical school. GPA and MCAT are important quantifiable metrics, though prior reports have demonstrated that up to 8 percent of applicants with undergraduate GPA’s ranging from 3.80 – 4.0 and MCAT scores above the 99th percentile were rejected by all schools to which they applied.21 Participation in programs such as CERC likely provides applicants important experience in clinical and academic medicine in addition to access to strong mentorship relationships. As applicants seeking employment to CERC represent a subset of enthusiastic and motivated applicants, additional study will be needed to fully the determine the extent to which such research programs enhance career progression over other gap year activities.
Our program not only aided many coordinators in achieving their original goals, but it also provided valuable experience that facilitated positive changes for those who chose differently. Over 20% of survey respondents reported that their career goals changed or shifted in some way during their employment, which included the 4 past coordinators who had the initial goal of attending medical school but did not end up matriculating. Nonetheless, these coordinators still rated CERC highly in advancing their career goals. Free text responses revealed that through their employment, coordinators were able to better identify what types of activities they did or did not like, thereby helping them inform future plans.
While our analysis revealed that CERC was able to effectively aid coordinators with many aspects of career development, we were also able to identify areas for future improvement. Figure 2A demonstrates that over 50% of coordinators rated “producing publications” and “working in an environment with a strong sense of community” as being “highly important”, but less than 50% of coordinators rated CERC as being “highly helpful” with these objectives. Thoughtfully addressing these two values may lead to impactful changes in coordinator experiences. Our data also reveals that less than 20% of survey respondents viewed CERC as being “highly helpful” with their financial stability/ salary, though only around 30% rated this as being “highly important” to them. Even though this value was not particularly important to our survey respondents, we recognize that a program’s ability or inability to address financial needs may greatly impact the type of student who can participate in such programs. With three quarters of U.S. medical students coming from households with incomes in the top two quintiles,24 a greater awareness of financial factors is necessary to increase the accessibility of similar opportunities to lower income individuals.
In conjunction with other efforts, research programs can be an effective vehicle by which to expedite change in medicine and medical education. Study of postbaccalaureate premedical programs has demonstrated that interventions between college and medical school can have long-reaching impacts such as increasing medical school attendance for targeted populations.4-8 This potential to facilitate change combined with the increasing number of students pursuing gap year research experiences suggests that programs might be used for initiatives such as increasing opportunities and medical school enrollment for groups traditionally underrepresented in medicine.
Our study has several limitations. While our coordinators came from a wide variety of educational and geographic backgrounds, our results reflect one program at a single academic center. Future studies of research programs from multiple institutions including larger samples of participants will help further understand program impacts. Next, participants seeking out opportunities to participate in programs such as CERC likely represent a particularly motivated subset of applicants from which conclusions may not be generalizable. Additionally, though we carefully designed the survey and achieved a response rate over 80%, a response bias might have been present in that those who returned the survey might be more likely to have had favorable experiences. Furthermore choice-supportive bias in this subjective and retrospectively collected assessment may have led respondents to rate participation in a program in which they themselves participated more favorably. Additional study in this area may make use of pre-employment surveys to achieve higher survey response rates and less biased data.
Our study demonstrates that a structured research program within a clinical department can help coordinators progress toward short term and long term career goals. We identified key values important to this population as well as relative strengths and weaknesses of our program. As more prospective medical students pursue gap year research activities, similar structured research programs may be useful tools for both facilitating academic research and driving coordinator career success. Results of our CERC program and analysis should be of use to other subspecialty departments in academic medical centers.
