Residency programs aim to provide residents with the knowledge, skills, and surgical proficiency necessary to become a competent physician. However, duty hour requirements may limit a resident’s surgical experience. There is increasing concern that surgical residents may not achieve the training needed to produce confident, independent, and safe practitioners. Similarly, residents often lack the opportunity to adequately train for low volume procedures due to their limited first-hand exposure. This challenge demands the medical community’s attention.^1,2 Mattar et al.³ presented results of a survey of surgical fellowship program directors, in which only 33.7% felt that graduating general surgery residents could perform 30 minutes of a major procedure independently. Nearly one third felt that incoming fellows could not independently perform a basic procedure such as laparoscopic cholecystectomy, and over 20% of respondents believed that graduating residents were unable to recognize the early signs of complications.

Simulation training has been shown to be valuable for surgical trainees, reducing error rates and increasing efficiency.⁴ When simulation is not available either due to time or logistical constraints, high-quality surgical videos, albeit an imperfect substitute for hands-on training, provide an alternative means of surgical education that was found to be useful for surgical training. They provide residents with a better view of the procedure, allow for an increased number of observers, and offer exposure to a wide range of procedures.⁵ However, it is important to note that surgical videos can never replace in person training. They may serve as a resource for residents to view procedures, augment their experience in the operating room and potentially expand the breadth of surgical exposure. It is particularly useful in practices with a high ratio of residents to attendings, where observation may be a role for residents in the operating room.⁵

Surgical video offers advantages for attendings as well. Intra-disciplinary communication and collaboration have always been a cornerstone of scientific progress, but surgeons are limited by physical constraints. In order to observe new procedures, they have to travel to different institutions. This barrier ultimately restricts advancement in surgical technique and, more importantly, hinders surgeons in their attempt to adopt new, more successful methods in their own practices. Surgical video offers a medium for the dissemination of new surgical procedures, fostering further collaboration and innovation.

Today, many resources for surgical videos are accessible online, each with its own purpose and intended audience. The heterogeneity of surgical video demonstrates the need for an evaluation method. This review examines several surgical video libraries with an emphasis on orthopedic surgery, although all specialties provided by each library were assessed. The goal of this review is to:

1. Develop a system for evaluating surgical video libraries
2. Apply this system to a small cohort of libraries
3. Discuss the different qualities of said libraries

A small cohort of eight surgical video libraries (Table 1) was selected by identifying libraries that met the following criteria:

1. Free access or subscription available through the author’s institution
2. Emphasized or at least contained orthopedic surgical videos
3. Not operated by a specific academic institution

TABLE 1

Resource information

All Rights Reserved. Permission For Use Required.

-View Large-

The Disaster Anatomy session took place at a Smith & Nephew facility in Andover, MA, A sample size of fifteen surgical videos were randomly selected from each of these libraries. An attempt was made to create a balanced sample by selecting videos with varying length, specialty, and from different time frames, with an emphasis on orthopedics. Other types of videos such as product advertisements and lectures were excluded. We have developed a rating scheme to evaluate the volume, quality, and value of surgical videos in an attempt to create a standardized means of comparing video libraries. The individual procedures used for each of the three categories are outlined below.

Volume

For most of the surgical libraries, the total number of videos is readily advertised. However, when this information was not available, the number of videos was counted. A volume rating was assigned based on the total number of surgical videos provided by each resource, according to (Table 2).

TABLE 2

Volume rating system

All Rights Reserved. Permission For Use Required.

-View Large-

Quality

To evaluate the quality of the videos hosted by each resource, the American Academy of Orthopedic Surgeons (AAOS) guidelines for Peer Review/Evaluation of Video Submissions⁶ were applied to the 15 sample videos. The AAOS guidelines include 28 points for evaluating surgical videos (Table 3). Because the number of points relevant to an individual video varied, each video was evaluated based on the percentage of relevant points that were satisfied. We developed this modification in an attempt to evaluate all surgical videos equally and to accommodate circumstances in which some points were not applicable. The median and interquartile range of this percentage was calculated across the 15 sample videos. A rating, marked on a scale of one to five, was assigned to each resource, based on Table 4.

TABLE 3

AAOS evaluation guidelines⁶

All Rights Reserved. Permission For Use Required.

-View Large-

TABLE 4

Quality rating system

All Rights Reserved. Permission For Use Required.

-View Large-

Value

The value rating, marked on a half-integer scale from 1-5, was determined by the fulfillment of ten criteria. Each criterion satisfied contributes 0.5 to the value rating. The criteria are as follows:

1. Resource includes a peer-review process
2. Majority of articles includes an adequately detailed written component
3. Resource has a quality rating ≥ 3
4. Majority of articles includes a didactic narration
5. ≤ 25% of videos were submitted before 2005
6. Resource has volume rating ≥ 3
7. Resource includes Editorial Board
8. Resource offers Continuing Medical Education (CME) credit
9. Resource avoids promotional content/industry influence
10. Resource includes at least 5 specialties

1. ClinicalKey – Procedures Consult

ClinicalKey was developed with input from over 2,000 doctors aiming to provide a medical database for physicians, healthcare executives, medical librarians, and students. It also provides an excellent search function and CME credit. It is important to note that this review only examines the 100 videos available through “Procedures Consult.” ClinicalKey also offers over 4,000 surgical video clips. Although many are high quality, they are typically less than a minute in length and were not considered relevant for the purposes of this review. The surgical videos offered through Procedures Consult are accompanied by a long, comprehensive outline detailing the procedure including an introduction, anatomical figures, fluoroscopic images, equipment, potential complications, and postoperative care. For these reasons, ClinicalKey scored the highest in both quality and value (4 for both, Table 5).

TABLE 5

Quantitative results

All Rights Reserved. Permission For Use Required.

-View Large-

2. MedlinePlus

MedLinePlus provides detailed articles on 959 diseases, conditions and drugs, 950 of which are available in Spanish as well. In terms of surgical video, MedLinePlus offers a page with nearly a thousand direct links to articles on ORLive (see below). MedlinePlus does not include advertisements nor does it link to the large branding component of ORLive. MedlinePlus scored the second highest in the quality (3) and value (3, Table 5). Due to the large volume of videos offered (scored 4), MedlinePlus had the highest combined quantitative score.

3. ORLive/BroadcastMed

ORLive primarily serves as a search engine for surgical videos published on BroadcastMed. BroadcastMed hosts live webcasts of numerous surgeries, with the audience’s questions and comments directly relayed to the surgeon via a reporter. ORLive provides access to a wide variety of videos, lectures, and slideshows covering numerous specialties. With its affiliation to over 40 prominent medical institutions, ORLive offers a wide variety of reliable, educational content. The mixture of high quality, educational videos with short or promotional clips resulted in intermediate quality (2) and value (2.5) scores for ORLive (Table 5).

4. SurgeryTheater

In addition to surgical video, SurgeryTheater includes a blog, photo albums, and audio recordings. It also provides the “Master Series Videos” that include high quality footage of full length procedures with didactic narration, offering superior educational and instructive value. Although SurgeryTheater boasts an impressive volume of surgical videos (over 12,000), the majority have poor video quality and are short or incomplete. Moreover, many do not include instructive narration, resulting in a high variability in educational value. Finally, SurgeryTheater includes a significant amount of promotional videos and advertisements. For these reasons, it scored the lowest (along with YouTube), in quality (1) and value (1.5, Table 5).

5. Video Journal of Orthopaedics (VJOrtho)

The Video Journal of Orthopaedics (VJOrtho) offers professionally edited, peer-reviewed surgical videos in the field of orthopedics, developed with an emphasis on education. A multimedia alliance with the Journal of Bone and Joint Surgery (JBJS) was formed in 2000, in which JBJS linked some of its published, relevant articles to a video synopsis on VJOrtho, performed by the author. Although this alliance was dissolved for unknown reasons, select JBJS articles are still available. Each JBJS video includes a link to the full JBJS article associated with it, providing access to peer-reviewed, reliable information. VJOrtho scored second highest in quality (3), tied with MedlinePlus, but had a relatively low value score (2, Table 5).

6. VuMedi

VuMedi, a free resource exclusively for physicians, residents, and fellows, was originally pitched as a “YouTube for surgeons.” With features such as polling, private chats, forums, and weekly discussions on published articles, VuMedi excels at promoting discussion within its large international community of over 100,000 physicians. VuMedi offers over 5,000 surgical videos, some of which are high quality and include thorough, didactic narration. Many of the best videos on the site are from the 38 hospitals and clinics that are partnered with VuMedi. However, the majority of videos have poor video quality and provide only short summaries or outlines of the procedure rather than a comprehensive video. VuMedi is partnered with 16 surgical brands, leading to a significant amount of promotional content. VuMedi earned an intermediate score of 2 in quality and 2.5 in value (Table 5). Combined with the high volume score of 5, VuMedi had the second highest combined score (9.5).

7. WebSurg

WebSurg, a “virtual surgical university,” is currently the most widely subscribed international e-learning website, with over 323,000 members. This resource offers free access to minimally invasive surgical videos within 10 surgical specialties. WebSurg also offers customized “Master Classes” and 275 HD videos available for purchase. These training modules include comprehensive tools such as grand rounds, operating techniques, surgical videos, and lectures on a specific topic within a specialty. All content is assessed to meet Health on the Net Foundation (HONCode) standards to ensure reliability. However, the majority of videos are short and generally incomplete, ultimately earning a quality score of 1 (Table 5). It is important to note that the editorial board contributes significant amount of content to the site as does numerous medical brands.

8. YouTube

YouTube is a popular site for viewing surgical videos due the volume offered and free access. As one of the largest collections of online videos, the site offers a wide range of specialties and procedures, including a few yet elusive high quality, instructive videos. The lack of any submission requirements makes it difficult to distinguish academically valuable videos from outdated or unprofessional ones. Moreover, many surgical videos from around the world appear to be filmed in non-standard conditions with questionable methods. There is also a significant amount of promotional content from medical brands.

Figure 1 shows the quantitative results from this study. ClinicalKey scored the highest in both quality and value (4 for both), yet scored the lowest in volume (1). MedlinePlus scored the next highest in quality and value (3 for both), and with a higher score in volume (4), achieved the highest combined quantitative score (10). Several resources, SurgeryTheater, VuMedi, Websurg, and YouTube scored a 5 for the volume rating. However, SurgeryTheater and YouTube had the lowest scores in quality (1 for both) and value (1.5 for both).

FIGURE 1

Summary of results

This figure compares the quality, value, and volume ratings across different resources. The individual values for each category is inset within the bars, while the total quantitative score is represented by the height of the bar.

All Rights Reserved. Permission For Use Required.

-View Large-

This review reports on the type, volume, quality and value of surgical videos available using eight popular online resources. The value rating captures components of the resources that are not reflected in individual videos but are considered important to the overall instructive and educational value of the site, especially those aspects that are insufficiently represented in the AAOS guidelines. Of the eight resources studied, there was no single resource that scored the highest in all three categories, demonstrating the wide variability in value, quality, and volume in online surgical video libraries.

In terms of quality, ClinicalKey was rated the highest with a median score of 73% (Table 5). The surgical videos offered through Procedures Consult are didactic in nature, offering post procedure tests and specific training clips demonstrating certain techniques. The accompanying outlines provide a valuable resource for residents to prepare for cases or familiarize themselves with a new procedure. For these reasons, ClinicalKey also scored the highest in value. The primary limitation of this resource is the volume of videos contained in Procedures Consult, with the lowest volume score of 1 (Table 5).

VJOrtho had the second highest quality score, with a median rating of 59% (Table 5). Unlike the other resources evaluated in this review, this video library focuses solely on orthopedics. The Journal of Bone and Joint Surgery (JBJS) articles offer reliable, high quality, and educational videos through VJOrtho. However, there is a noticeable lack of video and narrative quality in the non-JBJS videos. Many of the videos are outdated, as roughly 50% of the procedures were submitted before 2005. As such VJOrtho achieved a relatively low value score of 2 (Table 5).

Like VJOrtho, MedlinePlus had a quality score of 3 (Table 5). MedlinePlus is not primarily a surgical video library, instead focusing on basic information on diseases and drugs. Moreover, it does not appear to target the medical community, as only 17% of its visitors are healthcare providers and 8% are students.⁷ MedlinePlus provides an index of surgical videos linked to articles on ORLive. It selects videos with detailed, didactic narration, and avoids advertisements and product endorsements. With high value (3) and volume (4), ratings, MedlinePlus had the highest combined score of the resources reviewed.

ORLive primarily serves as a search engine for the videos on BroadcastMed. Therefore, these two sites were evaluated together. Through live webcasts, BroadcastMed offers a unique opportunity for users to interact with the attending surgeon and to receive personalized feedback. Although it includes a variety of didactic videos, there is a wide variance in video/narration quality and length. This problem is exacerbated by its large branding component, with promotional videos from medical technology companies mixed in with the search results. As such ORLive/BroadcastMed had intermediate quality (2) and value (2.5) scores.

SurgeryTheater, WebSurg, and YouTube all scored the lowest in quality (1), yet achieved the highest rating in volume (5), providing thousands of surgical videos and thus offering an impressive range of procedures. Among these three sites, Websurg was rated the highest in value with a score of 2.5 (Table 5). Some of its articles include full length procedures and didactic narration, with the Master Classes offering the best quality. An important limitation is that it only includes minimally invasive procedures.

SurgeryTheater and YouTube are alike in many aspects. Both libraries suffer from significant variability in video quality and value, with a significant amount of promotional videos and advertisements. SurgeryTheater offers the advantage of its high quality “Master Series Videos,” although they only cover a small range of specialties and comprise a small fraction of its total content. YouTube is limited in that it does not target the medical community, as it is simply an online collection of videos.

This review has demonstrated the extreme heterogeneity in online surgical video libraries. Quantitatively, MedlinePlus had the greatest combined score (10) while VJOrtho had the lowest combined score (7). However, each resource has varying strengths and weaknesses that must be taken into account. While ClinicalKey offers superior quality and value, it is lacking in volume. Similarly, although YouTube and SurgeryTheater were rated the lowest in quality and value, they provide a tremendous volume of procedures. It is also important to consider the additional aspects of each resource that were not captured by the rating scheme, such as the individual specialties offered or the availability of master classes. For example, an orthopedic resident may find the outlines available through ClinicalKey’s Procedures Consult to be most useful in preparing for cases, while he may prefer Websurg for finding arthroscopy videos.

The primary limitation of this review is that value is inherently subjective and will vary widely based on the experience and goals of the individual viewer. While a resident may find the basic principles and anatomical features of a procedure of greatest value, attendings might alternatively value subtle variations in technique or instrumentation. Therefore, our development of the ten value criteria is a significant source of bias. Ideally, a group of surgeons with varying levels of training would have been polled to better determine criteria for evaluating value. The randomization process also introduced a source of bias. Although an attempt was made to collect a balanced sample, it is possible that the samples included a disproportionate number of high or low quality videos.

It is important to consider that this review only examines a small cohort of resources. Countless other options are available. While many sites may be primarily dedicated to surgical video, the majority of online videos can be found scattered throughout sites with other purposes. Medical schools and medical centers often have their own means of posting surgical videos. Whether through the institution’s own site or through a YouTube channel, these videos have a wide variance in quality and are typically short clips of surgical footage. Similarly, most medical device brands publish videos advertising their product and demonstrating its use. There appears to be an overlap between the sites noted in this report and other sites, due to the very nature of the internet and user’s unlimited ability to download and upload videos at will.

Finally, although we believe that the quality, volume, and value ratings capture the most important qualities for a surgical video resource, it is important to recognize that much of what makes each site unique could not be completely represented by its scores. For example, although WebSurg scored a 1 in quality, it is still a valuable resource for users wanting to watch minimally invasive procedures exclusively.

Appreciation for surgical video’s valuable potential has led to an overwhelming volume of videos available online. The extreme variance in quality, length, and educational value means that some videos are more beneficial than others. For the sake of time, professionalism, and training, it is necessary to separate worthwhile content from the remainder. The purpose of this report is to develop a means of evaluating surgical video libraries and to apply it a small cohort of resources. We found that none of these resources can be considered the best. Rather, sites vary in the volume, quality, and value of their content. Some resources, such as SurgeryTheater, offer a tremendous volume of surgical videos, covering nearly every specialty. However, these sites may suffer from inferior quality or value. In contrast, some libraries, such as ClinicalKey, provide a few excellent quality videos with superb educational value. Therefore, the best resource is highly dependent upon each user’s individual preferences.

No other reviews of this nature are currently available. However, future reviews could benefit from assessing different resources like academic journals that provide surgical videos to accompany their content. Future studies should also examine the quantitative benefits of surgical video at the student and resident level.