Cost effectiveness and quality of care are integral to healthcare reform strategies. In attempts to improve value, private business ventures, government agencies, and commercial insurance companies have begun to report cost and quality metrics. Private businesses have created what are known as physician-rating websites (PRWs) such as RateMDs, HealthGrades, and Yelp. Meanwhile, government agencies such as the Centers for Medicare and Medicaid Services (CMS) and commercial insurance companies have created what our group calls cost and quality-of-care websites (CQCWs). Examples of CQCWs include Hospital Compare, Physician Compare, Blue Cross Blue Shield’s Cost and Quality Tools, and Aetna’s Member Payment Estimator.

PRWs have been an intense focus of recent literature. These websites allow anonymous users to comment on and rate physicians.^1,2 The prevalence of PRW use in the general population has been found to range from 23% to 28%.^3-7 It has been reported that between 3% and 32% of physicians are rated on any individual PRW and that the vast majority of ratings are positive.^8-14 While PRWs do provide increased transparency in the healthcare system, this transparency comes with several shortcomings. Ratings may not come from verified patients,^1,2 may be based on a low number of reviews,^9-11 and have not been associated with better outcomes.^15,16

In contrast, little has been written about CQCWs. CQCWs, simply stated, are governmental or commercial insurance websites that attempt to provide information regarding cost and/or quality-of-care metrics from verified patient encounters.^17,18 For example, Physician Compare displays performance data from the Physician Quality Rating System, which tracks over 200 quality of care metrics as well as biographic information about physicians who participate in Medicare.¹⁸

The focus of this study was to evaluate the use of CQCWs within the orthopaedic new patient population, and to examine patients’ perceptions regarding the utility of these websites. Specifically, we examined the frequency with which new orthopaedic surgery patients use CQCWs, which websites were utilized, and whether patient demographics affected website use. Additionally, we evaluated patients’ perceptions about the utility of information reported on these websites with regard to their selection of the orthopaedic surgeon they saw.

Patients

Following approval from the hospital’s Institutional Review Board, 300 adult orthopaedic outpatients were prospectively recruited at a tertiary academic medical center. All patients were seeing their provider for the first time. Patients were excluded only if they were younger than 18 or did not speak English. All participants gave informed consent before participating. Enrollment occurred during June and July of 2015 and compensation was a $5 gift card.

Websites

The CQCWs eligible for inclusion were determined a priori. Websites were included in analysis if:

1. a publicized cost or quality-of-care website affiliated with governmental agencies, or
2. a cost or quality-of-care website offered through the most common commercial insurers within our patient population

Generally, websites affiliated with commercial insurers required users to create a profile and were restricted to only those patients of the same insurance type. The final list of websites included was agreed upon by a consensus among the authors and is shown in Table 1. These websites contain a variety of information that is either factual (e.g. reports from the most recent Joint Commission visit in the case of Joint Commission Quality Check) or subjective but based on verified patient encounters (e.g. physician ratings within one of the commercial insurance websites).

TABLE 1

Prevalence of individual CQCW use amongst the 83 patients who reported to use at least one CQCW prior to their appointment. All CQCWs considered for this study are included. Some patients visited several websites prior to their appointment.

All Rights Reserved. Permission For Use Required.

-View Large-

Survey

Participants were asked to complete a 19-question written survey (Appendix 1*). This survey included questions about patient awareness of and use of individual CQCWs, as well as their perception regarding the importance of information provided by such websites when selecting an orthopaedic surgeon. The survey also included questions about other factors that may have influenced their choice of provider (e.g. provider’s reputation, physician referral, location, and friend/family recommendation). Use of CQCWs was determined both for CQCWs as a whole and for each individual website. Demographic cohorts were compared on the basis of those who used one or more CQCWs versus those who used none. Perceived importance results were stratified using a 5-point Likert scale, anchored with Unimportant and Very Important. Responses of Important or Very Important were grouped together as top-box responses and used to determine relative importance of these factors. Responses to perceived importance questions were considered only from participants who had used one or more CQCWs.

Statistics

Two-tailed Fisher’s Exact Tests were used to determine whether use of CQCWs differed by patient demographic factors. 95% confidence intervals were calculated for comparative importance of factors in selecting a new orthopaedic surgeon. Results with P<0.05 were considered statistically significant.

Three hundred of the 344 new orthopaedic surgery patients approached agreed to participate in the study (87.2% response rate). The study cohort was comprised of 93 males and 207 females with an average age of 55 years (Table 2). Seventy-two patients (24.0%) were insured by Medicare. Five (1.7%) were insured by Medicaid.

TABLE 2

Demographics of our orthopaedic new patient population

All Rights Reserved. Permission For Use Required.

-View Large-

Overall, 167 patients (55.7%) reported being aware of at least one CQCW while 133 (44.3%) had never heard of any. Eighty-three patients (27.7%) reported using at least one CQCW in selecting their new orthopaedic surgeon.

Individual website use by the total cohort ranged from 1 to 9%, with Blue Cross Blue Shield’s Cost and Quality Tools being the most frequently used (Table 1). The most frequently used government sponsored CQCW was the Commonwealth of Massachusetts Board of Registration in Medicine, although this was only used by 4.3% of our patients (Table 1).

Use of CQCWs by demographic is reported in Table 3. Those insured by Medicare, those older than 70, and females were significantly less likely to use CQCWs than their respective comparative cohorts (P=0.011, 0.019, 0.026 respectively (Table 3). Race, orthopaedic subspecialty, and income were not significantly associated with CQCW use.

TABLE 3

CQCW use by various demographics in our patient population

All Rights Reserved. Permission For Use Required.

-View Large-

Factors that influenced a patient’s decision to select a new orthopaedic surgeon are listed in Figure 1. Overall, 83 patients used at least one CQCW and were therefore selected for this analysis. Reputation of a surgeon, insurance coverage, physician ratings (such as those found on PRWs) and referral from another provider were considered the most important factors in selecting a new orthopaedic surgeon. Quality-of-care information provided by commercial and government websites was deemed important by patients, with 73% and 68% of patients indicating that these factors were important or very important, respectively. Fewer patients reported that cost-of-care information was important. Cost to insurance company was deemed important by the fewest number of patients.

FIGURE 1

Importance of various factors in selecting a new orthopaedic surgeon amongst those who accessed CQCWs. The percentage represents the number of patients who selected top-box responses of Important or Very Important for a given factor.

All Rights Reserved. Permission For Use Required.

-View Large-

While much has been written about physician rating websites (PRWs), less is known about cost and quality-of-care websites (CQCWs). This distinction is important because recent literature has suggested that the ratings found on PRWs may not come from verified patients,^1,2 may be based on a low number of reviews,^9-11 and have not been associated with better outcomes.^15,16 By contrast, CQCWs offer the potential to convey either factual information (such as in the case of CMS Provider Charge Data) or subjective data that is based on confirmed patient encounters (such as in the physician rating portions of commercial insurance websites). As such, CQCWs may deliver meaningfully different information to patients than they would receive on PRWs. Within this context, we sought to characterize the use and perceived value of CQCWs as defined by patients seeing an orthopaedic provider for the first time.

In the population we studied, 55.7% of patients were aware of and 27.7% had actually used a CQCW. To our knowledge, the rates of awareness and use of these websites among orthopaedic patients has not been previously investigated. Meanwhile, it has been documented that approximately 65-74% of patients have heard of and 23-28% of patients have used PRWs when selecting a physician.^3-7 Thus, while slightly more patients have heard of PRWs than CQCWs, the prevalence of use of both are roughly equivalent at least based on historical standards.

Individual CQCW use was fairly infrequent across our patient population. With the exception of BCBS’s Cost and Quality Tools, no individual website was used by more than 13 (4.3%) of our patients. Although BCBS’s Cost and Quality Tools website was used by 27 (9.0%) of our patients, BCBS did insure a plurality (30.3%) of our patient population. This is important context because most commercial insurance websites are only offered to those patients with that specific insurance. Subgroup analysis amongst commercial insurers revealed wide heterogeneity in the utilization of their websites by their respective patient populations. For example, Cigna’s Find Doctors and Services website performed comparatively well as it was used by 100% of Cigna patients, whereas Harvard Pilgrim’s Now iKnow website performed relatively poorly as it was used by only 25.6% of those insured by Harvard Pilgrim. (Table 1). We suspect this heterogeneity may be due to a combination of websites being marketed differently by individual insurers, website quality dissimilarities, different market dynamics (i.e. co-pays) amongst insurers, and disparate plans (i.e. health maintenance organizations versus preferred provider organizations) utilized by differently insured subgroups.

However, even amongst the open access CQCWs (e.g. Physician Compare) utilization remained quite low (ranging from 1 to 4.3% of our patient population). We believe this could be due to lack of advertising of open access CQCWs, though the evidence of this is currently lacking. We also suspect that the substantial number of different open access CQCWs makes it difficult for patients to keep up with all of the available information, which is often unique on each website. The implication of this finding is that patients who search for cost and quality-of-care information online are likely accessing different information based on the website they visit. We believe the large number of unique CQCWs may be exhaustive to patients. Perhaps one centralized open access CQCW with all appropriate information would achieve more traction.

We found that patients insured by Medicare accessed CQCWs at a significantly lower rate than did all of our other patients (P=0.011). One possible explanation for this is that the Medicare patient population skews older. Prior literature has demonstrated that elderly patients are less Internet proficient¹⁹ and that they use PRWs at a lower rate^5,6 than younger patients. Indeed, we found this to be the case amongst our patient population; those who were 70 years old or older used these websites significantly less frequently than the rest of the patient population (P=0.019). Therefore, some of the difference between CQCW use by Medicare patients versus the rest of our patient population might be explained by age. Additionally, although no Medicaid patients used CQCWs, too few were surveyed to determine the significance of this finding (P=0.327).

We additionally found men to use CQCWs significantly more than women (P=0.026). This finding is not well documented in the existing literature. In fact, studies have found that women access PRWs⁶ and general health websites at significantly higher rates than men.²⁰ None of these studies, however, investigated how gender influenced website use amongst the orthopaedic new patient population.

Among those who accessed the websites investigated, factors other than CQCWs were reported by many patients to be more important than CQCWs. These factors included surgeon’s reputation, insurance coverage, physician ratings, and referral. Physician ratings, which can be found on some hospitals’ websites but are mostly found on PRWs, were still considered important or very important by the majority (80.3%) of our patients who used CQCWs^1,21 (Figure 1). This is concerning because it has been documented that ratings on PRWs are determined by very few reviews, are not from verified patients, and are not associated with better outcome measures.^{1,2,9-11,15,16} Though aspects of CQCWs have been called into question, they do provide mostly objective measures from verified patient encounters and may provide a better alternative to PRWs.^17,22-24 Further investigation into the public’s perception of PRWs versus CQCWs is recommended.

Our study was limited to new patients visiting an academic tertiary care facility. No attempts were made to normalize our study population or to weight results based on representative demographic. Moreover, some of the websites investigated are relevant only to the Massachusetts market. Therefore, the generalizability of our findings is not known. Nevertheless, we believe the data presented here highlight the need for more, and larger studies to be performed across the country regarding patient behavior as it relates to cost and quality-of-care information.

In summary, most orthopaedic new patients have heard of CQCWs but a majority-particularly those insured by Medicare, those older than 70, and females-do not use them. Even amongst those who actively seek out CQCWs before selecting a new orthopaedic surgeon, other factors are considered more important than much of the information found on CQCWs. Improving quality and minimizing cost are desirable goals for healthcare reform. However, patient behavior as it relates to cost and quality-of-care information should be considered prior to designing further reform initiatives.

http://www.orthojournalhms.org/22/supplementaryMaterial.html