This article is more than 5 years old. Information may no longer be current.
Physician-rating websites do not reflect quality, outcome of care
Click Here to Manage Email Alerts
Within health care, there has been a greater emphasis on transparency and the improvement of the quality of care. Health care reform measures from CMS have led to the measurement and public reporting of hospital performance. Nationally organized physician-reporting initiatives have failed to gain the same momentum.
The private sector has sought to create mechanisms for similarly reporting physician quality. Physician-rating websites have become popular among patients. Some have suggested patients can be regarded as consumers of health care and services, such as surgery and perioperative care. Therefore, patients should review their physician the same way they review hotels or restaurants.
Increasing influence
Regardless of whether one agrees with the rationale, physician-rating websites have an increasing influence on physician practices and patient referrals. A survey study published in the Journal of the American Medical Association found online reviews were highly influential for patients. Among survey respondents, 65% were aware of physician-rating websites and 35% had used a physician-rating website within the past year. Among respondents seeking physician online rating websites, 35% subsequently selected a physician based on good ratings while 37% avoided a physician with bad ratings.
The physician-rating landscape constantly evolves. Online ratings generally involve an overall rating for a single surgeon. In addition to assigning their physician a certain number of stars, patients can leave written comments describing their experience. The physician is then rated with a one- to five-star system on various categories. The process of care and staff are also graded on the same scale in various categories, such as helpfulness, knowledge and punctuality.
Validity assessments
Several investigators have sought to assess the validity of these websites within orthopedic surgery. A recent study looked at online ratings for 275 sports medicine physicians across three high-traffic websites. The authors found 99% of the surgeons had online ratings and most surgeons were highly rated on average. However, scores across the three websites did not have a high degree of correlation. Scores were moderately well correlated for surgeons scoring between 4 to 5, but for surgeons scoring between 2.5 to 3.5 on any one website there was low correlation of these scores across sites. The authors suggested this finding may be explained by lack of aggregation wherein patients with a negative experience post a negative review on one website thereby impacting the surgeon’s rating on that website but not affecting the surgeon’s rating on other websites.
Another similar study of hand surgeons found positive online ratings were associated with a higher number of ratings and increased online presence. Hypothetically, surgeons with a high number of ratings and online presence may be involved in some form of marketing to elicit positive reviews.
As ratings websites have become popular, more private companies have sought to provide a mechanism for rating physicians. With many available options, physician ratings can become diluted in a large pool of offerings, thereby leading to a heterogeneity of reviews. Health policy experts have warned that physician-ratings websites based ratings on too few reviews. On the patient side, having many options does the patient a disservice as they are left scrounging through different rating websites to develop a collective grade for their intended physician.
Beyond the problematically high number of available rating websites, it is unclear if the reviews themselves are fundamentally trustworthy. For many websites, there is no requirement for reviewer identity verification. As such, physician ratings could be conceivably — both positively and negatively — skewed by paid reviewers and false reviews. “Reputation management” agencies work with surgeons to encourage satisfied patients to provide strong reviews of the surgeon across multiple websites. Similarly, patients thought to be dissatisfied are identified early in the process to provide mediation.
A study looking at physician ratings for arthroplasty surgeons sought to assess whether rating websites assessed the quality measures deemed important by CMS through release of Consensus Core of Orthopedic Measures. The authors found the Consensus Core domains were infrequently captured and therefore questioned the reliability of the websites.
The fundamental question remains: Do these websites measure the quality and outcome of care? While a five-star rating mechanism may be appropriate for judging the quality of a meal or hotel experience, the delivery and improvement of health is a more complicated task. Accepted measures of clinical outcome are often not asked on physician-rating websites and it is frequently unclear whether a rating is related to the process vs. outcome of care. While the process and outcome of care are inter-related and clearly go hand in hand, maximizing the outcome of care is paramount.
Need for a sophisticated repository
A multidisciplinary effort is needed to address the deficiencies in the current physician-rating system. Public ratings for physicians will continue to evolve. Orthopedic surgeons must show initiative in determining the method with which we are graded and the content that is valuable to patients and other health care providers. The current system could be improved through the creation of an independent central repository for physician ratings, although the financial stability of this program is likely to be challenging without support from CMS or another non-profit organization. The components of this central rating mechanism should be established and monitored by orthopedic health policy advocates such that quality measures deemed important by surgeons and patients are included. Surgeons can then provide their patients with the information necessary to access this central repository. Patients provide their rating of the surgeon and surgeons can be tracked within the repository for measurement of objective quality measures.
A more sophisticated repository could provide patients with patient-reported outcome measures such that the functional outcome of care is also made available to guide the surgeon’s overall rating. However, funding this type of program, minimizing patient response burden and achieving a fair representation of responses from patients are obstacles that need to be resolved.
Such a repository would address the validity and reliability issue, and reliance on one source would address issues of variability across websites while also mitigating problems with having a small number of reviews per website. Such a proposal would be highly disruptive to the established private sector business incentives associated with current physician-ratings websites. Physician-driven legislative advocacy, as well as leadership from the American Academy of Orthopaedic Surgery is necessary to ensure surgeons are portrayed correctly for their work and patients have the information that matters most to them.
- References:
- Hanauer DA, et al. JAMA. 2014;doi:10.1001/jama.2013.283194.
- Kamal RN, et al. J Hand Surg Am. 2016;doi:10.1016/j.jhsa.2015.10.002.
- Nwachukwu BU, et al. HSS Journal. 2016;12:272-277.
- Okike K, et al. J Med Internet Res. 2016;18:e324.
- Ramkumar PN, et al. J Arthroplasty. 2017 Apr 4. pii: S0883-5403(17)30290-5. doi: 10.1016/j.arth.2017.03.053.
- Sabin JE. Virtual Mentor. 2013:15):932-936.
- Tanne JH. BMJ. 2013; doi:10.1136/bmj.f295.
- Trehan SK, et al. J Hand Surg Am. 2016;doi:10.1016/j.jhsa.2015.10.006.
- For more information:
- Anthony A. Romeo, MD, is the Chief Medical Editor of Orthopedics Today. He can be reached at Orthopedics Today, 6900 Grove Rd., Thorofare, NJ 08086; email: orthopedics@healio.com.
- Benedict U. Nwachukwu, MD, MBA, is an orthopedic surgery chief resident at Hospital for Special Surgery.
Disclosures: Romeo reports he receives royalties, is on the speaker’s bureau and a consultant for Arthrex; does contracted research for Arthrex and DJO Surgical; receives institutional grants from AANA and MLB; and receives institutional research support from Arthrex, Ossur, Smith & Nephew, ConMed Linvatec, Athletico and Miomed. Nwachukwu reports no relevant financial disclosures.