Dialysis patient experience, clinical star ratings diverge in practice
Click Here to Manage Email Alerts
CMS has released new data for its Dialysis Facility Compare Star Rating system for Medicare- certified dialysis facilities. There are new “clinical” ratings, based on facility performance on the eight clinical measures used in this rating system for the period of 2015 to 2018, as well as updated patient experience of care ratings based on the In-Center Hemodialysis Consumer Assessment of Healthcare Providers and Systems (ICH-CAHPS) surveys conducted between April 2018 and January 2019.
The information is published on the Dialysis Facility Compare website (www.medicare.gov/dialysisfacilitycompare).
While the periods covered by the two types of star rating systems essentially overlap, facility star ratings based on clinical measures have low correlation to star ratings based on patients’ expressions of their experience of care at the facility and from their nephrologists (Table 1). While the two systems are directionally consistent (higher ratings on one system are associated with higher average ratings on the other system), the overall correlation between patient star ratings and the clinical star ratings is 0.23. In fact, there were 10 facilities that received five stars from CMS on clinical measures but received one star from their patients.
Patient stars, CMS stars
Of note are the relatively small differences in average patient star ratings across the five clinical star categories. Facilities with one star (clinical) have an average of 2.8 stars from patients, while five-star facilities have an average of 3.6 stars from patients. Looked at another way, whereas 27% of the facilities included in this analysis got five clinical stars from CMS, 11% got five stars from patients. This may be partly due to differences in scoring methodology because the patient star ratings are actually an average of six-star ratings on measures related to the dialysis facility, the facility staff and the nephrologist. Averaging six scores between one and five will likely tend to reduce variation in the composite score.
However, given that the goal of the star ratings is to make it easier for consumers to spot excellence in health care quality when choosing or comparing dialysis facilities, the wide divergence in scores on the two five-star rating systems is somewhat problematic. Clearly, factors that matter most matter to patients as reflected in the ICH-CAHPS surveys are often different than what CMS uses to rate the clinical quality of care at a facility. Studies have shown patient beliefs and values and physician perceptions of such are not always the same.1-3 While a facility may look good on clinical measures, patients may not feel that they are getting the attention they want from staff.
Facility characteristics
What are the characteristics of facilities receiving five stars from patients that set them apart from facilities that got fewer stars? We looked at several dimensions of facility characteristics to see how five-star facilities compared, including facility size (stations), ownership type and geography. Table 2 highlights some key characteristics where patient-rated five-star facilities differ from facilities with fewer stars. Five-star facilities are more likely to be small, non-profit, independent or hospital-based and in rural areas.
The apparent negative association between dialysis organization/facility size and patient star ratings has an interesting parallel in the hospital setting, where in a study recently published in JAMA Internal Medicine,4 patient quality ratings of hospitals were found to diverge from more objective criteria for hospital service quality such as comprehensiveness of services. Hospital patients gave higher ratings to smaller hospitals that tend to take care of patients with simpler needs. Think of how, similarly, one might prefer the local family owned hardware store for simple things compared to the more impersonal and sometimes bewildering experience typical in a mega-store like a Home Depot.
Socioeconomic disparities
It is certainly possible that the characteristics of the patients themselves may play an important role in perceptions of the quality of their experience of care and have an effect on the star ratings. Some research has shown patient characteristics are more important in how facilities are rated than the actual quality of the facility.
To examine this, we looked at the characteristics of patients and of the neighborhoods where dialysis facilities are located for clues about the makeup of the patient base in facilities rated five stars compared to those with lower ratings. We linked each facility ZIP code to U.S. Census Bureau statistics from the latest (2017) American Community Survey 5-year estimates5 to examine the socioeconomic characteristics of the neighborhoods where facilities are located, and also looked at the demographic characteristics of the actual patients at each facility, as of Dec. 31, 2017, pulled from the FY2019 Dialysis Facility Reports (Tables 3 and 4).6
The facilities rated five stars by patients tend to be in neighborhoods that are predominantly higher in average income, with higher average education and residents who are older and with fewer black patients and minorities. The patients themselves at these facilities also tend to be older, white, male, with a shorter time on dialysis, but also more likely to be in a nursing home. Conversely, one- and two-star facilities are in communities with more black patients, and overall higher minority populations and lower incomes.
Similar economic and racial disparities were found in a recently published study of the role of social risk factors on quality incentive program (QIP) performance.7 In the study, independent facilities in urban areas with the highest QIP penalties also have a higher percentage of low income, minority and dual-eligible patients. It may be that these patients are more difficult to manage clinically, which can hurt QIP and star rating scores.
Patient satisfaction
Several studies have looked at the link between patient characteristics and better experience of care. Results of a Dialysis Clinic Inc. internal analysis from 2011 reported that older, white patients with shorter dialysis vintage were more satisfied.8 A follow-up study in 2012 also found some associations between patient age, race and gender with ICH-CAHPS ratings that were directionally similar to this study, but found the mode of survey administration (telephone vs. mail) was by far the strongest determinant of how patients reported their care experiences.9 Telephone administration led to higher ratings than mailed surveys, again emphasizing the importance to patients of the personalized experience. Other research emphasizes the role of racial and gender concordance between patient and physician.10 The physician-patient relationship is strengthened when patients see themselves as similar to their physicians, which is in turn associated with higher ratings of trust and satisfaction. Race concordance was the primary factor affecting perceived personal similarity.
Conclusions
Five-star rating systems for consumers have inherent flaws regardless of what services are being rated. Reviewer bias is clearly one potential problem, as is ratings creep, which makes it hard to distinguish the top providers due to clustering in the four- to five-star categories.11 However, as CMS has now given equal visibility to its clinical star ratings of dialysis centers and to patient experience of care star ratings, further research may help disentangle the reasons for the wide divergence in the two rating systems, which will hopefully lead to a clearer, more understandable broadened view of dialysis care quality.
- References:
- 1. Kennedy BM, et al. Healthcare providers versus patients’ understanding of health beliefs and values. Patient Exp J. 2017;4(3):29-37.
- 2. Street RL, et al. J Gen Intern Med. 2011;doi:101007/s11606-010-1453-3.
- 3. Rodriguez KL, et al. Soc Sci Med. 1982;doi:10.1016/j.socscimed.2005.05.023.
- 4. Siddiqui ZK, et al. JAMA Intern Med. 2019;doi:10.1017/ice.2018.324
- 5. US Census Bureau. American Factfinder Download Center. https://factfinder.census.gov/faces/nav/jsf/pages/community_facts.xhtml. Published 2019. Accessed June 27, 2019.
- 6. CMS. Data.CMS.gov. https://data.cms.gov/dialysis-facility-reports. Accessed Nov. 11, 2019.
- 7. Qi AC, et al. Health Aff (Millwood). 2019;38(7)doi:10.1377/hlthaff.2018.05406.
- 8. Richardson MM, et al. Clin J Am Soc Nephrol. 2015;doi:10.2215/CJN.11241114.
- 9. Dad T, et al. BMC Nephrol. 2018:19:doi:10.1186/s12882-018-1147-3.
- 10. Street RL, O’Malley K. Ann Fam Med. 2008;6:198-205.
- 11. Breinlinger J. Harv Bus Rev. July 2019. https://hbr.org/2019/07/the-problems-with-5-star-rating-systems-and-how-to-fix-them. Accessed July 8, 2019.
- For more information:
- Mark Stephens is owner of Prima Health Analytics (www.primahealthanalytics.com), based in Weymouth, Massachusetts. He can be reached at 781-588-0248; email: jmarkstephens @gmail.com.
Disclosure: Stephens reports no relevant financial disclosures.
Collapse
Click Here to Manage Email Alerts