Evidence to show results of data reporting limited

Issue: February 2008

Data on the effect of publicly reporting performance data on patient safety and quality of care remain limited.

Researchers analyzed data on study design, provider selection and outcomes from 45 peer-reviewed articles in order to combine evidence for the use of publicly reported performance data to improve quality of care.

The results were published in the Annals of Internal Medicine.

Confirmation of the effect of publicly reported data on patient-centered care was scarce, according to the researchers. Gathered data from eight studies on the health plan level implied a weak link between reporting and plan selection. Combined data from 11 hospital-level studies indicated motivation for action that would improve quality; however, the correlation between reporting and selection of hospitals and providers was inconsistent between hospital-level and individual provider-level studies.

The researchers concluded that further evidence demonstrating the effect of publicly reported performance data on safety and patient-focused care is needed, especially in terms of individual providers and practices. – by Stacey L. Adams

Ann Intern Med. 2008;148:111-123.

In this paper, the discussion section is almost like an apology; the researchers are saying that they thoroughly looked through the available data, and did not find much support for this type of approach public reporting. The government is basically rushing headlong in this direction; however, the government does not have evidence to support this approach. Based on this information, I thought this paper was interesting. There are many factors that are impairing our ability to provide the best care for our patients. Some of these include:

1. We are working with very scarce resources. In such an environment, is error reporting the best use of limited healthcare dollars? Should we be telling nurses that now we want them to record extra data because we are trying to measure this, that or the other thing? I think Twila Brase, from the Citizens Council on Health Care, summed it up well, when she said, 'At a time when hospitals are facing staffing shortages and healthcare premiums are skyrocketing, every minute and every healthcare dollar should be used for taking care of patients in an attempt to avoid medical errors rather than reporting statistics to show how few there really are.'

2. The 'performance measures' and reporting adds another layer of complexity to the medical decision-making process. Because of them, physicians when treating a patient will ask themselves, 'Will this patient help or hinder my statistics?' The answer to that question might impact whether the patient then receives treatment or gets shuffled to the other side of town.

3. In order to have a statistically valid measurement constants are needed. But patients are not identical. They are not constant. So how valid can these reported numbers be?

4. People seem to be under the impression that the information to help them choose a physician, hospital, or treatment is not available. But, in reality, that information is readily available in the community. For example, if a patient sees a primary care physician and is referred by him to a cardiologist, that primary care physician will select the cardiologist who he thinks would be best for him. Because he has worked with and gotten to know the cardiologists in his community, he has a grasp of who does what and how well they treat their patients. He receives feedback from many different sources. Previous patients have reported back to him and other doctors have shared their thoughts with him. He might also have worked with him in the hospital on other cases. Likewise, if that cardiologist believes that you require surgery he in turn will send you to the cardiac surgeon who he believes would be best for you. Thus patients are guided through the healthcare maze based on medical reputations formed over many years. How does one condense such reputations down to a simple number or letter on a sheet of paper?

5. I am concerned that third party restrictions are impacting doctors’ referral patterns. Patients cannot always see the recommended doctors because their health care plans will not cover them. This sometimes results in the referring physician being forced to disregard the established referral patterns and simply tell a patient, 'Choose a cardiologist or cardiac surgeon from your health plan list.' Referral patterns have helped to promote quality by promoting those physicians and practices who have excelled in their fields. Attempting to replace these referral patterns by 'public reporting' of a score will prove to be elusive.

– Richard Dolinar, MD

Endocrine Today Editorial Board member

Senior Fellow in Healthcare Policy at the Heartland Institute, Chicago