Physician profiling may improve ratesetting and patient outcomes

‘Supplier-induced’ demand and demand-side factors influence the accuracy of area variation research in the orthopedic workforce.

Issue: June 2005

ByDouglas W. Jackson, MD

I am pleased to share with our readers this recent interview with Dr. Warren Reid Dunn focusing on topics I have heard discussed repeatedly in orthopedics — mostly in theoretical terms — over the past 30 years. The practical impact of some of those ideas is now closer than ever and we can expect to see some of the changes they are bringing to our specialty in the decade ahead.

What effect will these concepts have on our patients and our practices? The effects will include renewed workforce projections for anticipated musculoskeletal care and population variation needs in this country, data collection of “high volume” and “high rates” of surgery by individuals and institutions, and the potential negative and positive aspects of physician profiling. I found the current perspectives on these important issues presented by Dr. Dunn, one of the leading thinkers in this area, informative and thought provoking.

Frank M. Phillips, MD [photo] Warren Reid Dunn ---

Douglas W. Jackson, MD: What are the variations in distribution of orthopedic surgeons and how does this potentially impact “supplier-induced” demand?

Warren Reid Dunn, MD, MPH: The size of the orthopedic workforce varies considerably among geographic regions. The average for the United States is six per 100,000 residents and varies from three to 12 per 100,000 residents. There is a concern that there may be an overall physician surplus, including too many specialists. In fact, the American Orthopaedic Association and the American Academy of Orthopaedic Surgeons commissioned the RAND Corp. to study the orthopedic workforce, which concluded that there would be a surplus of more than 4000 full-time orthopedic surgeons by the year 2010. It is unclear whether this surplus is real or perceived because estimating the health care workforce is difficult, and these estimates are based on self-reports that are notoriously inaccurate.

A surgeon surplus has theoretical implications regarding “supplier-induced” demand for medical care – specifically, surgery. The fear is that physicians could manipulate the demand for services because of information asymmetry. Consumers in the medical market typically allow the physician to make decisions for them due to the complex nature of the information. This creates an unusual situation where the provider of a service (the surgeon) has the ability to influence the demand for the service (surgery) they personally provide. However, two studies have found no association between the number of surgeons and surgical rates of arthroplasty and shoulder surgery.

Jackson: How do we realistically measure the demand for musculoskeletal care?

Dunn: Demand for musculoskeletal care is very difficult to measure and is derived from the demand for musculoskeletal health. Health is influenced by many factors; for example, education has been found to be twice as important to health as medical care. Demand-sided factors are patient related, such as education level,socioeconomic status and disease prevalence and severity. The problem with demand estimates is that current utilization is an imperfect proxy for need. Two studies have found an inverse relationship between surgical rates and population density. Hence, it is unlikely that we will ever measure the demand for musculoskeletal health well. But, it is conceivable to measure disease prevalence, severity of disease and behaviors such as smoking, which likely play an important role in the demand for musculoskeletal care.

Jackson: How do “high rate” and “high volume” institutions and individuals differ? How do we interpret them in relation to outcome studies?

Dunn: Rates are ratios with a numerator containing the number of procedures occurring in a specified period, and a denominator consisting of the average population during the period. The use of rates rather than raw numbers allows for comparison between populations in different areas. A high-rate area refers to the per capita rate of a surgical procedure at some group level, such as an institution, a ZIP code or a hospital service over a specified period. The latter is based on a localization index, which is the amount of local hospitalizations of residents in certain ZIP codes divided by all the hospitalizations of those residents. In contrast, volume is the raw number of procedures performed at some level of aggregate (an individual level, institution level, city, state, country, etc.) over a specified period of time.

"There is a concern that there may be an overall physician surplus, including too many specialists."
— Warren Reid Dunn

When interpreting these studies, it is crucial to be aware of some fallacies that are associated with cross-level inferences. There are many inferential fallacies; the two most important fallacies related to interpreting these studies are the ecologic and atomistic fallacies, which are both related to extrapolating inferences based one level of data to a different level of data. The ecologic fallacy can occur when associations observed at the group (aggregate) level are misinterpreted to apply at the patient level. The atomistic fallacy is the reverse, and can occur when associations observed at the individual level are misinterpreted to apply at the group level.

An excellent example of problems with cross-level inferences is a study by Wen that investigated time to surgery and removal of a normal appendix. They examined the same data at different levels of aggregate. The proportion of normal appendix removal, measured at the patient level, was higher when surgery was delayed from the time of admission, but when measured at the hospital level, the proportion of normal appendix removal was lower when surgery was delayed.

Hence, imagine a study that seeks to decrease normal appendix removal to save money in a state. The study examines the relationship between the timing of surgery and normal appendix removal at the hospital level (which is readily available administrative data in that state). In this instance, the state would conclude that normal appendix removal is lower if surgery is delayed. In turn, the state implements a policy mandating a minimum observation period for patients with suspected appendicitis. Such a policy would have the opposite effect of its intention at the patient level and would lead to more normal appendectomies.

There is a growing body of volume-outcome studies in the literature, reporting a positive association between high volume and outcome. Such studies should be viewed with extreme caution due to the bias that can occur with cross-level inferences. These studies utilize typical group level outcomes such as length of stay, re-operation and mortality, which are readily available in administrative databases but may not be applicable at the patient level.

For instance, a study by Keller identified three service areas of high, medium and low utilization of spine surgery at the state level, and investigated outcomes at the patient level which demonstrated an inverse, graded response between volume and outcome. Patients in the high rate area were the least satisfied (49%), the patients in the medium rate area were 63% satisfied, while patients in the low rate area were the most satisfied (72%). This study demonstrates how volume at the group level may not apply to outcome at the patient level. However, if this same state were to have used a typical group level volume-outcome measure like length of stay or readmission they may have found that higher volume was associated with shorter lengths of stay and fewer readmissions. Payers and/or government could misinterpret such a volume-outcome studies at the group level as a ‘quality’ measure for individual patients. If implemented into policy/reimbursement, such a study could have a predictable paradoxical negative effect on health outcomes.

Jackson: What are the limitations in interpreting the Medicare and Maine studies, related to area and different population variations across our country?

Dunn: Our understanding on area variation mostly comes from the Dartmouth Atlas Project, which is a comprehensive descriptive analysis of area variation in the U.S. Medicare population. The Maine Medical Assessment Program, which began in 1980, monitors area variation using Maine discharge data. From these studies, it is clear that small area variation exists in orthopedics, but there is still much to learn. External validity is a limitation of existing research; for example, the Maine effort has been largely limited to spine procedures. The Medicare population has been over-represented in prior research, and findings from this population may not be applicable to non-Medicare populations like Medicaid, which now exceeds Medicare in expenditures. Much of the literature on area variation assumes that the observed variation is ‘too much’ without statistical analysis, and the amount of variation that would be expected by chance alone is high.

These studies have not adequately accounted for the influence of disease prevalence and severity. The lack of this data not only makes estimating demand for services difficult, but also weakens the inferences that can be drawn from the area variation literature. While it seems unlikely that this will explain all of the observed variations, it is equally unlikely that musculoskeletal diseases are distributed evenly across geopolitical boundaries.

Jackson: What are the different physician profiling types being gathered currently?

Dunn: There are two types of ‘physician profiling.’ Profiling refers to the dissemination of information about medical malpractice claims against physicians to the public. Alternatively, the term is used to describe data collection on individual providers’ patients. Massachusetts was the first state to pass legislation making information about physician practices, including education, training and malpractice settlements, publicly available to consumers. Many states have subsequently made this type of information available to the public.

Jackson: What is potentially beneficial about profiling and how can it be used in a positive manner?

Dunn: With respect to area variation research, physician or provider profiling refers to the analysis of patient data. This is area variation analysis at the individual provider level, instead of some larger level of aggregate, like comparing institutions or counties. This type of profiling has the potential to influence medical decision-making, because inferences apply at the provider level. Physicians are responsible for the majority of decisions regarding hospital resource allocation and could use this information accordingly. Data collected at the patient-level could be very beneficial, particularly if validated patient-relevant outcomes were to be incorporated into data collection, for instance the SF-36. Studies using data of this nature are better suited to make inferences that actually apply to individual patients.

Jackson: What are the potential problems with profiling?

Dunn: Provider profiling has been championed as a method of reducing variation, lowering cost and improving quality. These claims are similar to those made by the managed care industry, which is a proponent of physician profiling programs. Surveys have shown that up to 80% of group practices with capitated patients profile their providers. Length of stay is a common profiling target. In fact, the average length of stay in America is shorter than any other industrialized nation. This type of profiling has been used effectively in at least one area of orthopedics to establish a “center of excellence” designation for total joint replacement. However, physicians can game the system by altering patient selection (avoiding complicated cases) rather than by altering their practice style. Therefore, insurers that use profiling might unintentionally pass the problem of adverse selection onto the provider, which may undermine the doctor/patient relationship and potentially become a barrier to care for patients with more severe or complicated conditions.

Profiling has the potential to be overly intrusive to orthopedic surgeons. Statewide hospital discharge data are available in more than 30 states. For instance, in 1979, the New York State Department of Health joined with the health care industry and established the Statewide Planning and Research Cooperative System. This database tracks all hospitalizations and ambulatory surgery with unique patient identifiers and physician license numbers. These databases offer an opportunity to address some of the weaknesses of existing area variation studies in orthopedics by allowing for studies at the physician level. However, this information could potentially be abused if used to generate physician report cards that do not account for case mix. Poor profiles have led to the loss of managed care contracts and admitting privileges in some instances. Hence, many providers are cynical of profiling policy because it is potentially punitive.

Many orthopedic surgeons in New York may not realize that all of their admissions, readmissions and operative procedures are tracked (although, not with perfect accuracy) and that the volume of many common procedures that they performed can be accessed on the Internet by following the “more is better” link at the Center for Medical Consumers Web site (www.medicalconsumers.org/#MainIndex). This Web site currently provides information for the year 2002 by physician name or institution. For example, they report that a total of 43,462 meniscectomies and 9348 rotator cuff repairs were performed in 2002, and the highest volume physician for these procedures performed 808 meniscectomies and 244 rotator cuff repairs in 2002. It is not difficult to imagine the potential health regulatory policy that this data offers.

Jackson: What must be changed in the future to improve profiling?

Dunn: Physician profiling remains in its infancy and many aspects could be improved. The reliability of existing profiling systems is unknown and should be investigated. At the most basic level, accurate profiling is difficult because claims data are imperfect. Estimates of incorrect diagnostic coding in claims data range from to 10% to 30%, for example. Furthermore, the physician/patient mix varies appreciably with regard to care-seeking behavior, patient expectations and types and severity of illness. Future research should account for these factors to allow risk adjustment.

The myriad profiling limitations are yet to be reconciled with its regulatory potential, and cross-level inferences should be avoided. Where the correct data is collected and the appropriate level of aggregate is studied to answer a clinically relevant question, profiling can be a very powerful tool for improving patient outcomes.

Payers, both public and private, are very interested in area variation. Their motive is containment of escalating health care costs. The fact that the managed care industry is particularly attracted to this area of research is not surprising and the orthopedic community should heed this warning. Historically, the managed care industry has not been very successful at improving quality or managing care but has succeeded in managing costs. Hasty decisions, based on incorrect conclusions from area variation studies, could lead to policy initiatives that fail to achieve the desired effect and have negative repercussions on quality of care. Ignoring this emerging paradigm could result in trate setting and practice guidelines that are not evidence-based and potentially threaten physician autonomy. At the same time, the orthopedic community should exploit the strength of profiling.

For more information:

Keller RB, Atlas SJ, Soule DN, et al. Relationship between rates and outcomes of operative treatment for lumbar disc herniation and spinal stenosis. J Bone Joint Surg Am. 1999;81:752-762.

Wen SW, Naylor CD. Diagnostic accuracy and short-term surgical outcomes in cases of suspected acute appendicitis. CMAJ. 1995:152:1617-1626.