This article is more than 5 years old. Information may no longer be current.
Comparative effectiveness research: Recognize issues impacting your orthopedic practice
Click Here to Manage Email Alerts
Introduction
Comparative effectiveness research (CER) is one issue likely to impact future orthopedic practice and medical practice in general. Governmental agencies, such as the Centers for Medicare & Medicaid Services (CMS) and the Agency for Healthcare Research and Quality (AHRQ), as well as private third-party payers, have adopted CER as a principal factor in making decisions on reimbursement for technology and interventional procedures such as surgery and injections.
The Congressional Budget Office (CBO) has published the definition of CER used by governmental agencies as, “A rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients… The analysis may focus only on the relative medical benefits and risks of each option. Or it may also weigh both the costs and the benefits of those options.”
Key points to note in the definition are that both clinical benefit and cost may be considered in a CER evaluation. Orthopedic surgeons are experienced and comfortable with investigations of clinical benefits, but are generally not familiar with cost analysis. Clearly, the cost equation has received much more attention in recent time.
The “currency” of cost analysis in CER has been the Quality Adjusted Life Year (QALY). The relative value of an intervention is expressed as a monetary cost per QALY. This reflects the cost of a procedure to incrementally improve health status, with the baseline being perfect health for a period of a year.
When used to help patients and their surgeons determine treatment options, CER can clearly have a positive effect on health care. However, apprehension persists around the manner in which CER may be implemented by government agencies and third party payers.
Our expert panel will respond to questions to help clarify these issues.
David A. Wong MD, MSc, FRCS(C)
Moderator
|
||||||||
David A. Wong MD, MSc, FRCS(C)
Sigurd H. Berven, MD
Steven D. Glassman, MD
Kevin J. Bozic, MD, MBA
David W. Polly Jr., MD David A. Wong MD, MSc, FRCS(C): How influential do you think CER is going to be in formulating government policy and coverage decisions for orthopedic procedures and technology?
Kevin J. Bozic, MD, MBA: According to the United States CBO, “As applied in the health care sector, an analysis of comparative effectiveness is simply a rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients.” Ostensibly a relatively simple and intuitive concept which seeks to evaluate and compare the relative merits of different treatment alternatives for a particular condition (ie, which treatment works best), the term “comparative effectiveness research” has become the subject of great controversy and debate among patients, providers, payers and policymakers.
Opponents of CER have associated it with “rationing,” and have even used the term “death panel” to describe the federal government’s first major policy foray into CER, known as the Patient Centered Outcomes Research Institute (PCORI). PCORI was created through the 2010 Patient Protection and Affordable Care Act (PPACA) to conduct research to provide information about the best available evidence to help patients and their health care providers make more informed decisions. By law, CMS and other payers are prohibited from using the results of PCORI-funded research as the sole justification for denying payment for a particular treatment or technology. However, it seems obvious that if a particular treatment is shown to be less effective or equally effective but more expensive, payers, including the federal government, would question the rationale for paying for it.
Ideally, CER would be used to inform shared medical decision making between patients and their physicians, and modify insurance benefit designs, whereby less efficacious and/or more costly treatment alternatives would be associated with a higher patient copays and/or lower provider reimbursement.
David W. Polly Jr., MD: Even if the federal government does not use it, private payers already are beginning to use it.
Wong: Any concerns about CER implementation as a driver of policy?
Steven D. Glassman, MD: Given the assumption that health care spending needs to decline, I believe that CER is, in theory, a reasonable way to allocate resources. The problem lies in the wide range of information that is described as “comparative effectiveness research.” Our national leadership organizations in this arena (American Academy of Orthopaedic Surgeons [AAOS], North American Spine Society [NASS], American Association of Neurological Surgeons/Congress of Neurological Surgeons [AANS/CNS]) have generated CER-based guidance using carefully constructed and rigorously applied evidence-based medicine standards, and this is the right thing to do. In contrast, many private payers are presently using nonevidence-based guidelines which are falsely labeled as CER. If we are going to take the high road, and support high quality research as a driver of policy decisions, then we need to hold the other players to the same quality standards. I believe that our leadership organizations need to call out and strongly criticize for-profit sources of nonevidence-based guidance.
Wong: Have you seen CER used in any private sector initiatives?
Polly: The first instance was the denial of coverage for lumbar artificial discs. The clinical data started out as non-inferiority studies. The push back was twofold. There was denial of any increased payment for the new technology and a concomitant attack on the efficacy of the control group.
We are beginning to see some pushback on vertebral augmentation for vertebral compression fractures as in a recent the New England Journal of Medicine perspective piece, which interestingly left out the SAVES study demonstrating increased survival at 48 months of those who underwent such augmentation.
Wong: The standard measurement for CER is cost per QALY. Could you give us a brief description of how a clinical study determines a utility score which is then used to calculate a QALY?
Sigurd H. Berven, MD: In orthopedic surgery, we have several outcome measures that are used commonly to assess health-related quality of life. These measures generally include general health status measures, such as the SF-36, or disease specific measures including the DASH or Oswestry Disability Index. The limitation of these outcome instruments is that the resultant measure has no intrinsic value or unit of measure. A utility score is a quantifiable measure of the preference of an individual, or of society, for a specific state of health.
The utility of a health state is a measure of the value that an individual or society places on that specific health state. Utility scores span a range from 0 to 1, with 0 representing the state of death, and 1 representing perfect health. The unit of a utility score is well-years per year. Utility scores represent the value of a given health state at a specific point in time. However, it is often the goal to describe differences in health states that are sustained over time. For this purpose, physicians may speak in terms of QALYs. Remaining in a health state of 0.5 for 1 year will result in a loss of 0.5 well-years, or QALY, compared to a health state of perfect health. Improving from a health state of 0.4 to 0.65 and maintaining that improvement for 4 years will result in a gain of 1 QALY. Therefore, utility scores have a unit that has intrinsic value.
Utility of a health status may be measured directly with techniques including a time-utility trade-off or standard gamble. Direct measurement techniques assess the utility of a health state under conditions of uncertainty. Indirect measure of utility scores is the most common method for calculating health-state preferences. Indirect measures include patient-based questionnaires in which answers to questions about health status are translated to a score that represents the preference of society or a larger community for that health state, rather than the preference of the individual. Questionnaires that may be used to derive a utility score based upon societal health status preference include the Health Utilities Index, the EuroQol-5D (EQ-5D), and the SF-6-D.
Wong: The other data point in CER measurement is cost. How difficult is it to obtain this information and how often are estimates used?
Bozic: Comparing the “value” of health care interventions requires an accurate assessment of both the costs and the benefits of a particular intervention. Measuring health care costs can be challenging, but not as difficult as defining and measuring the benefits of care in terms of risk-adjusted outcomes and patient experience. In fact, many stakeholders are concerned that in the absence of well-defined, clinically relevant outcome measures, health care costs will be used as a proxy for the value of a particular intervention.
Identifying and evaluating costs are important steps in any economic evaluation. Which costs are included in the analysis will vary based on the time frame and perspective being considered. Ideally, a thorough economic analysis will attempt to measure direct, indirect or time, and opportunity costs. Direct costs include all costs that are directly related to the intervention, including personnel, supplies and facility costs involved in the treatment. Indirect or time costs include costs associated with lost productivity, usually valued as lost wages or an imputed monetary value of time.
Opportunity costs are the health benefits lost because the next-best alternative was not selected. Although billed charges or reimbursement are often used to approximate costs, they are a poor proxy for economic value. Both direct and time costs should be measured using activity based costing methodologies, which attempt to assign the cost of each activity resource to all products and services delivered according to the actual consumption by each. Unfortunately, actual cost data is often considered proprietary information, which hospitals and physician groups are reluctant to share, due to concerns about the potential impact on their reimbursement. Increased transparency around the actual cost of delivering care is needed in order to enhance the validity and utility of comparative effectiveness research.
Wong: Your group has been instrumental in determining the formula for converting the ODI score into a utility score so that QALYs can be calculated. Will this give us good data from existing studies to use in comparisons between treatments?
Glassman: The critical elements of cost effectiveness analysis (CEA) are a preference based health state value (utility score), an accurate measure of costs, and an assessment of durability. An intervention which generates maintained clinical benefit, without added cost, becomes increasingly cost effective over time. Realistically, collection of cost effectiveness data for lumbar spinal disorders is difficult, and thus CEA is unavailable for most lumbar spine interventions. Ideally, data for CEA would be generated by randomized controlled trials (RCTs), allowing an assessment of incremental cost effectiveness for the intervention being studied as compared to the best alternative treatment.
Unfortunately, we will never have RCT data for the wide array of lumbar spine treatments in question. For that reason, a simplified measure of cost effectiveness, the cost per QALY gained for a specific intervention (cost/QALY) is likely to be one of the critical measures of value by which our interventions are judged. Cost/QALY can be determined based upon cohort data, facilitating comparison among parallel study populations for which demographic, diagnostic and cost data are available. The option to use ODI as a proxy for a health utility measure greatly simplifies the development of cost/QALY data for a wide array of existing interventions. In the future a health utility score, most likely EQ-5D, will become part of our standard data set for lumbar spine disorders.
Wong: Do you think orthopedics has a sufficient cadre of researchers and appropriate funding to generate the data needed to compete for health care dollars using CER?
Polly: Absolutely not. There is a growing awareness of this need. I am encouraged by the recent publication in the Journal of Bone and Joint Surgery using decision modeling analysis to compare two-stage vs. single-stage re-implantation for infected total joints. I would encourage young orthopedic investigators to partner with methodologists who are expert in this area.
We have so many questions that need to be answered, such as for each CPT code. The AAOS is now engaging in appropriate use criteria development and, at least one specialty society that I know of (Scoliosis Research Society), is preparing to fund an educational process for its leadership and future leaders. I also believe that funding opportunities for this will emerge as well.
Wong: CER and cost effectiveness are sometimes incorrectly used interchangeably. Could you point out the key differences?
Berven: CER and cost effectiveness research share a common goal of informing patients and providers about the benefits and risks of specific alternative treatment options, including pharmacologics, surgical interventions, medical devices and diagnostic tests. There are important differences in the methodology and outcome measures used in comparative effectiveness research and cost-effectiveness research. CER encompasses systematic reviews and prospective investigation designed to assess the clinical benefit as well as the harm of alternatives, and the outcome of measure is difference in clinical outcome and harm without explicit consideration of cost. Cost-effectiveness research is a focused economic tool that is designed to compare value of alternative health care choices using the specific measure of cost per QALY. There are important and significant concerns regarding the introduction of cost considerations into CER.
Clearly, the clinical efficacy of interventions have an intrinsic value which may not be adequately expressed in measures of dollars per QALY. Similarly, the harms of alternatives are not adequately encompassed by cost consideration alone. For interventions that may not have a direct and immediate impact on health-related quality of life, but rather an impact on avoiding future consequences of disease progression, cost-effectiveness analysis may underestimate the value of treatments. Examples may include preventative health programs and treatment of idiopathic scoliosis. Therefore, CER and outcomes have been a focus rather than cost-specific research. However, in a health care economy with limited resources, consideration of cost must be transparent and cannot be ignored.
Wong: NICE in the United Kingdom uses a cost per QALY threshold of £30,000 (approximately $48,000) for reimbursement decisions. What is your sense of what Americans would be willing to pay for procedures and new technology?
Polly: The benchmark for cost effectiveness is the cost of renal dialysis. In 1972, Medicare chose to cover all patients needing renal dialysis. This set the standard and will be hard to roll back. In the United Kingdom, renal dialysis is not routinely covered for older patients being viewed as not cost effective. So, anytime we can demonstrate our value at less than $50,000/QALY, we should be in a good position.
Wong: Do you think a cost per QALY threshold will lead to health care rationing in the United States?
Glassman: Rationing is an inevitable consequence, if and when our society determines that we cannot afford unlimited health care. We may call it something else, but limiting resource utilization means rationing in some form. The real question is how the resources will be allocated, and allocation of resources to those interventions which have proven to be effective seems like a pretty reasonable place to start. Rather than trying to fight this inevitable trend, we need to concentrate on generating data which demonstrate the value of our interventions. The good news is that surgical treatments in well selected patients have generally proven to be cost effective.
Wong: Epidural steroid injections for herniated discs and facet injections are examples of procedures that have been largely eliminated in the United Kingdom. Given your research in the CER area to date, are there specific orthopedic procedures that you feel are at risk of not being reimbursed?
Berven: Establishing a threshold of value as a criteria for reimbursement has significant flaws. Specifically, many interventions in orthopedics may be preventative interventions and measuring value as cost per QALY will underestimate true value. Similarly, expensive interventions may show value only after long-term follow-up, and conclusions based upon limited follow-up will again underestimate the true value of an intervention.
Epidural steroid injections for treatment of acute and chronic sciatica did not meet the threshold of NICE for cost-effectiveness. On average, epidural steroid injections resulted in a gain of 2.2 days of health. This translated to a cost to the purchaser of nearly $700,000 per QALY. However, epidural steroids and facet blocks have significant value as diagnostic techniques and may be useful in guiding further more durable care alternatives. Therefore, a calculation of value based upon improvement of health status and cost alone may underestimate real value.
If reimbursement in the United States were based upon cost per QALY alone, then interventions that would be most at risk include procedures that have a high initial cost and interventions that are designed to prevent future consequences of disease progression would be most at risk. Examples may include surgery for scoliosis in the child, osteotomies of the hip and pelvis, and major reconstructive procedures of the hip, knee and spine. If cost per QALY were the criteria for reimbursement in the United States, then future studies would have to have adequate follow-up, including greater than 5 years for major reconstructive procedures, and a detailed and accurate assessment of the cost of alternative care options.
Wong: Should we embrace or fight the adoption of CER as a key principle of health care?
Polly: Clearly our resources are constrained. We need to spend wisely as a society. If something is not effective, we should not do it. If we have a choice between two options, then we want to pick the one that is better. Also we will learn to find ways to take unnecessary costs out of the system which will be challenging. This is a paradigm shift where we are now being asked to look at health from a population perspective rather than from an individual patient perspective. This is the substantial conflict of interest challenge for this decade and following. It also tells us how to target future research.
We will have to answer two questions. Is any new treatment beneficial? Similarly important — is it cost effective? I think that it will profoundly change how pharmaceuticals and devices are priced. Currently, it seems as if the average selling price was whatever the market will bear. Now, it will be the change in health-related quality of life times the durability of the intervention. That will determine the price we as a society are willing to pay for the intervention. The intervention cost will then have to be apportioned between the doctors, the facility and the manufacturers.
Glassman: I absolutely believe that CER, clearly defined and administered on a level playing field, is in our long-term interest. The drivers for success underlying CER are patient-based outcomes measures and preference weighted health utility scores. These measures emphasize the benefits achieved in successful surgical interventions, including decreased pain and improved quality of life. Existing health utility data clearly demonstrates the severity of impairment in patients with hip or knee osteoarthritis and spinal stenosis. We have effective surgical treatments for these disorders, and those interventions will compete successfully for available resources. Importantly, our support of CER as a key component of health care policy must include an aggressive insistence that payers do not deny appropriate care based on lesser standards of evidence.
Berven: As a community of physicians, we should embrace and engage in comparative effectiveness research in guiding our practices in providing evidence for optimal alternatives to care for specific conditions. CER with a focus on outcomes including change in health status, long-term benefits of care, and harms and complications of alternative health care options will provide useful information to patients and physicians in guiding an evidence-based approach to care.
At the level of informing the individual physician and patient, CER will have significant value and should be a foundation for decision making. However, the role of CER in providing key information on guiding health care policy should be more limited, and outcomes of comparative effectiveness studies should be included as one of many components in decision making on allocation of resources and reimbursement.
The value of CER is critically dependent upon the quality of the data collected. Data elements that have proven difficult to collect in prospective research and in systematic review of published research include adequate and complete follow-up at time points up to and beyond 2 years, information on the cost and outcomes of alternative options to care, and data on the significant indirect costs of alternative options to care. Analyses based upon administrative databases have been especially barren and void of useful data regarding clinical outcome, measures of benefit and cost of alternatives. Based upon these limitations, my enthusiasm for adoption of CER or cost-effectiveness research as a key principle for guiding health care policy and reimbursement is significantly tempered.
Bozic: It is incumbent upon us as stewards of our health care system to help develop strategies to “bend the cost curve” in order to ensure that future generations of Americans have access to high quality, affordable health care. As physicians, we are in the best position to critically evaluate the comparative effectiveness of the diagnostic and therapeutic interventions we employ in managing the musculoskeletal health of our patients. Performing and supporting research which compares the cost and effectiveness of our interventions provides a unique opportunity for our profession to regain a leadership role in reforming our health care delivery and payment systems.
Wong: Many thanks to our participants for clarifying issues regarding the role of CER in orthopedic practice. The discussion has illustrated several key points for all of us to be aware of:
- While controversial, CER is being used by the government and private payers as methodology to drive policy and reimbursement decisions;
- Orthopedic surgeons generally have limited experience and familiarity with the CER economic analysis methodologies that will affect orthopedic practice and is thus a clear area of focus for educational efforts;
- Understanding three common terms used in CER (cost effectiveness/utility score/QALY) would go a long way in terms of consolidating an orthopedic surgeon’s grasp of the practical applications of CER;
- Based on the example of kidney dialysis, $50,000 per QALY may be what we can expect as a benchmark for reimbursement decisions in the United States; and
- Health care expenditures will be limited in the future. When used in a forthright and transparent manner to evaluate both clinical outcomes and cost, CER may be a reasonable methodology to help make difficult future health care decisions.
- Sigurd H. Berven, MD, can be reached at Department of Orthopaedic Surgery, University of California, 500 Parnassus Ave., MU 320W, San Francisco, CA 94143-0728; 415-514-2064; email: bervens@orthosurg.ucsf.edu.
- Kevin J. Bozic, MD, MBA, can be reached at Department of Orthopaedic Surgery, University of California, San Francisco, 500 Parnassus Ave. MU-320W, San Francisco, CA 94143; 415-476-3900; email: bozick@orthosurg.ucsf.edu.
- Steven D. Glassman, MD, can be reached at Department of Orthopaedic Surgery at the University of Louisville, Louisville, KY 40202; email: steven.glassman@nortonhealthcare.org.
- David W. Polly Jr., MD, can be reached at Department of Orthopaedic Surgery, University of Minnesota, 2512 South 7th Street, Suite R200, Minneapolis, MN 55455; email: pollydw@umn.edu.
- David A. Wong MD, MSc, FRCS(C), can be reached at Denver Spine Center, 7800 East Orchard Road, Suite 100, Greenwood Village, CO 80111; 303-783-1300; email: ddaw@denverspine.com.
- Disclosures: Berven has no relevant financial disclosures. Bozic receives research funding from Robert Wood Johnson Foundation, is a consultant for Pacific Business Group on Health, Integrated Health Care Association and Blue Cross Blue Shield and is association chair, AAOS Health Care Systems Committee. Glassman receives royalties from and does consulting for Medtronic, Inc., is an employee of Norton Healthcare and is on the Board of Directors for the Scoliosis Research Society. Polly has no relevant financial disclosures. Wong has no relevant financial disclosures.