This article is more than 5 years old. Information may no longer be current.
‘Significance of numbers’ key to finding, understanding good literature
Click Here to Manage Email Alerts
Clinical literature has been a foundation for evidence-based medicine for centuries; however, identifying good material can be a challenge. Clinicians must critically consider studies for good medical evidence that can be safely and effectively applied in a clinical setting to help achieve better outcomes.
“The key to good literature is to understand the significance of numbers,” J. James Thimons, OD, said in an interview with Primary Care Optometry News. “It’s really not pertinent to do a glaucoma study that lasts for a few months or has 25 patients. That’s a trend-line, which is interesting, but not a definitive statement. The nature of the disease state, to a large degree, speaks to how and when the statistical significance can be developed.
“Conditions such as age-related macular degeneration and glaucoma, which are generally very slow, progressive diseases, can take a period of several years, maybe up to 10, to be able to show statistical significance,” he continued. “The time period is necessary because of the variation of the disease. You need to be able to see its expression over that time period in its natural state to determine whether a drug or surgical intervention effects actual changes.”
Clinicians have to ask themselves, he said, what the numbers and evidence actually demonstrate.
A slow, progressive disease such as glaucoma (pictured here) must be studied for a period of many years to indicate statistical significance for drug intervention.
Image: Thimons JJ
“A clinical trial of a drug or surgical intervention that showed a consistent decrease in IOP in 100 glaucoma patients is reasonable, but only if it follows the efficacy over an extended period of time,” Thimons said. “Conversely, in trials that involve infectious disease, inflammation or other acute therapies, efficacy can be measured in defined periods, but the cohort of the study becomes important to show efficacy over a wide range of patients (gender, race, age, etc.).
“So, it’s critical for doctors to utilize their knowledge of the disease state and apply it to the data that’s being presented,” he added.
Keeping pace
To provide the best care in the most efficient way means that there must be an established medical necessity for any test the clinician considers, Anthony A. Cavallerano, OD, said during a lecture at the SECO meeting in Atlanta. The best way to determine medical necessity is through the latest and best clinical evidence that says this test is called for, he said.
Anthony A.
Cavallerano
“Take the Macular Photocoagulation Study, for example,” Cavallerano told attendees of his lecture. “Remember how important it was? Is it subfoveal? Is it juxtafoveal? Is it extrafoveal? Now it doesn’t matter. If there’s membrane, or if there’s bleeding, or if there’s exudative maculopathy, the patient gets an intravitreal injection. We’ve seen this major evolution away from all of these important diagnostic procedures and disease classification and toward intravitreal injection.”
Literature, evidence-based medicine
Evidence-based medicine involves more than just looking at the number of people that have been entered into a study, what the results are or the percentage of patients that benefited from a particular intervention, according to Cavallerano. Studies need to be considered in a much more intensive and comprehensive way.
“The goal of evidence-based medicine is to provide individual patients with care that’s based on the best available information that’s not only timely, but accurate and up-to-date and that reflects thorough investigation and evaluation,” Cavallerano said. “And that doesn’t come from a trial of six patients, two of whom showed improvement, two of whom got worse and two of whom stayed the same. That doesn’t help us.
“Time and time again we see limited studies that are reported like that, though, and the results often drive treatment philosophies,” he added.
Case reports and case series, for example, are easy to perform and relatively easy to get published. Case control or cohort studies are more useful than those, but random clinical trials are what clinicians should be looking at, he said.
“Systematic review and meta-analysis is the best way to approach dealing with evidence-based information on a patient. Thoroughly look at information and data in a number of studies, not just one, and combine those results using sophisticated statistical techniques,” Cavallerano said.
“Then amalgamate the data and come up with a meaningful summary,” he continued. “This will allow you to make much more juried interpretations of the evidence and decisions on what to do with that evidence.
For example, according to Cavallerano’s co-lecturer, David Shein, MD, one aspect that requires diversified, aggregated data is a study’s patient population.
“It involves understanding what the population was in the study as it relates to that particular study and your patient’s specific needs,” Shein said.
For instance, a study done at the Veterans Affairs would typically have a population that is mostly, if not completely, males, because military personnel are predominantly male, he said.
“Will those results apply to female patients?” Shein said. “Often studies in the U.S. are of Caucasian populations. How might those results apply to someone of a different race, or from a different place, or if you’re practicing in a different country?
“You can’t let your clinical impression of a surrogate endpoint outshine the evidence,” he continued. “Early studies of various drugs demonstrated improvement in cholesterol profiles. However, cholesterol level is a surrogate endpoint — the goal for our patients is to reduce heart disease and death. When longer studies looked at the impact on deaths, there was no benefit.”
Good reading practices
According to Thimons, there are two levels of information. The first level is trade journals, which discuss trends and concepts that are upcoming and important. Clinicians can go there first to find topics of interest.
“They’re valuable,” he said, “but are typically not going to have the type of statistically significant, placebo-controlled, prospective, multicenter trials that are necessary to show real impact.”
Once a topic is found, the second level is to create a larger base of knowledge through the use of the peer-reviewed publications, Thimons said.
Abstracting services are also good resources to have when reviewing literature, he said.
“One of the prime services I subscribe to is a company called JournalWatch out of Massachusetts,” Thimons said. “They review and abstract key articles and provde an editorial comment about the validity of the study, statistical significance, outcomes, relative value that it may add to or subtract from previous literature or the need for a larger study to show that this trend-line is actually applicable to patients in a broader context.
“PubMed also does a nice job of creating abstract material,” he added.
J. James Thimons
It is also important to apply these critical reading skills to information provided by speakers at meetings, he said.
“It’s reasonable to ask individuals who are at the podium to speak from their clinical experience, referenced through peer-reviewed literature,” Thimons said.
Industry influence
The industry has had, and to a degree still has, a significant influence on how clinical evidence is presented to doctors, Thimons said. A study published in the New England Journal of Medicine in 2008 examined trials on 12 antidepressant drugs reported in literature during the time period of 1987 to 2004 and compared the published outcomes with the U.S. Food and Drug Administration’s reviews of each of the 12 drugs.
The study found that 31% of 74 FDA-registered trials associated with the drugs during the measured time period were not published. It also determined that “whether or how the studies were published were associated with the study outcome.”
According to the study, 38 of the trials were viewed by the FDA as having positive results and, of that number, only one was not published. Of the 36 trials that were viewed by the FDA as having negative or questionable results, 22 were not published.
Eleven additional studies were published in a way – in the opinion of the authors – that conveyed a positive outcome by downplaying or omitting negative pre-specified primary endpoints or highlighting secondary endpoints as if they were primary.
“You really need to consider how this information is being filtered to you,” Thimons said. “Look to the great peer-reviewed journals. They take pride in the fact that articles that get published in their journals are highly vetted to a level that the vast majority will be successfully capable of withstanding public scrutiny and rebuttal.
The same goes for speakers at meetings, Thimons said. Presenters should note that their concepts are based on their clinical experience and peer-reviewed literature and not supported by industry.
“Once I know his or her disclosures, I can then put into perspective the message I’m hearing,” Thimons said.
Federal incentives
Practicing evidence-based medicine and improving outcomes has become even more important to clinical practice through health care reform.
In past years, clinicians have been incentivized for reporting quality outcomes to the Centers for Medicare and Medicaid Services (CMS). Beginning in 2015, the federal government will be imposing new legislation that will shift the Physician Quality Reporting System (PQRS) from an incentive-based system to a penalty-based system, according to John M. B. Rumpakis OD, MBA.
“The PQRS requires that clinicians identify certain quality measures that you practice in order to get a bonus as a percentage of your CMS billing if you fulfill a minimum number of measures,” Rumpakis said. “With new changes in 2015, you’ll be subject to a 1.5% deduction in reimbursement if you’re not meeting those quality measures. And in 2016, you’ll get a 2% deduction.”
Also, clinicians who do not report clinical quality measures (CQM) to CMS – in addition to the 1.5% to 2% deduction in reimbursements – could be relegated to a second tier level of providers when the legislation comes into effect, Rumpakis said.
In essence, a Tier 2 provider is simply a clinician who does not report CQM to CMS, he said.
“So, the patient may be inclined to see a Tier 1 provider,” Rumpakis said.
Patients will be further inclined to see Tier 1 providers by possibly being eligible for a lower deductible and lower copays if they choose one, he added.
“We just don’t know what we don’t know yet,” Rumpakis said. – by Daniel R. Morgan