PEDIG study methodology reviewed

At the AAO, a presenter discussed limitations of the group’s study methodology. One of the PEDIG chairmen responds.

Issue: December 15, 2006

ByErin L. Boyle

The Pediatric Eye Disease Investigator Group has conducted numerous multicenter studies over the past 9 years. At this year’s American Academy of Ophthalmology meeting, one presenter outlined some possible limitations in these studies’ methodology.

Pamela J. Kutschke, CO, raised concerns about results of the network’s studies at an AAO symposium on treatment of amblyopia. She confined most of her remarks to the PEDIG study of atropine vs. patching for the treatment of moderate amblyopia.

Ms. Kutschke said techniques used by investigators in PEDIG studies can vary from center to center.

“They tried to look at all angles and all aspects of amblyopia treatment, which is fine, except they’re hampered by the fact that all people don’t treat amblyopia the same way,” Ms. Kutschke said in a follow-up telephone interview with Ocular Surgery News. “So you have some people doing it one way, and other people doing it another way, and you’re trying to compare all those groups because you need a certain amount of patients to do a prospective study that has sufficient statistical power.”

Michael X. Repka, MD, one of the two PEDIG chairmen, responded to Ms. Kutschke’s remarks via e-mail and in a telephone interview with OSN. He also spoke during the same AAO session. (See related article.)

“A clinical trial has a protocol that stipulates how the treatments were to be delivered,” Dr. Repka said by telephone. “These were developed by the investigator group to be consistent with typical clinical care. In addition, it would be an unusual trial that could answer every clinical question about a disease. Every study leads to questions, and this study led to studies in which we addressed the question of dosage. For instance, that’s where we found that fewer drops of atropine were necessary per week than we prescribed in the first trial.”

Atropine vs. patching

The PEDIG atropine vs. patching study was a randomized clinical trial including 419 children younger than 7 years with amblyopia and visual acuity in the range of 20/40 to 20/100. Patients at 47 clinical sites were assigned to receive either patching or atropine.

The main outcome measure was visual acuity in the amblyopic eye and the sound eye after 6 months. Researchers found that visual acuity in the amblyopic eye improved in both groups (improvement from baseline to 6 months was 3.16 lines in the patching group and 2.84 lines in the atropine group). Improvement was initially faster in the patching group, but after 6 months the difference in acuity between treatment groups was clinically inconsequential: mean difference at 6 months, 0.034 logMAR units (95% confidence interval, 0.005-0.064 logMAR units).

At 6 months, acuity in the amblyopic eye was 20/30 or better and/or improved from baseline by three or more lines in 79% of the patching group and 74% of the atropine group, the study found. Both treatments were well tolerated. More patients in the atropine group than in the patching group had reduced acuity in the sound eye at 6 months, but this did not persist with further follow-up.

Methodology

Ms. Kutschke said ambiguity in the study’s methodology made it difficult to interpret the results. For instance, in the patching study, the authors found that much of the improvement seen at 6 months was present at 5 weeks, she noted. That result could be interpreted to mean that patching could be stopped after 5 weeks, she said.

“Actually, the data presented in the paper appear to show continued improvement throughout the 6-month period,” Ms. Kutschke said in her AAO presentation. “It is not known how much patching was really done during this time. Information is not presented in the paper as to how many patients increased or decreased their patching during the course of the study, except to say that patients with lower starting acuity were prescribed more patching at onset. It is not clear how much patching was done when left to the ‘investigator’s discretion.’”

She said “investigator’s discretion” could allow great variability, and this was not accounted for in the study.

“Let’s say that a patient reaches the success level of 20/30 or three lines of improvement,” Ms. Kutschke said at the AAO. “One investigator says ‘great’ and stops patching except for the mandated 1 hour per day. Another investigator says ‘good, but not good enough’ and continues patching. … No differentiation was made in the results. Maybe those who stopped patching would have increased vision if they had continued.”

In the study, patients were randomized to one drop of atropine per day, or occlusion for 6 hours a day minimum or all but 1 hour that patients were awake.

“While the atropine was very controlled, the patching was not,” Ms. Kutschke said. “I felt that this discrepancy in the amount of patching that could be done led to unreliable results.”

Reply to critique

In an e-mail reply to Ms. Kutschke’s critique, Dr. Repka said the study authors “used standard methods of encouraging compliance.”

“We reported that improvement was rapid in the first 5 weeks and continued for 6 months, and we did not suggest this to mean that treatment could be stopped after 5 weeks,” he said. “In fact, we acknowledged in the manuscript that maximum improvement could take longer than 6 months for many patients.”

“This was a real-world study, known most properly as an effectiveness study,” he continued. “In such a study we compare the effects of prescribed treatments, in contrast to an efficacy study that compares the effects of treatment actually done.”

Dr. Repka said the investigators knew the amount of patching prescribed was always at least 6 hours per day and that it could be as much as full-time during the patients’ waking hours. If the children were not successfully treated, the protocol stipulated for the final 8 weeks patching was for 12 or more hours.

“Since that study was performed, we have refined the prescribed patching dose with subsequent studies,” Dr. Repka told to OSN. “Investigator discretion allowed us to deal with the huge diversity of patching doses prescribed in the community at the time the first PEDIG amblyopia study was launched.”

Because some physicians prefer full-time occlusion, which has not been clinically proven to be the most effective treatment, a “floor” of 6 hours was set for patching, but investigators were allowed to use more if they wished, Dr. Repka said. Allowing discretion with prescribed hours of patching improved the comfort of investigators and patients with the protocol, he said, and made the results more generalizable.

Dr. Repka said he agreed that the initial study comparing atropine to patching raised the issue of how many hours of patching are necessary for moderate amblyopia. However, he said, the PEDIG authors conducted a second randomized trial evaluating various patching regimens and found no “meaningful” difference based on dosage.

For more information:

Pamela J. Kutschke, CO, is director of the orthoptic teaching program and a faculty adjunct lecturer in the Department of Ophthalmology & Visual Science at the University of Iowa. She can be reached at 200 Hawkins Drive, Iowa City, IA 52242-1091; 319-356-3863; fax: 319-356-0363; pamela-kutschke@uiowa.edu.

Michael X. Repka, MD, can be reached at 233 Wilmer Ophthalmological Institute, 600 N. Wolfe St, John Hopkins Hospital, Baltimore, MD 21287-9028; 410-955-8314; fax: 410-955-0809; e-mail: mrepka@jhmi.edu.

Erin L. Boyle is an OSN Staff Writer who covers all aspects of ophthalmology.