This article is more than 5 years old. Information may no longer be current.
Automated algorithm and retina specialists show similar ability to recognize fluid on OCT scans
The Notal OCT Analyzer distinguishes normal morphologic features from elevated or distorted features that indicate the presence of intraretinal fluid.
Automated analysis and human observation showed significant agreement in the identification of retinal fluid on OCT scans from patients with neovascular age-related macular degeneration, a study found.
Investigators assessed the accuracy of the Notal OCT Analyzer (NOA) compared with retina specialists in identifying fluid on OCT.
“What we found was that the NOA performed as well as the retina specialists, indicating that automation is a feasible objective,” corresponding author Usha Chakravarthy, PhD, told Ocular Surgery News.
Reading the scans
The prospective study, published in Ophthalmology, included 155 spectral-domain OCT volume scans obtained with the Cirrus OCT system (Carl Zeiss Meditec) that were stripped of all identifying information. Thirteen scans from eyes with both AMD and vitreomacular interface disease were excluded, leaving 142 scans eligible for analysis.
The NOA and three independent retina specialists analyzed each scan for the presence of intraretinal fluid, subretinal fluid and subretinal pigment epithelium fluid. The NOA ranked individual B-scans for likelihood of choroidal neovascularization. In a second stage, the three retina specialists were asked to review the B-scans in the rank order that the NOA had assigned for signs of lesion activity.
The NOA comprises an image recognition computational technique that distinguishes normal morphologic features from elevated or distorted features that indicate the presence of intraretinal fluid. Pixel-graph optimization delineates the internal limiting membrane and retinal pigment epithelium.
Chakravarthy said that the NOA algorithm was developed specifically to analyze scans from the Cirrus OCT platform, so adaptation of the software would be needed if it were to be used with other OCT systems.
Results
Stage 1 of the study involved the primary analysis of 142 scans in which the outcome was the presence or absence of lesion activity. Ten scans were duplicated and presented randomly and masked to the reader. The duplicate scans were removed, and the 142 original scans served as the primary analysis cohort.
The NOA showed a sensitivity of 92%, specificity of 91% and diagnostic accuracy of 91% relative to the majority grading by the three retina specialists.
In the 121 scans in which results from all three retina specialists agreed, the NOA showed a sensitivity of 94%, specificity of 89% and accuracy of 92%.
The three retina specialists disagreed on 21 of the 142 scans; for these scans, the sensitivity of the NOA was 83%, specificity was 100% and accuracy was 92%. The NOA outcome agreed with the majority vote in 19 of 21 cases (90%).
When the average success rate of a retina specialist against the panel of three readers and an in-house reader was calculated, average sensitivity was 94%, specificity was 92% and accuracy was 93%.
Stage 2 involved the assessment of the lowest number of B-scans required to identify lesion activity. When the subset of active scans within the 121 cube scans that attained consensus grading by all three retina specialists was tested, the retina specialist needed an average of 1.08 B-scans to identify fluid.
On average, the three readers were able to identify fluid in 95% of scans by reviewing a single B-scan with the highest NOA score, 95.5% of consensus scans by viewing the top three B-scans selected by the NOA and 100% of scans by reviewing up to six B-scans.
For the complete set of positive scans showing lesion activity, 91% of lesions were identified by reviewing a single B-scan, 98.2% of lesions were identified by viewing the top four B-scans and 99.1% of lesions were identified by reviewing up to six B-scans.
“All we needed to know was, did the automated analysis system perform in a manner similar to that of a retina specialist in detecting signs of these activities?” Chakravarthy said. “Our hypothesis was that it wouldn’t be any different. Had it been different, in the sense that had the retina specialists performed badly and the NOA performed better, now that would have been a surprise. Or if the analyzer performed really badly, we would have just said, ‘It’s not a very good algorithm,’ but it wouldn’t have been a surprise.” – by Matt Hasson
- Reference:
- Chakravarthy U, et al. Ophthalmology. 2016;doi:10.1016/j.ophtha.2016.04.005.
- For more information:
- Usha Chakravarthy, PhD, can be reached at Centre for Vascular and Vision Sciences, Institute of Clinical Science, Queen’s University of Belfast, Royal Victoria Hospital, Grosvenor Road, Belfast, Northern Ireland BT12 6BA, United Kingdom; email: u.chakravarthy@qub.ac.uk.
Disclosure: Chakravarthy reports she has been a consultant for and has received travel funding from Notal Vision.
Perspective
Back to Top
Chakravarthy and colleagues studied the accuracy of a commercial device/software, the Notal OCT Analyzer, and compared it with the ability of a retina specialist to detect fluid on the OCT in patients with AMD, so it was a test of the software vs. the retina specialists review and grading of the OCT (and not their ability to detect fluid clinically on the exam). Detecting fluid on an OCT matters because it may be considered at the time of initial diagnosis of wet AMD and during follow-up. If one is treating using an as-needed or treat-and-extend algorithm, the OCT findings are an integral part of the decision-making process about treatment. The software and the retina specialists agreed 91% of the time. The authors showed that one can automate the OCT grading for presence of fluid and do a good job of this. Health care providers are searching for improved efficiency, predictability and reproducibility. It is likely there will be a role for software to help guide and, I hope, improve care for our patients with wet AMD and similar conditions.