This article is more than 5 years old. Information may no longer be current.
Natural language processing system accurately measures colonoscopy quality
A natural language processing program provided accurate tracking of colonoscopy quality measures and assignment of surveillance intervals, according to recent study data.
“We found that rapid and inexpensive natural language processing (NLP), which utilizes free-text data that was previously unusable for efficient computer-based analysis, was extremely accurate in measuring adenoma detection rate during colonoscopy,” Timothy Imler, MD, a Regenstrief Institute investigator and assistant professor of medicine in the division of gastroenterology and hepatology at Indiana University School of Medicine, said in a press release.
Timothy Imler
Imler and colleagues created and validated the performance of an NLP system with 19 measures for quantifying adenoma detection rate and providing surveillance recommendations using data from 13 Veterans Affairs endoscopy units. From 42,569 colonoscopy reports with pathology records, they randomly selected 250 for the training set to refine the NLP and 500 for the test set. Masked, paired, annotated expert manual review was used to create the reference standard, and the remaining 41,819 nonannotated records were processed through the NLP system to evaluate consistency.
Resulting from paired annotation were 176 (23.5%) documents with 252 (1.8%) discrepant content points. Within the 500 test documents, the error rate was 31.2% for NLP and 25.4% for the paired annotators (P = .001); at the content point level it was 3.5% for NLP and 1.9% for the paired annotators (P = .04). Eight vaguely worded documents were removed in the post hoc analysis, resulting in 125 of 492 (25.4%) incorrect by NLP and 104 of 492 (21.1%) by the initial annotator (P = .07). NLP and annotation resulted in similar rates of pathologic findings.
Accuracy of the test set was 99.6% for colorectal cancer, 95% for advanced adenoma, 94.6% for nonadvanced adenoma, 99.8% for advanced sessile serrated polyps, 99.2% for nonadvanced sessile serrated polyps, 96.8% for large hyperplastic polyps and 96% for small hyperplastic polyps. Lesion location demonstrated 87% to 99.8% accuracy, and accuracy for number of adenomas was 92%.
“NLP for colonoscopy has a bright future for both clinicians and the patients we serve,” Imler told Healio Gastroenterology. “The optimal utilization within colonoscopy would be to have a procedure performed, pathology potentially generated, NLP of the procedure and pathologic records, automated reporting to the provider/patient/primary care/quality reporting agency/payer for quality measures, and generation of a guideline-appropriate surveillance interval. While we must individualize patient care, NLP will allow for a glimpse into the practice patterns of individuals and potentially allow for a means for intervention to improve care.” – by Adam Leitenberger
Disclosure: Imler and another researcher report they have filed for provisional patent for this work. The other authors report no relevant financial disclosures.