This article is more than 5 years old. Information may no longer be current.
Online resource enhances grading of fundus images in DR screening
Click Here to Manage Email Alerts
PHILADELPHIA — The use of anonymous workers online may enhance the grading of fundus photographs in screening patients for diabetic retinopathy, a speaker said here.
“With minimal training, the Amazon workforce can rapidly and correctly categorize fundus photos as normal or abnormal,” Christopher J. Brady, MD, said at the Wills Eye Annual Conference. “They did that excellently from the outside. With some modifications to our interface, they did a much better job grading the level of retinopathy. This is a new and inexpensive way to possibly screen for diabetic retinopathy.”
Christopher J. Brady
Researchers used crowdsourcing, which involves the use of anonymous workers to perform specific tasks online. They accessed Amazon Mechanical Turk, a resource for online employment. The researchers used 19 fundus images from teaching libraries, grading 12 images as abnormal and seven images as normal. They then asked online workers to grade the images as normal or abnormal. Online workers were paid $1 per image.
The average time needed by online workers to grade each image was 25 seconds.
Online workers reached a correct diagnosis of diabetic retinopathy 81.5% of the time. Grading by online workers agreed with that of expert graders almost 90% of the time, Brady said.
“Critically, there were no false-negative results, so all of the errors were overcalls rather than undercalls, which would be critical for any type of screening program. You can’t miss diagnoses,” Brady said. “Between seven and 10 graders gives you the ideal balance, sensitivity and specificity. We felt confident that 10 graders was a good number moving forward.”
Disclosure: Brady has no relevant financial disclosures.