This article is more than 5 years old. Information may no longer be current.

Physicians 'vastly outperform' online symptom checkers in diagnostic accuracy

Physicians were more than 30% more accurate compared to symptom checker websites and apps in a comparison published in JAMA Internal Medicine.

Hannah L. Semigran, BA, from Harvard Medical School, and colleagues, noted, however, that physicians did diagnose incorrectly in about 15% of cases.

"The Institute of Medicine recently highlighted that physician diagnostic error is common and information technology may be part of the solution," Semigran and colleagues wrote. "Given advancements in computer science, computers may be able to independently make accurate clinical diagnoses. While studies have compared computer vs. physician performance for reading electrocardiograms, the diagnostic accuracy of computers vs. physicians remains unknown. To fill this gap in knowledge, we compared the diagnostic accuracy of physicians with computer algorithms called symptom checkers."

They provided internists, pediatricians and family practice physicians with 45 clinical vignettes via Human Dx, a web- and app-based platform where physicians can provide diagnoses. The physicians submitted diagnoses for the vignettes, which were classified as 15 low, 15 medium and 15 high-acuity conditions for 26 common and 19 uncommon conditions between December 2015 and May 2016.

The researchers also included diagnostic accuracy data of the vignettes for 23 symptom checkers that they had assessed in a previous study.

Semigran and colleagues wrote that 234 physicians solved at least one of the vignettes, with 211 (90%) reporting internal medicine training and 121 (52%) reporting fellow or resident status.

Results showed that, compared with symptom checkers, physicians were more likely to list the correct diagnosis first (72.1% vs. 34%; P < .001) and were more likely to list the correct diagnosis in the top three diagnoses (84.3% vs. 51.2%; P < .001).

In addition, symptom checkers most often listed the correct diagnosis for low-acuity vignettes and common conditions, while physicians most often listed the correct diagnosis for high-acuity vignettes and uncommon conditions.

"In what we believe to be the first direct comparison of diagnostic accuracy, physicians vastly outperformed computer algorithms in diagnostic accuracy (84.3% vs. 51.2% correct diagnosis in the top three listed)," Semigran and colleagues concluded. "Despite physicians’ superior performance, they provided the incorrect diagnosis in about 15% of cases, similar to prior estimates (10% to 15%) for physician diagnostic error. While in this project we compared diagnostic performance, future work should test whether computer algorithms can augment physician diagnostic accuracy." – by Chelsea Frajerman Pardes

Disclosure: One researcher is an equity holder of The Human Diagnosis Project, the creators of Human Dx. None of the authors reported any relevant financial disclosures.