AI model using voice recognition outperforms endocrinologists for diagnosing acromegaly
Click Here to Manage Email Alerts
Key takeaways:
- An AI model correctly identified 22 of 31 adults as having acromegaly strictly based on voice recordings.
- The model had a higher area under the receiver operating curve than a group of endocrinologists.
A machine learning model was more accurate than 12 experienced endocrinologists in identifying adults with acromegaly based on voice recordings, according to findings published in The Journal of Clinical Endocrinology & Metabolism.
“Voice is a personal characteristic which may tell us more than we think and has potential as a biomarker of several diseases,” Per Dahlqvist, MD, PhD, associate professor at Umeå University in Sweden and Konstantina Vouzouneraki, MD, a PhD candidate at Umeå University, told Healio. “We have presented the first study using digital voice analysis as a biomarker to identify acromegaly. We analyzed short recordings of a sustained [letter A] recorded with a mobile phone and trained a classification model using machine learning. The machine learning model showed a higher accuracy than experienced endocrinologists in distinguishing acromegaly patients from controls in our population by listening to their voice samples.”
Dahlqvist, Vouzouneraki and colleagues collected voice recordings from 151 adults with acromegaly (median age, 57 years; 39% women) and 151 controls without acromegaly matched by sex and age. Participants voiced a sustained and stable pronunciation of the letter A and read a 160-word text. Voice Handicap Index was administered to assess psychosocial impact of a voice disorder, with a score of 20 or greater indicating a significant level of voice limitations.
AI more accurate than endocrinologists
Researchers used 76% of the voice recordings to train three machine learning models, which were combined together through model stacking. Voice recordings from 31 adults with acromegaly and 31 matched controls were used for the machine learning model’s test set. The AI model correctly diagnosed 22 adults with acromegaly and 24 controls who did not have acromegaly. The model had a sensitivity of 71%, a specificity of 77%, and an area under the receiver operating curve of 0.84.
Twelve of the study’s co-authors, all of whom were endocrinologists experienced in treating pituitary diseases, also attempted to identify which adults had acromegaly. The endocrinologists assessed 50 people with acromegaly and 50 controls by listening to one recording of a sustained letter A sound and four short sentences for each participant.
The endocrinologists finished with a mean sensitivity of 41% and specificity of 75%. The machine learning model finished with a higher area under the receiver operating curve than the endocrinologists (0.84 vs. 0.69; P = .0042). The endocrinologists’ performance when listening only to sentences was similar to their overall performance, with an area under the receiver operating curve of 0.7. When listening to the sustained letter A recordings, the endocrinologists’ area under the receiver operating curve dropped to 0.57.
“There is, still, a substantial diagnostic delay in acromegaly and we believe that easily accessible technology and the current development of machine learning may make voice a useful biomarker for screening for acromegaly in primary care or high-risk populations (like sleep apnea clinics) and awareness campaigns,” Dahlqvist and Vouzouneraki told Healio.
Voice as a future biomarker
Voice Handicap Index scores were higher among the acromegaly groups vs. controls. A higher proportion of adults with acromegaly had a Voice Handicap Index score of 20 points or higher than controls (22.5% vs. 3.6%; P <.001). Among those with acromegaly, Voice Handicap Index scores were similar between those who were biochemically controlled and adults who were biochemically active.
Dahlqvist and Vouzouneraki said machine learning models using voice as a biomarker could not only be used to identify acromegaly, but potentially other diseases in the future. However, they said more training and validation studies are needed before an AI model is used as a diagnostic tool for acromegaly in a real-world setting.
“Digitalization and machine learning shows a tremendous progress with the potential to assist physicians and patients, but it requires large amounts of data,” Dahlqvist and Vouzouneraki told Healio. “For acromegaly and other rare diseases, we need to unite our powers and make international collaborations to make progress.”
For more information:
Per Dahlqvist, MD, PhD, can be reached at per.dahlqvist@umu.se.
Konstantina Vouzouneraki, MD, can be reached at konstantina.vouzouneraki@umu.se.
Collapse
Read more about
Click Here to Manage Email Alerts