Clinician input essential in entire cardiology AI tool development process
Click Here to Manage Email Alerts
Key takeaways:
- Solicitation of clinician input during AI tool development often happens late stage, if at all.
- Clinicians are “most aware of the clinical needs and deficiencies in existing technology.”
Editor's Note: This is part three of a three-part Healio Exclusive series on the development and use of AI to improve clinical outcomes in cardiovascular medicine and considerations for regulatory labeling and patient privacy .
Part 1 can be viewed here. Part 2 can be viewed here.
Physician involvement in the development of AI tools is imperative to ensuring their success in not only improving clinical outcomes but also improving daily workflow and reducing physician workload.
“AI has a potential to help facilitate our work. We like to call it ‘augmented intelligence’ because it can augment our care. ... AI offers an opportunity for us to do this right. We dismally failed when it came to electronic health records and we paid for it. This is an opportunity for us to drive it,” Healio | Cardiology Today Editorial Board Member Dipti Itchhaporia, MD, MACC, FESC, the Eric & Sheila Samson Endowed Chair in Cardiovascular Health and director of disease management for Jeffrey M. Carlton Heart & Vascular Institute at Hoag Memorial Hospital, Newport Beach, California, clinical professor of medicine at University of California, Irvine, and past president of the American College of Cardiology, said in an interview. “There is more clinician involvement now than there was in the past. For a long time that hadn’t been the case. ... Some of these [engineers] were just developing products and then figuring out what the clinical need was.”
Clinician input in development of AI tools
In a systemic literature review published in Frontiers in Psychology, researchers evaluated the rate of clinician involvement in the development and evaluation of clinical AI models.
They found that among 24 studies aimed to develop new AI tools for clinical medicine, clinician input was solicited in less than one-quarter and of those that did solicit input, 88% did not do so until later stages of AI model development.
In a review published in BMJ Health & Care Informatics, an international group of researchers put forward their nine stages for developing predictive AI models for use in clinical medicine:
- clarifying the clinical question of interest;
- features selection;
- choosing relevant datasets;
- developing the AI model;
- validating the model;
- interpreting the model’s predictions;
- licensing model;
- maintaining the model; and
- ongoing evaluation of the impact of the predictive model.
The researchers wrote that in stage 1 of development, it is vital to seek out a multidisciplinary team of clinical specialists when clarifying the clinical question of interest for the AI model to answer.
Real-world clinician participation
An example of clinician involvement was the participation of Paul A. Friedman, MD, FHRS, cardiac electrophysiologist and chair of cardiovascular medicine at Mayo Clinic in Rochester, Minnesota, and Patricia A. Pellikka, MD, The Betty Knight Scripps Professor of Cardiovascular Disease Clinical Research, president-elect of the Mayo Clinic Officers and Councilors and consultant in the department of cardiovascular medicine at Mayo Clinic, in the EAGLE trial.
EAGLE was a randomized controlled trial that evaluated an AI-powered clinical decision support tool for the early diagnosis of low left ventricular ejection fraction. The results were published in Nature Medicine (a discussion of the results is in part 1 of this series) and the algorithm subsequently received FDA 510(k) clearance for the identification of low LVEF.
“If we really want to improve health, we need to make sure that we’re asking impactful questions,” Friedman told Healio. “Clinicians are best suited to identify what are the questions, what are the unsolved problems and what are the clinical gaps that we need addressed.”
Pellikka, who is also editor-in-chief of the Journal of the American Society of Echocardiography, spearheaded a project to develop a 3D convolutional neural network (EchoGo Heart Failure, Ultromics) to detect HF with preserved ejection fraction from a single standard echocardiogram. The positive results were published in JACC: Advances.
The AI model demonstrated excellent discrimination with an area under the curve of 0.97 during training and 0.95 on validation, with high sensitivity (87.8%) and specificity (81.9%).
As Healio previously reported, the AI model was cleared by the FDA in December 2022.
“We are extending this collaboration to develop other models for disease detection from limited echocardiographic images,” Pellikka told Healio. “Given the shortage of cardiac sonographer staff in the U.S. and the increasing complexity of patients undergoing echocardiography, these tools offer great potential to benefit clinical practice.”
AI models to predict arrhythmia events are also being actively investigated.
In 2021, Shaan Khurshid, MD, MPH, cardiac electrophysiologist and assistant in medicine at the Telemachus and Irene Demoulas Family Foundation Center for Cardiac Arrhythmias at Massachusetts General Hospital, instructor of medicine at Harvard Medical School and affiliated scientist at Broad Institute of Harvard and MIT, and colleagues published in Circulation their findings from a study to utilize ECG-based deep learning combined with clinical risk factors to predict future atrial fibrillation.
The researchers reported that the AI model predicted 5-year AF risk with a similar AUC compared with the CHARGE-AF clinical risk factor model (CHARGE-AF, 0.752; AI model, 0.747).
‘Be part of development’
“This is a real opportunity and an important point in the history of AI in cardiology. ... Clinicians have to be involved early to help cocreate, codevelop all of the things we need to gather if we’re going to get the true value of AI,” Itchhaporia told Healio. “Educate yourself. Get involved. Don’t be afraid of it, but also play a role. Giving clinical input into what it is that we need from the products being developed with embedded AI is going to be invaluable.”
“For clinicians who are research-oriented, the best way to get involved is to actually be part of development. For clinicians not interested in research, I still think it’s important for them to be part of the process,” Khurshid told Healio. “There are some data that suggest that non-MD providers such as nurse practitioners and physician assistants had quicker uptake of AI tools compared with some MDs. Factors like that are important because those are key determinants of whether clinicians are actually going to use the AI tools. They help us design AI tools that better serve the end user. That’s an area of research that’s relatively understudied.”
In the American Heart Association Scientific Statement on the use of AI to improve CV outcomes, published in February in Circulation, the authors stated that there is an urgent need to develop implementation science for AI tools into clinical practice that address core unmet clinical needs, and clinicians are best suited to identify those needs.
“Clinicians should have a central role in the development of AI tools,” Pellikka said. “It is clinicians who are most aware of the clinical needs and deficiencies in existing technology.”
In the recent state-of-the-art review, published in July in the Journal of the American College of Cardiology, researchers provided an overview of the current landscape of AI across cardiology clinical practice and biomedical discovery. The authors wrote that an understanding of the current AI landscape is a prerequisite to clinician participation in future developments.
“Their involvement begins with awareness of emerging technologies,” Rohan Khera, MD, MS, assistant professor in the section of cardiovascular medicine at Yale School of Medicine and cardiologist at Yale New Haven Hospital and first author of the review, told Healio. “It will be useful for the community to also have a degree of skepticism about AI tools as they would for new drugs and devices. In the end, changes in care are a certainty, but ensuring safety and security of our patients requires awareness of the evidence behind the tools we use in our care.”
We want to hear from you:
Healio wants to hear from you: Are you or your colleagues working on any novel projects involving AI in cardiovascular medicine? Share your thoughts with Healio by emailing the author at sbuzby@healio.com or tagging @CardiologyToday on X (Twitter). We will contact you if we wish to publish any part of your story.
For more information:
Paul A. Friedman, MD, can be reached at friedman.paul@mayo.edu; X (Twitter): @drpaulfriedman.
Dipti N. Itchhaporia, MD, can be reached at drdipti@yahoo.com; X (Twitter): @ditchhaporia.
Rohan Khera, MD, MS, can be reached at rohan.khera@yale.edu.
Shaan Khurshid, MD, MPH, can be reached at skhurshid@mgb.org; X (Twitter): @shaan_khurshid.
Patricia A. Pellikka, MD, can be reached at pellikka.patricia@mayo.edu; X (Twitter): @pattypellikka.
References:
- Akerman PA, et al. JACC Adv. 2023;doi:10.1016/j.jacadv.2023.100452.
- Armoundas AA, et al. Circulation. 2024;doi:10.1161/CIR.0000000000001201.
- Hassan N, et al. BMJ Health Care Inform. 2023;doi:10.1136/bmjhci-2023-100784.
- Jesso ST, et al. Front Psychol. 2022;doi:10.3389/fpsyg.2022.830345.
- Khera R, et al. J Am Coll Cardiol. 2024;doi:10.1016/j.jacc.2024.05.003.
- Khurshid S, et al. Circulation. 2021;doi:10.1161/CIRCULATIONAHA.121.057480.
- Yao X, et al. Nat Med. 2021;doi:10.1038/s41591-021-01335-4.
Collapse
Healio Interviews
Click Here to Manage Email Alerts