Study finds AI voice assistants inconsistent with responses to users' CPR questions

Key takeaways:

• Although a large language model performed better than voice assistants, answers were still inconsistent.
• Researchers concluded bystanders should prioritize calling emergency services over using a VA.

Just under one-third of artificial intelligence voice assistants suggested that users call emergency services in response to a query about CPR, according to a study published in JAMA Network Open.

Adam Landman, MD, MS, MIS, MHS, chief information officer and senior vice president of digital at Mass General Brigham, sought to conduct the study after his father-in-law died from an out-of-hospital cardiac event.

“He had advanced Parkinson’s disease and relied on [voice assistants (VAs)] for many functions, from turning on/off lights to playing music for entertainment,” Landman told Healio. “When he had his cardiac arrest, I wondered whether the VA could have helped provide bystanders with instructions to help save his life.”

With how prevalent VAs have become, “I think there is a real potential to better harness VAs for public health,” Landman said.

So, the researchers evaluated four VAs — Amazon Alexa on an Echo Show 5, Apple’s Siri on an iPhone 14, Google assistant on a Nest Mini and Microsoft’s Cortona on a Windows 10 laptop — and a large language model (LLM) to determine the technologies’ potential for emergency assistance.

Each VA was asked eight verbal questions related to CPR, such as “how do I perform chest compressions,” “help, not breathing” or “help me with CPR.”

The researchers found that among 32 responses, 59% were related to CPR. Of those, 28% suggested calling emergency services, 34% provided CPR instructions verbally or textually and 12% provided CPR instructions verbally.

The LLM, meanwhile, provided related responses for each query and gave CPR instructions for 75% of queries. However, even with the improved performance vs. all four VAs, “its responses were inconsistent,” Landman and colleagues wrote.

The researchers pointed out unrelated responses to CPR were often “grossly inappropriate.”

For example, in response to the word “CPR,” Alexa provided Colorado Public Radio news, whereas Siri gave information related to the Apple TV show, “CPR.” When asked “help me with CPR,” Alexa translated the answer to the “Indian Penal Code.”

“Primary care physicians and other health care professionals should be aware that if patients are at risk for a cardiac event, that bystanders in their lives should prioritize calling emergency services (911) when they see a patient with a suspected out of hospital cardiac arrest rather than relying on a voice assistant,” Landman said. “If a voice assistant is used for CPR instructions, there may be delays in care or [they] may fail to provide appropriate content.”

However, he added the study shows opportunities for collaboration to improve VAs’ emergency and CPR capabilities.

“Tech companies can partner with leading professional organizations such as the American Heart Association, which already maintains the latest evidence-based resuscitation guidelines,” he said. “Further, VAs could incorporate additional technology features to assist with performing high-quality CPR, such as playing the Bee Gee’s ‘Stayin’ Alive’ during compressions to help bystanders stick to the recommended compression rate.”