‘Reassuring’ findings show that patients are open to AI-drafted messages

ByAndrew (Drew) Rhoades

Fact checked byMindy Valcarcel, MS

Key takeaways:

Patients were more satisfied with AI-drafted messages than those from clinicians.
Disclosure of AI involvement “won’t alienate patients,” a researcher said.

Patients showed a preference for messages created by AI over those by clinicians, an analysis in JAMA Network Open showed.

Patients were less enthused when they were told AI was involved, but the results were still “reassuring,” Anand Chowdhury, MD, MMCi, an assistant professor of medicine at Duke University School of Medicine, told Healio.

“We wanted to understand what patients think about using automated tools in communication with their doctors,” he said. “Our goal was to hear directly from patients about their opinions on the quality, usefulness and care of AI-generated messages compared to human messages, especially after they knew AI was involved.”

Chowdhury and colleagues sent multiple surveys to members of the Duke University Health System patient advisory committee — a mixture of past and current patients and community members who “provide input on various topics related to health care, patient experience and service enhancements,” according to the study.

In the survey, the participants assessed three types of realistic responses to questions that might come up in a patient portal, which varied in seriousness: a routine medication refill request, a medication side effect question and possible cancer on imaging. A multidisciplinary team of doctors created the human responses to those questions, and a version of ChatGPT wrote the AI-generated responses. The team made sure to review the AI messages for accuracy before participants had an opportunity to rate them. The author of the message — AI or human — was randomly disclosed to participants.

Respondents rated their satisfaction with the message, its usefulness and their perceived level of care on a 5-point Likert scale.

A total of 1,455 people responded to at least one of the surveys.

The researchers found that patients preferred AI-generated responses over those drafted by clinicians, with mean differences (MD) of:

−0.3 (95% CI, −0.37 to −0.23) points for satisfaction;
−0.43 (95% CI, −0.5 to −0.37) points for perceived level of care; and
−0.28 (95% CI, −0.34 to −0.22) points for usefulness.

Chowdhury and colleagues also pointed out that participants tended to have higher satisfaction with the message if they learned that it was written by a human (MD = 0.13 points; 95% CI, 0.05-0.22) or if the author was not disclosed at all rather than discovering that the message was generated by AI (MD = 0.09 points; 95% CI, 0.01-0.17).

“Interestingly, there wasn't much difference in satisfaction when patients were told a human wrote the response and when there was no disclosure statement,” Chowdhury told Healio. “This suggests that patients might assume a human is involved unless explicitly told otherwise. If this is true, this type of assumption may change as patients become more accustomed to AI interactions.”

He highlighted two key takeaways, one of which being that “within the limits of our study, patients have a high degree of satisfaction with AI-drafted responses that are edited by their human clinician.

“Second, our results reassure us that health systems won't alienate patients by disclosing the use of AI,” he added. “This is crucial as health care looks to AI as a tool to deliver more and higher-quality care to patients, because it enables us to continue to be transparent with patients about the use of AI in their care.”

Chowdhury concluded that further research on this topic “should include more voices that better reflect the population as a whole... since our results came from a group of patients who volunteered to be part of an advisory board.

“Attitudes toward AI will likely evolve over time, and future studies will need to account for this,” he added. “Finally, we only looked at one instance of the use of AI in patient-clinician communication. As more applications are found, the research on patient preferences will have to keep pace.”