AI identified patient messages sent by proxies, but also broke confidentiality
Click Here to Manage Email Alerts
Key takeaways:
- LLMs drafting responses to patient messages can mistake caregivers for patients.
- The LLM in this study correctly identified 76% of messages sent by proxies, but also broke confidentiality twice.
AI identified messages sent through a patient portal by adolescents’ caregivers — and not the patients themselves — three-quarters of the time, but also broke confidentiality in two cases, according to findings published in JAMA Pediatrics.
“Clinicians responding to adolescent patient messages understand that patients’ parents or legal guardians (ie, proxies) have access to most of their children’s health information and can send messages directly on their behalf,” Gabriel Tse, MBChB, MS, pediatric hospitalist at Lucile Packard Children’s Hospital in Palo Alto, California, and clinical assistant professor of pediatrics at Stanford School of Medicine, and colleagues wrote.
“Pediatric clinicians are therefore required to identify whether messages are being sent from the patient or proxy user (ie, proxy user identification) to communicate effectively and protect adolescent confidentiality,” they wrote.
The researchers evaluated a large language model’s (LLM) ability to discern who was sending messages and to send appropriate responses. They randomly selected messages sent through patient portals for adolescents aged 12 to 17 years at Stanford Medicine Children’s Health system and made the LLM generate a response.
Overall, 213 (71%) of the messages were sent by proxies and 25 (12%) were from the adolescent patients, Tse and colleagues found. There were 52 (17%) messages where the sender could not be verified based on content.
The LLM correctly identified most of the messages sent by proxy users (76%), according to Tse and colleagues.
“Of the 300 patient messages that our study analyzed, two AI responses breached patient privacy by providing unsolicited confidential information to the patient’s caregiver,” Tse told Healio. “While this is a relatively uncommon occurrence (<1%), this becomes a major issue for health systems managing thousands of patient messages per week.”
Most of the responses were easy to understand (91%), and 67% were considered clinically useful, the researchers wrote.
“Pediatricians using AI tools to draft patient message responses should remain vigilant to ensure that accurate and safe information is ultimately provided to the patient,” Tse said. “My hope is that future research on AI will specifically focus on the needs of pediatric populations since there are unique attributes in pediatrics that do not exist in adult populations.”
Collapse
Click Here to Manage Email Alerts