Large language models making an impact in ophthalmology
Key takeaways:
- AI programs are capable of mimicking ophthalmic journal articles.
- A major weakness of large language models is that they are not able to understand truthfulness.
KOLOA, Hawaii — Large language models are impacting ophthalmology in a myriad of ways, according to a speaker here.
During a presentation at Retina 2025, Aaron Y. Lee, MD, MSCI, discussed how AI models are able to generate ophthalmology journal articles and pose a threat to Zoom-administered oral board exams, offering the potential for cheating.
To test ChatGPT-3’s capabilities, Lee and colleagues aggregated more than 23,000 ophthalmic research articles and had the program generate fake, full-text manuscripts.
“We were sort of shocked to see how well this worked,” he said.
Lee noted that the AI-generated articles included reasonable numbers, ranges and statistical outcomes. Some of the generated articles also included new hypotheses that had not been previously tested.
However, large language models have a notable Achilles heel. Deep learning models need a mathematical function that is differentiable to work properly, Lee said, and truthfulness is not something that can be currently coded as that function.
“At the end of the day, all that these large language models are doing is mimicking human language and human reasoning patterns,” he said. “There is nothing that actually ensures these models will be truthful.”
Lee also demonstrated how ChatGPT could potentially be used to cheat on an oral board exam. He used two computers: One to simulate the examiner and one to simulate the exam taker. ChatGPT was used to read the test prompts and automatically generate answers.
“What’s sort of amazing to see is that the model is able to hit these questions out of the park,” he said. “It gives a differential diagnosis, what tests it would order and it tells you how quickly those interventions need to come.”
One way this type of cheating is being combated is via the use of a separate web camera to monitor the exam taker’s screen, Lee explained.
“We are entering into a brave new world, and I think if you haven’t already tried to push the envelope with these models, you should go home and try it,” he said. “You might be very impressed.”
Collapse
Lee AY. How large language models are transforming ophthalmology. Presented at: Retina 2025; Jan. 18-24, 2025; Koloa, Hawaii.
Read more about