ChatGPT accurately, satisfactorily answers most glaucoma-related questions
Click Here to Manage Email Alerts
Key takeaways:
- ChatGPT answered 88.7% of 53 common glaucoma-related questions correctly.
- ChatGPT did not answer any questions completely incorrectly, although researchers deemed some responses insufficient or misleading.
ChatGPT correctly and satisfactorily answered nine out of 10 frequently asked questions about glaucoma concerning general information, diagnosis, treatment and prevention, according to an analysis published in Journal of Glaucoma.
Social media applications, including X, Instagram, YouTube and ChatGPT play an important role in medicine, according to the researchers.
“Widespread and accurate use of artificial intelligence can increase awareness about diseases, enable early diagnosis of diseases and increase patient compliance with treatment,” Suleyman G. Kerci, MD, and Berna Sahan, MD, with the department of ophthalmology at Medicana International Izmir Hospital in Turkey, wrote in the study.
Previous studies have investigated ChatGPT’s role in the field of medicine as an AI application that uses OpenAI to process natural language. The researchers’ goal with this study was to evaluate ChatGPT’s knowledge of glaucoma via its ability to effectively and adequately answer glaucoma-related questions.
To do so, researchers came up with a list of glaucoma questions based on those frequently asked on hospital and ophthalmology association websites and social media apps, including YouTube and Instagram. The researchers did not assess questions that had subjective answers, were repetitive or grammatically inadequate.
Two experienced ophthalmologists formulated the questions, and two different specialist ophthalmologists independently evaluated ChatGPT’s answers, scoring them based on their adherence to the 2021 “Terminology and Guidelines for Glaucoma” created by the European Glaucoma Society. They scored them from one to four, where one indicated the answer was completely correct, two correct but insufficient, three correct but misleading and four completely incorrect.
Notably, each question was asked twice on different computers to determine the reproducibility of the answers.
Ultimately, the study included 53 questions, 14 of which related to general information on glaucoma, 12 to diagnosis, 15 to treatment and 12 to prevention.
ChatGPT answered 88.7% of the questions completely correctly, and no question was answered completely incorrectly. The researchers found that 7.5% of ChatGPT’s answers were correct but insufficient, and 3.8% contained correct but misleading information.
Further, ChatGPT performed better in some question categories than others. It answered 91.7% of the diagnosis and prevention questions completely correctly, followed by 86.8% of the treatment questions and 85.8% of the general knowledge questions.
The researchers also formulated 16 questions based on the European Glaucoma Society guidelines. ChatGPT answered 75% of these questions completely correctly. However, it gave three (18.8%) correct but insufficient answers and one response (6.3%) that contained both false and correct information.
There were some limitations to this study, the researchers wrote, including that the information sources used in the study are constantly updated, and the questions were posed in English. They also noted that the understanding of ChatGPT based on different education levels was not assessed in this study.
Based on their findings, Kerci and Sahan described ChatGPT as a source of “accurate information” about glaucoma.
“We believe that the use of ChatGPT in ophthalmology practice under the supervision of health care professionals will enable patients and their relatives to better understand the diagnosis and treatment of glaucoma,” they wrote.
Reference:
- Bagnasco L, et al. Br J Ophthalmol. 2021;doi:10.1136/bjophthalmol-2021-egsguidelines.
Perspective
Back to Top
AI is one of the most interesting developments in technology, and it is no surprise that it would be employed in health care. That can seem scary to doctors due to a natural fear of being replaced.
There is also hesitation about the accuracy of AI and whether it should be trusted with health care decisions. We have all been frustrated by EHRs that permit mistakes that are so absurd they could never occur on paper. It is hard to believe “smart” technology is here when many health-related programs and their interfaces look like something from the 1990s.
However, Chat GPT is a next-level application of AI. It is impressive and does appear to possess extraordinary research and writing ability.
Whether or not we like it, patients will go to the internet and social media to get answers to health-related questions. If we can employ modern technology to make accurate content, it will benefit both doctors and patients. We can use this study’s promising results from ChatGPT to assist us while still carefully vetting what information is received by our patients.
Collapse
Read more about
Click Here to Manage Email Alerts