Limitations of Obtaining Medical Information about Age-Related Macular Degeneration from Artificial Intelligence Chatbots
10.21561/jor.2024.9.2.119
- Author:
Dong Gyu NA
1
;
Yi Sang YOON
;
Jae Hui KIM
Author Information
1. Department of Ophthalmology, Kim’s Eye Hospital, Seoul, Korea
- Publication Type:ORIGINAL ARTICLE
- From:
Journal of Retina
2024;9(2):119-126
- CountryRepublic of Korea
- Language:English
-
Abstract:
Purpose:This study aims to evaluate the quality of and trends in artificial intelligence (AI) chatbot responses to questions related to age-related macular degeneration (AMD) and to analyze the frequency of incorrect key information.
Methods:Three chatbots, ChatGPT 3.5, ChatGPT 4.0, and Gemini, were used in this study. Nine questions were formulated covering general information about: 1) AMD, 2) AMD treatment options, 3) effects and side effects of intraocular injections. Each question was queried three times with each of the three chatbots using two different accounts. Responses to items 1–3 were rated as poor/acceptable/good, and the frequency of incorrect key information was noted.
Results:Overall, the majority of the queries received acceptable or good responses. Poor-quality responses were noted in 16.7% of ChatGPT 3.5 responses. Incorrect key information was present in 6.5% of all responses.
Conclusions:While AI chatbots generally provided acceptable responses to questions regarding AMD, some responses contained incorrect key information, suggesting the need for caution when accessing medical information through AI chatbots.
