Investigating the utility of large language models for image-based rare disease phenotyping
- Author:
Jihoon G. YOON
1
Author Information
- Publication Type:Original Article
- From:Journal of Genetic Medicine 2025;22(1):7-15
- CountryRepublic of Korea
- Language:English
-
Abstract:
Purpose:Artificial intelligence has been applied across various fields of medicine, with large language models (LLMs) demonstrating potential to assist in clinical decision-making for rare diseases. This study assessed the performance of LLMs in aiding the phenotyping process and guiding the identification of correct genetic conditions.
Materials and Methods:Clinical images from 10 Korean individuals with genetically confirmed rare diseases were collected through a literature review. Using identical prompts, the top 10 Human Phenotype Ontology (HPO) terms and suspected genetic conditions were queried across three LLMs: generative pre-trained transformers (GPTs) models GPT-4o and GPT o1, and Claude 3.5 Sonnet. Concordance between 5 manually curated key HPO terms and the top 10 predicted terms were assessed, and the accuracy of genetic diagnoses among the LLMs was analyzed.
Results:Clinical images, ranging from 3 to 9 per case, were used as input for 10 Korean rare disease cases. The average number of key HPO terms correctly matched among the top 10 predictions was 3.0 (1.99-4.01) for GPT-4o, 2.8 (2.06-3.54) for GPT o1, and 1.7 (0.80-2.60) for Claude 3.5 (mean, [95% confidence interval]). GPT models provided more specific HPO terms than Claude in these cases. The accuracy of genetic diagnosis within the top 10 predictions was 2/10 for GPT-4o, 3/10 for GPT o1, and 0/10 for Claude 3.5, with frequent hallucination events observed.
Conclusion:LLMs demonstrate potential as a supportive tool for image-based rare disease phenotyping, while the frequent hallucinations highlight the need for further investigation and caution in clinical application.
