Efficiency of large language models in answering questions about childhood asthma
- VernacularTitle:应用大语言模型解答儿童哮喘问题的效果研究
- Author:
Weipeng HAN
1
;
Xiaomei YIN
;
Jian WANG
;
Xuejun LI
;
Jijiang YANG
Author Information
- Keywords: large language models; childhood asthma; medical artificial intelligence; intelli-gent assistant; Wenxin Yiyan; Zhipu Qingyan
- From: Journal of Clinical Medicine in Practice 2024;28(11):6-11,17
- CountryChina
- Language:Chinese
- Abstract: Objective To evaluate the performance of large language models in answering ques-tions about childhood asthma,comprehensively understand the quality of their provision of information on children's health,and identify their limitations to facilitate model improvement.Methods Sixty common questions related to childhood asthma were formulated and put to two large language models known as Wenxin Yiyan and Zhipu Qingyan,which were publicly available in China.Three pediatric asthma specialists assessed the quality of the large language models'responses by using a blind meth-od.Results Wenxin Yiyan scored higher in terms of accuracy,understanding,reliability,and logi-cality;Zhipu Qingyan scored higher in term of safety.Comparing the scores of the five different di-mensions,it was found that large language models scored higher in terms of understanding,reliability and logicality,but relatively insufficient in terms of accuracy and safety.Conclusion Application of large language models in the education of children with asthma can provide useful references for asth-ma children and their parents.However,the current large language model technology still has certain limitations in terms of accuracy and safety,which requires further improvement and optimization.
