Efficiency of different large language models in China in response to consultations about PCa-related perioperative nursing and health education
10.13263/j.cnki.nja.2024.02.010
- VernacularTitle:国内不同大型语言模型对前列腺癌围术期护理与健康教育相关问题的查询响应与效果评价
- Author:
Xiao-Wen TAN
1
;
Wen-Fang CHEN
;
Na-Na WANG
;
Hui-Yu LI
;
Juan LI
;
Yu-Mei CAO
;
Meng-Qi ZHU
;
Kun LI
;
Ting-Ling ZHANG
;
Dian FU
Author Information
1. 东部战区总医院泌尿外科,江苏南京 210002
- Keywords:
prostate cancer;
large language model;
artificial intelligence;
health education
- From:
National Journal of Andrology
2024;30(2):151-156
- CountryChina
- Language:Chinese
-
Abstract:
Objective:To evaluate the efficiency of the four domestic language models,ERNIE Bot,ChatGLM2,Spark Desk and Qwen-14B-Chat,all with a massive user base and significant social attention,in response to consultations about PCa-related perio-perative nursing and health education.Methods:We designed a questionnaire that includes 15 questions commonly concerned by patients undergoing radical prostatectomy and 2 common nursing cases,and inputted the questions into each of the four language models for simulation consultation.Three nursing experts assessed the model responses based on a pre-designed Likert 5-point scale in terms of accuracy,comprehensiveness,understandability,humanistic care,and case analysis.We evaluated and compared the performance of the four models using visualization tools and statistical analyses.Results:All the models generated high-quality texts with no mis-leading information and exhibited satisfactory performance.Qwen-14B-Chat scored the highest in all aspects and showed relatively sta-ble outputs in multiple tests compared with ChatGLM2.Spark Desk performed well in terms of understandability but lacked comprehen-siveness and humanistic care.Both Qwen-14B-Chat and ChatGLM2 demonstrated excellent performance in case analysis.The overall performance of ERNIE Bot was slightly inferior.All things considered,Qwen-14B-Chat was superior to the other three models in con-sultations about PCa-related perioperative nursing and health education.Conclusion:In PCa-related perioperative nursing,large language models represented by Qwen-14B-Chat are expected to become powerful auxiliary tools to provide patients with more medical expertise and information support,so as to improve the patient compliance and the quality of clinical treatment and nursing.
