1.A Comparative Study of Artificial Intelligence-based Classification Versus Manual Classification of Medical Adverse Events: Taking the DeepSeek Large Language Model As an Example
Rui WANG ; Xutong TAN ; Congpu ZHAO ; Shuchang WANG ; Zheng CHEN ; Xiaojun MA ; Zhiling CAI
Medical Journal of Peking Union Medical College Hospital 2026;17(3):828-833
To analyze the application value of artificial intelligence (AI)-based classification in the categorization of medical adverse events. Medical adverse events reported to the Adverse Event Reporting System of Peking Union Medical College Hospital from September 1, 2023, to August 31, 2024, were retrospectively collected as the study subjects. After de-identification of adverse events meeting the inclusion criteria, conventional manual classification and AI-based classification using a large language model (DeepSeek-R1 Full-Context Internet Edition) were performed. The time required for classification using both methods was recorded, and the consistency and discrepancies between the two methods were compared. Using manual classification as the gold standard, the accuracy of AI-based classification was comprehensively evaluated. A total of 273 medical adverse events were analyzed. Manual classification took 38 838 seconds in total, with an average of 14.22 seconds per event. AI-based classification took 600 seconds in total, with an average of 2.19 seconds per event. The two methods showed consistent classification in 202 events and inconsistent classification in 71 events, yielding an overall agreement rate of 73.99% and a Kappa coefficient of 0.646 (95% CI: 0.575-0.717), with a standard error of 0.0362. Using manual classification as the gold standard, AI-based classification achieved accuracy ranging from 80% to 100%, precision from 30% to 100%, recall from 40% to 100%, F1 scores from 0.46 to 0.79, and specificity from 46% to 98%. Notably, AI-based classification demonstrated balanced and overall excellent performance in the categorization of device-related and drug-related adverse events. The DeepSeek large language model can assist in improving the efficiency of medical adverse event classification, showing promising application potential, particularly in the categorization of device-related and drug-related adverse events.
2.Study and Practice on Intelligent Classification of Medical Safety Incidents Based on BERT Model
Congpu ZHAO ; Da YUAN ; Pujue ZHU ; Jiong ZHOU ; Zheng CHEN ; Hua PENG
Journal of Medical Informatics 2024;45(1):27-32,38
Purpose/Significance To improve the classification and evaluation mode of medical safety incidents,and to improve work efficiency and timeliness.Method/Process The data of previous medical safety incidents are pre-processed,BERT model is used for training,testing and iterative optimization,and an intelligent classification and prediction model for medical safety incidents is built.Re-sult/Conclusion The model is used to classify 466 medical safety incidents reported by clinical departments from January to November 2022,and F1 value reaches 0.66.The application of BERT model in the classification and evaluation of medical safety incidents can im-prove work efficiency and timeliness,and help timely intervene in medical safety risks.
3.The Construction Status and Development Trend of Smart Hospital in China
Da YUAN ; Congpu ZHAO ; Pujue ZHU ; Jieshi ZHANG ; Zheng CHEN ; Jiong ZHOU ; Xiaojun MA ; Hua PENG
Journal of Medical Informatics 2024;45(7):33-36
Purpose/Significance To expound the development status,difficulties and challenges of smart hospital in China,so as to pro-vide references for the subsequent related research.Method/Process By using the methods of bibliometrics and literature review,the definition of smart hospital is summarized and feasible suggestions on the construction of smart hospital are put forward.Result/Conclusion Smart hospital in China has initially established a"trinity"structural framework of smart healthcare,smart service and smart management,playing a positive role in improving patient satisfaction and promoting high-quality development of hospitals.It is necessary for the government,hospitals,social capital and other multi-party cooperation to jointly promote the construction of smart hospital in China and better protect people's health.
Result Analysis
Print
Save
E-mail