1.Ruibin Agent versus mainstream large language models: A comparative study on medical literature comprehension with esophageal cancer as a case study
Pinghua WEN ; Zhijie JIANG ; Huan JIANG ; Xianglei YUAN ; Yu ZHOU ; Hu MA ; Chao LU ; Bing HU
Chinese Journal of Clinical Thoracic and Cardiovascular Surgery 2025;32(10):1404-1410
Objective To explore the application value of artificial intelligence in medical research assistance, and analyze the key paths to achieve precise execution of model instructions, improvement of model interpretation completeness, and control of hallucinations. Methods Taking esophageal cancer research as the scenario, five types of literature including research articles, case reports, reviews, editorials, and guidelines were selected for model interpretation tests. The model performance was systematically evaluated from five dimensions: recognition accuracy, format accuracy, instruction execution accuracy, content reliability rate, and content completeness index. The performance differences of Ruibin Agent, GPT-4o, Claude 3.7 Sonnet, DeepSeek V3, and DouBao-pro models in medical literature interpretation tasks were compared. Results A total of 15 studies were included, with 3 studies of each type. The five models collectively conducted 1 875 tests. Due to the poor recognition accuracy of the editorial type, the overall recognition accuracy of Ruibin Agent was significantly lower than other models (92.0% vs. 100.0%, P<0.001). In terms of format accuracy, Ruibin Agent was significantly better than Claude 3.7 Sonnet (98.7% vs. 92.0%, P=0.002) and GPT-4o (98.7% vs. 78.9%, P<0.001). In terms of instruction execution accuracy, Ruibin Agent was better than GPT-4o (97.3% vs. 80.0%, P<0.001). In terms of content reliability rate, Ruibin Agent was significantly lower than Claude 3.7 Sonnet (84.0% vs. 92.0%, P=0.010) and DeepSeek V3 (84.0% vs. 94.7%, P<0.001). In terms of content completeness index, the median scores of Ruibin Agent, GPT-4o, Claude 3.7 Sonnet, DeepSeek V3, and DouBao-pro were 0.71, 0.60, 0.85, 0.74, and 0.77, respectively. Conclusion Ruibin Agent has significant advantages in terms of formatted interpretation of medical literature and instruction execution accuracy. In the future, it is necessary to focus on optimizing the recognition ability of editorial types, strengthening the coverage ability of core elements of various types of literature to improve interpretation completeness, and improving content reliability through optimizing the confidence mechanism to ensure the rigor of medical literature interpretation.
2.Analysis on current situation of ordinary medical college undergraduates' contact with scientific research at early stage
Huihao MA ; Xuanwen LU ; Jiaojiao YU ; Juju LIU ; Yakun LI ; Lei WANG ; Chao ZHAO
Chinese Journal of Medical Education Research 2012;11(10):1075-1078
Objective To analyze the current situation and influence factors of ordinary medical college undergraduates' contact with scientific research at early stage in order to provide references for scientific research.Methods Totally 1940 students majoring in clinical medicine,imaging,traditional Chinese medicine and nursing (2008 -2010 grade) in China Three Gorges University were enrolled to do questionnaine and SPSS 17.0 was used to do statistical analysis.Results Totally 1653copies of questionnaires were collected from 1940 students,the recovery rate was 85.21%.Two hundred and nineteen students ( 13.25% ) participated in scientific research,65.28% students thought college propaganda to be ordinary,95.43% students got benefits from scientific research.The main influence factors of scientific research were lack of time (23.73%),insufficient knowledge reserves (22.03%) and researchers' own problems (39.73%).Conclusions Medical school should expand the range of scientific research and strengthen propaganda.Medical students should arrange research time and constantly improve their comprehensive ability so as to achieve good results.

Result Analysis
Print
Save
E-mail