A chest CT report conclusion generation system based on mT5 large language model for residency training
10.3760/cma.j.cn116021-20240710-02089
- VernacularTitle:基于mT5大语言模型建立用于住院医师培训的胸部CT报告结论生成系统
- Author:
Yanfei HU
1
;
Ai WANG
1
;
Yaping ZHANG
1
;
Keke ZHAO
1
;
Zhijie PAN
1
;
Qingyao LI
1
;
Min XU
1
;
Xifu WANG
1
;
Xueqian XIE
1
Author Information
1. 上海交通大学医学院附属第一人民医院放射科，上海 200080
- Publication Type:Journal Article
- Keywords:
Artificial intelligence;
Large language model;
Text summarization;
mT5;
Medical image analysis;
Residency training
- From:
Chinese Journal of Medical Education Research
2025;24(8):1016-1021
- CountryChina
- Language:Chinese
-
Abstract:
Objective:To fine-tune the mT5 (massively multilingual pre-trained text-to-text transformer) large language model, automatically generate report conclusions for teaching purposes from chest CT image descriptions, and assess the quality of automatically generated conclusions.Methods:The training set included 3 000 high-quality physical examination chest CT reports from one hospital, and the external validation set consisted of 600 physical examination chest CT reports from two other hospitals. Experienced radiology teaching physicians assessed the consistency between the generated conclusions and the original physician-written conclusions in the external validation set using a 5-point Likert scale across five linguistic indicators (correctness of examination information, correctness of lesion detection, standardization of terminology, applicability of the conclusions, and simplicity of conclusions). Using the original report conclusions as the reference, the accuracy of the conclusions generated based on the external validation set in describing four major thoracic conditions (pulmonary nodules, pneumonia, emphysema, pleural effusion) was evaluated. Perform chi square test using SPSS 25.0.Results:In the external validation set, the mean consistency score between the generated conclusions and the original conclusions given by the radiology teaching physicians was >4 points, indicating agreement with the original conclusions. In the generated conclusions, the description of the four major thoracic conditions demonstrated 0.95-1.00 (95% CI=0.91-1.00) accuracy, 0.76-1.00 (95% CI=0.59-1.00) sensitivity, and 0.97-1.00 (95% CI=0.91-1.00) specificity. Conclusions:The chest CT report conclusion generation system based on the mT5 large language model demonstrated high accuracy and is expected to provide immediate and efficient automated guidance for standardized residency training.
