1.Intelligent head and neck CT angiography report quality detection using large language models
Liping TIAN ; Xiaolu FEI ; Dan SONG ; Yao LU ; Jie LU
Chinese Journal of Radiology 2025;59(10):1118-1125
Objective:To assess common errors in head and neck CT angiography (CTA) image reports using four types of large language models (LLM), namely GPT-4, DeepSeek, ERNIE Bot and SparkDesk, and to assess the feasibility of using existing LLMs to support quality control of radiology reports in Chinese.Methods:The study was a cross-sectional study. Totally 1 000 head and neck CTA image reports were randomly selected using the simple random sampling method from Xuanwu Hospital, Capital Medical University in 2023, including 500 primary reports and 500 finalized reports. Two radiologists collaboratively identified six types of errors in the reports: description errors, writing errors, left-right confusion errors, diagnostic omissions, logical sequence errors, and other errors. The overall quality of the reports was assessed using a 5-point Likert scale. Subsequently, GPT-4, DeepSeek, ERNIE Bot and SparkDesk models were employed to detect the same six types of errors in the imaging reports and to provide overall scoring. The results from manual review were considered the gold standard for calculating F1 score to evaluate model performance. Intra-class correlation coefficients ( ICC) were used to assess the consistency between manual scores and the overall scores from the four LLMs. Results:In the primary imaging reports, the proportions of manually detected errors were as follows: descriptive errors 2.6% (13/500), writing errors 0.6% (3/500), left-right confusion errors 0, diagnostic omissions 6.4% (32/500), logical sequence errors 5.2% (26/500), and other errors 0. In the finalized imaging reports, the proportions of errors across the six categories were 0.2% (1/500), 0, 0, 0, 0, and 0.2% (1/500), respectively. For error detection in the primary imaging reports, the F1 scores of GPT-4 for the six error types were 0.992, 0.997, 0.997, 0.967, 0.980, and 0.992, respectively. DeepSeek achieved F1 scores of 0.980, 0.955, 0.981, 0.920, 0.995, and 0.960; ERNIE Bot scored 0.982, 0.990, 1.000, 0.956, 0.976, and 0.999; and SparkDesk achieved 0.985, 0.995, 1.000, 0.961, 0.982, and 1.000. In the detection of errors in finalized imaging reports, GPT-4′s F1 scores were 0.994, 0.995, 0.998, 0.973, 0.989, and 0.993; DeepSeek scored 0.968, 0.965, 0.985, 0.971, 0.991, and 0.983; ERNIE Bot achieved 0.996, 0.992, 1.000, 0.983, 0.999, and 0.997; and SparkDesk achieved 0.999, 0.999, 1.000, 1.000, 1.000, and 0.999. The consistency between GPT-4, DeepSeek, and SparkDesk models and human ratings was moderate, with ICC values of 0.514, 0.560, and 0.515 respectively (all P0.001); in contrast, the overall score of ERNIE Bot showed poor consistency with human ratings, with an ICC of 0.221 ( P0.001). Conclusion:LLMs demonstrate high accuracy in detecting errors in head and neck CTA imaging reports. The overall scoring of report quality shows moderate consistency with manual assessments, indicating a certain feasibility for automated quality control in reporting.
2.Hypoperfusion intensity ratio of CT perfusion for predicting infarct core progression and prognosis of acute ischemic stroke
Yao LU ; Wenbo CAO ; Jingkai LI ; Miao ZHANG ; Xiaolu FEI ; Jie LU
Chinese Journal of Medical Imaging Technology 2025;41(5):718-722
Objective To observe the value of hypoperfusion intensity ratio(HIR)of CT perfusion(CTP)for predicting infarct core progression and prognosis of acute ischemic stroke(AIS).Methods Totally 271 AIS patients were retrospectively enrolled and divided into rapid progression group(group A,n=92)and slow progression group(group B,n=179)according to infarction growth rate(IGR).Clinical data,CTP parameters,treatment strategies and patients' outcome were compared between groups.Receiver operating characteristic curve was drawn,the area under the curve(AUC)was calculated to evaluate the efficacy of HIR for predicting rapid progression in infarct core of AIS.The mediating relationships among HIR,IGR and modified Rankin scale(mRS)90 days after treatment were analyzed.Results Significant differences of National Institute of Health stroke scale(NIHSS)score,Alberta stroke program early CT score(ASPECTS),also of interval time between onset and CTP,infarct core volume,hypoperfusion volume,HIR,whether intravenous thrombolysis and mRS score 90 days after treatments were found between groups(all P<0.05).The AUC of HIR for predicting infarct core progression of AIS was 0.856,with sensitivity and specificity was 73.91%and 81.56%,respectively,when the optimal cutoff value was 0.42.IGR was a complete mediating variable between HIR and mRS score 90 days after treatment.Conclusion HIR of CTP could be used to effectively predict infarct core progression of AIS,which completely affected prognosis through mediating variable IGR.
3.Intelligent head and neck CT angiography report quality detection using large language models
Liping TIAN ; Xiaolu FEI ; Dan SONG ; Yao LU ; Jie LU
Chinese Journal of Radiology 2025;59(10):1118-1125
Objective:To assess common errors in head and neck CT angiography (CTA) image reports using four types of large language models (LLM), namely GPT-4, DeepSeek, ERNIE Bot and SparkDesk, and to assess the feasibility of using existing LLMs to support quality control of radiology reports in Chinese.Methods:The study was a cross-sectional study. Totally 1 000 head and neck CTA image reports were randomly selected using the simple random sampling method from Xuanwu Hospital, Capital Medical University in 2023, including 500 primary reports and 500 finalized reports. Two radiologists collaboratively identified six types of errors in the reports: description errors, writing errors, left-right confusion errors, diagnostic omissions, logical sequence errors, and other errors. The overall quality of the reports was assessed using a 5-point Likert scale. Subsequently, GPT-4, DeepSeek, ERNIE Bot and SparkDesk models were employed to detect the same six types of errors in the imaging reports and to provide overall scoring. The results from manual review were considered the gold standard for calculating F1 score to evaluate model performance. Intra-class correlation coefficients ( ICC) were used to assess the consistency between manual scores and the overall scores from the four LLMs. Results:In the primary imaging reports, the proportions of manually detected errors were as follows: descriptive errors 2.6% (13/500), writing errors 0.6% (3/500), left-right confusion errors 0, diagnostic omissions 6.4% (32/500), logical sequence errors 5.2% (26/500), and other errors 0. In the finalized imaging reports, the proportions of errors across the six categories were 0.2% (1/500), 0, 0, 0, 0, and 0.2% (1/500), respectively. For error detection in the primary imaging reports, the F1 scores of GPT-4 for the six error types were 0.992, 0.997, 0.997, 0.967, 0.980, and 0.992, respectively. DeepSeek achieved F1 scores of 0.980, 0.955, 0.981, 0.920, 0.995, and 0.960; ERNIE Bot scored 0.982, 0.990, 1.000, 0.956, 0.976, and 0.999; and SparkDesk achieved 0.985, 0.995, 1.000, 0.961, 0.982, and 1.000. In the detection of errors in finalized imaging reports, GPT-4′s F1 scores were 0.994, 0.995, 0.998, 0.973, 0.989, and 0.993; DeepSeek scored 0.968, 0.965, 0.985, 0.971, 0.991, and 0.983; ERNIE Bot achieved 0.996, 0.992, 1.000, 0.983, 0.999, and 0.997; and SparkDesk achieved 0.999, 0.999, 1.000, 1.000, 1.000, and 0.999. The consistency between GPT-4, DeepSeek, and SparkDesk models and human ratings was moderate, with ICC values of 0.514, 0.560, and 0.515 respectively (all P0.001); in contrast, the overall score of ERNIE Bot showed poor consistency with human ratings, with an ICC of 0.221 ( P0.001). Conclusion:LLMs demonstrate high accuracy in detecting errors in head and neck CTA imaging reports. The overall scoring of report quality shows moderate consistency with manual assessments, indicating a certain feasibility for automated quality control in reporting.
4.Application of health big data in hospital-based cancer screening study
Chenran WANG ; Zeming GUO ; Xiaoyue SHI ; Yadi ZHENG ; Zilin LUO ; Jiaxin XIE ; Xiaolu CHEN ; Jibin LI ; Yongjie XU ; Wei CAO ; Fei WANG ; Xuesi DONG ; Ni LI ; Jie HE
Chinese Journal of Epidemiology 2025;46(7):1297-1303
This paper focuses on the application of health big data in cancer screening. Firstly, the sources and characteristics of health big data are introduced, then the commonly used epidemiological designs and analytical techniques in hospital-based cancer screening studies are summarized and the application scenarios of such studies are described. Finally, the challenges and future development in the application of health big data are analyzed to provide reference for the future studies.
5.USP20 as a super-enhancer-regulated gene drives T-ALL progression via HIF1A deubiquitination.
Ling XU ; Zimu ZHANG ; Juanjuan YU ; Tongting JI ; Jia CHENG ; Xiaodong FEI ; Xinran CHU ; Yanfang TAO ; Yan XU ; Pengju YANG ; Wenyuan LIU ; Gen LI ; Yongping ZHANG ; Yan LI ; Fenli ZHANG ; Ying YANG ; Bi ZHOU ; Yumeng WU ; Zhongling WEI ; Yanling CHEN ; Jianwei WANG ; Di WU ; Xiaolu LI ; Yang YANG ; Guanghui QIAN ; Hongli YIN ; Shuiyan WU ; Shuqi ZHANG ; Dan LIU ; Jun-Jie FAN ; Lei SHI ; Xiaodong WANG ; Shaoyan HU ; Jun LU ; Jian PAN
Acta Pharmaceutica Sinica B 2025;15(9):4751-4771
T-cell acute lymphoblastic leukemia (T-ALL) is a highly aggressive hematologic malignancy with a poor prognosis, despite advancements in treatment. Many patients struggle with relapse or refractory disease. Investigating the role of the super-enhancer (SE) regulated gene ubiquitin-specific protease 20 (USP20) in T-ALL could enhance targeted therapies and improve clinical outcomes. Analysis of histone H3 lysine 27 acetylation (H3K27ac) chromatin immunoprecipitation sequencing (ChIP-seq) data from six T-ALL cell lines and seven pediatric samples identified USP20 as an SE-regulated driver gene. Utilizing the Cancer Cell Line Encyclopedia (CCLE) and BloodSpot databases, it was found that USP20 is specifically highly expressed in T-ALL. Knocking down USP20 with short hairpin RNA (shRNA) increased apoptosis and inhibited proliferation in T-ALL cells. In vivo studies showed that USP20 knockdown reduced tumor growth and improved survival. The USP20 inhibitor GSK2643943A demonstrated similar anti-tumor effects. Mass spectrometry, RNA-Seq, and immunoprecipitation revealed that USP20 interacted with hypoxia-inducible factor 1 subunit alpha (HIF1A) and stabilized it by deubiquitination. Cleavage under targets and tagmentation (CUT&Tag) results indicated that USP20 co-localized with HIF1A, jointly modulating target genes in T-ALL. This study identifies USP20 as a therapeutic target in T-ALL and suggests GSK2643943A as a potential treatment strategy.
6.Application of health big data in hospital-based cancer screening study
Chenran WANG ; Zeming GUO ; Xiaoyue SHI ; Yadi ZHENG ; Zilin LUO ; Jiaxin XIE ; Xiaolu CHEN ; Jibin LI ; Yongjie XU ; Wei CAO ; Fei WANG ; Xuesi DONG ; Ni LI ; Jie HE
Chinese Journal of Epidemiology 2025;46(7):1297-1303
This paper focuses on the application of health big data in cancer screening. Firstly, the sources and characteristics of health big data are introduced, then the commonly used epidemiological designs and analytical techniques in hospital-based cancer screening studies are summarized and the application scenarios of such studies are described. Finally, the challenges and future development in the application of health big data are analyzed to provide reference for the future studies.
7.Hypoperfusion intensity ratio of CT perfusion for predicting infarct core progression and prognosis of acute ischemic stroke
Yao LU ; Wenbo CAO ; Jingkai LI ; Miao ZHANG ; Xiaolu FEI ; Jie LU
Chinese Journal of Medical Imaging Technology 2025;41(5):718-722
Objective To observe the value of hypoperfusion intensity ratio(HIR)of CT perfusion(CTP)for predicting infarct core progression and prognosis of acute ischemic stroke(AIS).Methods Totally 271 AIS patients were retrospectively enrolled and divided into rapid progression group(group A,n=92)and slow progression group(group B,n=179)according to infarction growth rate(IGR).Clinical data,CTP parameters,treatment strategies and patients' outcome were compared between groups.Receiver operating characteristic curve was drawn,the area under the curve(AUC)was calculated to evaluate the efficacy of HIR for predicting rapid progression in infarct core of AIS.The mediating relationships among HIR,IGR and modified Rankin scale(mRS)90 days after treatment were analyzed.Results Significant differences of National Institute of Health stroke scale(NIHSS)score,Alberta stroke program early CT score(ASPECTS),also of interval time between onset and CTP,infarct core volume,hypoperfusion volume,HIR,whether intravenous thrombolysis and mRS score 90 days after treatments were found between groups(all P<0.05).The AUC of HIR for predicting infarct core progression of AIS was 0.856,with sensitivity and specificity was 73.91%and 81.56%,respectively,when the optimal cutoff value was 0.42.IGR was a complete mediating variable between HIR and mRS score 90 days after treatment.Conclusion HIR of CTP could be used to effectively predict infarct core progression of AIS,which completely affected prognosis through mediating variable IGR.
8.Repeated transcranial magnetic stimulation for post-stroke depression
Xiangzhu FAN ; Chenchen LI ; Ziwei CAO ; Xiaolu HE ; Fei LI ; Zhi ZHANG
International Journal of Cerebrovascular Diseases 2024;32(5):374-379
Post-stroke depression (PSD) is an important mental complication of stroke, affecting nearly 1/3 of stroke patients, seriously affecting patients' functional recovery and quality of life, and is associated with increased mortality of stroke patients. Traditional antidepressant treatments include medication and psychotherapy, but there may be problems with adverse reactions, tolerance, or limited effectiveness. Repetitive transcranial magnetic stimulation (rTMS), as a non-invasive neuroregulatory technique, offers a new treatment option for patients with PSD. This article reviews the application of rTMS in the treatment of PSD and its possible mechanism.
9.Machine learning predicts poor outcome in patients with acute minor ischemic stroke
Fei XIE ; Qiuwan LIU ; Xiaolu HE ; Zhuqing WU ; Juncang WU
International Journal of Cerebrovascular Diseases 2024;32(6):421-427
Objectives:To develop a machine learning prediction model for poor outcome of acute minor ischemic stroke (AMIS) at 90 days after onset and to explain the importance of various risk factors.Methods:Patients with AMIS admitted to the Second People's Hospital of Hefei from June 2022 to December 2023 were included retrospectively. AMIS was defined as the National Institutes of Health Stroke Scale (NIHSS) score ≤5 at admission. According to the modified Rankin Scale score at 90 days after onset, the patients were divided into a good outcome group (<2) and a poor outcome group (≥2). Recursive feature elimination (RFE) method was used to screen characteristic variables of poor outcome. Based on logistic regression (LR), supported vector machine (SVM), and extreme Gradient Boosting (XGBoost) machine learning algorithms, prediction models for poor outcome of AMIS were developed, and the predictive performance of the models was compared by the area under the curve (AUC) of receiver operating characteristic (ROC) curve and the calibration curve. Shapley Additive exPlanations (SHAP) algorithm was used to explain the role of characteristic variables in the optimal prediction model. Results:A total of 225 patients with AMIS were included, of which 152 (67.56%) had good outcome and 73 (32.44%) had poor outcome. Multivariate analysis showed that baseline NIHSS score, baseline systolic blood pressure, hypertension, diabetes, low-density lipoprotein cholesterol, homocysteine, body mass index, D-dimer, and age were the characteristic variables associated with poor outcome in patients with AMIS. The ROC curve analysis shows that the LR model had the best predictive performance (AUC=0.888, 95% confidence interval [ CI] 0.807-0.970), the next was the XGBoost model (AUC=0.888, 95% CI 0.796-0.980), while the SVM model had the lowest performance (AUC=0.849, 95% CI 0.754-0.944). The calibration curve showed that the LR model performed the best in terms of calibration accuracy. SHAP showed that baseline systolic blood pressure, baseline NIHSS score, diabetes, hypertension and body mass index were the top five risk factors for poor outcome of patients with AMIS. Conclusions:The LR algorithm has stable and superior performance in predicting poor outcome of patients with AMIS. Baseline systolic blood pressure, baseline NIHSS score, diabetes, hypertension and body mass index are the important risk factors for poor outcome of patients with AMIS.
10.Semantic information retrieval based on the case report dataset of Adverse Drug Reactions Journal
Yayi XIAO ; Yi LEI ; Xin WANG ; Xiangrong BAI ; Qingxia ZHANG ; Xiaolu FEI
Adverse Drug Reactions Journal 2024;26(3):170-177
Objective:To explore the application value of semantic information retrieval (semantic retrieval) based on case reports dataset of Adverse Drug Reactions Journal. Methods:The dataset used in this study consists of 2 597 PDF files of case reports published on Adverse Drug Reactions Journal from 1999 to 2022. The semantic retrieval system is built by Baidu PaddlePaddle′s deep learning framework, the code was written in Python, and the text encoding model was Baidu RocketQA model. The precision at position k (P@k), recall at position k (R@k), mean reciprocal rank (MRR), mean average precision (MAP) and precision-recall (P-R) curve were used to evaluate the performance of semantic retrieval. The performance of semantic retrieval and keyword matching retrieval were compared by calculating the recall. Results:The set of preprocessed theme fields as items to be retrieved contained 2 597 documents, the set of search terms (queries) after removing deplicates and reorganizing included 1 388 drug name queries and 1 118 adverse reactions/events queries. The precision of drug name queries and adverse reactions/events queries by semantic retrieval were 0.667-1 and 0.566-1, and their recall were 0.667-0.871 and 0.566-0.863, respectively. The P-R curves of the top 1, 3, 5 and 10 documents in the semantic retrieval results using drug names queries and adverse reactions/events search terms showed that the precision decreased slowly in top 1 and 3 documents but significantly in top 5 and 10 documents with the increase of recall. The MRR of the 2 types of search terms were 0.854 and 0.871, and the MAP were 0.778 and 0.773, respectively. Using adverse reactions/events as search terms, semantic retrieval has a higher recall rate than keyword matching retrieval; using drug names as search terms, the recall rate of keyword matching retrieval is generally higher than that of semantic retrieval.Conclusions:The semantic retrieval system based on Baidu PaddlePaddle deep learning framework has good retrieval performance on the case reports dataset of Adverse Drug Reactions Journal. The semantic retrieval performs better with adverse reactions/events queries, while the keyword matching retrieval performs better with drug name queries.

Result Analysis
Print
Save
E-mail