Application of a multimodal model based on radiomics and 3D deep learning in predicting severe acute pancreatitis
- VernacularTitle:基于影像组学与3D深度学习的多模态模型在重症急性胰腺炎预测中的应用
- Author:
Xianglin DING
1
;
Xin CHEN
1
;
Meiyu CHEN
2
;
Yiping SHEN
2
;
Yu WANG
2
;
Minyue YIN
3
;
Kai ZHAO
4
;
Jinzhou ZHU
2
Author Information
- Publication Type:Journal Article
- Keywords: Pancreatitis， Acute Necrotizing; XGBoost Algorithm; Imaging Genomics; Multimodal Imaging; Deep Learning
- From: Journal of Clinical Hepatology 2025;41(10):2110-2117
- CountryChina
- Language:Chinese
- Abstract: ObjectiveTo investigate the application value of a multimodal model integrating radiomics features， deep learning features， and clinical structured data in predicting severe acute pancreatitis （SAP）， and to provide more accurate tools for the early identification of SAP in clinical practice. MethodsThe patients with acute pancreatitis （AP） who attended The First Affiliated Hospital of Soochow University， Jintan Hospital Affiliated to Jiangsu University， and Suzhou Yongding Hospital from January 1， 2017 to December 31， 2023 were included. Related data were collected， including demographic information， previous medical history， etiology， laboratory test data， and systemic inflammatory response syndrome （SIRS） within 24 hours after admission， as well as imaging data within 72 hours after admission， while related scores were calculated， including Ranson score， modified CT severity index （MCTSI）， bedside index for severity in acute pancreatitis （BISAP）， and systemic inflammatory response syndrome， albumin， blood urea nitrogen and pleural effusion （SABP） score. The model was constructed in the following process： （1） three-dimensional CT images were used to extract and identify radiomics features， and a radiomics classification model was established based on the extreme gradient Boost （XGBoost） algorithm； （2） U-Net is used to perform semantic segmentation of three-dimensional CT images， and then the results of segmentation were imported into 3D ResNet50 to construct a deep learning classification model； （3） the predicted values of the above two models were integrated with clinical structured data to establish a multimodal model based on the XGBoost algorithm. The variable importance plot and local interpretability plot were used to perform visual interpretation of the model. The independent samples t-test was used for comparison of normally distributed continuous data between groups， and the Mann-Whitney U test was used for comparison of non-normally distributed continuous data between groups； the chi-square test or Fisher’s exact test was used for comparison of categorical data between groups. The receiver operating characteristic （ROC） curve was plotted for each model and existing scoring systems， and the area under the ROC curve （AUC） was calculated to assess their performance； the Delong test was used for comparison of AUC. ResultsA total of 609 patients who met the criteria were included， among whom 114 （18.7%） developed SAP. In this study， the data of 426 patients from The First Affiliated Hospital of Soochow University was used as the training set， and the data of 183 patients from Jintan Hospital Affiliated to Jiangsu University and Suzhou Yongding Hospital were used as the independent test set. The multimodal model had an AUC of 0.914 in the test set， which was significantly higher than the AUC of traditional scoring systems such as MCTSI （AUC=0.827）， Ranson score （AUC=0.675）， BISAP （AUC=0.791）， and SABP score （AUC=0.648）； in addition， the multimodal model showed a significant improvement in performance compared with the radiomics classification model （AUC=0.739） and the deep learning classification model （AUC=0.685） （the Delong test： Z=-3.23， -4.83， -3.48， -4.92， -4.31， and -4.59， all P <0.01）. The top 10 variables in terms of importance in the multimodal model were pleural effusion， predicted value of the deep learning model， predicted value of the radiomics model， triglycerides， calcium ions， SIRS， white blood cell count， age， platelets， and C-reactive protein， suggesting that the above variables had significant contributions to the performance of the model in predicting SAP. ConclusionBased on structured data， radiomic features， and deep learning features， this study constructs a multicenter prediction model for SAP based on the XGBoost algorithm， which has a better predictive performance than existing traditional scoring systems and unimodal models.
