ResNet-Vision Transformer based MRI-endoscopy fusion model for predicting treatment response to neoadjuvant chemoradiotherapy in locally advanced rectal cancer: A multicenter study.
10.1097/CM9.0000000000003391
- Author:
Junhao ZHANG
1
;
Ruiqing LIU
1
;
Di HAO
2
;
Guangye TIAN
2
;
Shiwei ZHANG
3
;
Sen ZHANG
4
;
Yitong ZANG
5
;
Kai PANG
6
;
Xuhua HU
7
;
Keyu REN
8
;
Mingjuan CUI
8
;
Shuhao LIU
9
;
Jinhui WU
10
;
Quan WANG
3
;
Bo FENG
4
;
Weidong TONG
5
;
Yingchi YANG
6
;
Guiying WANG
7
;
Yun LU
1
Author Information
1. Department of Gastrointestinal Surgery, The Affiliated Hospital of Qingdao University, Qingdao, Shandong 266000, China.
2. School of Control Science and Engineering, Shandong University, Jinan, Shandong 250061, China.
3. Department of Gastric and Colorectal Surgery, General Surgery Center, The First Hospital of Jilin University, Changchun, Jilin 130021, China.
4. Department of General Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 20025, China.
5. Department of General Surgery, Colorectal Division, Army Medical Center, Army Medical University, Chongqing 400038, China.
6. Department of General Surgery, Beijing Friendship Hospital, Capital Medical University, Beijing 100050, China.
7. The Second Department of General Surgery, The Fourth Hospital of Hebei Medical University, Shijiazhuang, Hebei 050001, China.
8. Department of Gastroenterology, The Affiliated Hospital of Qingdao University, Qingdao, Shandong 266000, China.
9. Department of Colorectal and Anal Surgery, The First Affiliated Hospital of Shandong Second Medical University, Weifang, Shandong 261000, China.
10. Department of Gastrointestinal Surgery Ward II, Yantai Yuhuangding Hospital, Yantai, Shandong 264009, China.
- Publication Type:Multicenter Study
- Keywords:
Artificial intelligence;
Deep learning;
Locally advanced rectal cancer;
Neoadjuvant chemoradiotherapy;
Rectal neoplasm;
Treatment response
- MeSH:
Humans;
Rectal Neoplasms/diagnostic imaging*;
Magnetic Resonance Imaging/methods*;
Male;
Female;
Middle Aged;
Neoadjuvant Therapy/methods*;
Aged;
Adult;
Chemoradiotherapy/methods*;
Endoscopy/methods*;
Treatment Outcome
- From:
Chinese Medical Journal
2025;138(21):2793-2803
- CountryChina
- Language:English
-
Abstract:
BACKGROUND:Neoadjuvant chemoradiotherapy followed by radical surgery has been a common practice for patients with locally advanced rectal cancer, but the response rate varies among patients. This study aimed to develop a ResNet-Vision Transformer based magnetic resonance imaging (MRI)-endoscopy fusion model to precisely predict treatment response and provide personalized treatment.
METHODS:In this multicenter study, 366 eligible patients who had undergone neoadjuvant chemoradiotherapy followed by radical surgery at eight Chinese tertiary hospitals between January 2017 and June 2024 were recruited, with 2928 pretreatment colonic endoscopic images and 366 pelvic MRI images. An MRI-endoscopy fusion model was constructed based on the ResNet backbone and Transformer network using pretreatment MRI and endoscopic images. Treatment response was defined as good response or non-good response based on the tumor regression grade. The Delong test and the Hanley-McNeil test were utilized to compare prediction performance among different models and different subgroups, respectively. The predictive performance of the MRI-endoscopy fusion model was comprehensively validated in the test sets and was further compared to that of the single-modal MRI model and single-modal endoscopy model.
RESULTS:The MRI-endoscopy fusion model demonstrated favorable prediction performance. In the internal validation set, the area under the curve (AUC) and accuracy were 0.852 (95% confidence interval [CI]: 0.744-0.940) and 0.737 (95% CI: 0.712-0.844), respectively. Moreover, the AUC and accuracy reached 0.769 (95% CI: 0.678-0.861) and 0.729 (95% CI: 0.628-0.821), respectively, in the external test set. In addition, the MRI-endoscopy fusion model outperformed the single-modal MRI model (AUC: 0.692 [95% CI: 0.609-0.783], accuracy: 0.659 [95% CI: 0.565-0.775]) and the single-modal endoscopy model (AUC: 0.720 [95% CI: 0.617-0.823], accuracy: 0.713 [95% CI: 0.612-0.809]) in the external test set.
CONCLUSION:The MRI-endoscopy fusion model based on ResNet-Vision Transformer achieved favorable performance in predicting treatment response to neoadjuvant chemoradiotherapy and holds tremendous potential for enabling personalized treatment regimens for locally advanced rectal cancer patients.
