Prediction of Pulmonary Nodule Progression Based on Multi-modal Data Fusion of CCNet-DGNN Model
10.13422/j.cnki.syfjx.20251565
- VernacularTitle:基于CCNet-DGNN模型的多模态数据融合的肺结节进展预测
- Author:
Lehua YU
1
;
Yehui PENG
1
;
Wei YANG
2
;
Xinghua XIANG
2
;
Rui LIU
3
;
Xiongjun ZHAO
4
;
Maolan AYIDANA
3
;
Yue LI
3
;
Wenyuan XU
2
;
Min JIN
4
;
Shaoliang PENG
4
;
Baojin HUA
3
Author Information
1. School of Mathematics and Statistics，Hunan University of Science and Technology， Xiangtan 411201，China
2. Institute of Basic Research in Clinical Medicine， China Academy of Chinese Medical Sciences，Beijing 100700，China
3. Guang'anmen Hospital，China Academy of Chinese Medical Sciences，Beijing 100053，China
4. College of Computer Science and Electronic Engineering，Hunan University，Changsha 410082，China
- Publication Type:Journal Article
- Keywords:
pulmonary nodule;
progression prediction;
multimodal data fusion;
criss-cross attention mechanism;
directed graph neural network;
high-risk lung cancer population;
traditional Chinese and western clinical assessment
- From:
Chinese Journal of Experimental Traditional Medical Formulae
2025;31(24):135-143
- CountryChina
- Language:Chinese
-
Abstract:
ObjectiveThis study aims to develop and validate a novel multimodal predictive model， termed criss-cross network（CCNet）-directed graph neural network（DGNN）（CGN）， for accurate assessment of pulmonary nodule progression in high-risk individuals for lung cancer， by integrating longitudinal chest computed tomography（CT） imaging with both traditional Chinese and western clinical evaluation data. MethodsA cohort of 4 432 patients with pulmonary nodules was retrospectively analyzed. A twin CCNet was employed to extract spatiotemporal representations from paired sequential CT scans. Structured clinical assessment and imaging-derived features were encoded via a multilayer perceptron， and a similarity-based alignment strategy was adopted to harmonize multimodal imaging features across temporal dimensions. Subsequently， a DGNN was constructed to integrate heterogeneous features， where nodes represented modality-specific embeddings and edges denoted inter-modal information flow. Finally， model optimization was performed using a joint loss function combining cross-entropy and cosine similarity loss， facilitating robust classification of nodule progression status. ResultsThe proposed CGN model demonstrated superior predictive performance on the held-out test set， achieving an area under the receiver operating characteristic curve（AUC） of 0.830， accuracy of 0.843， sensitivity of 0.657， specificity of 0.712， Cohen's Kappa of 0.417， and F1 score of 0.544. Compared with unimodal baselines， the CGN model yielded a 36%-48% relative improvement in AUC. Ablation studies revealed a 2%-22% increase in AUC when compared to simplified architectures lacking key components， substantiating the efficacy of the proposed multimodal fusion strategy and modular design. Incorporation of traditional Chinese medicine （TCM）-specific symptomatology led to an additional 5% improvement in AUC， underscoring the complementary value of integrating TCM and western clinical data. Through gradient-weighted activation mapping visualization analysis， it was found that the model's attention predominantly focused on nodule regions and effectively captured dynamic associations between clinical data and imaging-derived features. ConclusionThe CGN model， by synergistically combining cross-attention encoding with directed graph-based feature integration， enables effective alignment and fusion of heterogeneous multimodal data. The incorporation of both TCM and western clinical information facilitates complementary feature enrichment， thereby enhancing predictive accuracy for pulmonary nodule progression. This approach holds significant potential for supporting intelligent risk stratification and personalized surveillance strategies in lung cancer prevention.
