Incomplete multimodal bone tumor image classification based on feature decoupling and fusion.
10.12122/j.issn.1673-4254.2025.06.22
- Author:
Qinghai ZENG
1
;
Chuanpu LI
1
;
Wei YANG
1
;
Liwen SONG
2
;
Yinghua ZHAO
2
;
Yi YANG
1
Author Information
1. School of Biomedical Engineering, Southern Medical University, Guangzhou 510515, China.
2. Department of Radiology, Third Affiliated Hospital of Southern Medical University, Guangzhou 510630, China.
- Publication Type:Journal Article
- Keywords:
attention fusion;
bone tumor classification;
feature decoupling;
modality missing;
multimodal imaging
- MeSH:
Humans;
Bone Neoplasms/diagnosis*;
Multimodal Imaging/methods*;
Magnetic Resonance Imaging;
Tomography, X-Ray Computed;
Image Processing, Computer-Assisted/methods*;
Algorithms
- From:
Journal of Southern Medical University
2025;45(6):1327-1335
- CountryChina
- Language:Chinese
-
Abstract:
OBJECTIVES:To construct a bone tumor classification model based on feature decoupling and fusion for processing modality loss and fusing multimodal information to improve classification accuracy.
METHODS:A decoupling completion module was designed to extract local and global bone tumor image features from available modalities. These features were then decomposed into shared and modality-specific features, which were used to complete the missing modality features, thereby reducing completion bias caused by modality differences. To address the challenge of modality differences that hinder multimodal information fusion, a cross-attention-based fusion module was introduced to enhance the model's ability to learn cross-modal information and fully integrate specific features, thereby improving the accuracy of bone tumor classification.
RESULTS:The experiment was conducted using a bone tumor dataset collected from the Third Affiliated Hospital of Southern Medical University for training and testing. Among the 7 available modality combinations, the proposed method achieved an average AUC, accuracy, and specificity of 0.766, 0.621, and 0.793, respectively, which represent improvements of 2.6%, 3.5%, and 1.7% over existing methods for handling missing modalities. The best performance was observed when all the modalities were available, resulting in an AUC of 0.837, which still reached 0.826 even with MRI alone.
CONCLUSIONS:The proposed method can effectively handle missing modalities and successfully integrate multimodal information, and show robust performance in bone tumor classification under various complex missing modality scenarios.
