Research on lung function prediction methodology combining transfer learning and multimodal feature fusion
10.3760/cma.j.cn121382-20230920-00605
- VernacularTitle:结合迁移学习和多模态特征融合的肺功能预测方法研究
- Author:
Jian MA
1
;
Honglin ZHU
;
Jian LI
;
Wenhui WU
;
Shouqiang JIA
;
Shengdong NIE
Author Information
1. 上海理工大学健康科学与工程学院，上海 200093
- Keywords:
Idiopathic pulmonary fibrosis;
Pulmonary function;
Self-adaption;
Transfer learning;
Multimodality
- From:
International Journal of Biomedical Engineering
2023;46(6):506-513
- CountryChina
- Language:Chinese
-
Abstract:
Objective:To design a lung function prediction method that combines transfer learning and multimodal feature fusion, aiming to improve the accuracy of lung function prediction in patients with idiopathic pulmonary fibrosis (IPF).Methods:CT images and clinical text data were reprocessed, and an adaptive module was designed to find the most suitable lung function attenuation function for IPF patients. The feature extraction module was utilized to comprehensively extract features. The feature extraction module comprises three sub-modules, including CT feature extraction, clinical text feature extraction, and lung function feature extraction. A multimodal feature prediction network was used to comprehensively evaluate the attenuation of lung function. The pre-trained model was fine-tuned to improve the predictive performance of the model.Results:Based on the OSIC pulmonary fibrosis progression competition dataset, it is found through the adaptive module that the linear attenuation hypothesis is more in line with the trend of pulmonary function decline in patients. Different modal data prediction experiments show that the model incorporating clinical text features has better predictive ability than the model using only CT images. The model combining CT images, clinical text features, and lung function features have optimal predictive results. The lung function prediction method combining transfer learning and multimodal feature fusion has modified version of the Laplace log likelihood (LLLm) of ?6.706 5, root mean squared error (RMSE) of 184.5, and mean absolute error (MAE) of 146.2, which outperforms other methods in terms of performance. The pre-trained model has higher prediction accuracy compared to the zero base training model.Conclusions:The lung function prediction method designed by combining transfer learning and multimodal feature fusion can effectively predict the lung function status of IPF patients at different weeks, providing important support for patient health management and disease diagnosis.
