Construction of a Chinese Medicine Zhengsu Differentiation Model for Type 2 Diabetes Based on Deep Learning Multimodal Fusion
- VernacularTitle:基于深度学习多模态融合的2型糖尿病中医证素辨证模型的构建
- Author:
Zhihui ZHAO
1
;
Yi ZHOU
;
Weihong LI
;
Zhaohui TANG
;
Qiang GUO
;
Rigao CHEN
Author Information
- Keywords: Zhengsu differentiation; Type 2 diabetes; Deep learning; Multimodal fusion
- From: World Science and Technology-Modernization of Traditional Chinese Medicine 2024;26(4):908-918
- CountryChina
- Language:Chinese
- Abstract: Objective To construct a TCM Zhengsu differentiation model for type 2 diabetes based on deep learning and multimodal fusion,thus providing algorithmic support for full intelligence in TCM Zhengsu differentiation.Methods A total of 2585 patients with type 2 diabetes were recruited.Three experts were invited to perform the Zhengsu differentiation separately.Deep fully connected neural networks,U2-Net and ResNet34 networks were applied to construct the symptom-based differentiation model(S-Model)and the tongue image-based differentiation model(T-Model),respectively,while multimodal fusion techniques were employed to build the multimodal fusion model(TS-Model)with the above two as co-inputs.Finally,the prediction performance of the above models was compared by F1 value,accuracy,and recall.Results The predicted F1 values of the T-Model fluctuated from 0.000%to 86.726%,while those in the S-Model and TS-Model fluctuated from 0.000%to 97.826%and from 55.556%to 99.065%,respectively.A stable and high F1 value was found in the TS-Model.Conclusion The multimodal fusion technique was demonstrated to be applicable in the TCM Zhengsu differentiation model,which provided methodological support for developingof a fully intelligent Zhengsu differentiation model with high objective four diagnostic information.
