Application of multimodal weakly-supervised learning in image synthesis and segmentation of liver cancer
10.3969/j.issn.1005-202X.2024.01.002
- VernacularTitle:多模态弱监督学习在肝癌图像生成与分割中的应用
- Author:
Yile PAN
1
;
Yongbin GAO
Author Information
1. 上海工程技术大学电子电气工程学院,上海 201620
- Keywords:
medical image processing;
mode conversion;
liver tumor segmentation;
generative adversarial network;
mixed
- From:
Chinese Journal of Medical Physics
2024;41(1):8-17
- CountryChina
- Language:Chinese
-
Abstract:
Although it has high resolution for soft tissues,magnetic resonance imaging(MRI)is not the standard for chest imaging,which results in an insufficient amount of expert-annotated MRI data.Therefore,CT image is usually converted into MRI image.To overcome the difficulty of obtaining the corresponding modal CT and MRI images,a CSCGAN model with CycleGAN as the framework is proposed based on the structural characteristics of generative adversarial networks.Considering the possibility of mode collapse in CycleGAN,StyleGan2 which can control the style and feature details of the synthetic image and realize the synthesis of high-resolution images is integrated into CycleGAN for reconstructing the generator.A noise module is introduced to reduce external interference.In addition,in order to prevent the loss of tumors during conversion,the discriminator structure of the network is modified,and a mixed attention mechanism is added.Experimental results show that compared with the images generated by other methods,those generated by the proposed model are improved in Dice similarity coefficient,Hausdorff distance,volume ratio and mean intersection over union,indicating that the proposed method can effectively realize the mode conversion of liver tumor images,and that the generated data can improve the segmentation accuracy.
