An Exploratory Study of Multimodality in Medical Imaging and Natural Language Processing

VernacularTitle:医学影像与自然语言处理多模态探索研究
Author: Yuxin GONG ¹ ; Fei XIANG ; Kui YING
Author Information

1. 华中科技大学同济医学院医药卫生管理学院武汉 430000
Keywords: chest radiograph; multimodal; automatic report generation; attention mechanism; natural language processing
From: Journal of Medical Informatics 2024;45(1):33-38
CountryChina
Language:Chinese
Abstract: Purpose/Significance Achieving automatic generation of medical imaging reports is important for reducing the workload of radiologists and promoting the standardization of clinical workflow.Method/Process Focusing on finding the chest report generation mod-els with open source code in recent years,the paper develops an automatic medical image report generation method based on the CDGPT2 model.Result/Conclusion The advantages of the model in report generation are still to be explored,the quality of reports generated after modifications to the decoder inputs of the model is not high.Future research could improve the performance of the model by using large datasets and incorporating more clinical information.