Comparison on efficiency of three artificial intelligence-based models to read pneumoconiosis chest radiographs
- VernacularTitle:尘肺胸片人工智能读片三种模型效能比较研究
- Author:
Wei WANG
1
;
Baoping LI
2
;
Fuhai SHEN
1
;
Zhiping SUN
2
;
Bowen HOU
2
;
Lini GAO
2
;
Congxia YAN
1
Author Information
- Publication Type:Methodology
- Keywords: pneumoconiosis; convolutional neural network; artificial intelligence; deep learning
- From: Journal of Environmental and Occupational Medicine 2022;39(1):41-46
- CountryChina
- Language:Chinese
- Abstract: Background Diagnosis of pneumoconiosis by radiologist reading chest X-ray images is affected by many factors and is prone to misdiagnosis/missed diagnosis. With the rapid development of artificial intelligence in the field of medical imaging, whether artificial intelligence can be used to read images of pneumoconiosis deserves consideration. Objective Three deep learning models for identifying presence of pneumoconiosis were constructed based on deep convolutional neural network. An optimal model was selected by comparing diagnostic efficiency of the three models. Methods Digital radiography (DR) chest images were collected between June 2017 and December 2020 from 7 hospitals and standard radiograph quality control protocol was also followed. The DR chest images with positive results were classified into the positive group, while those without pneumoconiosis were classified into the negative group. The collected chest radiographs were labeled by experts who had passed the assessment of reading radiographs,and the experts were constantly assessed for consistency in the labeling process based on an expectation-maximization algorithm. The labeled data were cleaned, archived, and preprocessed, and then were grouped into a training set and a verification set. Three deep convolutional neural network models TMNet, ResNet-50, and ResNeXt-50 were constructed and trained by ten-fold cross-validation method to obtain an optimal model. Five hundred cases of DR chest radiographs that were not included in the training set and the validation set were collected, and identified by five senior experts as the gold standard, named the test set. The accuracy rate, sensitivity, specificity, area under curve (AUC), and other indexes of the three models were derived after testing, and the efficiency of the three models was evaluated and compared. Results A total of 24867 DR chest radiographs of the training set and the validation set were collected in this study, including 6978 images in the positive group and 17889 images in the negative group. There were 312 cases of pulmonary abnormalities such as pneumothorax and pulmonary tuberculosis. A total of nine experts labeled the chest radiographs, the labeling consistency rate of pneumoconiosis (non-staging) was above 88%, and the labeling consistency rate of pneumoconiosis staging ranged from 84.68% to 93.66%. The diagnostic accuracy, sensitivity, specificity, and AUC of TMNet were 95.20%, 99.66%, 88.61%, and 0.987, respectively. The indicators of ResNeXt were 87.00%, 89.93%, 82.67%, and 0.911, respectively. Those of ResNet were 84.00%, 85.91%, 81.19%, and 0.912, respectively. All these indexes of TMNet were higher than those of ResNeXt-50 and ResNet-50 models. The AUC differences between TMNet and the other two models were both statistically significant (P<0.001). Conclusion All the three convolutional neural network models can effectively diagnose the presence of pneumoconiosis, among which TMNet provides the best efficiency.
