Diagnostic performance evaluation of artificial intelligence-assisted diagnostic systems in cervical cytopathological examination
10.3760/cma.j.cn112338-20240711-00412
- VernacularTitle:人工智能辅助诊断系统在宫颈细胞病理检查中的诊断性能评价
- Author:
Zichen YE
1
;
Yihui YANG
;
Lian XU
;
Ronggan WEI
;
Xiling RUAN
;
Peng XUE
;
Yu JIANG
;
Youlin QIAO
Author Information
1. 中国医学科学院北京协和医学院群医学及公共卫生学院，北京 100730
- Publication Type:Journal Article
- Keywords:
Artificial intelligence;
Cervical cancer;
Cytopathology;
External validation;
Human-machine assistance
- From:
Chinese Journal of Epidemiology
2025;46(3):499-505
- CountryChina
- Language:Chinese
-
Abstract:
Objective:To evaluate the diagnostic performance of artificial intelligence-assisted diagnostic systems in cervical cytopathological examination.Methods:Cervical cytology slide data were retrospectively collected from four hospitals for the external validation of the developed artificial intelligence-assisted diagnostic system. Subsequently, prospective data collection was conducted for human-machine assisted studies.Results:In the retrospective study, a total of 3 162 valid samples were collected as external validation data. The system showed an area under the curve (AUC) of 0.890 (95% CI: 0.878-0.902), accuracy of 0.885 (95% CI: 0.873-0.896), sensitivity of 0.928 (95% CI: 0.914-0.941), and specificity of 0.852 (95% CI: 0.834-0.867). In the prospective study, 212 valid samples were collected, and five junior cytologists participated in the human-machine assisted study. Without artificial intelligence assistance, the average AUC for the five cytologists was 0.686 (95% CI: 0.650-0.722), the accuracy was 0.699 (95% CI: 0.671-0.727), the sensitivity was 0.653 (95% CI: 0.599-0.703), the specificity was 0.719 (95% CI: 0.685-0.750), the Fleiss κ value was 0.510, and the reading time was 223 seconds. With artificial intelligence assistance, the AUC, accuracy, sensitivity, and specificity increased by 0.166, 0.143, 0.225, and 0.107, respectively. Additionally, Fleiss κ was 0.730 and the reading time decreased by 188 seconds. All differences were statistically significant (all P<0.001). Conclusions:Artificial intelligence-assisted diagnosis system shows excellent performance and good generalizability, significantly improving the diagnostic accuracy, consistency, and efficiency of junior cytologists. It can be an effective auxiliary tool for junior cytologists in clinical practice.
