Application of standardized manual labeling on identification of retinopathy of prematurity images in deep learning
10.3760/cma.j.issn.2095-0160.2019.08.013
- VernacularTitle:深度学习中标准化人工标注在早产儿视网膜病变图像识别中的应用
- Author:
Ji WANG
1
;
Guihua ZHANG
;
Jianwei LIN
;
Jie JI
;
Kunliang QIU
;
Mingzhi ZHANG
Author Information
1. 汕头大学·香港中文大学联合汕头国际眼科中心
- Keywords:
Artificial intelligence;
Standardized label;
Deep learning;
Retinopathy of prematurity;
Stage disease;
Plus disease
- From:
Chinese Journal of Experimental Ophthalmology
2019;37(8):653-657
- CountryChina
- Language:Chinese
-
Abstract:
Objective To evaluate the application of the standard manual labeling on identification of retinopathy of prematurity ( ROP) images in deep learning. Methods According to the International Classification of ROP,different periods of ROP were classified into stage disease and plus disease in this study. From Joint Shantou International Eye Center from August 2009 to July 2018, a total of 1464 labeled fundus retinal photographs were divided randomly by stratified sampling into 3 groups:stage disease group(subgroup 1:173,subgroup 2:117) was used to train for labeling stage disease,whereas plus disease group(subgroup 1:163,subgroup 2:116) was used to train for labeling plus disease,and consistent labels group consisted of 895 consistent labeled images on both disease. Graders consisted of senior experts,3 senior ophthalmologists and 2 interns,and received training for classification and labeling on ROP fundus images. The results were compared among the doctors and doctors with deep learning,and the agreement between non-experts doctors and the reference standards, and deep learning and the reference standards were tested. Results After the first training,the overall agreement rate of the senior ophthalmologist group and the intern group were lower than 90％ for both two disease labeling. After two to three times of training, in image of consistent labels group,overall agreement rates of senior ophthalmologists and intern doctor's were 98. 99％ ( Kappa=0. 979),99. 22％ (Kappa=0. 984) on stage disease,and 97. 43％ (Kappa=0. 914),98. 11％ (Kappa=0. 935) on plus disease,respectively. The agreement on stage disease using deep learning based on human-machine combination was 94. 08％,Kappa value was 0. 880,which achieved good degree. Conclusions Standardized manual labeling can improve the intelligentization of deep learning on identification of ROP images,and be considered as an innovative method of homogenization and standardized training for doctors in ophthalmology.
