OBJECTIVE: To validate the effectiveness of a novel comprehensive classification for intertrochanteric fracture (ITF). METHODS: The study included 616 patients with ITF, including 279 males (45.29%) and 337 females (54.71%); the age ranged from 23 to 100 years, with an average of 72.5 years. Two orthopaedic residents (observers Ⅰ and Ⅱ) and two senior orthopaedic surgeons (observers Ⅲ and Ⅳ) were selected to classify the CT imaging data of 616 patients in a random order by using the AO/Orthopaedic Trauma Association (AO/OTA) classification of 1996/2007 edition, the AO/OTA classification of 2018 edition, and the novel comprehensive classification method at an interval of 1 month. Kappa consistency test was used to evaluate the intra-observer and inter-observer consistency of the three ITF classification systems. RESULTS: The inter-observer consistency of the three classification systems evaluated by 4 observers twice showed that the 3 classification systems had strong inter-observer consistency. Among them, the κ value of the novel comprehensive classification was higher than that of the AO/OTA classification of 1996/2007 edition and 2018 edition, and the experience of observers had a certain impact on the classification results, and the inter-observer consistency of orthopaedic residents was slightly better than that of senior orthopaedic surgeons. The intra-observer consistency of two evaluations of three classification systems by 4 observers showed that the consistency of the novel comprehensive classification was better for the other 3 observers, except that the consistency of observer Ⅳ in the AO/OTA classification of 2018 version was slightly higher than that of the novel comprehensive classification. The results showed that the novel comprehensive classification has higher repeatability, and the intra-observer consistency of senior orthopaedic surgeons was better than that of orthopaedic residents. CONCLUSION The novel comprehensive classification system has good intra- and inter-observer consistency, and has high validity in the classification of CT images of ITF patients; the experience of observers has a certain impact on the results of the three classification systems, and those with more experiences have higher intra-observer consistency.
Male ; Female ; Humans ; Young Adult ; Adult ; Middle Aged ; Aged ; Aged, 80 and over ; Observer Variation ; Reproducibility of Results ; Hip Fractures/surgery* ; Tomography, X-Ray Computed/methods* ; Radiography

INTRODUCTION: In view of the important role of the environment in improving population health, implementation of health promotion programmes is recommended in living and working environments. Assessing the prevalence of such community health-promoting practices is important to identify gaps and make continuous and tangible improvements to health-promoting environments. We aimed to evaluate the inter-rater reliability of a composite scorecard used to assess the prevalence of community health-promoting practices in Singapore. METHODS: Inter-rater reliability for the use of the composite health promotion scorecards was evaluated in eight residential zones in the western region of Singapore. The assessment involved three raters, and each zone was evaluated by two raters. Health-promoting practices in residential zones were assessed based on 44 measurable elements under five domains - community support and resources, healthy behaviours, chronic conditions, mental health and common medical emergencies - in the composite scorecard using weighted kappa. The strength of agreement was determined based on Landis and Koch's classification method. RESULTS: A high degree of agreement (almost perfect-to-perfect) was observed between both raters for the measurable elements from most domains and subdomains. An exception was observed for the community support and resources domain, where there was a lower degree of agreement between the raters for a few elements. CONCLUSION The composite scorecard demonstrated a high degree of reliability and yielded similar scores for the same residential zone, even when used by different raters.
Health Promotion ; Humans ; Observer Variation ; Public Health ; Reproducibility of Results ; Singapore

Objective: To evaluate the inter-observer agreement of the severity of oral epithelial dysplasia in oral leukoplakia, providing a theoretical basis for the development of a more objective grading system. Methods: This study included 60 digital pathological slides of oral leukoplakia from Oral Medicine Department of West China Hospital of Stomatology, Sichuan University, and 239 tissue microarray images of oral leukoplakia from State Key Laboratory of Oral Diseases, Sichuan University, to evaluate the agreement of grading. Besides, 1 000 patches were generated from the 60 digital pathological slides and were divided into 500 small-sized patches (224 pixel×224 pixel) and 500 large-sized patches (1 024 pixel×1 024 pixel), to evaluate the agreement of feature detection. Gradings and feature detections were completed by three pathological experts from the oral pathology departments of two Grade 3, Class A stomatological hospitals in China. Kappa coefficient was used to quantify the inter-observer agreement among pathologists. Results: Minimal agreement was found in the grading of oral epithelial dysplasia among pathologists (Kappa=0.30 in the pathological slide group, Kappa=0.30 in the tissue microarray group). None agreement was found in feature detection within the small-sized patches group (median Kappa=0.14 for architectural features, median Kappa=0.18 for cytological features), and minimal agreement was found in feature detection within the large-sized patches group (median Kappa=0.25 for architectural features, median Kappa=0.25 for cytological features). Conclusions: Generally, the agreement of grading and feature detection of oral epithelial dysplasia in oral leukoplakia is poor. Development of a more objective grading system of oral epithelial dysplasia based on artificial intelligence may be helpful to improve the agreement.
Artificial Intelligence ; China ; Humans ; Leukoplakia, Oral ; Observer Variation ; Precancerous Conditions

Objectives: To evaluate the diagnostic consistency of working lengths by observing endodontic files in root canals and periapical subtle structures in digital intraoral radiographs presented in two smartphones, a tablet and a laptop computer. Methods: A dried human skull embedded in an acrylic compound was used for exposing radiographs of the upper and lower second premolars and first molars with two endodontic files (Kerr files size 10 and 15) positioned to the full length of the roots or 1.5 mm short of apexes. A total of 100 radiographs were taken for each of the file sizes. Five observers were asked to assess all the 200 digital radiographs according to a 5-category scale in smartphone A (HUAWEI P9 Plus), smartphjone B (Apple iPhone 7), tablet (Apple iPad 2018) and laptop computer (Lenovo Thinkpad E480), respectively. The gold standard for receiver operating characteristic curve (ROC) analysis was determined with the endodontic Kerr file size 20. A total of 150 roots with files were radiographed, 75 of which with files reaching the radiographic apexes of the respective roots and 75 of which with files 1.5 mm short of the radiographic apexes for each endodontic file size. Results from ROC analysis was analyzed with one-way ANOVA and independent sample t test. Results: For the Kerr file size 10, the area under the ROC curve for laptop, tablet and two smartphones were 0.891±0.037, 0.869±0.037, 0.870±0.017 and 0.849±0.037, while for the Kerr file size 15 the ROC values were 0.957±0.02, 0.961±0.02, 0.961±0.01 and 0.961±0.02, respectively. There were no significant differences for diagnostic accuracy for observing endodontic file positions among digital radiographs presented in the two smartphones, one tablet and one laptop devices (endodontic file size 10: F=1.39, P=0.281; endodontic file size 15: F=0.05, P=0.985). A significant difference was found in the diagnostic accuracy of endodontic file positions between size 10 and 15 files in different display devices (t=-10.65, P<0.001). Conclusions: There was a high diagnostic consistency in the determination of working length and periapical subtle structures of roots by observing digital radiographs displayed on smartphones, tablet and laptop computer.
Dental Instruments ; Dental Pulp Cavity/diagnostic imaging* ; Electronics ; Humans ; Molar ; Observer Variation ; Root Canal Preparation

Objective To investigate the intra-and inter-observer reproducibility of iodine concentrations of abdominal parenchymal organs based on spectral CT.Methods The water-free iodine images of the venous phase were retrospectively obtained from 50 patients with abdominal dynamic spectral CT scans.The iodine concentrations were measured in the left,right and caudate lobes of liver,spleen,pancreas and bilateral kidneys.Intraclass correlation coefficient(ICC)and Bland-Altman plot were employed to analyze the intra-and inter-observer reproducibility.Results The intra-observer ICCs of the left,right and caudate lobes of liver,spleen,pancreas,and left and right kidneys were 0.938(0.894,0.965),0.932(0.884,0.961),0.939(0.895,0.965),0.947(0.909,0.970),0.912(0.851,0.949),0.946(0.906,0.969)and 0.907(0.842,0.946),which indicated good intra-observer reproducibility.The inter-observer ICCs of the left,right and caudate lobes of liver,spleen,pancreas,and left and right kidneys were 0.947(0.909,0.970),0.927(0.875,0.958),0.943(0.902,0.968),0.956(0.924,0.975),0.934(0.887,0.962),0.927(0.875,0.958)and 0.892(0.818,0.937),which indicated good inter-observer reproducibility.Bland-Altman plots presented that more than 95% points of the intra-observer differences located within 95% CI of limits of agreement for the caudate lobe of liver,spleen,pancreas and bilateral kidneys,which was same as inter-observer differences of the caudate lobe of liver,spleen and right kidney.Conclusion The iodine concentration measurement based on the spectral CT presented good intra-and inter-observer reproducibility for the caudate lobe of liver and spleen.
Humans ; Iodine ; Observer Variation ; Reproducibility of Results ; Retrospective Studies ; Tomography, X-Ray Computed

BACKGROUND: Previously, we developed a novel Coronary Artery Tree description and Lesion EvaluaTion (CatLet©) angiographic scoring system, which was capable of accounting for the variability in the coronary anatomy and assisting in the risk-stratification of patients with acute myocardial infarction (AMI). Our preliminary study revealed that the CatLet score better predicted clinical outcomes for AMI patients than the Synergy between Percutaneous Coronary Intervention with Taxus and Cardiac Surgery score. However, the reproducibility of the CatLet score in both inter- and intra-observer remains to be evaluated. METHODS: A total of 30 consecutive AMI patients, admitted in September of 2015, were independently assessed by two experienced interventional cardiologists to evaluate the inter-observer reproducibility of the CatLet score. Another set of 49 consecutive AMI patients, admitted between September and October in 2014, were assessed by one of the two interventional cardiologists on two occasions 3 months apart to evaluate the intra-observer reproducibility of the CatLet score. The weighted kappa was used to express the degree of agreement. RESULTS: The weighted kappa values (95% confidence interval) for the intra- and inter-observer reproducibility of the CatLet Score were 0.82 (0.59-1.00, Z = 7.23, P < 0.001) and 0.86 (0.54-1.00, Z = 5.20, P < 0.001), respectively, according to the tertile analysis (≤14, 15-22, >22). Regarding the adverse characteristics pertinent to lesions and dominance parameters, the kappa values for the inter-observer variability were 0.80 (0.56-1.00, Z = 6.47, P < 0.001) for total number of lesions, 0.57 (0.28-0.85, Z = 3.03, P < 0.001) for bifurcation, 0.69 (0.43-0.96, Z = 5.06, P < 0.001) for heavy calcification, 1.00 (0.72-1.00, Z = 6.93, P < 0.001) for tortuosity, 0.54 (0.26-0.82, Z = 3.78, P < 0.001) for thrombus, 0.69 (0.48-0.91, Z = 6.29, P < 0.001) for right coronary artery dominance, 0.69 (0.41-0.96, Z = 4.91, P < 0.001) for left anterior descending artery length, and 0.22 (0.06-0.51, Z = 1.56, P = 0.06) for diagonal size. Equivalent values for the intra-observer variability were moderate to almost perfect (range 0.54-1.00). CONCLUSIONS The reproducibility of the CatLet angiographic scoring system for evaluation of the coronary angiograms ranged from substantial to excellent. The high reproducibility of the CatLet angiographic scoring system will boost its clinical application to patients with AMI.
Coronary Angiography ; Coronary Artery Disease ; Humans ; Myocardial Infarction/diagnostic imaging* ; Observer Variation ; Reproducibility of Results ; Treatment Outcome ; Trees

Although microscopic analysis of tissue slides has been the basis for disease diagnosis for decades, intra- and inter-observer variabilities remain issues to be resolved. The recent introduction of digital scanners has allowed for using deep learning in the analysis of tissue images because many whole slide images (WSIs) are accessible to researchers. In the present study, we investigated the possibility of a deep learning-based, fully automated, computer-aided diagnosis system with WSIs from a stomach adenocarcinoma dataset. Three different convolutional neural network architectures were tested to determine the better architecture for tissue classifier. Each network was trained to classify small tissue patches into normal or tumor. Based on the patch-level classification, tumor probability heatmaps can be overlaid on tissue images. We observed three different tissue patterns, including clear normal, clear tumor and ambiguous cases. We suggest that longer inspection time can be assigned to ambiguous cases compared to clear normal cases, increasing the accuracy and efficiency of histopathologic diagnosis by pre-evaluating the status of the WSIs. When the classifier was tested with completely different WSI dataset, the performance was not optimal because of the different tissue preparation quality. By including a small amount of data from the new dataset for training, the performance for the new dataset was much enhanced. These results indicated that WSI dataset should include tissues prepared from many different preparation conditions to construct a generalized tissue classifier. Thus, multi-national/multi-center dataset should be built for the application of deep learning in the real world medical practice.
Adenocarcinoma ; Classification ; Dataset ; Diagnosis ; Learning ; Observer Variation ; Stomach

BACKGROUND: Assessment of programmed cell death-ligand 1 (PD-L1) immunohistochemical staining is used for treatment decisions in non-small cell lung cancer (NSCLC) regarding use of PD-L1/programmed cell death protein 1 (PD-1) immunotherapy. The reliability of the PD-L1 22C3 pharmDx assay is critical in guiding clinical practice. The Cardiopulmonary Pathology Study Group of the Korean Society of Pathologists investigated the interobserver reproducibility of PD-L1 staining with 22C3 pharmDx in NSCLC samples.METHODS: Twenty-seven pathologists individually assessed the tumor proportion score (TPS) for 107 NSCLC samples. Each case was divided into three levels based on TPS: <1%, 1%–49%, and ≥50%.RESULTS: The intraclass correlation coefficient for TPS was 0.902±0.058. Weighted κ coefficient for 3-step assessment was 0.748±0.093. The κ coefficients for 1% and 50% cut-offs were 0.633 and 0.834, respectively. There was a significant association between interobserver reproducibility and experience (formal PD-L1 training, more experience for PD-L1 assessment, and longer practice duration on surgical pathology), histologic subtype, and specimen type.CONCLUSIONS: Our results indicate that PD-L1 immunohistochemical staining provides a reproducible basis for decisions on anti–PD-1 therapy in NSCLC.
Carcinoma, Non-Small-Cell Lung ; Cell Death ; Immunohistochemistry ; Immunotherapy ; Observer Variation ; Pathology

Lack of standardized, reproducible protocols and reference values is among the challenges faced when using new or upgraded versions of instruments in reproductive laboratories and flow cytometry. Terminal deoxynucleotidyl transferase dUTP nick end labeling (TUNEL) assay combined with flow cytometry routinely used for diagnostic measurement of sperm DNA fragmentation (SDF) is a unique example. Any change in the setting of the standard instrument, including upgrades of hardware or software, can lead to different results and may affect clinicians' decision for treatment. Therefore, we compared TUNEL results of SDF obtained from a standard (C6) flow cytometer with a newer version of the same instrument (C6 Plus) and examined the cutoff, sensitivity, and specificity without calibration (adjustment) and after adjustment. Identical sperm preparation and matched acquisition settings were used to examine the performance of two flow cytometers. The strength of agreement of the results between the two observers was also assessed. After adjustment of the settings, overall concordance became high and the two cytometers showed 100% positive and negative predictive value with 100% area under the curve. The overall correlation coefficient observed between C6 and C6 Plus was highly significant (P < 0.0001; r = 0.992; 95% confidence interval [CI]: 0.982-0.997). After adjustment, the two cytometers showed very high precision of 98% and accuracy of >99%. The interobserver agreement on C6 flow cytometer for the two observers was 0.801 ± 0.062 and 0.746 ± 0.044 for C6 Plus. We demonstrated a strong agreement between the samples tested on the two flow cytometers after calibration and established the robustness of both instruments.
Adult ; Calibration ; DNA Fragmentation ; Flow Cytometry/instrumentation* ; Humans ; In Situ Nick-End Labeling ; Male ; Observer Variation ; Reference Values ; Reproducibility of Results ; Semen Analysis/methods* ; Sensitivity and Specificity ; Spermatozoa/chemistry*

OBJECTIVE: To evaluate the interpretive performance and inter-observer agreement on digital mammographs among radiologists and to investigate whether radiologist characteristics affect performance and agreement. MATERIALS AND METHODS: The test sets consisted of full-field digital mammograms and contained 12 cancer cases among 1000 total cases. Twelve radiologists independently interpreted all mammograms. Performance indicators included the recall rate, cancer detection rate (CDR), positive predictive value (PPV), sensitivity, specificity, false positive rate (FPR), and area under the receiver operating characteristic curve (AUC). Inter-radiologist agreement was measured. The reporting radiologist characteristics included number of years of experience interpreting mammography, fellowship training in breast imaging, and annual volume of mammography interpretation. RESULTS: The mean and range of interpretive performance were as follows: recall rate, 7.5% (3.3–10.2%); CDR, 10.6 (8.0–12.0 per 1000 examinations); PPV, 15.9% (8.8–33.3%); sensitivity, 88.2% (66.7–100%); specificity, 93.5% (90.6–97.8%); FPR, 6.5% (2.2–9.4%); and AUC, 0.93 (0.82–0.99). Radiologists who annually interpreted more than 3000 screening mammograms tended to exhibit higher CDRs and sensitivities than those who interpreted fewer than 3000 mammograms (p = 0.064). The inter-radiologist agreement showed a percent agreement of 77.2–88.8% and a kappa value of 0.27–0.34. Radiologist characteristics did not affect agreement. CONCLUSION: The interpretative performance of the radiologists fulfilled the mammography screening goal of the American College of Radiology, although there was inter-observer variability. Radiologists who interpreted more than 3000 screening mammograms annually tended to perform better than radiologists who did not.
Area Under Curve ; Breast ; Fellowships and Scholarships ; Mammography ; Mass Screening ; Medical Audit ; Observer Variation ; ROC Curve ; Sensitivity and Specificity