Selection and Reporting of Statistical Methods to Assess Reliability of a Diagnostic Test: Conformity to Recommended Methods in a Peer-Reviewed Journal.
10.3348/kjr.2017.18.6.888
- Author:
Ji Eun PARK
1
;
Kyunghwa HAN
;
Yu Sub SUNG
;
Mi Sun CHUNG
;
Hyun Jung KOO
;
Hee Mang YOON
;
Young Jun CHOI
;
Seung Soo LEE
;
Kyung Won KIM
;
Youngbin SHIN
;
Suah AN
;
Hyo Min CHO
;
Seong Ho PARK
Author Information
1. Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul 05505, Korea. parksh.radiology@gmail.com
- Publication Type:Original Article
- Keywords:
Reliability;
Reproducibility;
Repeatability;
Agreement;
Statistical method;
Statistical analysis;
Software program;
Repeatability coefficient
- MeSH:
Biomarkers;
Diagnostic Tests, Routine*;
Methods*
- From:Korean Journal of Radiology
2017;18(6):888-897
- CountryRepublic of Korea
- Language:English
-
Abstract:
OBJECTIVE: To evaluate the frequency and adequacy of statistical analyses in a general radiology journal when reporting a reliability analysis for a diagnostic test. MATERIALS AND METHODS: Sixty-three studies of diagnostic test accuracy (DTA) and 36 studies reporting reliability analyses published in the Korean Journal of Radiology between 2012 and 2016 were analyzed. Studies were judged using the methodological guidelines of the Radiological Society of North America-Quantitative Imaging Biomarkers Alliance (RSNA-QIBA), and COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) initiative. DTA studies were evaluated by nine editorial board members of the journal. Reliability studies were evaluated by study reviewers experienced with reliability analysis. RESULTS: Thirty-one (49.2%) of the 63 DTA studies did not include a reliability analysis when deemed necessary. Among the 36 reliability studies, proper statistical methods were used in all (5/5) studies dealing with dichotomous/nominal data, 46.7% (7/15) of studies dealing with ordinal data, and 95.2% (20/21) of studies dealing with continuous data. Statistical methods were described in sufficient detail regarding weighted kappa in 28.6% (2/7) of studies and regarding the model and assumptions of intraclass correlation coefficient in 35.3% (6/17) and 29.4% (5/17) of studies, respectively. Reliability parameters were used as if they were agreement parameters in 23.1% (3/13) of studies. Reproducibility and repeatability were used incorrectly in 20% (3/15) of studies. CONCLUSION: Greater attention to the importance of reporting reliability, thorough description of the related statistical methods, efforts not to neglect agreement parameters, and better use of relevant terminology is necessary.
