OBJECTIVE: To evaluate the design characteristics of studies that evaluated the performance of artificial intelligence (AI) algorithms for the diagnostic analysis of medical images. MATERIALS AND METHODS: PubMed MEDLINE and Embase databases were searched to identify original research articles published between January 1, 2018 and August 17, 2018 that investigated the performance of AI algorithms that analyze medical images to provide diagnostic decisions. Eligible articles were evaluated to determine 1) whether the study used external validation rather than internal validation, and in case of external validation, whether the data for validation were collected, 2) with diagnostic cohort design instead of diagnostic case-control design, 3) from multiple institutions, and 4) in a prospective manner. These are fundamental methodologic features recommended for clinical validation of AI performance in real-world practice. The studies that fulfilled the above criteria were identified. We classified the publishing journals into medical vs. non-medical journal groups. Then, the results were compared between medical and non-medical journals. RESULTS: Of 516 eligible published studies, only 6% (31 studies) performed external validation. None of the 31 studies adopted all three design features: diagnostic cohort design, the inclusion of multiple institutions, and prospective data collection for external validation. No significant difference was found between medical and non-medical journals. CONCLUSION: Nearly all of the studies published in the study period that evaluated the performance of AI algorithms for diagnostic analysis of medical images were designed as proof-of-concept technical feasibility studies and did not have the design features that are recommended for robust validation of the real-world clinical performance of AI algorithms.
Artificial Intelligence ; Case-Control Studies ; Cohort Studies ; Data Collection ; Feasibility Studies ; Machine Learning ; Prospective Studies

Objective: This study verifies the practicality of the delta neutrophil index to lymphocyte ratio for the prognostic evaluation of sepsis patients. Methods: Records of 2,233 patients diagnosed with sepsis were reviewed; 1,042 patients were included in the final analysis. Receiver operating characteristic (ROC) curve studies were used to calculate the area under the curve (AUC) to determine the neutrophil-to-lymphocyte ratio (NLR) and the delta neutrophil-to-lymphocyte ratio (Delta-NLR). To adjust for skewed distributions, the NLR and Delta-NLR were analyzed after natural logarithm transformations. Multivariate logistic regression was applied to determine potential predictors for mortality. Results: To predict 30-day mortality, AUCs were performed using the values of days 0, 1, and 2 (0.604, P<0.0001; 0.648, P<0.0001; and 0.684, P<0.0001, respectively). The NLR results were 0.504 (P=0.8624), 0.553 (P=0.0191), and 0.598 (P<0.0001), respectively. The AUC increased significantly when the Delta-NLR at day 0 was combined with age, hemoglobin levels, and lactate levels. Further subgroup analysis was performed by dividing patients into an upper respiratory infection (URI) group, a gastrointestinal tract infection (GI) (including hepatobiliary infection) group, and a urinary tract infection (UTI) group. The predictive ability of the GI group was determined to be much higher than the other two groups. Conclusion Increase in the Delta-NLR of sepsis patients was found to be an independent predictor of mortality within 30 days.

Noonan syndrome (NS) is a genetic disorder caused by autosomal dominant inheritance and is characterized by a distinctive facial appearance, short stature, chest deformity, and congenital heart disease. In individuals with NS, germline mutations have been identified in several genes involved in the RAS/mitogen-activated protein kinase signal transduction pathway. Because of its clinical and genetic heterogeneity, the conventional diagnostic protocol with Sanger sequencing requires a multistep approach. Therefore, molecular genetic diagnosis using targeted exome sequencing (TES) is considered a less expensive and faster method, particularly for patients who do not fulfill the clinical diagnostic criteria of NS. In this case, the patient showed short stature, dysmorphic facial features suggestive of NS, feeding intolerance, cryptorchidism, and intellectual disability in early childhood. At the age of 16, the patient still showed extreme short stature with delayed puberty and characteristic facial features suggestive of NS. Although the patient had no cardiac problems or chest wall deformities, which are commonly present in NS and are major concerns for patients and clinicians, the patient showed several other characteristic clinical features of NS. Considering the possibility of a genetic disorder, including NS, a molecular genetic study with TES was performed. With TES analysis, we detected a pathogenic variant of c.458A > T in KRAS in this patient with atypical NS phenotype and provided appropriate clinical management and genetic counseling. The application of TES enables accurate molecular diagnosis of patients with nonspecific or atypical features in genetic diseases with several responsible genes, such as NS.
Congenital Abnormalities ; Cryptorchidism ; Diagnosis ; Exome* ; Genetic Counseling ; Genetic Heterogeneity ; Germ-Line Mutation ; Heart Defects, Congenital ; Humans ; Intellectual Disability ; Male ; Methods ; Molecular Biology ; Noonan Syndrome* ; Phenotype ; Protein Kinases ; Puberty, Delayed ; Signal Transduction ; Thoracic Wall ; Thorax ; Wills