Recent advancements in artificial intelligence have led to increased interest in predictive modeling across various domains, including medicine. Although numerous metrics have been established for binary classification, the growing adoption of multi-class systems necessitates robust evaluation measures. However, comprehensive simulation studies comparing the performance of existing multi-class metrics under diverse data conditions remain limited. In this study, we first provide a concise overview of commonly used accuracy metrics for multi-class classification. Then, we report a simulation study that systematically evaluates several diagnostic accuracy measures under a wide range of scenarios, including three- and five-class settings, balanced and imbalanced sample sizes, and different distributional assumptions for predictors. We assessed each metric’s performance in terms of bias and 95% confidence interval coverage. Under balanced conditions, most metrics demonstrated stable and unbiased performance, closely approximating the true values. However, under imbalanced conditions, greater bias was observed, with the M-index and polytomous discrimination index exhibiting comparatively more stable performance across various scenarios. The micro-averaged receiver operating characteristic curve area consistently showed higher bias under class imbalance. Finally, we applied these metrics to a glioma tumor grading task using external datasets. This study provides a systematic comparison of commonly used metrics and offers practical guidance for selecting appropriate measures in multi-class classification tasks.

Purpose: This study aimed to investigate the effect of respiratory motion on hepatic steatosis quantification using ultrasound attenuation imaging (ATI) or ultrasound-guided attenuation parameter (UGAP) in pediatric patients. Materials and Methods: Pediatric patients (aged ≤18 years) who underwent liver ultrasonography (US) with ATI or UGAP between May 2022 and February 2023 were included retrospectively. Median, interquartile range (IQR), and IQR/median values were calculated in both free-breathing (FB) and breath-holding (BH) states. Subjects were divided into normal and fatty liver groups according to grayscale US. Wilcoxon signed rank test, intraclass correlation coefficient (ICC), and linear regression test were used. Results: A total of 83 patients (M:F=46:37, median age 10 years, range 6–17 years) was included, with 55 patients in the ATI group and 28 patients in the UGAP group. The measured values of ATI and UGAP were not significantly different between FB and BH.The ICC values between FB and BH states were 0.950 [95% confidence interval (CI) 0.916–0.971] for median ATI and 0.786 (95% CI 0.591–0.894) for median UGAP. FB and BH status did not significantly affect the median ATI and UGAP (p=0.852, 0.531, respectively). The IQR/median value showed a significant association with age only in the FB status of the normal group using ATI (β= -0.014, p=0.042). Conclusion Respiratory motion does not significantly affect the measurement of ATI or UGAP. Median ATI value showed excellent agreement in FB and BH status, while UGAP showed good agreement. Younger age may affect measurement variability in FB status of the normal group using ATI.