Recent advancements in artificial intelligence have led to increased interest in predictive modeling across various domains, including medicine. Although numerous metrics have been established for binary classification, the growing adoption of multi-class systems necessitates robust evaluation measures. However, comprehensive simulation studies comparing the performance of existing multi-class metrics under diverse data conditions remain limited. In this study, we first provide a concise overview of commonly used accuracy metrics for multi-class classification. Then, we report a simulation study that systematically evaluates several diagnostic accuracy measures under a wide range of scenarios, including three- and five-class settings, balanced and imbalanced sample sizes, and different distributional assumptions for predictors. We assessed each metric’s performance in terms of bias and 95% confidence interval coverage. Under balanced conditions, most metrics demonstrated stable and unbiased performance, closely approximating the true values. However, under imbalanced conditions, greater bias was observed, with the M-index and polytomous discrimination index exhibiting comparatively more stable performance across various scenarios. The micro-averaged receiver operating characteristic curve area consistently showed higher bias under class imbalance. Finally, we applied these metrics to a glioma tumor grading task using external datasets. This study provides a systematic comparison of commonly used metrics and offers practical guidance for selecting appropriate measures in multi-class classification tasks.

Deep learning has recently achieved remarkable results in the field of medical imaging. However, as a deep learning network becomes deeper to improve its performance, it becomes more difficult to interpret the processes within. This can especially be a critical problem in medical fields where diagnostic decisions are directly related to a patient's survival. In order to solve this, explainable artificial intelligence techniques are being widely studied, and an attention mechanism was developed as part of this approach. In this paper, attention techniques are divided into two types: post hoc attention, which aims to analyze a network that has already been trained, and trainable attention, which further improves network performance. Detailed comparisons of each method, examples of applications in medical imaging, and future perspectives will be covered.

Deep learning has recently achieved remarkable results in the field of medical imaging. However, as a deep learning network becomes deeper to improve its performance, it becomes more difficult to interpret the processes within. This can especially be a critical problem in medical fields where diagnostic decisions are directly related to a patient's survival. In order to solve this, explainable artificial intelligence techniques are being widely studied, and an attention mechanism was developed as part of this approach. In this paper, attention techniques are divided into two types: post hoc attention, which aims to analyze a network that has already been trained, and trainable attention, which further improves network performance. Detailed comparisons of each method, examples of applications in medical imaging, and future perspectives will be covered.