Twelve practical recommendations for enhancing the reliability of medical simulation-based assessment
10.3760/cma.j.cn116021-20230503-02099
- VernacularTitle:提升医学模拟评价信度的12项实用建议
- Author:
Li LI
1
;
Jihai LIU
;
Yadian XIE
;
Di SHI
;
Xiaoyun HU
;
Zhiqiao CHEN
;
Haiyan AN
Author Information
1. 武汉大学中南医院重症医学科/湖北省重症医学临床医学研究中心，武汉 430071
- Publication Type:Journal Article
- Keywords:
Medical simulation;
Assessment;
Reliability;
Quality control
- From:
Chinese Journal of Medical Education Research
2025;24(7):865-871
- CountryChina
- Language:Chinese
-
Abstract:
Medical simulation-based assessments have emerged as a vital method for evaluating the clinical competence of healthcare professionals. These assessments provide examinees with opportunities to practice clinical skills in a controlled environment that mirrors real-life medical scenarios. However, assessment organizers must implement specific measures to ensure that these assessments yield valid and reliable results. The reliability of simulation-based assessments is a crucial component of quality assurance. This article delineates twelve practical recommendations to enhance the reliability of medical simulation-based assessments across the three key dimensions of raters, simulation environment, and assessment tools. These recommendations include ① selecting raters with relevant professional backgrounds and extensive experience in simulation-based teaching and assessment; ②providing raters with standardized training for the assessment; ③conducting mock ratings and piloting to reach consensus among raters; ④preventing excessive work hours or cognitively overload while rating; ⑤providing standardized personnel with adequate training and re-training; ⑥implementing immediate and long-term quality assurance measures for standardized patient role performance; ⑦maintaining consistent operation of simulation equipment; ⑧guaranteeing the simulation provides adequate functional fidelity for the examinees; ⑨clearly defining each element and item in assessment tools while controlling their quantity; ⑩ensuring consistency when multiple assessment tools are used; 11using assessment tools with evidence of construct validity when possible; and 12establishing systems to ensure consistency in data (scoring) collection while safeguarding data security. These strategies aim to assist medical educators in developing structured and reliable simulation-based assessment schemes.
