Objective:To explore the construction of an assessment tool for the implementation of scenario-based simulation courses, and provide a reference for the standardized implementation of scenario-based simulation courses and the key direction of teacher training.Methods:Based on the elements of the implementation of scenario-based simulation courses, the Scenario-Based Simulation Course Implementation Quality Scale was developed through expert interviews and expert panel meetings. This scale was used to rate the implementation of scenario-based simulation courses by 12 teaching teams in Shanghai, China. The evaluation scores were collected and analyzed using R language.Results:The reliability of the evaluation scale was assessed using Cronbach's α (0.893) and inter-rater consistency was assessed using the intraclass correlation coefficient (0.226). Among the 12 teaching teams, the lowest score rate was 67.07% for "debriefing". A correlation analysis between scenario-based simulation teaching design and course implementation showed a Pearson correlation coefficient r of -0.08 ( P=0.809), which indicated no significant correlation between them. Conclusions:The assessment of the implementation of scenario-based simulation courses should include all aspects of course implementation. The rater's understanding of scenario-based simulation courses can affect the results. Debriefing remains a challenge in implementing scenario-based simulation courses. A comprehensive assessment of the teaching ability of simulation teaching teams should include both teaching design and course implementation.

Objective:To explore the construction of an assessment tool for the implementation of scenario-based simulation courses, and provide a reference for the standardized implementation of scenario-based simulation courses and the key direction of teacher training.Methods:Based on the elements of the implementation of scenario-based simulation courses, the Scenario-Based Simulation Course Implementation Quality Scale was developed through expert interviews and expert panel meetings. This scale was used to rate the implementation of scenario-based simulation courses by 12 teaching teams in Shanghai, China. The evaluation scores were collected and analyzed using R language.Results:The reliability of the evaluation scale was assessed using Cronbach's α (0.893) and inter-rater consistency was assessed using the intraclass correlation coefficient (0.226). Among the 12 teaching teams, the lowest score rate was 67.07% for "debriefing". A correlation analysis between scenario-based simulation teaching design and course implementation showed a Pearson correlation coefficient r of -0.08 ( P=0.809), which indicated no significant correlation between them. Conclusions:The assessment of the implementation of scenario-based simulation courses should include all aspects of course implementation. The rater's understanding of scenario-based simulation courses can affect the results. Debriefing remains a challenge in implementing scenario-based simulation courses. A comprehensive assessment of the teaching ability of simulation teaching teams should include both teaching design and course implementation.

ObjectiveTo analyze the scores of the second-tier assessment on theory of competency of Traditional Chinese Medicine （TCM） physicians based on cognitive diagnostic model， and to provide a reference for cognitive diagnostic studies of large-scale medical examinations. MethodsCognitive diagnostic analyses were conducted by selecting the responses of 16，310 candidates who took the theory examination of the second-tier assessment on the competency of TCM physicians in 2023. The degree of fit between the topological deterministic input noise and gate model （G-DINA model） and the examination data was examined； item parameter analysis and reliability analysis were conducted under the cognitive diagnostic framework； the candidates' mastery of cognitive attributes was comprehensively analysed at different levels. ResultsThe mean score of 16，310 candidates in this study was （189.76 ± 40.86）， and a total of 9，994 candidates passed this examination， with a pass rate of 61.28 %. The total number of questions in the examination was 300， and among the five modules， the frequency of assessment of TCM fundamentals was the highest （132 questions， 44%）， followed by TCM clinic （87 questions， 29%）， Western medicine fundamentals （60 questions， 20%）， and TCM classics （12 questions， 4%） and medical humanities （9 questions， 3%） was less. The results of the model fit index showed that the area under the curve （AUC） of the G-DINA model was 0.98， indicating that the model fit the study data excellently. The results of item parameter analysis showed that all the discriminations of the cognitive diagnostic model for the 300 test questions were greater than 0， with an average discrimination of 0.25， and the overall performance of the question discriminations was good. The results of the reliability analysis showed that the test level classification accuracy was 0.75， and the attribute level classification accuracy was 0.83 to 0.99， indicating that the G-DINA model has high attribute classification accuracy. The results of the cognitive diagnostic analyses showed that the proportion of candidates who had passed the exam was higher for the cognitive attributes of each module compared to those who had not passed the exam. ConclusionThe analysis of large-scale TCM examinations based on cognitive diagnosis can provide support for improving the quality of TCM talent education and cultivation.