PURPOSE: This study examined the use of the Tucker linear equating method in producing an individual student's score in 3 groups with bridging stations over 3 consecutive days of the clinical performance examination (CPX) and compared the differences in scoring patterns by bridging number. METHODS: Data were drawn from 88 examinees from 3 different CPX groups-DAY1, DAY2, and DAY3-each of which comprised of 6 stations. Each group had 3 common stations, and each group had 2 or 3 stations that differed from other groups. DAY1 and DAY3 were equated to DAY2. Equated mean scores and standard deviations were compared with the originals. DAY1 and DAY3 were equated again, and the differences in scores (equated score-raw score) were compared between the 3 sets of equated scores. RESULTS: By equating to DAY2, DAY1 decreased in mean score from 58.188 to 56.549 and in standard deviation from 4.991 to 5.046, and DAY3 fell in mean score from 58.351 to 58.057 and in standard deviation from 5.546 to 5.856, which demonstrates that the scores of examinees in DAY1 and DAY2 were accentuated after use of the equation. The patterns in score differences between the equated sets to DAY1, DAY2, and DAY3 yielded information on the soundness of the equating results from individual and overall comparisons. CONCLUSION: To generate equated scores between 3 groups on 3 consecutive days of the CPX, we applied the Tucker linear equating method. We also present a method of equating reciprocal days to the anchoring day as much as bridging stations.
Clinical Competence ; Educational Measurement

Medical specialty systems were launched in 1951 by the National Medical Services Law. The following year, the specialty certifying examination had implemented in the form of portfolio evaluation. A paper-and-pencil type examination was implemented in 1960, and the 55th examination was carried out in January 2012. Currently, 26 specialties are represented, and the overall pass rate is over 90%. The examination consists of a step 1 paper-and-pencil test and step 2 skills test. In the step 1 test, the test items are multiple choice questions and short answer questions. Clinical performance examination is partially applied to the step 2 test. To cope with changes in the social situation and the growth of medical services, developmental changes are needed in the specialty certifying examinations. Performance assessment is an alternative worth considering. CPX should be a major part of the skill test. A computer-based test should be introduced as soon as possible, and it could eventually be developed into an adaptive test.
Educational Measurement ; Jurisprudence ; Phosphatidylethanolamines

PURPOSE: Teaching evaluation tools can be used to provide feedback to medical educators and help them improve their teaching skills. The purpose of this study was to develop a clinical teaching evaluation tool and test its efficacy. METHODS: The draft tool was developed based on clinical education literature and was modified by the stakeholder groups. To examine the quality of the instrument, content and construct validity, as well as reliability and fitness were analyzed. Content validity was tested by the stakeholder groups using a survey, and construct validity was verified by confirmatory factor analysis using LISREL 8.8. Internal consistency of items was assessed thorough Cronbach alpha estimation. Rasch analysis using Winstep 3.65 was performed to estimate the fitness of the tool. RESULTS: The resulting tool consisted of 4 large categories, 25 small categories, and 43 items. According to the test results, the average importance of all 43 items was 4.03 (3.63 to 4.29). Cronbach alpha was 0.9689, and the correlation coefficients between the items were high. With regard to construct validity, 10 items needed minor modifications in the category setting. As the infit (0.76 to 1.23) and outfit (0.75 to 1.40) indices show, 42 items were fit to the item response theory. CONCLUSION: The clinical teaching evaluation tool that has been developed in this study is valid and reliable and fits the item response theory. It can be used as an evaluation method in a variety of clinical teaching settings.
Educational Measurement ; Program Evaluation

Dentistry ; Educational Measurement ; Licensure, Dental ; Prosthodontics ; education

PURPOSE: The purpose of this study is to analyze the development and implementation of the Objective Structured Clinical Examination (OSCE) for the residency applicant examination at Daegu-Catholic Hospital. METHODS: Fifty-four Daegu-Catholic residency applicants were evaluated by written examination, internship scores, and OSCE. The correlation between written examination scores, internship scores, and OSCE scores was assessed. RESULTS: The correlation between OSCE and internship scores showed a tendency to be more significant than that between the written test and internship scores, but OSCE-internship correlation coefficients and written test and internship correlation coefficients was not statistically significant. The distribution of OSCE and internship scores on a graph corroborated this relationship between the two variables. CONCLUSION: The OSCE presents more objective criteria for residency application tests.
Clinical Competence ; Educational Measurement ; Internship and Residency

Formative assessment aims at cultivating and improving students' comprehensive qualities and abilities. It acquires and analyzes the dynamic changes of students' learning process through flexible and diverse assessment methods. The assessment contents cover multiple aspects including knowledge, abilities, attitudes, literacy, etc., which help teachers and students to formulate practical and efficient improvement strategies and to meet the inherent needs of students for comprehensive development. This review describes the connotation of formative assessment and summarizes its application in higher education of basic medical science in China. The existing shortcomings and the reasonable advice are proposed. We also introduce our experience in the application of formative assessment in pathophysiology education. This review will provide reference and enlightenment for the reform and innovation of the assessment system in higher education of basic medical science.
China ; Education, Medical ; Educational Measurement ; Humans

PURPOSE: Test equating studies in medical education have been conducted only for high-stake exams or to compare two tests given in a single course. Based on item response theory, we equated computer-based test (CBT) results from the basic medical education curriculum at the College of Medicine, the Catholic University of Korea and evaluated the validity of using fixed passing scores. METHODS: We collected 232 CBTs (28,636 items) for 40 courses administered over a study period of 9 years. The final data used for test equating included 12 pairs of tests. After test equating, Wilcoxon rank-sum tests were utilized to identify changes in item difficulty between previous tests and subsequent tests. Then, we identified gaps between equated passing scores and actual passing scores in subsequent tests through an observed-score equating method. RESULTS: The results of Wilcoxon rank-sum tests indicated that there were no significant differences in item difficulty distribution by year for seven pairs. In the other five pairs, however, the items were significantly more difficult in subsequent years than in previous years. Concerning the gaps between equated passing scores and actual passing scores, equated passing scores in 10 pairs were found to be lower than actual passing scores. In the other two pairs, equated passing scores were higher than actual passing scores. CONCLUSION: Our results suggest that the item difficulty distributions of tests taught in the same course during successive terms can differ significantly. It may therefore be problematic to use fixed passing scores without considering this possibility.
Curriculum ; Education, Medical ; Educational Measurement ; Korea ; Methods

PURPOSE: Evaluation and feedback are fundamental components in a residency program. A portfolio assessment system is an appropriate option for making formative assessments, particularly for a family medicine resident whose training includes rotations through various disciplines at multiple locations. However, a paper-based portfolio assessment system has limitations when applied to a clinical environment due to how it is managed and cared. This study assessed the satisfaction and utility of a paper-based portfolio assessment system, and then designed and implemented a web-based portfolio assessment system for family medicine residents. METHODS: A self-administered questionnaire focusing on the satisfaction and utility of a paperbased portfolio assessment system was given to 23 family medicine residents. A web-based portfolio assessment system was then designed using the component-based development approach. RESULTS: The overall satisfaction and utility with the paper-based portfolio assessment system were low. Fifteen residents (62.5%) reported that following a paper-based portfolio assessment system was sometimes a waste of time and only three (13.0%) reported having no problems. The web-based portfolio assessment system was successfully designed to be easy to implement and user-friendly. CONCLUSION: The web-based portfolio assessment system is expected to overcome the shortcomings of a paper-based portfolio assessment system and improve the level of satisfaction of medical residents. It is expected that the web-based portfolio assessment system designed in this study will be an additional tool for family medicine residencies.
Education ; Educational Measurement ; Humans ; Internship and Residency* ; Korea* ; Surveys and Questionnaires

PURPOSE: The aim of this research was to compare different methods of calibrating multiple choice question (MCQ) and clinical decision making (CDM) components for the Medical Council of Canada's Qualifying Examination Part I (MCCQEI) based on item response theory. METHODS: Our data consisted of test results from 8,213 first time applicants to MCCQEI in spring and fall 2010 and 2011 test administrations. The data set contained several thousand multiple choice items and several hundred CDM cases. Four dichotomous calibrations were run using BILOG-MG 3.0. All 3 mixed item format (dichotomous MCQ responses and polytomous CDM case scores) calibrations were conducted using PARSCALE 4. RESULTS: The 2-PL model had identical numbers of items with chi-square values at or below a Type I error rate of 0.01 (83/3,499 or 0.02). In all 3 polytomous models, whether the MCQs were either anchored or concurrently run with the CDM cases, results suggest very poor fit. All IRT abilities estimated from dichotomous calibration designs correlated very highly with each other. IRT-based pass-fail rates were extremely similar, not only across calibration designs and methods, but also with regard to the actual reported decision to candidates. The largest difference noted in pass rates was 4.78%, which occurred between the mixed format concurrent 2-PL graded response model (pass rate= 80.43%) and the dichotomous anchored 1-PL calibrations (pass rate= 85.21%). CONCLUSION: Simpler calibration designs with dichotomized items should be implemented. The dichotomous calibrations provided better fit of the item response matrix than more complex, polytomous calibrations.
Calibration ; Canada ; Clinical Decision-Making ; Dataset ; Educational Measurement ; Licensure

PURPOSE: The purpose of this research is to evaluate whether quizzes using moodle are useful for academic achievement in a medical English class and to introduce moodle to educators based on the author's teaching experience. METHODS: After a final examination in a medical English class, the author surveyed (scale of 1 low to 5 high) the degree of satisfaction of students and the usefulness of quizzes provided on the author's homepage using moodle. Students had been recommended to solve the quizzes on the homepage voluntarily. The author analyzed statistical differences of the final examination scores between the students who completed the quizzes and those who did not. RESULTS: A total of 59 students completed the survey (collection rate=81.9%). On the question of satisfaction about the medical English class and the question of usefulness of quizzes, scores of mean, maximum, and minimum were 4.29 (SD=0.56), 5, and 3, and 4.03 (SD=0.72), 5, and 2, respectively. Statistically significant differences in the final examination scores were observed between the students who completed quizzes and those who did not. CONCLUSION: A tool for students' self-directed study is needed for improving academic achievement. In particular, various educational programs and environments provided by moodle are thought to be very useful. The quizzes the author made with moodle were very effective in the aspect of achievement.
Achievement ; Computer-Assisted Instruction ; Educational Measurement ; Humans ; Programmed Instruction as Topic