PURPOSE: Test equating studies in medical education have been conducted only for high-stake exams or to compare two tests given in a single course. Based on item response theory, we equated computer-based test (CBT) results from the basic medical education curriculum at the College of Medicine, the Catholic University of Korea and evaluated the validity of using fixed passing scores. METHODS: We collected 232 CBTs (28,636 items) for 40 courses administered over a study period of 9 years. The final data used for test equating included 12 pairs of tests. After test equating, Wilcoxon rank-sum tests were utilized to identify changes in item difficulty between previous tests and subsequent tests. Then, we identified gaps between equated passing scores and actual passing scores in subsequent tests through an observed-score equating method. RESULTS: The results of Wilcoxon rank-sum tests indicated that there were no significant differences in item difficulty distribution by year for seven pairs. In the other five pairs, however, the items were significantly more difficult in subsequent years than in previous years. Concerning the gaps between equated passing scores and actual passing scores, equated passing scores in 10 pairs were found to be lower than actual passing scores. In the other two pairs, equated passing scores were higher than actual passing scores. CONCLUSION: Our results suggest that the item difficulty distributions of tests taught in the same course during successive terms can differ significantly. It may therefore be problematic to use fixed passing scores without considering this possibility.
Curriculum ; Education, Medical ; Educational Measurement ; Korea ; Methods

INTRODUCTIONThis paper reports the relationship between the difficulty level and the discrimination power of true/false-type multiple-choice questions (MCQs) in a multidisciplinary paper for the para-clinical year of an undergraduate medical programme.

MATERIALS AND METHODSMCQ items in papers taken from Year II Parts A, B and C examinations for Sessions 2001/02, and Part B examinations for 2002/03 and 2003/04, were analysed to obtain their difficulty indices and discrimination indices. Each paper consisted of 250 true/false items (50 questions of 5 items each) on topics drawn from different disciplines. The questions were first constructed and vetted by the individual departments before being submitted to a central committee, where the final selection of the MCQs was made, based purely on the academic judgement of the committee.

RESULTSThere was a wide distribution of item difficulty indices in all the MCQ papers analysed. Furthermore, the relationship between the difficulty index (P) and discrimination index (D) of the MCQ items in a paper was not linear, but more dome-shaped. Maximal discrimination (D = 51% to 71%) occurred with moderately easy/difficult items (P = 40% to 74%). On average, about 38% of the MCQ items in each paper were "very easy" (P > or =75%), while about 9% were "very difficult" (P <25%). About two-thirds of these very easy/difficult items had "very poor" or even negative discrimination (D < or =20%).

CONCLUSIONSMCQ items that demonstrate good discriminating potential tend to be moderately difficult items, and the moderately-to-very difficult items are more likely to show negative discrimination. There is a need to evaluate the effectiveness of our MCQ items.

Education, Medical ; standards ; Educational Measurement ; methods ; Humans ; Taiwan

PURPOSE: This study aims to develop a system of course monitoring by students and evaluate the course quality management system (CQMS) implemented as an educational assessment tool. METHODS: This research was conducted in accordance with the ADDIE model which is a well-known instructional design model. The ADDIE process includes needs analysis, design of the course monitoring and course evaluation, development of evaluation forms, implementation of course monitoring, and evaluation of the program. RESULTS: To meet the need for a system that can replace the traditional lecture evaluation approach, this study developed and implemented a new course evaluation system. In comparison with the quantitative evaluation method, course monitoring by students provided more qualitative information on classes and courses from the students' perspective. The students' realistic description helped know how student felt the atmosphere of class and what kinds of teaching style students preferred. However, some view that the evaluation by the four monitoring members was less reliable. CONCLUSION: This study proposed a course quality management based on students' monitoring which emphasized the narrative evaluation to help to identify the strengths and weaknesses of the classes and gather qualitative information from the students' perspective that can be used to improve the courses. It is expected that providing the monitoring members with better orientation could help manage the quality of the courses using the monitoring system.
Atmosphere ; Educational Measurement ; Evaluation Studies as Topic ; Humans ; Methods ; Schools, Medical*

INTRODUCTIONMany different strategies exist to try and encourage students to increase their knowledge and understanding of a subject. This study was undertaken to measure the effect of student-based construction of multiple choice questions (MCQs) as a stimulus for the learning and understanding of topics in clinical surgery.

MATERIALS AND METHODSThe study was carried out at the University of Adelaide, Australia and had 2 components. Fourth-year students were required to provide a case study during a surgical attachment and half of the group was asked to supplement this with MCQs. These students were pre- and post-tested and the effect of the additional intervention (MCQ-construction) measured. Fifth-year students were polled on their preferred methods of learning before and after a learning exercise in which they were asked to undertake a case presentation and create some MCQs.

RESULTSThe MCQ questions designed by the students were of a high standard and clearly displayed an understanding of the topic concerned. The 4th-year students in the MCQ construction group showed equivalent outcomes as the case study control group. Students initially ranked MCQ-construction amongst the least stimulating methods of learning, but after the exercise their opinion was significantly more favourable, although still much less than traditional learning methodologies (tutorials, books).

CONCLUSIONSConstruction of MCQs as a learning tool is an unfamiliar exercise to most students and is an unpopular learning strategy. However, students are capable of producing high quality questions, and the challenge for medical faculties is how best to use this initiative to the students' advantage.

Argentina ; Developing Countries ; Education, Medical, Undergraduate ; methods ; Educational Measurement ; methods ; Humans ; Problem-Based Learning

Clinical Competence ; standards ; Education, Medical ; methods ; standards ; Educational Measurement ; methods ; Humans

In 1995, the Colombo Medical Faculty changed its curriculum from a traditional model to an integrated one. The major challenge to the Faculty was obtaining students' feedback on their learning activities. To overcome this, a new method where staff and student groups from different years of study engage in an interactive discussion relating to their learning environment was developed. This feedback was then processed and forwarded to the relevant authorities for necessary action.
Curriculum ; standards ; Education, Medical ; methods ; Educational Measurement ; methods ; Feedback ; Humans ; Students, Medical ; Surveys and Questionnaires

BACKGROUNDComputer-based testing (CBT) has become increasingly popular as a testing modality in under- and postgraduate medical education. Since 2004, our medical school has utilised CBT to conduct 2 papers for the third- and final-year assessments - Paper 3, with 30 multiple choice questions featuring clinical vignettes, and the modified essay question (MEQ) paper.

AIMSTo obtain feedback from final-year students on their preferred mode of testing for Paper 3 and MEQ components of the Medicine track examination, and the reasons underlying their preferences.

METHODSAn online survey was carried out on 213 final-year undergraduates, in which they were asked to provide feedback on Paper 3 and MEQ papers. Students were asked if they thought that the CBT format was preferable to the pen-and-paper (PNP) format for Paper 3 and the MEQ, and why.

RESULTSOne hundred and fourteen out of 213 (53.5%) students completed the online survey. For Paper 3, 91 (79.8%) felt that CBT was preferable to PNP, 11 (9.6%) preferred the PNP format and 12 (10.5%) were unsure. For the MEQ, 62 (54.4%) preferred CBT over PNP, 30 (26.3%) preferred the PNP format and 22 (19.3%) were unsure. Reasons given to explain preference for CBT over PNP for Paper 3 included independence from seating position, better image quality (as images were shown on personal computer screens instead of projected onto a common screen) and the fact that CBT allowed them to proceed at their own pace. For the MEQ, better image quality, neater answer scripts and better indication of answer length in CBT format were cited as reasons for their preference.

CONCLUSIONSOur survey indicated that whereas the majority of students preferred CBT over PNP for Paper 3, a smaller margin had the same preference for the MEQ.

INTRODUCTIONThe Thai medical students sat for the Medical Licensing Examination of Thailand (MLET) Step 2 for the first time in 2008. This paper analysed the first batch of Ramathibodi students taking the MLET Steps 1 and 2 in 2006 and 2008 respectively.

MATERIALS AND METHODSThe scores from the MLET Steps1 and 2, and fifth-year cumulative grade point averages (GPAX) of 108 students were analysed.

RESULTSOnly 6 (5.6%) students failed the MLET Step 2 examination. Students who failed the MLET Step1 were more likely to fail their MLET Step 2 (relative risk, 5.8; 95% confidence interval, 1.3-26.0). Students with low GPAX or scoring in the lowest quintile or tertile on the MLET Step1 were also at increased risk of failing the LET Step 2.

CONCLUSIONSThe data suggest that performance on the MLET Step 1 and GPAX are important predictors of a student's chances of passing the MLET Step 2. Students with poor academic achievement or failing the MLET Step1 should be given intensive tutorials to pass the medical licensing examination.