1.Issues to conduct randomized controlled trials in medical education area
Hirotaka ONISHI ; Atsushi WATANABE ; Hirono ISHIKAWA ; Yasutomo ODA ; Sugimoto SUGIMOTO ; Rika MORIYA ; Motofumi YOSHIDA ; Takeshi MORIMOTO ; Akinobu YOSHIMURA ; Ryoko ASO ; Toshiro SHIMURA
Medical Education 2010;41(1):65-71
1) We conducted a randomized controlled trial in medical education area and explored practical issues through reflection on the processes.
2) In February 2007, 39 fourth-year medical students in Nippon Medical School listened to the lecture about how to ask key questions for the diagnosis. Shortly after they had medical interview with a standardized patient for measurement purpose. They were randomly allocated to study and control groups. The lecture content for the intervention group corresponded to the interview but the one for the control group did not correspond to the interview.
3) We identified the issues related with ethical review for research, how to mask the information of randomization out of assessors, and equity of educational intervention and assessment offered to both groups.
2.Results of a Survey on Clinical Competence to Be Evaluated by the National Physicians' License Examination.
Takao MORITA ; Masahiko HATAO ; Takeshi Aso ; Kensuke HARADA ; Nobuya HASHIMOTO ; Kimitaka KAGA ; Shunzo KOIZUMI ; Kei MATSUEDA ; Makiko OSAWA ; Toshikazu SAITO ; Hiroyuki TOYOKAWA ; Tsukasa TSUDA ; Motokazu HORI
Medical Education 1999;30(6):405-412
The clinical competence needed by every beginning resident and the present status of such competencewere examined in August 1998 through questionnaires distributed to clinical educators and the nursing staff of university hospitals and clinical training hospitals designated by the Ministry of Health and Welfare. Completed questionnaires were returned by 576 (65.9%) of clinical educators and nursing staff. With a cluster analysis of the necessity and the present status of clinical competence, 21 items for clinical competence were identified as those most requiring evaluation by the national examination. These 21 items included 11 items for clinical competence in the cognitive domain, 8 items in the psychomotor domain, and 2 in the affective domain. In about half of the direct answers obtained from clinical educators, evaluations were considered necessary for 15 items of clinical competence, of which 13 belonged to the cognitive domain. These results were consistent with the present status. However, practical examinations have also attracted increasing attention, as the results included strong demands that the national examination evaluate some basic clinical skills, such as physical examination and measurement of vital signs. However, about 30 % of authorities governing the national examination thought no changes are needed in the national examination.
3.Effect of Workshops for Preparing Multiple-Choice Questions for Computer-Based Testing Used in the Nationwide Common Examination Before Clinical Clerkships
Nobuo NARA ; Nobuhiko SAITO ; Shu KURAMOTO ; Eiji GOTOH ; Hiroaki NAKAJIMA ; Osamu FUKUSIMA ; Saburo HORIUCHI ; Toshimasa YOSHIOKA ; Yoshio NITTA ; Tatsuki ISHIDA ; Takeshi ASO ; Yasuichiro FUKUDA ; Fumimaro TAKAKU
Medical Education 2005;36(1):11-16
Computer-based testing (CBT) has been used in Japan since 2002 to assess medical students' basic and clinical medical knowledge, based on the model core-curriculum, before they start clinical clerkships. For effective CBT, multiplechoice questions must accurately assess the knowledge of students. Questions for CBT are submitted by all medical schools in Japan. However, only 40% of questions are chosen for CBT and used at random; the other 60% of questions are rejected because of poor quality. Toimprove the ability of medical staff to devise questions, workshops were held at 30 medical schools. The acceptance rate of questions from schools where workshops were held was significantly increased. The workshops were extremely effbctive for improving the quality of questions.
4.Statistical Analysis with the Item-Response Theory of the First Trial of the Computer-Based Nationwide Common Achievement Test in Medicine
Yoshio NITTA ; Shinichi MAEKAWA ; Takemi YANAGIMOTO ; Tadahiko MAEDA ; Motofumi YOSHIDA ; Nobuo NARA ; Tatsuki ISHIDA ; Osamu FUKUSHIMA ; Nobuhiko SAITO ; Yasuichiro FUKUDA ; Fumimaro TAKAKU ; Takeshi ASO
Medical Education 2005;36(1):3-9
Data from the first trial of the computer-based nationwide common achievement test in medicine, carried out from February through July in 2002, were analyzed to evaluate the applicability of the item-response theory. The trial test was designed to cover 6 areas of the core curriculum and included a total of 2791 items. For each area, 3 to 40 items were chosen randomly and administered to 5693 students in the fourth to sixth years; the responses of 5676 of these students were analyzed with specifically designed computer systems. Each student was presented with 100 items. The itemresponse patterns were analyzed with a 3-parameter logistic model (item discrimination, item difficulty, and guessing parameter). The main findings were: 1) Item difficulty and the percentage of correct answers were strongly correlated (r=-0.969to-0.982). 2) Item discrimination and the point-biserial correlation were moderately strongly correlated (r=0.304 to 0.511). 3) The estimated abilities and the percentage of correct answers were strongly correlated (r=0.810 to 0.945). 4) The mean ability increased with school year. 5) The correlation coefficients among the 6 curriculum area ability scores were less than 0.6. Because the nationwide common achievement test was designed to randomly present items to each student, the item-response theory can be used to adjust the differences among test sets. The first trial test was designed without considering the item-response theory, but the second trial test was administered with a design better suited for comparison. Results of an analysis of the second trial will be reported soon.
5.An Investigation into Admission Systems in Medical Departments of Universities in Japan
Atsuhiro HAYASHI ; Hidetoki ISHII ; Kei ITO ; Kumiko SHIINA ; Haruo YANAI ; Atsushi AKANE ; Takeshi ASO ; Jun'ichiro IWAHORI ; Chiyoko UCHIDA ; Masaru KAWASAKI ; Nobuhiko SAITO ; Ryuji TAKEDA
Medical Education 2006;37(5):285-291
A nationwide survey of Japanese teachers belonging to departments of medicine was conducted to investigate their opinions about: 1) expanding medical departments into medical schools and, 2) the required subjects and selection criteria for admitting students to the faculty of medicine. We found that responses to both questions depended largely on the specialty of the teachers. With regard to the medical school design, which will cause a dramatic change in medical education in Japan, 60% of the teachers were in favor of expanding departments into medical schools.