INTRODUCTIONPlanning a high-stake clinical examination requires the evaluation of several psychometric and logistical variables. The authors conducted generalisability and decision studies to answer the following research questions in the context of the surgical long case: (1) Does the addition of a third examiner have any added benefit, vis-à-vis reliability, to the examination? (2) Is global marking more reliable than an itemised marking template? (3) What would be the impact on reliability if there was a reduction in the number of examinees that each panel of examiners is required to assess?

METHODSA third examiner and global marking were introduced. Separate generalisability and decision studies were carried out for both the two- and three-examiner models as well as for itemised and global scores.

RESULTSThe introduction of a third examiner resulted in a modest gain of reliability by 0.05-0.07. Gain in reliability was higher when each candidate was allowed to undertake a higher number of clinical cases. Both the global and itemised scores provided equivalent reliability (generalisability coefficient 0.74-0.89).

CONCLUSIONOur results showed that only a modest improvement in reliability of the surgical long case is achieved through the introduction of an additional examiner. Although the reliability of global scoring and the itemised marking template was comparable, the latter may provide opportunities for individualised feedback to examinees.

Clinical Competence ; Education, Medical, Undergraduate ; methods ; standards ; Educational Measurement ; methods ; Humans ; Medical History Taking ; methods ; Observation ; Professional-Patient Relations ; Psychometrics ; methods ; Reproducibility of Results ; Schools, Medical ; Singapore

BACKGROUND/AIMS: There is an increased risk for inflammatory bowel disease (IBD) patients to develop infections due to the use of immunomodulators and biologics. Several infections are preventable by immunizations. This study investigated the knowledge and awareness of Korean gastroenterologists regarding the vaccination of patients with IBD. METHODS: A self-reported questionnaire was sent by e-mail to the faculty members of tertiary hospitals. Gastroenterologists were asked ten questions regarding the immunization of patients with IBD. A total of 56 gastroenterologists completed the questionnaire. RESULTS: A majority of gastroenterologists (>60%) had rarely or never recorded an immunization history from their patients with IBD. Moreover, 50% to 70% of the gastroenterologists did not know that live vaccines should be avoided in immunosuppressed patients. The most commonly mentioned resistance to vaccinations was "the lack of concern and knowledge regarding vaccination." Gastroenterologists more frequently asked about the immunization history of influenza, pneumococcal, hepatitis A, and hepatitis B vaccines and recommended these vaccines more often than others. CONCLUSIONS: Korean gastroenterologists' awareness and knowledge regarding the vaccination of patients with IBD were very poor. Intensive educational programs on immunization guidelines directed toward gastroenterologists who care for patients with IBD are required to ensure that these patients receive the necessary vaccinations.
Clinical Competence/*standards ; Female ; Gastroenterology/*standards ; Health Knowledge, Attitudes, Practice ; Humans ; Immunocompetence/physiology ; Immunocompromised Host/physiology ; Inflammatory Bowel Diseases/*complications ; Male ; Medical History Taking/standards ; Questionnaires ; Republic of Korea ; Vaccination/*standards

PURPOSE: The purpose of this study was to test the validity of a modified clinical performance examination (CPX) for preclinical students in nursing. METHOD: 70 nursing students in their second semester of the junior year at C University participated in CPX. Scenarios and checklists were developed by our research team from September to October 2005. Six stations were organized. Evaluation included physical examination of a patient with lung cancer, education on usage of a metered dosage inhaler, and lobectomy postoperative care. Students were randomly assigned to a station. RESULT: There was a difference in the CPX scores according to stations. The agreement of scoring between trained faculty members and SPs was more than moderate (r=.647). The correlation between the CPX score and the average grade in the previous semester and between the CPX score and the average grade of a paper and pen test of the pulmonary system of adults was low (r=.276; r=.048). CONCLUSION: Traditional CPX is generally recommended, however, modified CPX is appropriate for preclinical students in the current Korean Nursing school setting if there are additional scoring systems to balance the testing level at each station.
Administration, Inhalation ; Adult ; Education, Nursing, Baccalaureate/*standards ; Educational Measurement ; Female ; Humans ; Lung Neoplasms/nursing ; Male ; Medical History Taking ; Physical Examination ; Postoperative Care ; Reproducibility of Results ; Task Performance and Analysis

PURPOSE: The purpose of this study is to investigate the reliability and validity of new clinical performance examination (CPX) for assessing clinical reasoning skills and evaluating clinical reasoning ability of the students. METHODS: Third-year medical school students (n=313) in Busan-Gyeongnam consortium in 2014 were included in the study. One of 12 stations was developed to assess clinical reasoning abilities. The scenario and checklists of the station were revised by six experts. Chief complaint of the case was rhinorrhea, accompanied by fever, headache, and vomiting. Checklists focused on identifying of the main problem and systematic approach to the problem. Students interviewed the patient and recorded subjective and objective findings, assessments, plans (SOAP) note for 15 minutes. Two professors assessed students simultaneously. We performed statistical analysis on their scores and survey. RESULTS: The Cronbach α of subject station was 0.878 and Cohen κ coefficient between graders was 0.785. Students agreed on CPX as an adequate tool to evaluate students' performance, but some graders argued that the CPX failed to secure its validity due to their lack of understanding the case. One hundred eight students (34.5%) identified essential problem early and only 58 (18.5%) performed systematic history taking and physical examination. One hundred seventy-three of them (55.3%) communicated correct diagnosis with the patient. Most of them had trouble in writing SOAP notes. CONCLUSION: To gain reliability and validity, interrater agreement should be secured. Students' clinical reasoning skills were not enough. Students need to be trained on problem identification, reasoning skills and accurate record-keeping.
Checklist ; *Clinical Competence ; Communication ; Comprehension ; *Education, Medical, Undergraduate ; Educational Measurement/*standards ; Humans ; Medical History Taking ; Medical Records ; Observer Variation ; Physical Examination ; Physician-Patient Relations ; *Problem-Based Learning ; Reproducibility of Results ; Republic of Korea ; *Schools, Medical ; *Students, Medical ; Surveys and Questionnaires ; *Thinking ; Universities