Clinical reasoning is situation-dependent and case-specific; therefore, assessments incorporating different patient presentations are warranted. The present study aimed to determine the reliability of a multi-station case-based viva assessment of clinical reasoning in an Australian pre-registration osteopathy program using generalizability theory. Students (from years 4 and 5) and examiners were recruited from the osteopathy program at Southern Cross University, Lismore, Australia. The study took place on a single day in the student teaching clinic. Examiners were trained before the examination. Students were allocated to 1 of 3 rounds consisting of 5 10-minute stations in an objective structured clinical examination-style. Generalizability analysis was used to explore the reliability of the examination. Fifteen students and 5 faculty members participated in the study. The examination produced a generalizability coefficient of 0.53, with 18 stations required to achieve a generalizability coefficient of 0.80. The reliability estimations were acceptable and the psychometric findings related to the marking rubric and overall scores were acceptable; however, further work is required in examiner training and ensuring consistent case difficulty to improve the reliability of the examination.
Australia ; Humans ; Osteopathic Medicine ; Physical Examination ; Psychometrics ; Reproducibility of Results

PURPOSE: Peer assessment provides a framework for developing expected skills and receiving feedback appropriate to the learner's level. Near-peer (NP) assessment may elevate expectations and motivate learning. Feedback from peers and NPs may be a sustainable way to enhance student assessment feedback. This study analysed relationships among self, peer, NP, and faculty marking of an assessment and students' attitudes towards marking by those various groups. METHODS: A cross-sectional study design was used. Year 2 osteopathy students (n=86) were invited to perform self and peer assessments of a clinical history-taking and communication skills assessment. NPs and faculty also marked the assessment. Year 2 students also completed a questionnaire on their attitudes to peer/NP marking. Descriptive statistics and the Spearman rho coefficient were used to evaluate relationships across marker groups. RESULTS: Year 2 students (n=9), NPs (n=3), and faculty (n=5) were recruited. Correlations between self and peer (r=0.38) and self and faculty (r=0.43) marks were moderate. A weak correlation was observed between self and NP marks (r=0.25). Perceptions of peer and NP marking varied, with over half of the cohort suggesting that peer or NP assessments should not contribute to their grade. CONCLUSION: Framing peer and NP assessment as another feedback source may offer a sustainable method for enhancing feedback without overloading faculty resources. Multiple sources of feedback may assist in developing assessment literacy and calibrating students' self-assessment capability. The small number of students recruited suggests some acceptability of peer and NP assessment; however, further work is required to increase its acceptability.
Australia ; Cohort Studies ; Cross-Sectional Studies ; Educational Measurement ; Humans ; Learning ; Literacy ; Methods ; Osteopathic Medicine ; Peer Review ; Self-Assessment