INTRODUCTIONOver the years, performance assessment (PA) has been widely employed in medical education, Objective Structured Clinical Examination (OSCE) being an excellent example. Typically, performance assessment involves multiple raters, and therefore, consistency among the scores provided by the auditors is a precondition to ensure the accuracy of the assessment. Inter-rater agreement and inter-rater reliability are two indices that are used to ensure such scoring consistency. This research primarily examined the relationship between inter-rater agreement and inter-rater reliability.

MATERIALS AND METHODSThis study used 3 sets of simulated data that was based on raters' evaluation of student performance to examine the relationship between inter-rater agreement and inter-rater reliability.

RESULTSData set 1 had high inter-rater agreement but low inter-rater reliability, data set 2 had high inter-rater reliability but low inter-rater agreement, and data set 3 had high inter-rater agreement and high inter-rater reliability.

CONCLUSIONInter-rater agreement and inter-rater reliability can but do not necessarily coexist. The presence of one does not guarantee that of the other. Inter-rater agreement and inter-rater reliability are both important for PA. The former shows stability of scores a student receives from different raters, while the latter shows the consistence of scores across different students from different raters.

Clinical Competence ; Curriculum ; Education, Medical ; Humans ; Psychometrics ; Reproducibility of Results ; Statistics as Topic ; Statistics, Nonparametric ; Students, Medical ; Task Performance and Analysis ; Validation Studies as Topic

INTRODUCTIONMany students, while performing clinical skills such as medical interviewing/ communication, physical examination, and procedural tasks, have never been observed by faculty members or residents. This study aimed to explore the relationships between final-year medical students' self-reported confidence and the frequency of direct observation by faculty member or resident while conducting these clinical skills.

MATERIALS AND METHODSMedical students at China Medical University in Taiwan participated in the survey. Before graduating, they were asked to answer a questionnaire about (1) their confidence in performing 17 clinical skills including medical interviewing/communication, physical examination, and procedural tasks, and (2) the number of times they had been directly observed by faculty members or residents during student-patient encounters.

RESULTSMany students reported never having been observed by a faculty member while they performed history taking/communication (46% to 84%), physical examination (36% to 42%), or procedural tasks (41% to 81%). It was found that residents had observed the students more frequently than the faculty members. The correlations between self-reported confidence and the corresponded direct observation were small to medium but significant. However, no difference was found between observation by a faculty member and by a resident.

CONCLUSIONSThis study confirmed that many medical students have not been directly observed in clinical training; and that those who were observed more often, expressed more self-reported confidence. Some assessment measures, which focus on direct observation and feedback during student-patient encounters, may improve the students' confidence.

Adult ; Clinical Competence ; standards ; Data Collection ; Female ; Humans ; Internship and Residency ; Male ; Observation ; Self Efficacy ; Students, Medical ; Taiwan

Adult ; Aged ; Aged, 80 and over ; Female ; Hepatitis B ; blood ; epidemiology ; Hepatitis B Surface Antigens ; blood ; Hepatitis C ; blood ; epidemiology ; Hepatitis C Antibodies ; blood ; Hospitals ; Humans ; Male ; Middle Aged ; Seroepidemiologic Studies ; Young Adult