Our institution has introduced the objective structured clinical examination as a means of objectively assessing the learning status of students. An annual task is to increase the reliability of assessment tables at medical interview stations. To increase the reliability of assessment tables, it might be necessary to increase the concordance among assessors. Thus, we examined the reproducibility of the assessment items.
1)With the cooperation of 4 training institutes for acupuncture and moxa cautery, medical interview examinations by 89 students were videotaped after they had heard a lecture on medical interviews, had been provided with information about the examination, and had given consent.
2)We asked 6 institute members with experience in assessing medical interview examinations to assess the same recorded videotaped interviews individually and in pairs. We asked them to assess the same video again after a certain period of time.
3)High concordance was observed for every item by single assessors but not for many items among assessors.
4)Single–assessor concordance was not necessarily correlated with that among assessors.
5)When agreement was not expected in advance, the assessors performed assessments on the basis of their own educational criteria, which may result in decreased concordance among assessors.