To evaluate the achievement of medical or dental education the criterion-referenced tests must be used. In this paper we proposed a new setting method of the difficulty level of the multiple-choice questions (MCQs) used in the National Board Examinations for Medical or Dental Practice. The difficulty levels should be determined by using the idea of the “Correct Terminal” which is consisting of the minimal sets of items to answer correctly. The difficulty level, the percentage of correct answer and the phi-coefficient of a question vary depending on the arrangement of the items in the question. Therefore, it is very difficult to get the real evaluation by the criterion-referenced measurement as far as we use MCQs. We should deepen our understanding of the limit of MCQ tests rather than we might expect the limitless possibility.

The 82 nd and 83 rd National Examination for Physicians' License, which were held in 1988 and 1989 respectively, were evaluated from question to question as well as in all the questions as a whole to set minimum pass scores and analysis “relevance” and “difficulty” in a matrix utilizing a modified Ebel's method.
The evaluators were teachers in different disciplines in nationwide medical schools and teaching hospitals and clinical trainees who had taken and passed the immediate past examinations.
Following data processing, the questionable and difficult questions were on the decrease compared with the preceding year, and it was tentatively concluded that the National Examinations have gradually improved year by year.