The pattern of variability in the pass rate of the national examination for physicians' license of Japan was analyzed. The variation in the pass rate of a medical school in a given year was moderately dependent on the variation in the pass rate in the previous year. There was a moderate negative correlation (r =-0.454, n=1262) between the variations in the pass rates in consecutive years. We termed this phenomenon “the previous-year effect.” The previous-year effect was greatest (r =-0.494, n=475) when the average pass rate of all medical schools changed from decrease to increase, but was smaller (r =-0.393, n=551) when the average pass rate changed from increase to decrease. These phenomena can be explained by interactions between the factors of examinees and examiners.

With multiple-choice questions presenting one, two, or three of five choices (types A, K or X), the expected probability of a correct answer (P) can be obtained with a five-dimensional equation for the knowledge level (q), which is the probability of discriminating correct and incorrect items. From equations for each question type, we inversely estimated the q value (the estimated knowledge level), replacing P with the raw score rate in the actual examinations for promotion. The distribution and mean value of the raw score rate of type X differed from those for types A and K. However, distributions of estimated q values for the three question types were similar. This method can be used to estimate the actual knowledge level of students without the question type affecting the raw score but cannot be used to estimate incorrect knowledge.