PURPOSE: This study was aimed to investigate the possibility of utilizing smart device-based test (SBT) for competency evaluation in dental education and to analyze the student responses on overall competency evaluation using SBT method, in comparison to ubiquitous-based test (UBT). MATERIALS AND METHODS: Questionnaire surveys have been conducted at Yonsei University College of Dentistry from 2015 to 2018 to obtain students' feedback on the application of SBT to competency evaluation. In addition, in order to supplement the competency evaluation procedure, considerations were explored by comparing the expected and actual difficulty of each item when preparing items for competency evaluation with SBT.RESULT: According to the survey results, student responses between the initial two years (2015 and 2016) differed from those in next two years (2017 and 2018). Students in 2017 and 2018 had more positive responses on competency evaluation with SBT. To determine the test validity, criterion-referenced evaluation was adopted to compare the data in 2017 and 2018 and slight differences in test difficulty in 2018 between the expected and actual difficulty of items were found. CONCLUSION The results indicated that SBT was more appropriate for competency evaluation than UBT, based on four-year period of competency evaluation. The SBT was not affected by either the file size or the number of test-takers. Interestingly, students were not sensitive to test version of competency evaluation (paper-based test and SBT). This study suggests that the quality of the test items should be measured by continuous monitoring of the expected and actual difficulty of items for determining test validity. More detailed results and discussions of the findings are given for the development of test procedure and further potential research directions in dental education.

Purpose: The practical examination portion of the National Dental Licensing Examination (NDLE) is slated to be administered in the latter half of 2021 in the form of a clinical performance examination that comprehensively evaluates the patient-dentist interaction using standardized patients (SPs). The SPs should be equipped with the basic qualities and capacity as evaluators for a fair and reliable administration of the test. Materials and Methods: In this study, we analyzed the existing training materials for SPs who participated in domestic and overseas practical tests for the development of training materials for SPs through seminars and surveys of 11 dentistry schools and colleges.Result: First, SPs should be selected according to the basic quality criteria and capacity, which they must possess, and the preliminary basic training about the details which they must have knowledge of and be provided through videorecorded cases before the implementation of the preliminary field training. Second, the roles of SPs and the calibration process of the evaluation result forms are needed when conducting the preliminary field training for SPs. After watching video-recorded scenario cases, the SPs participate in discussions about the watched videos before proceeding to calibration practices of evaluation result forms. Third, because the Type A questionnaire of the practical examination of the NDLE is dependent on the SPs’ capacity and training, the fairness of the practical test is largely dependent on the SPs. Therefore, practicing the roles as evaluators and evaluation training should be provided using practical test items that can improve the reliability of the test and show a high level of reproducibility about the same case. Conclusion The findings of this study will be utilized for the development of training materials for SPs, so they can participate in the administration of a fair and reliable practical examination of the NDLE.