BACKGROUND

Due to the COVID-19 outbreak, schools had to switch online. The sudden transition to blended teaching and learning (BTL) poses challenges for students and teachers, especially for health science programs that require hands-on practical experience. The validity, reliability, and responsiveness of these self-report questionnaires (SRQs) should be established to ensure the accuracy of the results as intended by the SRQ.

OBJECTIVES

This study critically appraised, compared, and summarized the psychometric properties of SRQ evaluating BTL among health science university students. This review determined the SRQ’s reliability, internal consistency, various forms of validity (content, criterion, construct), and responsiveness.

METHODS

Following a 10-step procedure based on COSMIN guidelines, we conducted a systematic review of SRQs used by health science university students to evaluate blended teaching and learning. Studies were eligible if they reported psychometric properties of SRQs related to blended learning among university health science students; exclusions included studies focusing on perceptions, attitudes, self-efficacy, and satisfaction, as well as articles such as biographies, editorials, and conference materials. Searches covered multiple electronic databases until April 26, 2023, including PubMed, EMBASE, Web of Science, MEDLINE (OVID), PsycInfo, CINAHL, EBSCOHOST, ERIC, Scopus, Science Direct, Google Scholar, JSTOR, Acta Medica Philippina, Philippine Journal of Health Research and Development, and HERDIN, managed through Zotero. Two independent reviewers performed database searches, title and abstract screening, and full-text evaluations, with a third reviewer resolving any disputes. The COSMIN Risk of Bias Checklist was employed to evaluate included studies on the development and various measurement properties of SRQs. The reviewers assessed SRQ standards, including validity, reliability, internal consistency, measurement error, responsiveness, interpretability, and feasibility. Data extraction and result tabulation were independently completed, with content comparison by two health education experts. This evaluation categorized the SRQs into three quality and validity levels.

RESULTS

The study examined five articles; four were rated as 'doubtful' and one as 'inadequate' in the overall development of SRQ. All four 'doubtful' studies demonstrated questionable content validity when university students were asked about the questionnaire's relevance, comprehensiveness, and comprehensibility. Only half of these studies achieved an 'adequate' rating for content validity based on expert opinions on relevance and comprehensiveness. All but one study scored from 'very good' to 'adequate' in structural validity. Three out of the four studies scored a very good rating for internal consistency, while one was deemed 'inadequate' in internal consistency, cross-cultural validity, and reliability. Three out of four studies scored 'very good' on construct validity, but all overlooked criterion validity and responsiveness. Conducted in various locations, including Australia, Romania, Turkey, and Taiwan, these studies highlighted both common characteristics and limitations in questionnaire development according to the COSMIN guidelines. Four studies were deemed reliable and valid for BTL constructs (Category A); Wu et al. requires further validation (Category B). Study limitations included heterogeneity in populations, settings, and questionnaire versions, potential subjective bias in SRQ content comparison, and the evolving nature of SRQs in blended learning contexts.

CONCLUSION

The systematic review reports the development and evaluation of SRQs for BTL while identifying gaps in their applicability to health science programs. The Blended Learning Scale (BLS) of Lazar et al. and the Blended Learning Questionnaire (BLQ) of Ballouk et al. showed an ‘adequate' rating for content validity. BLS revealed very good structural validity, internal consistency, and adequate content validation. Although the BLQ lacked Confirmatory Factor Analysis, it yielded valuable constructs for evaluating health sciences students' experiences in BTL. Both tools require improvements on recall period, completion time, interpretability, and feasibility. The review underscores the necessity for cont inuous assessment and enhancement of such instruments in BTL, advocating a rigorous scale development process. Furthermore, it encourages the customization of teaching and learning evaluation tools to suit specific institutional contexts while promoting further validation of these questionnaires across different populations in future research.

Human ; Psychometrics ; Checklist ; Self Report ; Universities ; Health Education

BACKGROUND

Due to the COVID-19 outbreak, schools had to switch online. The sudden transition to blended teaching and learning (BTL) poses challenges for students and teachers, especially for health science programs that require hands-on practical experience. The validity, reliability, and responsiveness of these self-report questionnaires (SRQs) should be established to ensure the accuracy of the results as intended by the SRQ.

OBJECTIVES

This study critically appraised, compared, and summarized the psychometric properties of SRQ evaluating BTL among health science university students. This review determined the SRQ’s reliability, internal consistency, various forms of validity (content, criterion, construct), and responsiveness.

METHODS

Following a 10-step procedure based on COSMIN guidelines, we conducted a systematic review of SRQs used by health science university students to evaluate blended teaching and learning. Studies were eligible if they reported psychometric properties of SRQs related to blended learning among university health science students; exclusions included studies focusing on perceptions, attitudes, self-efficacy, and satisfaction, as well as articles such as biographies, editorials, and conference materials. Searches covered multiple electronic databases until April 26, 2023, including PubMed, EMBASE, Web of Science, MEDLINE (OVID), PsycInfo, CINAHL, EBSCOHOST, ERIC, Scopus, Science Direct, Google Scholar, JSTOR, Acta Medica Philippina, Philippine Journal of Health Research and Development, and HERDIN, managed through Zotero. Two independent reviewers performed database searches, title and abstract screening, and full-text evaluations, with a third reviewer resolving any disputes. The COSMIN Risk of Bias Checklist was employed to evaluate included studies on the development and various measurement properties of SRQs. The reviewers assessed SRQ standards, including validity, reliability, internal consistency, measurement error, responsiveness, interpretability, and feasibility. Data extraction and result tabulation were independently completed, with content comparison by two health education experts. This evaluation categorized the SRQs into three quality and validity levels.

RESULTS

The study examined five articles; four were rated as 'doubtful' and one as 'inadequate' in the overall development of SRQ. All four 'doubtful' studies demonstrated questionable content validity when university students were asked about the questionnaire's relevance, comprehensiveness, and comprehensibility. Only half of these studies achieved an 'adequate' rating for content validity based on expert opinions on relevance and comprehensiveness. All but one study scored from 'very good' to 'adequate' in structural validity. Three out of the four studies scored a very good rating for internal consistency, while one was deemed 'inadequate' in internal consistency, cross-cultural validity, and reliability. Three out of four studies scored 'very good' on construct validity, but all overlooked criterion validity and responsiveness. Conducted in various locations, including Australia, Romania, Turkey, and Taiwan, these studies highlighted both common characteristics and limitations in questionnaire development according to the COSMIN guidelines. Four studies were deemed reliable and valid for BTL constructs (Category A); Wu et al. requires further validation (Category B). Study limitations included heterogeneity in populations, settings, and questionnaire versions, potential subjective bias in SRQ content comparison, and the evolving nature of SRQs in blended learning contexts.

CONCLUSION

The systematic review reports the development and evaluation of SRQs for BTL while identifying gaps in their applicability to health science programs. The Blended Learning Scale (BLS) of Lazar et al. and the Blended Learning Questionnaire (BLQ) of Ballouk et al. showed an ‘adequate' rating for content validity. BLS revealed very good structural validity, internal consistency, and adequate content validation. Although the BLQ lacked Confirmatory Factor Analysis, it yielded valuable constructs for evaluating health sciences students' experiences in BTL. Both tools require improvements on recall period, completion time, interpretability, and feasibility. The review underscores the necessity for cont inuous assessment and enhancement of such instruments in BTL, advocating a rigorous scale development process. Furthermore, it encourages the customization of teaching and learning evaluation tools to suit specific institutional contexts while promoting further validation of these questionnaires across different populations in future research.

Human ; Psychometrics ; Checklist ; Self Report ; Universities ; Health Education

Objective: Our study aims to establish interrater reliability in performing the step-by-step procedure of selected pain provocation tests for hamstrings and special tests for lower extremity musculoskeletal injuries. Study Design: An interrater reliability study Setting: University of Santo Tomas - Sports Science Laboratory Participants: Ten healthy adults (five females, five males; age = 22.2 ± 0.42) from the university community. Main outcome measures: Interrater reliability of performing step-by-step procedures for selected pain provocation tests for hamstrings (painful resisted knee flexion 90°, painful resisted knee flexion 30°, active slump test, Puranen-Orava Test, bent knee stretch) and special tests for lower extremity musculoskeletal injuries (Lachman’s test, McMurray’s test, posterior drawer test, valgus, and varus stress test). Results: Fleiss kappa showed perfect agreement (κ = 1.00) for all test procedures except for Lachman’s test procedure 1 (κ= -0.11 [95% CI, -0.36 to 0.14]), active slump test procedure 4 (κ= -0.03 [95% CI, -0.28 to 0.23]), active slump test procedure 5 (κ= -0.11 [95% CI, -0.28 to 0.23]), and active slump test procedure 6 (κ= -0.05 [95% CI, -0.31 to 0.20]), which resulted in negative agreements. Conclusions The researcher developed protocols for each special and provocative test were consistent in measuring the intended procedures, and the raters were generally consistent with their ability to measure these tests.