Content validity is the degree to which an instrument has an appropriate sample of items for the construct being measured and is an important procedure in scale development. Content validity index (CVI) is the most widely used index in quantitative evaluation. There are 2 kinds of CVI: I-CVI and S-CVI. A method to compute a modified kappa statistic (K*) can be used to adjust I-CVI for chance agreement. S-CVI/UA and S-CVI/Ave are both scale level CVI with different formulas. Researchers recommend that a scale with excellent content validity should be composed of I-CVIs of 0.78 or higher and S-CVI/UA and S-CVI/Ave of 0.8 and 0.9 or higher, respectively. The characteristics and qualifications of the experts, process and main results of content validity evaluation should be reported in scale-related manuscript.
Data Collection ; Humans ; Psychometrics ; methods ; Reproducibility of Results ; Surveys and Questionnaires ; standards

Psychosocial problems increase the risk for mental health problems and increase the need for health care services in children and adolescents. Primary care practice is a valuable avenue for identifying the need for more specialized mental health care. We hypothesized that Korean version of the pediatric symptom checklist (PSC) would be a useful tool for early detection of psychosocial problems in children and adolescents in Korea and we aimed to suggest cut-off scores for detecting meaningful psychosocial problems. A total of 397 children with their parents and 97 child patients with their parents were asked to complete the PSC Korean version and the child behavior checklist (CBCL). The internal reliability and test-retest reliability of the PSC as well as the cut-off score of the PSC was determined via receiver operating characteristic analysis of the CBCL score, clinical group scores and non-clinical group scores. The internal consistency of the PSC-Korean version was excellent (Cronbach's alpha = 0.95). The test-retest reliability was r = 0.73 (P < 0.001). Using clinical CBCL scores (total score, externalizing score, internalizing score, respectively > or = 60) and presence of clinical diagnosis, the recommended cut-off score of the PSC was 14. Using 494 Korean children aged 7-12 yr, the current study assessed the reliability and validity of a Korean version of the PSC and suggested a cut-off for recommending further clinical assessment. The present results suggest that the Korean version of the PSC has good internal consistency and validity using the standard of CBCL scores.
Checklist/methods/*standards ; Child ; Child Behavior Disorders/*diagnosis/*psychology ; Female ; Humans ; Male ; Mass Screening/standards ; Pediatrics/*standards ; Population ; Psychometrics/methods/*standards ; Reproducibility of Results ; Republic of Korea ; Sensitivity and Specificity ; Surveys and Questionnaires/standards ; Symptom Assessment/methods/*standards ; Translating ; United States

INTRODUCTIONPlanning a high-stake clinical examination requires the evaluation of several psychometric and logistical variables. The authors conducted generalisability and decision studies to answer the following research questions in the context of the surgical long case: (1) Does the addition of a third examiner have any added benefit, vis-à-vis reliability, to the examination? (2) Is global marking more reliable than an itemised marking template? (3) What would be the impact on reliability if there was a reduction in the number of examinees that each panel of examiners is required to assess?

METHODSA third examiner and global marking were introduced. Separate generalisability and decision studies were carried out for both the two- and three-examiner models as well as for itemised and global scores.

RESULTSThe introduction of a third examiner resulted in a modest gain of reliability by 0.05-0.07. Gain in reliability was higher when each candidate was allowed to undertake a higher number of clinical cases. Both the global and itemised scores provided equivalent reliability (generalisability coefficient 0.74-0.89).

CONCLUSIONOur results showed that only a modest improvement in reliability of the surgical long case is achieved through the introduction of an additional examiner. Although the reliability of global scoring and the itemised marking template was comparable, the latter may provide opportunities for individualised feedback to examinees.

Clinical Competence ; Education, Medical, Undergraduate ; methods ; standards ; Educational Measurement ; methods ; Humans ; Medical History Taking ; methods ; Observation ; Professional-Patient Relations ; Psychometrics ; methods ; Reproducibility of Results ; Schools, Medical ; Singapore

INTRODUCTIONAn accurate assessment of peer victimisation (i.e. bullying) is a necessary precondition for research and intervention. Most assessment instruments use the 'list of acts' measurement strategy, which does not account for the actual physical and psychological damage inflicted by bullying. To resolve this limitation, this study developed a peer victimisation scale (PVS) that includes harmful consequences for judgement and measurement of peer victimisation.

METHODSThe PVS is a 40-item self-report questionnaire designed to assess the four aspects of peer victimisation: physical, verbal, relational, and interference and control. A total of 1,469 Grade 3-8 students (49.9% male) were recruited to test the psychometric properties of the PVS. Another 420 Grade 3-8 students were examined by a modified PVS supplemented with a semi-structured interview for scale validation and establishment of the cut-off points for severe bullying. Incidence, age and gender distribution of peer victimisation were also analysed.

RESULTSThe PVS demonstrated good internal consistency reliability (Cronbach's alpha 0.73-0.83) and test-retest reliability two weeks later (correlation coefficient [r] = 0.71-0.80). The scores for each dimension were significantly and positively correlated with the scores from the questionnaire-interview sample (r = 0.73-0.78), and modestly correlated with the scores for symptoms of anxiety and depression (r = 0.36-0.54).

CONCLUSIONThe results were consistent with the measurement constructs, demonstrating that the PVS is a reliable and effective instrument for assessing peer victimisation in children. It may enable more reliable longitudinal studies assessing the impact of peer victimisation to be conducted.

Behavior Therapy ; methods ; Bullying ; prevention & control ; Child ; Depression ; diagnosis ; Factor Analysis, Statistical ; Female ; Humans ; Male ; Peer Group ; Psychometrics ; methods ; standards ; Reproducibility of Results ; Social Behavior ; Students ; Surveys and Questionnaires

PURPOSE: The purposes of this methodologic paper are to (1) describe theoretical background in conducting research across different cultures; (2) address measurement issues related to instrument administration; and (3) provide strategies to deal with measurement issues. METHODS: A thorough review of the literature was conducted. A theoretical background is provided, and examples of administering instrument in studies are described. RESULTS: When applying an instrument to different cultures, both equivalence and bias need to be established. Three levels of equivalence, i.e., construct equivalence, measurement unit equivalence, and full score comparability, need to be explained to maintain the same concept being measured. In this paper, sources of bias in construct, method, and item are discussed. Issues related to instrument administration in a cross-cultural study are described. CONCLUSION: Researchers need to acknowledge various group differences in concept and/or language that include a specific set of symbols and norms. There is a need to question the philosophical and conceptual appropriateness of an assessment measure that has been conceptualized and operationalized in a different culture. Additionally, testing different response formats such as narrowing response range can be considered to reduce bias.
Attitude of Health Personnel/ethnology ; Attitude to Health/ethnology ; Bias (Epidemiology) ; Communication Barriers ; *Cross-Cultural Comparison ; Data Collection/*methods/standards ; Humans ; Interviews as Topic/methods/standards ; Nursing Assessment ; Nursing Research/*methods/standards ; Philosophy, Nursing ; Psychometrics ; Reproducibility of Results ; Research Design/standards ; Researcher-Subject Relations/psychology ; Transcultural Nursing/*methods/standards ; Translating

PURPOSE: The aim of this research was to adapt the Sun Protection Behavior Scale (SPBS) to Turkish and to perform validity and reliability analyses. METHODS: The scale was administered to a total of 900 adolescents, the retest to 91 adolescents. The construct validity of the scale was evaluated using exploratory (EFA) and confirmatory factor analysis (CFA). The EFA and CFA were applied to sample groups of 449 and 451 people, respectively. RESULTS: The Cronbach alpha coefficients for the Turkish form of the SPBS (alpha = .74) and its sunscreen (alpha = .88) and hat use (alpha = .70) subscales were found to be > or =.70 while the sun avoidance subscale was calculated to be .67. The item-total score correlation between the scale and its subscales was > or =.26 and the test-retest correlations were found to be > or =.51. The CFA results verified the 8-item, 3-factor Turkish version of the SPBS. The confirmatory factor loadings for the scale were .45-.80 for sun avoidance, .72-.93 for sunscreen use, and .66-.83 for hat use. In particular, SPBS and sunscreen use (p < .001) exhibited significantly high mean scores among girls and economically better backgrounds (p = .007, p < .001, respectively). In addition, SPBS (p = .004) and hat use (p < .001) revealed that the mean scores were significantly high in younger adolescents. CONCLUSIONS: The SPBS was found to be valid and reliable and its psychometric characteristics acceptable. The scale can be used to measure the behavior of Turkish adolescent populations with respect to sun protection.
Adolescent ; Child ; Factor Analysis, Statistical ; Female ; *Health Behavior ; Humans ; Male ; Psychometrics/*methods ; Reproducibility of Results ; Sunscreening Agents/*therapeutic use ; Surveys and Questionnaires/*standards ; Turkey

PURPOSE: The present study was conducted to develop a mild vascular cognitive impairment (MVCI) assessment tool for patients with stroke and to examine its validity, reliability, and clinical adequacy. METHODS: Items of this tool were developed based on previously verified cognitive assessment tools. Face, content, and criterion (concurrent) validities, optimal cut-off score for differentiation of MVCI and normal cognitive function, clinical adequacy, internal consistency, and inter-rater reliability of the assessment tool were determined in 60 stroke patients at a university hospital located in Incheon, South Korea. RESULTS: The devised MVCI assessment tool contains 20 items which were designed to assess seven cognitive domains: orientation, memory, language, attention, reasoning/abstraction, visuospatial perception, and executive function/problem solving. Content, face, and construct validities were well supported. Clinical adequacy testing revealed that the overall probability of correctly discriminating MVCI using the MVCI assessment tool for stroke was 90.0%, which was statistically significant. Furthermore, a score of 23 was found to be the optimal cut-off score for MVCI. Internal consistency and inter-rater reliability were also well supported. CONCLUSIONS: The findings of this study indicate that the developed MVCI assessment tool for stroke could serve as a clinically useful tool for detecting MVCI and for properly assessing degree of cognitive impairment in stroke patients.
Aged ; Female ; Humans ; Male ; Middle Aged ; *Mild Cognitive Impairment/complications/diagnosis ; Neuropsychological Tests/*standards ; Psychometrics/*methods ; Reproducibility of Results ; Republic of Korea ; Sensitivity and Specificity ; Stroke/*complications

PURPOSE: To evaluate the diagnostic value of the Korean version of the Douleur Neuropathique 4 (DN4) questionnaire and to validate this questionnaire in terms of psychometric properties in patients with chronic pain due to degenerative spinal disease. MATERIALS AND METHODS: The Korean version of the DN4 questionnaire, which was translated and linguistically validated by the MAPI Research Group, was tested on 83 patients with lumbar or lumbar-radicular pain. Test-retest reliability was evaluated in a subsample of 40 patients who completed two assessments with an interval of 2 weeks. Nociceptive pain and neuropathic component pain were diagnosed in 40 and 43 patients, respectively. RESULTS: The Cronbach's alpha coefficient of internal consistency was 0.819, and the test-retest intraclass correlation coefficient (3, 1) (95% confidence interval) was 0.813 (0.776-0.847) (n=40). The area under the receiver-operator characteristics curve was 0.953 (p<0.001), with 95% confidence interval between 0.869 and 0.990. The Korean version of the DN4 questionnaire showed a sensitivity of 100% and 87.1%, and a specificity of 88.2% and 94.1% at the cutoff value of 3/10 and 4/10, respectively, for discriminating neuropathic component pain. CONCLUSION: The present study demonstrated the good discriminatory power of DN4 between nociceptive pain and neuropathic component pain in patients with lumbar or lumbar-radicular pain.
Adult ; Aged ; Aged, 80 and over ; Chronic Pain/*diagnosis ; Female ; Humans ; Male ; Middle Aged ; Neuralgia/*diagnosis ; Pain Measurement/*methods ; Psychometrics ; Reproducibility of Results ; Sensitivity and Specificity ; Surveys and Questionnaires/*standards ; Translating

OBJECTIVE: To introduce and revise the Violence Risk Scale (VRS) for assessing violence risk and risk change, and to examine the reliability of Violence Risk Scale of Chinese version(VRS-C). METHODS: The original English version of the VRS was translated into Chinese according to established translation procedures. To examine the scorer reliability the 14 cases assessed by 3 assessors separately. One hundred and twenty-five patients with mental disorders from 3 different institutions in Sichuan province (Refined Control Ward in Ankang Hospital, Department of Forensic Psychiatry, Institute of Forensic Science and Mental Health Center of West China Hospital) were collected to examine the reliability of VRS-C. RESULTS: The results showed moderately good scale reliability of the VRS-C, with 0.80 of ICC for scorer reliability. All items have significant consistence with Cronbach's alpha coefficient as 0.921, split-half reliability as 0.906 and item total correlation as 0.246-0.849. CONCLUSION The reliability of the VRS-C version is acceptable.
Adolescent ; Adult ; Age Factors ; China/epidemiology* ; Educational Status ; Female ; Forensic Psychiatry/methods* ; Humans ; Language ; Male ; Mental Disorders/psychology* ; Middle Aged ; Psychological Tests/standards* ; Psychometrics ; Reproducibility of Results ; Risk Assessment/standards* ; Surveys and Questionnaires ; Violence/statistics & numerical data* ; Young Adult

PURPOSE: This study was conducted to examine the relationship between the prevalence of burnout and its related factors in medical students in Korea. METHODS: All available medical students in the metropolitan city of Daejeon, Korea, were asked to answer self-administered questionnaires from July 1 to July 26 in 2013. A total of 534 medical students participated. The Maslach Burnout Inventory-Student Survey (MBI-SS) and structured questionnaires on related factors were used. Confirmatory factor analysis and Cronbach's alpha were used to verify the applicability of the MBI-SS to medical students in Korea. We also performed chi-square test and logistic regression analysis to identify the factors that were associated with burnout. RESULTS: The MBI-SS was reliable and valid in measuring burnout in Korean medical students. Our confirmatory factor analysis approved and explained the appropriateness of the model fit. The prevalence of burnout among medical students was 26.4% (n=141). Such rates were higher in students who were female, experienced greater levels of depression, had poor academic performance, feared dropping out, and were stressed by the poor quality of the class facilities. CONCLUSION: The MBI-SS is a valid instrument to measure academic burnout in Korean medical students. Further studies should be performed, because improvements in the mental health of medical students will benefit these doctors-to-be and their future patients.
Burnout, Professional/*epidemiology ; Factor Analysis, Statistical ; Female ; Humans ; Logistic Models ; Male ; Prevalence ; Psychometrics/*methods ; Reproducibility of Results ; Republic of Korea/epidemiology ; Risk Factors ; Stress, Psychological/*epidemiology ; Students, Medical/*psychology ; Surveys and Questionnaires/*standards