1.Verification of the Validity and Reliability of Therapeutic Communication Responses Generated by Large Language Models (LLMs): A Comparative Study of Prompt-Engineering Strategies
Seong Kwang KIM ; Geun-Myun KIM ; Sunkyung CHA ; Miran JUNG
Journal of Korean Academy of Psychiatric and Mental Health Nursing 2025;34(Special Issue):23-35
Purpose:
This study evaluated the validity and reliability of therapeutic communication responses generated by GPT-5 by comparing different prompt-engineering strategies and identifying optimal methods for psychiatric nursing education.
Methods:
A methodological design compared four strategies—Zero-shot, Few-shot, Chain-of-Thought (CoT), and Role-prompt—applied to ten psychiatric nursing communication questions. Fourteen experts with over ten years of combined clinical and educational experience rated 40 anonymized responses using a 4-point scale. Inter-rater reliability was assessed with Fleiss' κ and Krippendorff's ⍺ from 400 repeated outputs. Semantic consistency was examined using cosine similarity. Analyses were conducted in Python.
Results:
Role-prompt showed the highest content validity (mean=3.54±0.39, S-CVI/Ave=0.95), followed by CoT and Zero-shot. Inter-rater agreement was slight (κ=0.07; ⍺=0.15). Semantic consistency was highest for CoT (0.76±0.12) and Role-prompt (0.75±0.11). A Friedman test indicated significant differences among strategies (x2(3)=11.88, p=.008).
Conclusion
Role-prompt yielded the most valid therapeutic responses, while CoT produced the most consistent outputs. These findings support the use of Role-prompt and CoT strategies to enhance the validity and reliability of therapy.
2.Multilevel Analysis of Factors Affecting Health-Related Quality of Life of the Elderly
Journal of Korean Academy of Psychiatric and Mental Health Nursing 2022;31(3):391-401
Purpose:
This study identifies the individual and community level factors affecting the health-related quality of life (QOL) of the elderly based on an ecological model using multilevel analysis.
Methods:
As a secondary research, this study used the raw data, community health survey results, and database 1.0 data on community health outcomes and determinants from the same year. Survey participants were 62,710 respondents age ≥65 of 228,558, and database 1.0 data on community health outcomes and determinants were established targeting 254 regions nationally.
Results:
The health-related QOL showed differences according to the individual and community level factors. Additionally, estimating the effects of the community level factors through multilevel analysis was needed. In a model in which the individual and community level factors were simultaneously inserted, the significant community characteristic factor controlling the individual level factors was the number of hospital beds per thousand population (p=.028).
Conclusion
In the multilevel analysis considering the individual and community level factors, the number of hospital beds per thousand population, which is a factor concerning medical access, was identified as a factor significantly affecting health-related QOL. The results should be considered in allocating and developing community- based healthcare resources and establishing strategies to improve the health-related QOL.
3.Effects of a Well-dying Program on Nursing Students.
Hyunjung MOON ; Sunkyung CHA ; Sungwon JUNG
Korean Journal of Hospice and Palliative Care 2015;18(3):188-195
PURPOSE: We examined the effects of a well-dying program on nursing students in terms of death preparation, death recognition and perception of well-dying perception. METHODS: The design of this study was quasi-experimental and non-synchronized with a non-equivalent control group. The study was conducted with nursing students: 32 in the experimental group and 36 in the control group. The well-dying program was consisted of five sessions: introduction, thinking about meaning of death, organizing things to do before dying, looking back on my life, and leaving a trail of my life. Descriptive analysis, t-test, chi2 test and ANCOVA were used with SPSS 18.0 program to analyze the data. RESULTS: After attending the program, a difference was observed in death preparation of the experimental group (t=2.61, P=0.014). The death recognition (F=154.15, P<0.001) score of the experimental group was significantly higher than the control group. There was no significant difference between the groups in perception of well-dying (F=0.11, P=0.747). CONCLUSIONS: The well-dying program helped nursing students build positive death recognition. Therefore, this study is expected to contribute to development of a death education program for nursing students.
Education
;
Humans
;
Nursing*
;
Students, Nursing*
;
Thinking

Result Analysis
Print
Save
E-mail