Objectives: Given the rapidly growing expectations for large language models (LLMs) in healthcare, this study systematically collected perspectives from Korean experts on the potential benefits and risks of LLMs, aiming to promote their safe and effective utilization. Methods: A web-based mini-Delphi survey was conducted from August 27 to October 14, 2024, with 20 selected panelists. The expert questionnaire comprised 84 judgment items across five domains: potential applications, benefits, risks, reliability requirements, and safe usage. These items were developed through a literature review and expert consultation. Participants rated their agreement or perceived importance on a 5-point scale. Items meeting predefined thresholds (content validity ratio ≥0.49, degree of convergence ≤0.50, and degree of consensus ≥0.75) were prioritized. Results: Seventeen participants (85%) responded to the first round, and 16 participants (80%) completed the second round. Consensus was achieved on several potential applications, benefits, and reliability requirements for the use of LLMs in healthcare. However, significant heterogeneity was found regarding perceptions of associated risks and criteria for safe usage of LLMs. Of the 84 total items, 52 met the criteria for statistical validity, confirming the diversity of expert opinions. Conclusions Experts reached a consensus on certain aspects of LLM utilization in healthcare. Nonetheless, notable differences remained concerning risks and requirements for safe implementation, highlighting the need for further investigation. This study provides foundational insights to guide future research and inform policy development for the responsible introduction of LLMs into the healthcare field.

Objectives: This study proposes a mobile-based explainable artificial intelligence (XAI) platform designed for diagnosing febrile illnesses. Methods: We integrated the interpretability offered by local interpretable model-agnostic explanations (LIME) and the explainability provided by generative pre-trained transformers (GPT) to bridge the gap in understanding and trust often created by machine learning models in critical healthcare decision-making. The developed system employed random forest for disease diagnosis, LIME for interpretation of the results, and GPT-3.5 for generating explanations in easy-to-understand language. Results: Our model demonstrated robust performance in detecting malaria, achieving precision, recall, and F1-scores of 85%, 91%, and 88%, respectively. It performed moderately well in detecting urinary tract and respiratory tract infections, with precision, recall, and F1-scores of 80%, 65%, and 72%, and 77%, 68%, and 72%, respectively, maintaining an effective balance between sensitivity and specificity. However, the model exhibited limitations in detecting typhoid fever and human immunodeficiency virus/acquired immune deficiency syndrome, achieving lower precision, recall, and F1-scores of 69%, 53%, and 60%, and 75%, 39%, and 51%, respectively. These results indicate missed true-positive cases, necessitating further model fine-tuning. LIME and GPT-3.5 were integrated to enhance transparency and provide natural language explanations, thereby aiding decision-making and improving user comprehension of the diagnoses. Conclusions The LIME plots revealed key symptoms influencing the diagnoses, with bitter taste in the mouth and fever showing the highest negative influence on predictions, and GPT-3.5 provided natural language explanations that increased the reliability and trustworthiness of the system, promoting improved patient outcomes and reducing the healthcare burden.

Objectives: The integration of large language models (LLMs), particularly those based on the generative pre-trained transformer (GPT) architecture, has begun to revolutionize various fields, including dentistry. Despite these promising applications, the use of GPT in dentistry presents several challenges. Ongoing research and the development of robust ethical frameworks are essential to mitigate these issues and enhance the responsible deployment of GPT technologies in clinical settings. Hence, this systematic review aims to explore the trends, applications, strengths, and challenges associated with the use of GPT in dentistry. Methods: Articles were selected if they contained detailed information on the application of GPT in dentistry. The search strategy used in systematic reviews follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Our search of databases and other sources yielded a total of 704 studies. After removing duplicates and conducting a full-text screening, 16 articles were included in the review. The methodological quality of the research was evaluated using the Critical Appraisal Skills Programme (CASP) checklist. Results: Out of a total of 91 articles published on GPT in dentistry, 20 were editorials and 11 were narrative reviews; these were excluded, leaving 60 original research articles for further analysis. The articles were assessed based on the type of results they provided. Ultimately, 16 articles that reported positive findings with robust methodology were included in this review Conclusions The results highlight mixed responses; therefore, further research on integration into clinical workflows must be conducted with extensive methodological rigor.

Objectives: This research aimed to develop a retrieval-augmented generation (RAG) based large language model (LLM) system that offers personalized and reliable responses to a wide range of concerns raised by Korean adolescents. Our work focuses on building a culturally reflective dataset and on designing and validating the system’s effectiveness by comparing the answer quality of RAG-based models with non-RAG models. Methods: Data were collected from the NAVER Knowledge iN platform, concentrating on posts that featured adolescents’ questions and corresponding expert responses during the period 2014–2024. The dataset comprises 3,874 cases, categorized by key negative emotions and the primary sources of worry. The data were processed to remove irrelevant or redundant content and then classified into general and detailed causes. The RAG-based model employed FAISS for similarity-based retrieval of the top three reference cases and used GPT-4o mini for response generation. The responses generated with and without RAG were evaluated using several metrics. Results: RAG-based responses outperformed non-RAG responses across all evaluation metrics. Key findings indicate that RAG-based responses delivered more specific, empathetic, and actionable guidance, particularly when addressing complex emotional and situational concerns. The analysis revealed that family relationships, peer interactions, and academic stress are significant factors affecting adolescents’ worries, with depression and stress frequently co-occurring. Conclusions This study demonstrates the potential of RAG-based LLMs to address the diverse and culture-specific worries of Korean adolescents. By integrating external knowledge and offering personalized support, the proposed system provides a scalable approach to enhancing mental health interventions for adolescents. Future research should concentrate on expanding the dataset and improving multiturn conversational capabilities to deliver even more comprehensive support.

Objectives: This study presents a comprehensive review of the clinical applications, technical challenges, and ethical considerations associated with using large language models (LLMs) in medicine. Methods: A literature survey of peer-reviewed articles, technical reports, and expert commentary from relevant medical and artificial intelligence journals was conducted. Key clinical application areas, technical limitations (e.g., accuracy, validation, transparency), and ethical issues (e.g., bias, safety, accountability, privacy) were identified and analyzed. Results: LLMs have potential in clinical documentation assistance, decision support, patient communication, and workflow optimization. The level of supporting evidence varies; documentation support applications are relatively mature, whereas autonomous diagnostics continue to face notable limitations regarding accuracy and validation. Key technical challenges include model hallucination, lack of robust clinical validation, integration issues, and limited transparency. Ethical concerns involve algorithmic bias risking health inequities, threats to patient safety from inaccuracies, unclear accountability, data privacy, and impacts on clinician-patient interactions. Conclusions LLMs possess transformative potential for clinical medicine, particularly by augmenting clinician capabilities. However, substantial technical and ethical hurdles necessitate rigorous research, validation, clearly defined guidelines, and human oversight. Existing evidence supports an assistive rather than autonomous role, mandating careful, evidence-based integration that prioritizes patient safety and equity.

Objectives: Assessing medical disputes requires both medical and legal expertise, presenting challenges for patients seeking clarity regarding potential malpractice claims. This study aimed to develop and evaluate a chatbot based on a chain-of-thought pipeline using a large language model (LLM) for providing medical dispute counseling and compare its performance with responses from human experts. Methods: Retrospective counseling cases (n = 279) were collected from the Korea Medical Dispute Mediation and Arbitration Agency’s website, from which 50 cases were randomly selected as a validation dataset. The Claude 3.5 Sonnet model processed each counseling request through a five-step chain-of-thought pipeline. Thirty-eight experts evaluated the chatbot’s responses against the original human expert responses, rating them across four dimensions on a 5-point Likert scale. Statistical analyses were conducted using Wilcoxon signed-rank tests. Results: The chatbot significantly outperformed human experts in quality of information (p < 0.001), understanding and reasoning (p < 0.001), and overall satisfaction (p < 0.001). It also demonstrated a stronger tendency to produce opinion-driven content (p < 0.001). Despite generally high scores, evaluators noted specific instances where the chatbot encountered difficulties. Conclusions A chain-of-thought–based LLM chatbot shows promise for enhancing the quality of medical dispute counseling, outperforming human experts across key evaluation metrics. Future research should address inaccuracies resulting from legal and contextual variability, investigate patient acceptance, and further refine the chatbot’s performance in domain-specific applications.

Objectives: This study examined the symptoms and emotions expressed by older adults with cancer and their caregivers in South Korean online cancer communities. It aimed to identify narrative patterns and provide insights to inform personalized care strategies. Methods: We analyzed 6,908 user-generated posts collected from major online cancer communities in South Korea. Keyword frequency analysis, term frequency-inverse document frequency, 2-gram analysis, and latent Dirichlet allocation-based topic modeling were applied to explore language patterns. Sentiment analysis identified 12 emotional categories, and Pearson correlation coefficients were calculated to examine associations between symptoms and emotional expressions. All data were cleaned and standardized prior to analysis. Results: Many users expressed anxiety (20.63%) and depression (19.59%), frequently associated with chemotherapy and sleep disturbances. Among reported symptoms, sleep problems carried the highest negative sentiment (79.81%), underscoring their profound impact on well-being. Topic modeling consistently revealed seven recurring themes, including treatment decision-making, symptom management, and concerns about family, demonstrating the layered and personalized experiences of older cancer patients and their caregivers. Conclusions This study explored treatment-related and symptom-related difficulties faced by older adults with cancer. Many reported significant emotional strain, especially anxiety, depression, and sleep disturbances. These findings highlight the necessity for supportive strategies addressing both psychological and physical aspects of care. Future research could investigate the utility of large language models in analyzing these narratives, provided the data is ethically managed and appropriate for such use.

Objectives: The high incidence of sepsis necessitates the development of practical decision-making tools for intensivists, especially during the early, critical phases of management. This study evaluates a multi-agent system intended to assist clinicians with antibiotic therapy and adherence to current sepsis management guidelines before diagnostic results become available. Methods: A multi-agent system incorporating three specialized agents was developed: a sepsis management agent, an antibiotic recommendation agent, and a sepsis guidelines compliance agent. A sepsis case from the MIMIC IV database, organized as a clinical vignette, was used to integrate and test these agents for generating management recommendations. The system leverages retrieval-augmented generation to improve decision-making through the integration of current literature and guidelines. Results: The application produced management recommendations for a sepsis case associated with pneumonia, including early initiation of broad-spectrum antibiotics and close monitoring for clinical deterioration. Two expert intensivists evaluated these recommendations as “acceptable” and reported moderate interrater agreement (Cohen’s kappa = 0.622, p = 0.003) across various aspects of recommendation usefulness. Conclusions The multi-agent system shows promise in enhancing decision-making for sepsis management by optimizing antibiotic therapy and ensuring guideline compliance. However, reliance on a single case study limits the generalizability of the findings, highlighting the need for broader validation in diverse clinical settings to improve patient outcomes.

Objectives: Developing large language models (LLMs) in biomedicine requires access to high-quality training and alignment tuning datasets. However, publicly available Korean medical preference datasets are scarce, hindering the advancement of Korean medical LLMs. This study constructs and evaluates the efficacy of the Korean Medical Preference Dataset (KoMeP), an alignment tuning dataset constructed with an automated pipeline, minimizing the high costs of human annotation. Methods: KoMeP was generated using the DAHL score, an automated hallucination evaluation metric. Five LLMs (Dolly-v2-3B, MPT-7B, GPT-4o, Qwen-2-7B, Llama-3-8B) produced responses to 8,573 biomedical examination questions, from which 5,551 preference pairs were extracted. Each pair consisted of a “chosen” response and a “rejected” response, as determined by their DAHL scores. The dataset was evaluated when trained through two different alignment tuning methods, direct preference optimization (DPO) and odds ratio preference optimization (ORPO) respectively across five different models. The KorMedMCQA benchmark was employed to assess the effectiveness of alignment tuning. Results: Models trained with DPO consistently improved KorMedMCQA performance; notably, Llama-3.1-8B showed a 43.96% increase. In contrast, ORPO training produced inconsistent results. Additionally, English-to-Korean transfer learning proved effective, particularly for English-centric models like Gemma-2, whereas Korean-to-English transfer learning achieved limited success. Instruction tuning with KoMeP yielded mixed outcomes, which suggests challenges in dataset formatting. Conclusions KoMeP is the first publicly available Korean medical preference dataset and significantly improves alignment tuning performance in LLMs. The DPO method outperforms ORPO in alignment tuning. Future work should focus on expanding KoMeP, developing a Korean-native dataset, and refining alignment tuning methods to produce safer and more reliable Korean medical LLMs.

Objectives: Nursing documentation consumes approximately 30% of nurses’ professional time, making improvements in efficiency essential for patient safety and workflow optimization. This study compares traditional nursing documentation methods with a generative artificial intelligence (AI)-based system, evaluating its effectiveness in reducing documentation time and ensuring the accuracy of AI-suggested entries. Furthermore, the study aims to assess the system’s impact on overall documentation efficiency and quality. Methods: Forty nurses with a minimum of 6 months of clinical experience participated. In the pre-assessment phase, they documented a nursing scenario using traditional electronic nursing records (ENRs). In the post-assessment phase, they used the SmartENR AI version, developed with OpenAI’s ChatGPT 4.0 API and customized for domestic nursing standards; it supports NANDA, SOAPIE, Focus DAR, and narrative formats. Documentation was evaluated on a 5-point scale for accuracy, comprehensiveness, usability, ease of use, and fluency. Results: Participants averaged 64 months of clinical experience. Traditional documentation required 467.18 ± 314.77 seconds, whereas AI-assisted documentation took 182.68 ± 99.71 seconds, reducing documentation time by approximately 40%. AI-generated documentation received scores of 3.62 ± 1.29 for accuracy, 4.13 ± 1.07 for comprehensiveness, 3.50 ± 0.93 for usability, 4.80 ± 0.61 for ease of use, and 4.50 ± 0.88 for fluency. Conclusions Generative AI substantially reduces the nursing documentation workload and increases efficiency. Nevertheless, further refinement of AI models is necessary to improve accuracy and ensure seamless integration into clinical practice with minimal manual modifications. This study underscores AI’s potential to improve nursing documentation efficiency and accuracy in future clinical settings.