1.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
2.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
3.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
4.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
5.Pain Passport as a tool to improve analgesic use in children with suspected fractures in emergency departments
Soyun HWANG ; Yoo Jin CHOI ; Jae Yun JUNG ; Yeongho CHOI ; Eun Mi HAM ; Joong Wan PARK ; Hyuksool KWON ; Do Kyun KIM ; Young Ho KWAK
The Korean Journal of Pain 2020;33(4):386-394
Background:
In the emergency department (ED), adequate pain control is essential for managing patients; however, children with pain are known to receive less analgesia than adults with pain. We introduce the Pain Passport to improve pain management in paediatric patients with suspected fractures in the ED.
Methods:
This was a before-and-after study. We reviewed the medical records of paediatric patients who were primarily diagnosed with fractures from May to August 2015. After the introduction of the Pain Passport, eligible children were enrolled from May to August 2016. Demographics, analgesic administration rates, time intervals between ED arrival and analgesic administration, and satisfaction scores were obtained. We compared the analgesic prescription rate between the two periods using multiple logistic regression.
Results:
A total of 58 patients were analysed. The baseline characteristics of subjects during the two periods were not significantly different. Before the introduction of the Pain Passport, 9 children (31.0%) were given analgesics, while after the introduction of the Pain Passport, a significantly higher percentage of patients (24/29, 82.8%) were treated with analgesics (P < 0.001). The median administration times were 112 (interquartile range [IQR], 64-150) minutes in the pre-intervention period and 24 (IQR, 20-74) minutes in the post-intervention period. The median satisfaction score for the post-intervention period was 4 (IQR, 3-5). The adjusted odds ratio for providing analgesics in the post-intervention period was 25.91 (95% confidence interval, 4.36-154.02).
Conclusions
Patient-centred pain scoring with the Pain Passport improved pain management in patients with suspected fractures in the paediatric ED.
6.A retroperitoneal dedifferentiated liposarcoma mimicking an ovarian tumor.
Hyojin KIM ; Taewon JEONG ; Yeongho LEE ; Gyeonga KIM ; Sanggi HONG ; Sukyung BECK ; Jeongbeom MUN ; Kyongjin KIM ; Myeongjin JU
Obstetrics & Gynecology Science 2017;60(6):598-601
A 74-year-old postmenopausal woman visited our gynecology clinic complaining of a palpable abdominal mass. Physical and radiological evaluation indicated that the mass exhibited features of a left ovarian neoplasm showing heterogeneous enhancement. Surgical resection was performed to confirm this suspicion. During surgery, a mass was observed only in the left ovary with no invasive growth, but adhesions to the surrounding peritoneum were seen. Given the patient's age, large mass size, and accompanying uterine myoma and right ovarian cyst, total abdominal hysterectomy with bilateral salpingo-oophorectomy was performed. The final pathologic diagnosis was dedifferentiated liposarcoma. The liposarcoma was suspected to originate from retroperitoneal adipose tissue rather than the ovary. Radiotherapy was planned if a gross lesion indicating recurrence followed 6 months later. This case required a considerable multi-disciplinary approach for diagnosis and treatment because of its ambiguous clinical and radiological findings.
Aged
;
Diagnosis
;
Female
;
Gynecology
;
Humans
;
Hysterectomy
;
Intra-Abdominal Fat
;
Leiomyoma
;
Liposarcoma*
;
Ovarian Cysts
;
Ovarian Neoplasms
;
Ovary
;
Peritoneum
;
Radiotherapy
;
Recurrence
;
Retroperitoneal Neoplasms
7.Dynamics of Gut Microbiota According to the Delivery Mode in Healthy Korean Infants.
Eun LEE ; Byoung Ju KIM ; Mi Jin KANG ; Kil Yong CHOI ; Hyun Ju CHO ; Yeongho KIM ; Song I YANG ; Young Ho JUNG ; Hyung Young KIM ; Ju Hee SEO ; Ji Won KWON ; Hyo Bin KIM ; So Yeon LEE ; Soo Jong HONG
Allergy, Asthma & Immunology Research 2016;8(5):471-477
Microbial colonization of the infant gut is unstable and shows a wide range of diversity between individuals. Gut microbiota play an important role in the development of the immune system, and an imbalance in these organisms can affect health, including an increased risk of allergic diseases. Microbial colonization of young infants is affected by the delivery mode at birth and the consequent alterations of gut microbiota in early life affect the development of allergic diseases. We investigated the effects of the delivery mode on the temporal dynamics of gut microbiota in healthy Korean infants. Fecal samples were collected at 1-3 days, 1 month, and 6 months after birth in six healthy infants. Microbiota were characterized by 16S rRNA shotgun sequencing. At the first and third days of life, infants born by vaginal delivery showed a higher richness and diversity of gut microbiota compared with those born by cesarean section. However, these differences disappeared with age. The Bacteroides genus and Bacteroidetes phylum were abundant in infants born by vaginal delivery, whereas Bacilli and Clostridium g4 were increased in infants born by cesarean section. The Firmicutes phylum and Bacteroides genus showed convergent dynamics with age. This study demonstrated the effect of delivery mode on the dynamics of gut microbiota profiles in healthy Korean infants.
Bacteroides
;
Bacteroidetes
;
Cesarean Section
;
Clostridium
;
Colon
;
Female
;
Firmicutes
;
Gastrointestinal Microbiome*
;
Humans
;
Immune System
;
Infant*
;
Microbiota
;
Parturition
;
Pregnancy
8.Pediatric adverse drug reactions collected by an electronic reporting system in a single tertiary university hospital.
Geun Mi PARK ; Joo Hyun PARK ; Joo Won JUNG ; Hye Won HAN ; Jae Youn KIM ; Eun LEE ; Hyun Ju CHO ; Yeongho KIM ; Jisun YOON ; Jinho YU ; Tae Bum KIM ; Soo Jong HONG
Allergy, Asthma & Respiratory Disease 2016;4(5):354-359
PURPOSE: The incidence of adverse drug reactions (ADRs) is increasing. However, studies on the prevalence of ADRs in children are rare. The aim of this study was to investigate the causative drugs and clinical features of ADRs for children in a tertiary university hospital of Korea. METHODS: We retrospectively collected ADRs by a computerized self-reporting system in Asan Medical Center. ADRs of children under the age 18 were collected from January 2005 to August 2015, and we analyzed only ADRs containing current symptoms among total ADR data. RESULTS: A total of 1,408 ADR cases were reported, There were 764 male (54.3%) and 644 female patients (45.7%), and the mean age was 11.5±5.8 years (range, 0–18 years). Antibiotics (n=479, 34.0%) were the most common causative drugs, followed by tramadol (n=173, 12.3%), nonsteroidal anti-inflammatory agents (NSAIDs) and acetylsalicylic acid (n=103, 7.3%), narcotics (n=91, 6.5%), antineoplastics (n=87, 6.2%), and sedatives (n=82, 5.8%). The most common clinical features were skin manifestations (n=500, 34.4%). Gastrointestinal symptoms (n=435, 29.9%) were the second most common clinical features, followed by neuropsychiatric symptoms (n=155, 10.7%) and respiratory symptoms (n=123, 8.5%). Among antibiotics, glycopeptides (n=110, 23.0%), third-generation cephalosporins (n=83, 17.3%), and penicillin/β-lactamase inhibitors (n=60, 12.7%) were the most frequently reported causative drugs. CONCLUSION: Antibiotics were the most reported common causative drugs of ADRs in children, followed by tramadol, NSAID, and narcortics. Compared with adults, the prevalence of contrast medium-induced ADR was lower in children with a higher prevalence of sedative-associated ADR. Greater attention to possible ADRs in children is needed among medical personnel.
Adult
;
Anti-Bacterial Agents
;
Anti-Inflammatory Agents, Non-Steroidal
;
Antineoplastic Agents
;
Aspirin
;
Cephalosporins
;
Child
;
Chungcheongnam-do
;
Drug-Related Side Effects and Adverse Reactions*
;
Female
;
Glycopeptides
;
Humans
;
Hypnotics and Sedatives
;
Incidence
;
Korea
;
Male
;
Narcotics
;
Prevalence
;
Retrospective Studies
;
Skin Manifestations
;
Tramadol
9.The prevalence and risk factors of allergic rhinitis from a nationwide study of Korean elementary, middle, and high school students.
Yeongho KIM ; Ju Hee SEO ; Ji Won KWON ; Eun LEE ; Song I YANG ; Hyun Ju CHO ; Mina HA ; Eunae BURM ; Kee Jae LEE ; Hwan Cheol KIM ; Sinye LIM ; Hee Tae KANG ; Mia SON ; Soo Young KIM ; Hae Kwan CHEONG ; Yu Mi KIM ; Gyung Jae OH ; Joon SAKONG ; Chul Gab LEE ; Sue Jin KIM ; Yong Wook BEAK ; Soo Jong HONG
Allergy, Asthma & Respiratory Disease 2015;3(4):272-280
PURPOSE: We investigated the prevalence and risk factors of allergic rhinitis (AR), nationwide in random children and adolescents of Korea. METHODS: A modified International Study of Asthma and Allergies in Childhood (ISAAC) questionnaire survey was done in 1,820 children from elementary, middle, and high school nationwide in Korea. The subjects were selected by the stratifying sampling method by school grade and five regions. Current AR was defined as having AR symptoms during the last 12 months with a history of physician-diagnosed AR. Skin prick tests for 18 common allergens were performed. RESULTS: The number of males was 945, and that of females was 875. The mean age of the patients was 12.61+/-3.40 years. The prevalence of current AR and atopic current AR were 29.0% and 18.7%, respectively. Risk factors for current AR were male (adjusted odds ratio [aOR], 1.486; 95% confidence interval [CI], 1.189-1.856), family history of paternal AR (aOR, 3.208; 95% CI, 2.460-4.182), family history of maternal AR (aOR, 3.138; 95% CI, 2.446-4.025), antibiotic use in infancy (aOR, 1.547; 95% CI, 1.228-1.949), mold exposure during infancy (aOR, 1.416; 95% CI, 1.103-1.819), mold exposure during the last 12 months (aOR, 1.285; 95% CI, 1.012-1.630), and sensitization on skin prick tests (aOR, 2.596; 95% CI, 2.055-3.279). Risk factors for atopic current AR were the same as those of current AR, whereas breast-milk feeding (aOR, 0.720; 95% CI, 0.530-0.976) was a protective factor. Sensitized allergens as risk factors for current AR were Dermatophagoides pteronyssinus, Dermatophagoides farina, ragweed, mugwort, oak, alder, birch, Japanese hop, cat, and dog. CONCLUSION: The prevalences of current AR and atopic current AR were 29.0% and 18.7%, respectively. Male, sex parental AR, antibiotic use in infancy, mold exposure during the last 12 months, mold exposure during infancy, and atopic sensitization were risk factors for current AR. Breast-milk feeding was a protective factor for atopic current AR. Aeroallergen sensitization was an important risk factor for AR.
Adolescent
;
Allergens
;
Alnus
;
Ambrosia
;
Animals
;
Artemisia
;
Asian Continental Ancestry Group
;
Asthma
;
Betula
;
Cats
;
Child
;
Dermatophagoides pteronyssinus
;
Dogs
;
Female
;
Fungi
;
Humans
;
Humulus
;
Hypersensitivity
;
Korea
;
Male
;
Odds Ratio
;
Parents
;
Prevalence*
;
Pyroglyphidae
;
Rhinitis*
;
Risk Factors*
;
Skin
10.Computer-assisted Evaluation of Pedicle Screw Position on CT Images.
Jin Sup YEOM ; Moon Sang CHUNG ; Choon Ki LEE ; Bong Soon CHANG ; Yeongho KIM ; Namkug KIM ; Jae Bum LEE
The Journal of the Korean Orthopaedic Association 2003;38(2):165-171
PURPOSE: The purpose of this study was to develop a personal computer-based method to facilitate the evaluation of pedicle screw position on computed tomography (CT) scan images and to assess its diagnostic value. MATERIAL AND METHOD: A personal computer-based method was developed using the CT images of 17 patients having a total of 84 pedicle screws. Images with a window range of -2, 000 to +3, 000 were inverted; a multiplanar reconstruction viewer was then produced to create these images in the sagittal and coronal planes. Finally, lines circumscribing the threaded portion of screws were drawn on the images. For CT images of thirty-two pedicle screws placed in the lumbar vertebrae of four pigs, screw locations were evaluated by 6 orthopaedic surgeons by our method and by conventional bone window setting. The diagnostic values of the two methods were calculated and compared. RESULT: Our method significantly improved the specificity (from 82% to 94%, p=0.007), the positive predictive value (from 79% to 92%, p=0.015), and inter-observer agreement (from 0.61 to 0.78, p<0.001) in terms of identifying misplaced screws. CONCLUSION: The described method improves the diagnostic accuracy and inter-observer reliability for the identification of misplaced pedicle screws on CT scan images.
Humans
;
Lumbar Vertebrae
;
Sensitivity and Specificity
;
Swine
;
Tomography, X-Ray Computed

Result Analysis
Print
Save
E-mail