1.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
2.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
3.Breast Cancer Statistics in Korea, 2021
Chihwan David CHA ; Chan Sub PARK ; Hee-Chul SHIN ; Jaihong HAN ; Jung Eun CHOI ; Joo Heung KIM ; Kyu-Won JUNG ; Sae Byul LEE ; Sang Eun NAM ; Tae In YOON ; Young-Joon KANG ; Zisun KIM ; So-Youn JUNG ; Hyun-Ah KIM ;
Journal of Breast Cancer 2024;27(6):351-361
The Korean Breast Cancer Society (KBCS) has collected nationwide registry data on clinicopathologic characteristics and treatment since 1996. This study aimed to analyze the clinical characteristics of breast cancer in Korea and assess changes in breast cancer statistics for 2021 using data from the KBCS registry and the Korean Central Cancer Registry. In 2021, 34,628 women were newly diagnosed with breast cancer. The median age of women diagnosed with breast cancer was 53.4 years, with the highest incidence occurring in the 40–49 age group. The most common molecular subtype was hormone receptor-positive and human epidermal growth factor receptor 2 (HER2)-negative, accounting for 69.1% of cases, while HER2-positive subtypes comprised 19.3%. During the coronavirus disease 2019 pandemic, the national breast cancer screening rate declined. However, the incidence of early-stage breast cancer (stages 0 and I) continued to increase, accounting for 65.6% of newly diagnosed cases in 2021. Our results showed that the overall survival rate for patients with breast cancer has improved, primarily due to a rise in early-stage diagnoses and advancements in treatment.
4.Breast Cancer Statistics in Korea, 2021
Chihwan David CHA ; Chan Sub PARK ; Hee-Chul SHIN ; Jaihong HAN ; Jung Eun CHOI ; Joo Heung KIM ; Kyu-Won JUNG ; Sae Byul LEE ; Sang Eun NAM ; Tae In YOON ; Young-Joon KANG ; Zisun KIM ; So-Youn JUNG ; Hyun-Ah KIM ;
Journal of Breast Cancer 2024;27(6):351-361
The Korean Breast Cancer Society (KBCS) has collected nationwide registry data on clinicopathologic characteristics and treatment since 1996. This study aimed to analyze the clinical characteristics of breast cancer in Korea and assess changes in breast cancer statistics for 2021 using data from the KBCS registry and the Korean Central Cancer Registry. In 2021, 34,628 women were newly diagnosed with breast cancer. The median age of women diagnosed with breast cancer was 53.4 years, with the highest incidence occurring in the 40–49 age group. The most common molecular subtype was hormone receptor-positive and human epidermal growth factor receptor 2 (HER2)-negative, accounting for 69.1% of cases, while HER2-positive subtypes comprised 19.3%. During the coronavirus disease 2019 pandemic, the national breast cancer screening rate declined. However, the incidence of early-stage breast cancer (stages 0 and I) continued to increase, accounting for 65.6% of newly diagnosed cases in 2021. Our results showed that the overall survival rate for patients with breast cancer has improved, primarily due to a rise in early-stage diagnoses and advancements in treatment.
5.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
6.Breast Cancer Statistics in Korea, 2021
Chihwan David CHA ; Chan Sub PARK ; Hee-Chul SHIN ; Jaihong HAN ; Jung Eun CHOI ; Joo Heung KIM ; Kyu-Won JUNG ; Sae Byul LEE ; Sang Eun NAM ; Tae In YOON ; Young-Joon KANG ; Zisun KIM ; So-Youn JUNG ; Hyun-Ah KIM ;
Journal of Breast Cancer 2024;27(6):351-361
The Korean Breast Cancer Society (KBCS) has collected nationwide registry data on clinicopathologic characteristics and treatment since 1996. This study aimed to analyze the clinical characteristics of breast cancer in Korea and assess changes in breast cancer statistics for 2021 using data from the KBCS registry and the Korean Central Cancer Registry. In 2021, 34,628 women were newly diagnosed with breast cancer. The median age of women diagnosed with breast cancer was 53.4 years, with the highest incidence occurring in the 40–49 age group. The most common molecular subtype was hormone receptor-positive and human epidermal growth factor receptor 2 (HER2)-negative, accounting for 69.1% of cases, while HER2-positive subtypes comprised 19.3%. During the coronavirus disease 2019 pandemic, the national breast cancer screening rate declined. However, the incidence of early-stage breast cancer (stages 0 and I) continued to increase, accounting for 65.6% of newly diagnosed cases in 2021. Our results showed that the overall survival rate for patients with breast cancer has improved, primarily due to a rise in early-stage diagnoses and advancements in treatment.
7.Using Large Language Models to Extract Core Injury Information From Emergency Department Notes
Dong Hyun CHOI ; Yoonjic KIM ; Sae Won CHOI ; Ki Hong KIM ; Yeongho CHOI ; Sang Do SHIN
Journal of Korean Medical Science 2024;39(46):e291-
Background:
Injuries pose a significant global health challenge due to their high incidence and mortality rates. Although injury surveillance is essential for prevention, it is resource-intensive.This study aimed to develop and validate locally deployable large language models (LLMs) to extract core injury-related information from Emergency Department (ED) clinical notes.
Methods:
We conducted a diagnostic study using retrospectively collected data from January 2014 to December 2020 from two urban academic tertiary hospitals. One served as the derivation cohort and the other as the external test cohort. Adult patients presenting to the ED with injury-related complaints were included. Primary outcomes included classification accuracies for information extraction tasks related to injury mechanism, place of occurrence, activity, intent, and severity. We fine-tuned a single generalizable Llama-2 model and five distinct Bidirectional Encoder Representations from Transformers (BERT) models for each task to extract information from initial ED physician notes. The Llama-2 model was able to perform different tasks by modifying the instruction prompt. Data recorded in injury registries provided the gold standard labels. Model performance was assessed using accuracy and macro-average F1 scores.
Results:
The derivation and external test cohorts comprised 36,346 and 32,232 patients, respectively. In the derivation cohort’s test set, the Llama-2 model achieved accuracies (95% confidence intervals) of 0.899 (0.889–0.909) for injury mechanism, 0.774 (0.760–0.789) for place of occurrence, 0.679 (0.665–0.694) for activity, 0.972 (0.967–0.977) for intent, and 0.935 (0.926–0.943) for severity. The Llama-2 model outperformed the BERT models in accuracy and macro-average F1 scores across all tasks in both cohorts. Imposing constraints on the Llama-2 model to avoid uncertain predictions further improved its accuracy.
Conclusion
Locally deployable LLMs, trained to extract core injury-related information from free-text ED clinical notes, demonstrated good performance. Generative LLMs can serve as versatile solutions for various injury-related information extraction tasks.
8.Practice guidelines for managing extrahepatic biliary tract cancers
Hyung Sun KIM ; Mee Joo KANG ; Jingu KANG ; Kyubo KIM ; Bohyun KIM ; Seong-Hun KIM ; Soo Jin KIM ; Yong-Il KIM ; Joo Young KIM ; Jin Sil KIM ; Haeryoung KIM ; Hyo Jung KIM ; Ji Hae NAHM ; Won Suk PARK ; Eunkyu PARK ; Joo Kyung PARK ; Jin Myung PARK ; Byeong Jun SONG ; Yong Chan SHIN ; Keun Soo AHN ; Sang Myung WOO ; Jeong Il YU ; Changhoon YOO ; Kyoungbun LEE ; Dong Ho LEE ; Myung Ah LEE ; Seung Eun LEE ; Ik Jae LEE ; Huisong LEE ; Jung Ho IM ; Kee-Taek JANG ; Hye Young JANG ; Sun-Young JUN ; Hong Jae CHON ; Min Kyu JUNG ; Yong Eun CHUNG ; Jae Uk CHONG ; Eunae CHO ; Eui Kyu CHIE ; Sae Byeol CHOI ; Seo-Yeon CHOI ; Seong Ji CHOI ; Joon Young CHOI ; Hye-Jeong CHOI ; Seung-Mo HONG ; Ji Hyung HONG ; Tae Ho HONG ; Shin Hye HWANG ; In Gyu HWANG ; Joon Seong PARK
Annals of Hepato-Biliary-Pancreatic Surgery 2024;28(2):161-202
Background:
s/Aims: Reported incidence of extrahepatic bile duct cancer is higher in Asians than in Western populations. Korea, in particular, is one of the countries with the highest incidence rates of extrahepatic bile duct cancer in the world. Although research and innovative therapeutic modalities for extrahepatic bile duct cancer are emerging, clinical guidelines are currently unavailable in Korea. The Korean Society of Hepato-Biliary-Pancreatic Surgery in collaboration with related societies (Korean Pancreatic and Biliary Surgery Society, Korean Society of Abdominal Radiology, Korean Society of Medical Oncology, Korean Society of Radiation Oncology, Korean Society of Pathologists, and Korean Society of Nuclear Medicine) decided to establish clinical guideline for extrahepatic bile duct cancer in June 2021.
Methods:
Contents of the guidelines were developed through subgroup meetings for each key question and a preliminary draft was finalized through a Clinical Guidelines Committee workshop.
Results:
In November 2021, the finalized draft was presented for public scrutiny during a formal hearing.
Conclusions
The extrahepatic guideline committee believed that this guideline could be helpful in the treatment of patients.
9.A Phase 1b/2a Study of GC1118 with 5-Fluorouracil, Leucovorin and Irinotecan (FOLFIRI) in Patients with Recurrent or Metastatic Colorectal Cancer
Keun-Wook LEE ; Sae-Won HAN ; Tae Won KIM ; Joong Bae AHN ; Ji Yeon BAEK ; Sang Hee CHO ; Howard LEE ; Jin Won KIM ; Ji-Won KIM ; Tae-You KIM ; Yong Sang HONG ; Seung-Hoon BEOM ; Yongjun CHA ; Yoonjung CHOI ; Seonhui KIM ; Yung-Jue BANG
Cancer Research and Treatment 2024;56(2):590-601
Purpose:
GC1118 is a novel antibody targeting epidermal growth factor receptor (EGFR) with enhanced blocking activity against both low- and high-affinity EGFR ligands. A phase 1b/2a study was conducted to determine a recommended phase 2 dose (RP2D) of GC1118 in combination with 5-fluorouracil, leucovorin, and irinotecan (FOLFIRI) (phase 1b) and to assess the safety and efficacy of GC1118 plus FOLFIRI as a second-line therapy for recurrent/metastatic colorectal cancer (CRC) (phase 2a).
Materials and Methods:
Phase 1b was designed as a standard 3+3 dose-escalation study with a starting dose of GC1118 (3 mg/kg/week) in combination with biweekly FOLFIRI (irinotecan 180 mg/m2; leucovorin 400 mg/m2; 5-fluorouracil 400 mg/m2 bolus and 2,400 mg/m2 infusion over 46 hours) in patients with solid tumors refractory to standard treatments. The subsequent phase 2a part was conducted with objective response rate (ORR) as a primary endpoint. Patients with KRAS/NRAS/BRAF wild-type, EGFR-positive, recurrent/metastatic CRC resistant to the first-line treatment were enrolled in the phase 2a study.
Results:
RP2D of GC1118 was determined to be 3 mg/kg/wk in the phase 1b study (n=7). Common adverse drug reactions (ADRs) observed in the phase 2a study (n=24) were acneiform rash (95.8%), dry skin (66.7%), paronychia (58.3%), and stomatitis (50.0%). The most common ADR of ≥ grade 3 was neutropenia (33.3%). ORR was 42.5% (95% confidence interval [CI], 23.5 to 62.0), and median progression-free survival was 6.7 months (95% CI, 4.0-8.0).
Conclusion
GC1118 administered weekly at 3 mg/kg in combination with FOLFIRI appears as an effective and safe treatment option in recurrent/metastatic CRC.
10.Impacts of Tocolytics on Maternal and Neonatal Glucose Levels in Women With Gestational Diabetes Mellitus
Subeen HONG ; Hyun-Joo SEOL ; JoonHo LEE ; Han Sung HWANG ; Ji-Hee SUNG ; Ji Young KWON ; Seung Mi LEE ; Won Joon SEONG ; Soo Ran CHOI ; Seung Chul KIM ; Hee-Sun KIM ; Se Jin LEE ; Sae-Kyung CHOI ; Kyung A LEE ; Hyun Sun KO ; Hyun Soo PARK ;
Journal of Korean Medical Science 2024;39(34):e236-
Background:
We investigated the impacts of tocolytic agents on maternal and neonatal blood glucose levels in women with gestational diabetes mellitus (GDM) who used tocolytics for preterm labor.
Methods:
This multi-center, retrospective cohort study included women with GDM who were admitted for preterm labor from twelve hospitals in South Korea. We excluded women with multiple pregnancies, anomalies, overt DM diagnosed before pregnancy or 23 weeks of gestation, and women who received multiple tocolytics. The patients were divided according to the types of tocolytics; atosiban, ritodrine, and nifedipine group. We collected baseline maternal characteristics, pregnancy outcomes, maternal glucose levels during hospitalization, and neonatal glucose levels. We compared the frequency of maternal hyperglycemia and neonatal hypoglycemia among three groups. A multivariate logistic regression analysis was performed to evaluate the contributing factors to the occurrence of maternal hyperglycemia and neonatal hypoglycemia. Results: A total of 128 women were included: 44 (34.4%), 51 (39.8%), and 33 (25.8%) women received atosiban, ritodrine, and nifedipine, respectively. Mean fasting blood glucose (FBG) (112.3, 109.6, and 89.5 mg/dL, P < 0.001) and 2-hour postprandial glucose (PPG2) levels (145.4, 148.3, and 116.5 mg/dL, P = 0.004) were significantly higher in atosiban and ritodrine group than those in nifedipine group. Even after adjusting for covariates including antenatal steroid use, gestational age at admission, and pre-pregnancy body mass index, there was an increased risk of high maternal mean FBG (≥ 95 mg/dL) and PPG2 (≥ 120 mg/dL) levels in the atosiban and ritodrine group than in nifedipine group. The atosiban and ritodrine groups are also at increased risk of neonatal hypoglycemia (< 47 mg/dL) compared to the nifedipine group with the odds ratio of 4.58 and 4.67, respectively (P < 0.05).
Conclusion
There is an increased risk of maternal hyperglycemia and neonatal hypoglycemia in women with GDM using atosiban and ritodrine tocolytics for preterm labor compared to those using nifedipine.

Result Analysis
Print
Save
E-mail