Objectives: Developing large language models (LLMs) in biomedicine requires access to high-quality training and alignment tuning datasets. However, publicly available Korean medical preference datasets are scarce, hindering the advancement of Korean medical LLMs. This study constructs and evaluates the efficacy of the Korean Medical Preference Dataset (KoMeP), an alignment tuning dataset constructed with an automated pipeline, minimizing the high costs of human annotation. Methods: KoMeP was generated using the DAHL score, an automated hallucination evaluation metric. Five LLMs (Dolly-v2-3B, MPT-7B, GPT-4o, Qwen-2-7B, Llama-3-8B) produced responses to 8,573 biomedical examination questions, from which 5,551 preference pairs were extracted. Each pair consisted of a “chosen” response and a “rejected” response, as determined by their DAHL scores. The dataset was evaluated when trained through two different alignment tuning methods, direct preference optimization (DPO) and odds ratio preference optimization (ORPO) respectively across five different models. The KorMedMCQA benchmark was employed to assess the effectiveness of alignment tuning. Results: Models trained with DPO consistently improved KorMedMCQA performance; notably, Llama-3.1-8B showed a 43.96% increase. In contrast, ORPO training produced inconsistent results. Additionally, English-to-Korean transfer learning proved effective, particularly for English-centric models like Gemma-2, whereas Korean-to-English transfer learning achieved limited success. Instruction tuning with KoMeP yielded mixed outcomes, which suggests challenges in dataset formatting. Conclusions KoMeP is the first publicly available Korean medical preference dataset and significantly improves alignment tuning performance in LLMs. The DPO method outperforms ORPO in alignment tuning. Future work should focus on expanding KoMeP, developing a Korean-native dataset, and refining alignment tuning methods to produce safer and more reliable Korean medical LLMs.

Objectives: Developing large language models (LLMs) in biomedicine requires access to high-quality training and alignment tuning datasets. However, publicly available Korean medical preference datasets are scarce, hindering the advancement of Korean medical LLMs. This study constructs and evaluates the efficacy of the Korean Medical Preference Dataset (KoMeP), an alignment tuning dataset constructed with an automated pipeline, minimizing the high costs of human annotation. Methods: KoMeP was generated using the DAHL score, an automated hallucination evaluation metric. Five LLMs (Dolly-v2-3B, MPT-7B, GPT-4o, Qwen-2-7B, Llama-3-8B) produced responses to 8,573 biomedical examination questions, from which 5,551 preference pairs were extracted. Each pair consisted of a “chosen” response and a “rejected” response, as determined by their DAHL scores. The dataset was evaluated when trained through two different alignment tuning methods, direct preference optimization (DPO) and odds ratio preference optimization (ORPO) respectively across five different models. The KorMedMCQA benchmark was employed to assess the effectiveness of alignment tuning. Results: Models trained with DPO consistently improved KorMedMCQA performance; notably, Llama-3.1-8B showed a 43.96% increase. In contrast, ORPO training produced inconsistent results. Additionally, English-to-Korean transfer learning proved effective, particularly for English-centric models like Gemma-2, whereas Korean-to-English transfer learning achieved limited success. Instruction tuning with KoMeP yielded mixed outcomes, which suggests challenges in dataset formatting. Conclusions KoMeP is the first publicly available Korean medical preference dataset and significantly improves alignment tuning performance in LLMs. The DPO method outperforms ORPO in alignment tuning. Future work should focus on expanding KoMeP, developing a Korean-native dataset, and refining alignment tuning methods to produce safer and more reliable Korean medical LLMs.

Objectives: Developing large language models (LLMs) in biomedicine requires access to high-quality training and alignment tuning datasets. However, publicly available Korean medical preference datasets are scarce, hindering the advancement of Korean medical LLMs. This study constructs and evaluates the efficacy of the Korean Medical Preference Dataset (KoMeP), an alignment tuning dataset constructed with an automated pipeline, minimizing the high costs of human annotation. Methods: KoMeP was generated using the DAHL score, an automated hallucination evaluation metric. Five LLMs (Dolly-v2-3B, MPT-7B, GPT-4o, Qwen-2-7B, Llama-3-8B) produced responses to 8,573 biomedical examination questions, from which 5,551 preference pairs were extracted. Each pair consisted of a “chosen” response and a “rejected” response, as determined by their DAHL scores. The dataset was evaluated when trained through two different alignment tuning methods, direct preference optimization (DPO) and odds ratio preference optimization (ORPO) respectively across five different models. The KorMedMCQA benchmark was employed to assess the effectiveness of alignment tuning. Results: Models trained with DPO consistently improved KorMedMCQA performance; notably, Llama-3.1-8B showed a 43.96% increase. In contrast, ORPO training produced inconsistent results. Additionally, English-to-Korean transfer learning proved effective, particularly for English-centric models like Gemma-2, whereas Korean-to-English transfer learning achieved limited success. Instruction tuning with KoMeP yielded mixed outcomes, which suggests challenges in dataset formatting. Conclusions KoMeP is the first publicly available Korean medical preference dataset and significantly improves alignment tuning performance in LLMs. The DPO method outperforms ORPO in alignment tuning. Future work should focus on expanding KoMeP, developing a Korean-native dataset, and refining alignment tuning methods to produce safer and more reliable Korean medical LLMs.

Purpose: The incidence of gallstone disease and cholecystectomy is increasing worldwide. The aim of this study was to determine trends in the incidence of cholecystectomy in Korea. Methods: The National Health Insurance Services database was used to determine patterns in proportion of cholecystectomy and cholecystostomy in the total population of Korea from 2003 to 2017. The age-standardized rate (ASR) was calculated to compare the cholecystectomy and cholecystostomy according to changes in the population structure over time. The ASR was investigated according to patient age, sex, socioeconomic status, use of computed tomography, and type of hospital to identify trends. Results: The ASR per 100,000 based on the 2010 population of cholecystectomy cases increased markedly from 67.7 to 211.4 between 2003 and 2017. The ASR was consistently higher in female than male (71.9 vs. 63.6 in 2003, 221.8 vs. 201.8 in 2017). Furthermore, the ASR for cholecystectomy increased with age, and surgery for gallstone disease was performed more often at a specialized center than at other medical facilities. The length of hospital stay of cholecystectomy decreased steadily from 10.6 days in 2003 to 6.9 days in 2017. Conclusion This study shows that the incidence of cholecystectomy and cholecystostomy has steadily increased over the years in Korea, with a trend toward older age and higher socioeconomic status in patients undergoing cholecystectomy. Increasing use of computed tomography investigations could be a primary cause for this trend. An integrated strategy is needed to manage the increase in older patients undergoing cholecystectomy and shorten their hospital stay with medical safety.

Background: Advances in medicine and changes in the medical environment can affect the diagnosis and treatment of diseases. The main purpose of the present study was to investigate whether the difference in accessibility to diagnosis and treatment facilities influenced the occurrence of appendectomy in Korea. Methods: We collected data on 183,531 appendectomy patients between 2003 and 2017 using the National Health Insurance Services claims. Retrospective analysis of relationship between the age-standardized rate (ASR) of appendectomy and clinical variables affecting medical accessibility was performed. Pearson’s correlation analyses were used. Results: The incidence of appendectomy decreased from 30,164 cases in 2003 to 7,355 cases in 2017. The rate of computerized tomography (CT) scans for diagnosis of appendicitis increased from 4.73% in 2003 to 86.96% in 2017. The ASR of appendectomy in uncomplicated and complicated appendicitis decreased from 48.71 in 2005 to 13.40 in 2010 and 8.37 in 2005 to 2.96 in 2009, respectively. The ASR of appendectomy was higher in the high-income group.The proportion and ASR of appendectomy in older age group increased steadily with years.The total admission days continued to decrease from 6.02 days in 2003 to 4.96 days in 2017. Conclusion The incidence of appendectomy was seemingly associated with the rate of CT scan. In particular, the incidence of appendectomy in uncomplicated appendicitis was markedly reduced. Through enhanced accessibility to CT scans, accurate diagnosis and treatment of appendicitis can be facilitated.

BACKGROUND AND PURPOSE: Thrombolysis is underused in acute ischemic stroke, mainly due to the reluctance of physicians to treat thrombolysis patients. However, a computerized clinical decision support system can help physicians to develop individualized stroke treatments. METHODS: A consecutive series of 958 patients, hospitalized within 12 hours of ischemic stroke onset from a representative clinical center in Korea, was used to establish a prognostic model. Multivariable logistic regression was used to develop the model for global and safety outcomes. An external validation of developed model was performed using 954 patients data obtained from 5 university hospitals or regional stroke centers. RESULTS: Final global outcome predictors were age; previous modified Rankin scale score; initial National Institutes of Health Stroke Scale (NIHSS) score; previous stroke; diabetes; prior use of antiplatelet treatment, antihypertensive drugs, and statins; lacunae; thrombolysis; onset to treatment time; and systolic blood pressure. Final safety outcome predictors were age, initial NIHSS score, thrombolysis, onset to treatment time, systolic blood pressure, and glucose level. The discriminative ability of the prognostic model showed a C-statistic of 0.89 and 0.84 for the global and safety outcomes, respectively. Internal and external validation showed similar C-statistic results. After updating the model, calibration slopes were corrected from 0.68 to 1.0 and from 0.96 to 1.0 for the global and safety outcome models, respectively. CONCLUSIONS: A novel computerized outcome prediction model for thrombolysis after ischemic stroke was developed using large amounts of clinical information. After external validation and updating, the model's performance was deemed clinically satisfactory.
Antihypertensive Agents ; Blood Pressure ; Calibration ; Glucose ; Hospitals, University ; Humans ; Hydroxymethylglutaryl-CoA Reductase Inhibitors ; Korea ; Logistic Models ; National Institutes of Health (U.S.) ; Stroke*

OBJECTIVES: Adoption of hospital information systems offers distinctive advantages in healthcare delivery. First, implementation of consolidated hospital information system in Seoul National University Hospital led to significant improvements in quality of healthcare and efficiency of hospital management. METHODS: The hospital information system in Seoul National University Hospital consists of component applications: clinical information systems, clinical research support systems, administrative information systems, management information systems, education support systems, and referral systems that operate to generate utmost performance when delivering healthcare services. RESULTS: Clinical information systems, which consist of such applications as electronic medical records, picture archiving and communication systems, primarily support clinical activities. Clinical research support system provides valuable resources supporting various aspects of clinical activities, ranging from management of clinical laboratory tests to establishing care-giving procedures. CONCLUSIONS: Seoul National University Hospital strives to move its hospital information system to a whole new level, which enables customized healthcare service and fulfills individual requirements. The current information strategy is being formulated as an initial step of development, promoting the establishment of next-generation hospital information system.
Adoption ; Confidentiality ; Delivery of Health Care ; Electronic Health Records ; Hospital Information Systems ; Information Systems ; Management Information Systems ; Quality of Health Care ; Radiology Information Systems ; Referral and Consultation