1.ChatGPT goes to the operating room: evaluating GPT-4performance and its potential in surgical education and training in the era of large language models
Namkee OH ; Gyu-Seong CHOI ; Woo Yong LEE
Annals of Surgical Treatment and Research 2023;104(5):269-273
Purpose:
This study aimed to assess the performance of ChatGPT, specifically the GPT-3.5 and GPT-4 models, in understanding complex surgical clinical information and its potential implications for surgical education and training.
Methods:
The dataset comprised 280 questions from the Korean general surgery board exams conducted between 2020 and 2022. Both GPT-3.5 and GPT-4 models were evaluated, and their performances were compared using McNemar test.
Results:
GPT-3.5 achieved an overall accuracy of 46.8%, while GPT-4 demonstrated a significant improvement with an overall accuracy of 76.4%, indicating a notable difference in performance between the models (P < 0.001). GPT-4 also exhibited consistent performance across all subspecialties, with accuracy rates ranging from 63.6% to 83.3%.
Conclusion
ChatGPT, particularly GPT-4, demonstrates a remarkable ability to understand complex surgical clinical information, achieving an accuracy rate of 76.4% on the Korean general surgery board exam. However, it is important to recognize the limitations of large language models and ensure that they are used in conjunction with human expertise and judgment.
2.Characteristics and Distribution of Surgical Diseases in North Korean Research Papers Published between 2006 and 2017
Yo Han LEE ; Namkee OH ; Hyerim KIM ; Shin HA
Journal of Korean Medical Science 2021;36(12):e25-
Background:
Little is known about the surgical discipline in North Korea from the perspective of the outside world. This study aimed to examine the disease entities covered by articles published in the major medical journal in North Korea, “Surgery.”
Methods:
Content and frequency analyses of 2,132 articles published in “Surgery” from 2006 to 2017 were conducted. Two medical doctors who majored in Surgery and anesthesiology perused the articles and compiled the diseases being elucidated in each article. The diseases described in each article were stratified into 13 surgical subspecialties.
Results:
Articles from “Surgery,” similar to articles from the Western surgical community, also covered a wide variety of surgical diseases from different subspecialties, and the number of publications continued to grow consistently. Moreover, a number of studies focused on the fields of orthopedics and general Surgery dealing with benign diseases. Some articles focused on minimally invasive surgeries using laparoscopy.
Conclusion
The studies published in the North Korean journal “Surgery” encompass various clinical areas, but their quality is unclear.
3.Characteristics and Distribution of Surgical Diseases in North Korean Research Papers Published between 2006 and 2017
Yo Han LEE ; Namkee OH ; Hyerim KIM ; Shin HA
Journal of Korean Medical Science 2021;36(12):e25-
Background:
Little is known about the surgical discipline in North Korea from the perspective of the outside world. This study aimed to examine the disease entities covered by articles published in the major medical journal in North Korea, “Surgery.”
Methods:
Content and frequency analyses of 2,132 articles published in “Surgery” from 2006 to 2017 were conducted. Two medical doctors who majored in Surgery and anesthesiology perused the articles and compiled the diseases being elucidated in each article. The diseases described in each article were stratified into 13 surgical subspecialties.
Results:
Articles from “Surgery,” similar to articles from the Western surgical community, also covered a wide variety of surgical diseases from different subspecialties, and the number of publications continued to grow consistently. Moreover, a number of studies focused on the fields of orthopedics and general Surgery dealing with benign diseases. Some articles focused on minimally invasive surgeries using laparoscopy.
Conclusion
The studies published in the North Korean journal “Surgery” encompass various clinical areas, but their quality is unclear.
4.Digital Twins in Healthcare and Their Applicability in Rhinology: A Narrative Review
Minhae PARK ; Namkee OH ; Yong Gi JUNG
Journal of Rhinology 2023;30(2):80-86
Digital twins were initially introduced in the aerospace industry, but they have been applied to the medical field in the 2020s. The development of the Internet of Things, sensor technology, cloud computing, big data analysis, and simulation technology has made this idea feasible. Essentially, digital twins are virtual representations of real-world data that can generate virtual outcomes related to a patient based on their actual data. With this technology, doctors can predict treatment outcomes, plan surgery, and monitor patients’ medical conditions in real time. While digital twins have endless potential, challenges include the need to deal with vast amounts of data and ensure the security of personal information. In the field of rhinology, which deals with complex anatomy from the sinus to the skull base, the adoption of digital twins is just beginning. Digital twins have begun to be incorporated into surgical navigation and the management of chronic diseases such as chronic rhinosinusitis. Despite the limitless potential of digital twins, challenges related to dealing with vast amounts of data and enhancing the security of personal data need to be surmounted for this method to be more widely applied.
5.Left lobe living donor liver transplantation using the resection and partial liver segment 2–3 transplantation with delayed total hepatectomy (RAPID) procedure in cirrhotic patients:First case report in Korea
Jongman KIM ; Jinsoo RHU ; Eunjin LEE ; Youngju RYU ; Sunghyo AN ; Sung Jun JO ; Namkee OH ; Seungwook HAN ; Sunghae PARK ; Gyu-Seong CHOI
Annals of Hepato-Biliary-Pancreatic Surgery 2024;28(3):388-392
In liver transplantation, the primary concern is to ensure an adequate future liver remnant (FLR) volume for the donor, while selecting a graft of sufficient size for the recipient. The living donor–resection and partial liver segment 2−3 transplantation with delayed total hepatectomy (LD−RAPID) procedure offers a potential solution to expand the donor pool for living donor liver transplantation (LDLT).We report the first case involving a cirrhotic patient with autoimmune hepatitis and hepatocellular carcinoma, who underwent left lobe LDLT using the LD−RAPID procedure. The living liver donor (LLD) underwent a laparoscopic left hepatectomy, including middle hepatic vein. The resection on the recipient side was an extended left hepatectomy, including the middle hepatic vein orifice and caudate lobe. At postoperative day 7, a computed tomography scan showed hypertrophy of the left graft from 320 g to 465 mL (i.e., a 45.3% increase in graft volume body weight ratio from 0.60% to 0.77%). After a 7-day interval, the diseased right lobe was removed in the second stage surgery. The LD−RAPID procedure using left lobe graft allows for the use of a small liver graft or small FLR volume in LLD in LDLT, which expands the donor pool to minimize the risk to LLD by enabling the donation of a smaller liver portion.
6.Left lobe living donor liver transplantation using the resection and partial liver segment 2–3 transplantation with delayed total hepatectomy (RAPID) procedure in cirrhotic patients:First case report in Korea
Jongman KIM ; Jinsoo RHU ; Eunjin LEE ; Youngju RYU ; Sunghyo AN ; Sung Jun JO ; Namkee OH ; Seungwook HAN ; Sunghae PARK ; Gyu-Seong CHOI
Annals of Hepato-Biliary-Pancreatic Surgery 2024;28(3):388-392
In liver transplantation, the primary concern is to ensure an adequate future liver remnant (FLR) volume for the donor, while selecting a graft of sufficient size for the recipient. The living donor–resection and partial liver segment 2−3 transplantation with delayed total hepatectomy (LD−RAPID) procedure offers a potential solution to expand the donor pool for living donor liver transplantation (LDLT).We report the first case involving a cirrhotic patient with autoimmune hepatitis and hepatocellular carcinoma, who underwent left lobe LDLT using the LD−RAPID procedure. The living liver donor (LLD) underwent a laparoscopic left hepatectomy, including middle hepatic vein. The resection on the recipient side was an extended left hepatectomy, including the middle hepatic vein orifice and caudate lobe. At postoperative day 7, a computed tomography scan showed hypertrophy of the left graft from 320 g to 465 mL (i.e., a 45.3% increase in graft volume body weight ratio from 0.60% to 0.77%). After a 7-day interval, the diseased right lobe was removed in the second stage surgery. The LD−RAPID procedure using left lobe graft allows for the use of a small liver graft or small FLR volume in LLD in LDLT, which expands the donor pool to minimize the risk to LLD by enabling the donation of a smaller liver portion.
7.Left lobe living donor liver transplantation using the resection and partial liver segment 2–3 transplantation with delayed total hepatectomy (RAPID) procedure in cirrhotic patients:First case report in Korea
Jongman KIM ; Jinsoo RHU ; Eunjin LEE ; Youngju RYU ; Sunghyo AN ; Sung Jun JO ; Namkee OH ; Seungwook HAN ; Sunghae PARK ; Gyu-Seong CHOI
Annals of Hepato-Biliary-Pancreatic Surgery 2024;28(3):388-392
In liver transplantation, the primary concern is to ensure an adequate future liver remnant (FLR) volume for the donor, while selecting a graft of sufficient size for the recipient. The living donor–resection and partial liver segment 2−3 transplantation with delayed total hepatectomy (LD−RAPID) procedure offers a potential solution to expand the donor pool for living donor liver transplantation (LDLT).We report the first case involving a cirrhotic patient with autoimmune hepatitis and hepatocellular carcinoma, who underwent left lobe LDLT using the LD−RAPID procedure. The living liver donor (LLD) underwent a laparoscopic left hepatectomy, including middle hepatic vein. The resection on the recipient side was an extended left hepatectomy, including the middle hepatic vein orifice and caudate lobe. At postoperative day 7, a computed tomography scan showed hypertrophy of the left graft from 320 g to 465 mL (i.e., a 45.3% increase in graft volume body weight ratio from 0.60% to 0.77%). After a 7-day interval, the diseased right lobe was removed in the second stage surgery. The LD−RAPID procedure using left lobe graft allows for the use of a small liver graft or small FLR volume in LLD in LDLT, which expands the donor pool to minimize the risk to LLD by enabling the donation of a smaller liver portion.
8.Left lobe living donor liver transplantation using the resection and partial liver segment 2–3 transplantation with delayed total hepatectomy (RAPID) procedure in cirrhotic patients:First case report in Korea
Jongman KIM ; Jinsoo RHU ; Eunjin LEE ; Youngju RYU ; Sunghyo AN ; Sung Jun JO ; Namkee OH ; Seungwook HAN ; Sunghae PARK ; Gyu-Seong CHOI
Annals of Hepato-Biliary-Pancreatic Surgery 2024;28(3):388-392
In liver transplantation, the primary concern is to ensure an adequate future liver remnant (FLR) volume for the donor, while selecting a graft of sufficient size for the recipient. The living donor–resection and partial liver segment 2−3 transplantation with delayed total hepatectomy (LD−RAPID) procedure offers a potential solution to expand the donor pool for living donor liver transplantation (LDLT).We report the first case involving a cirrhotic patient with autoimmune hepatitis and hepatocellular carcinoma, who underwent left lobe LDLT using the LD−RAPID procedure. The living liver donor (LLD) underwent a laparoscopic left hepatectomy, including middle hepatic vein. The resection on the recipient side was an extended left hepatectomy, including the middle hepatic vein orifice and caudate lobe. At postoperative day 7, a computed tomography scan showed hypertrophy of the left graft from 320 g to 465 mL (i.e., a 45.3% increase in graft volume body weight ratio from 0.60% to 0.77%). After a 7-day interval, the diseased right lobe was removed in the second stage surgery. The LD−RAPID procedure using left lobe graft allows for the use of a small liver graft or small FLR volume in LLD in LDLT, which expands the donor pool to minimize the risk to LLD by enabling the donation of a smaller liver portion.
9.ChatGPT Predicts In-Hospital All-Cause Mortality for Sepsis: In-Context Learning with the Korean Sepsis Alliance Database
Namkee OH ; Won Chul CHA ; Jun Hyuk SEO ; Seong-Gyu CHOI ; Jong Man KIM ; Chi Ryang CHUNG ; Gee Young SUH ; Su Yeon LEE ; Dong Kyu OH ; Mi Hyeon PARK ; Chae-Man LIM ; Ryoung-Eun KO ;
Healthcare Informatics Research 2024;30(3):266-276
Objectives:
Sepsis is a leading global cause of mortality, and predicting its outcomes is vital for improving patient care. This study explored the capabilities of ChatGPT, a state-of-the-art natural language processing model, in predicting in-hospital mortality for sepsis patients.
Methods:
This study utilized data from the Korean Sepsis Alliance (KSA) database, collected between 2019 and 2021, focusing on adult intensive care unit (ICU) patients and aiming to determine whether ChatGPT could predict all-cause mortality after ICU admission at 7 and 30 days. Structured prompts enabled ChatGPT to engage in in-context learning, with the number of patient examples varying from zero to six. The predictive capabilities of ChatGPT-3.5-turbo and ChatGPT-4 were then compared against a gradient boosting model (GBM) using various performance metrics.
Results:
From the KSA database, 4,786 patients formed the 7-day mortality prediction dataset, of whom 718 died, and 4,025 patients formed the 30-day dataset, with 1,368 deaths. Age and clinical markers (e.g., Sequential Organ Failure Assessment score and lactic acid levels) showed significant differences between survivors and non-survivors in both datasets. For 7-day mortality predictions, the area under the receiver operating characteristic curve (AUROC) was 0.70–0.83 for GPT-4, 0.51–0.70 for GPT-3.5, and 0.79 for GBM. The AUROC for 30-day mortality was 0.51–0.59 for GPT-4, 0.47–0.57 for GPT-3.5, and 0.76 for GBM. Zero-shot predictions using GPT-4 for mortality from ICU admission to day 30 showed AUROCs from the mid-0.60s to 0.75 for GPT-4 and mainly from 0.47 to 0.63 for GPT-3.5.
Conclusions
GPT-4 demonstrated potential in predicting short-term in-hospital mortality, although its performance varied across different evaluation metrics.
10.ChatGPT Predicts In-Hospital All-Cause Mortality for Sepsis: In-Context Learning with the Korean Sepsis Alliance Database
Namkee OH ; Won Chul CHA ; Jun Hyuk SEO ; Seong-Gyu CHOI ; Jong Man KIM ; Chi Ryang CHUNG ; Gee Young SUH ; Su Yeon LEE ; Dong Kyu OH ; Mi Hyeon PARK ; Chae-Man LIM ; Ryoung-Eun KO ;
Healthcare Informatics Research 2024;30(3):266-276
Objectives:
Sepsis is a leading global cause of mortality, and predicting its outcomes is vital for improving patient care. This study explored the capabilities of ChatGPT, a state-of-the-art natural language processing model, in predicting in-hospital mortality for sepsis patients.
Methods:
This study utilized data from the Korean Sepsis Alliance (KSA) database, collected between 2019 and 2021, focusing on adult intensive care unit (ICU) patients and aiming to determine whether ChatGPT could predict all-cause mortality after ICU admission at 7 and 30 days. Structured prompts enabled ChatGPT to engage in in-context learning, with the number of patient examples varying from zero to six. The predictive capabilities of ChatGPT-3.5-turbo and ChatGPT-4 were then compared against a gradient boosting model (GBM) using various performance metrics.
Results:
From the KSA database, 4,786 patients formed the 7-day mortality prediction dataset, of whom 718 died, and 4,025 patients formed the 30-day dataset, with 1,368 deaths. Age and clinical markers (e.g., Sequential Organ Failure Assessment score and lactic acid levels) showed significant differences between survivors and non-survivors in both datasets. For 7-day mortality predictions, the area under the receiver operating characteristic curve (AUROC) was 0.70–0.83 for GPT-4, 0.51–0.70 for GPT-3.5, and 0.79 for GBM. The AUROC for 30-day mortality was 0.51–0.59 for GPT-4, 0.47–0.57 for GPT-3.5, and 0.76 for GBM. Zero-shot predictions using GPT-4 for mortality from ICU admission to day 30 showed AUROCs from the mid-0.60s to 0.75 for GPT-4 and mainly from 0.47 to 0.63 for GPT-3.5.
Conclusions
GPT-4 demonstrated potential in predicting short-term in-hospital mortality, although its performance varied across different evaluation metrics.