4.Effects of Expert-Determined Reference Standards in Evaluating the Diagnostic Performance of a Deep Learning Model: A Malignant Lung Nodule Detection Task on Chest Radiographs
Jung Eun HUH ; Jong Hyuk LEE ; Eui Jin HWANG ; Chang Min PARK
Korean Journal of Radiology 2023;24(2):155-165
Objective:
Little is known about the effects of using different expert-determined reference standards when evaluating the performance of deep learning-based automatic detection (DLAD) models and their added value to radiologists. We assessed the concordance of expert-determined standards with a clinical gold standard (herein, pathological confirmation) and the effects of different expert-determined reference standards on the estimates of radiologists’ diagnostic performance to detect malignant pulmonary nodules on chest radiographs with and without the assistance of a DLAD model.
Materials and Methods:
This study included chest radiographs from 50 patients with pathologically proven lung cancer and 50 controls. Five expert-determined standards were constructed using the interpretations of 10 experts: individual judgment by the most experienced expert, majority vote, consensus judgments of two and three experts, and a latent class analysis (LCA) model. In separate reader tests, additional 10 radiologists independently interpreted the radiographs and then assisted with the DLAD model. Their diagnostic performance was estimated using the clinical gold standard and various expertdetermined standards as the reference standard, and the results were compared using the t test with Bonferroni correction.
Results:
The LCA model (sensitivity, 72.6%; specificity, 100%) was most similar to the clinical gold standard. When expertdetermined standards were used, the sensitivities of radiologists and DLAD model alone were overestimated, and their specificities were underestimated (all p-values < 0.05). DLAD assistance diminished the overestimation of sensitivity but exaggerated the underestimation of specificity (all p-values < 0.001). The DLAD model improved sensitivity and specificity to a greater extent when using the clinical gold standard than when using the expert-determined standards (all p-values < 0.001), except for sensitivity with the LCA model (p = 0.094).
Conclusion
The LCA model was most similar to the clinical gold standard for malignant pulmonary nodule detection on chest radiographs. Expert-determined standards caused bias in measuring the diagnostic performance of the artificial intelligence model.
5.The Report for Emergency Resident's Training Status and Improvement.
Keun Jeong SONG ; Jung Bae PARK ; Hyuk Jun YANG ; Boo Soo LEE ; Joong Eui RHEE ; Yong Su LIM ; Seung Pil CHOI ; Tag HEO ; Sung Oh HWANG
Journal of the Korean Society of Emergency Medicine 2003;14(3):217-223
Emergency Medicine is a medical specialty that provides comprehensive emergency care. The spectrum of emergency medicine includes humanity as well as medical care for emergency patients. Since the Korean Society of Emergency Medicine has been established and the training course for emergency residents has begun, quality improvement of the training course still remains to be one of the most important issues in emergency medicine. This report is to understand the current status of training of emergency residents and to improve the quality of training of emergency residents. The survey was performed with questionnaires regarding current status of training and opinions to improve quality of training. The questionnaires were sent to all emergency physicians and emergency residents registered to the Korean Society of Emergency Medicine via an electronic mail or a printed letter. The response rates were 49.6% for emergency physicians and 35.8% for emergency residents. On the basis of the result of this survey, we suggest five proposals for improving the quality of resident training course. These include complete adherence of each institution to training guidelines presented by the Korean Society of Emergency Medicine, development of the education program compatible to each institution, shortening of length of training duration from 4 years to 3 years, promotion of the training environment, and introduction of evaluation examination to the training course.
Education
;
Electronic Mail
;
Emergencies*
;
Emergency Medical Services
;
Emergency Medicine
;
Humans
;
Internship and Residency
;
Quality Improvement
;
Surveys and Questionnaires
6.A Case of Primary Duodenal Carcinoid Tumor.
Joon Sang LEE ; Jong Hoon YOO ; Jeong Seok LEE ; Eui Hyuk CHOI ; Dong Yun LEE ; Soo Hee HONG ; Hyung Joo KWON ; Moo Yeol LEE ; Young Woo PARK ; Cheol Ho LEE ; Young Mug KIM ; Soon Chul HWANG
Korean Journal of Gastrointestinal Endoscopy 1999;19(2):275-280
Carcinoid tumors are slow growing, rare neoplasms that arise from enterochromaffin cells, with malignant potential. Primary duodenal carcinoid tumors are rare, the reported incidence being 2.0 ~8.9% of all gastrointestinal carcinoid tumors. Unless the carcinoid syndrome has developed, the non-specificity of the symptoms and physical findings, makes the clinical diagnosis of these tumors difficult. However, with the introduction of large and deep endoscopic biopsies, it is possible to diagnose duodenal carcinoid tumors at an early stage. Usually, local resection is the therapy of choice because of the very slow growth of the lesion and the low incidence of metastasis. Radical surgery is mandatory only in lesions more than 2 cm in diameter, or in the presence of muscular invasion. A 69-year-old female visited our medical department, having had symptoms of upper ab-dominal pain and intermittent melena for 1 week. Endoscopic examination showed 2.5 2.5 cm sized elevated mass lesion with central ulceration in the anterior wall side of the duodenal bulb. Immunohistochemical stains of the biopsy specimen showed that the tumor cells are positive reactivity for NSE (neuron-specific-enolase), chromogranin A, and cyto-keratin.
Aged
;
Biopsy
;
Carcinoid Tumor*
;
Chromogranin A
;
Coloring Agents
;
Diagnosis
;
Duodenum
;
Enterochromaffin Cells
;
Female
;
Humans
;
Incidence
;
Melena
;
Neoplasm Metastasis
;
Ulcer
7.A Case of Foreign Body Granuloma of the Posterior Mediastinum.
Eui Hyuk CHOI ; Jong Hoon YOO ; Jeong Seok LEE ; Su Hee HONG ; Hyung Joo GWON ; Young Woo PARK ; Moo Yeol LEE ; Chul Ho LEE ; Jung Hyun BANG ; Sung Gyun IM ; Soon Chul HWANG
Tuberculosis and Respiratory Diseases 1998;45(3):609-613
21-year-old male was admitted for evaluation of a mass shadow on chest film. On chest computed tomography showed 5 cm sized homogeneous low density based on the second thoracic vertebral body in the posterior mediastinum. The patient had been performed thoracic sympathectomy 6 months before admission and oxidized cellulose was used for hemostasis at that operation Surgical resection was performed and microscopic result was foreign body granuloma caused by oxidized cellulose. Oxidized cellulose is an absorbable sterile mesh and used to control capillary or venous bleeding. Although the manufacturer recommends its removal after hemostasis is achieved, in clinical practice it is usually left in situ to reabsorb spontaneously, usually with no untoward effect.
Capillaries
;
Cellulose, Oxidized
;
Foreign Bodies*
;
Granuloma, Foreign-Body*
;
Hemorrhage
;
Hemostasis
;
Humans
;
Male
;
Mediastinum*
;
Sympathectomy
;
Thorax
;
Young Adult
8.Changes of Mitral Inflow According to Position in Patients with Dilated Cardiomyopathy.
Woo Hyuk SONG ; Hong Eui LIM ; Sung Hee SHIN ; Eun Mi LEE ; Kyo Seung HWANG ; Jung Chun AHN ; Do Sun LIM ; Chang Gyu PARK ; Wan Joo SHIM ; Dong Joo OH ; Young Moo RO
Journal of the Korean Society of Echocardiography 1998;6(1):5-10
BACKGROUND AND OBJECTIVES: Dilated cardiomyopathy(DCMP) probably is the end result of myocardial damage produced by various causes and shows various clinical manifestations. Some patients with DCMP experience more shortness of breath when change their position especially in left lateral decubitus position. We investigated whether the symptomatic changes according to position in DCMP patients were related to the changes of mitral inflow pattern. MATERIALS AND METHOD: DCMP patients with New York Heart Association functional class III or IV, who felt more shortness of breath when changed their position, were studied. The patients with atrial fibrillation or with moderate and severe valvular heart disease were excluded. Early and late atrial left ventricular filling velocities and time velocity integrals(TVI) and it's ratios of mitral inflow, heart rates, isovolumic relaxation time(IVRT), decelaration time(DT) of early mitral inflow were analysed in each decubitus position. RESULTS: 4 men and 3 women were included and their mean age was 60.3 years. 6 patients had mild mitral regurgitation and 4 patients had mild aortic regurgition. All patients felt more shortness of breath in left lateral decubitus position. Peak velocity and TVI of early mitral inflow were not changed significantly according to position. Peak velocity of late mitral inflow was increased significantly with right lateral decubitus position(Lt; 39.0+/- 14.1cm/sec, Rt; 49. 4+/-23.6cm/sec, p<0.05). TVI of late mitral inflow was increased significantly with right lateral decubitus position(Lt; 2.8+/-1.1cm, Rt; 3.8+/-1.9cm, p<0.05). Ratio of early and late mitral inflow peak velocity was decreased significantly with right lateral decubitus position(Lt; 2.13+ 0.34, Rt: 1.62+/-0.57, p<0.05). Ratio of early and late mitral inflow TVI was decreased significantly with right lateral decubitus position(Lt; 3.13+/-1.49, Rt; 2.13+/-1.32, p<0.01). Heart rate, IVRT, DT were not changed significantly according to position. CONCLUSION: In DCMP patients with symptomatic relief in right lateral position, mitral inflow patterns were changed. And this finding suggests that symptomatic relief may be related to decreased left ventricular filling pressure.
Atrial Fibrillation
;
Cardiomyopathy, Dilated*
;
Deoxycytidine Monophosphate
;
Dyspnea
;
Female
;
Heart
;
Heart Rate
;
Heart Valve Diseases
;
Humans
;
Male
;
Mitral Valve Insufficiency
;
Relaxation
9.Significance of Serum Antibody Test for Toxocariasis in Healthy Healthcare Examinees with Eosinophilia in Seoul and Gyeongsangnam-do, Korea.
Hong Seok KIM ; Yan JIN ; Min Ho CHOI ; Jae Hwan KIM ; Young Ha LEE ; Cheong Ha YOON ; Eui Hyuk HWANG ; Hun KANG ; Sang Yong AHN ; Gi Jin KIM ; Sung Tae HONG
Journal of Korean Medical Science 2014;29(12):1618-1625
There have been numerous reports on the relationship between eosinophilia and toxocariasis. The present study investigated seropositive rates of toxocariasis among healthy people with or without eosinophilia in urban and rural areas, and assessed risk factors for positive antibody test. A total of 610 healthy people, who visited health check-up (Medicheck(R), Korea Association of Health Promotion), 310 from Seoul and 300 from Gyeongsangnam-do, were subjected for this study. Their serum samples were tested by ELISA with the crude antigen of Toxocara canis larvae. Cross-reactions with other tissue invading helminth antigens were also investigated. Total antibody positive rate of toxocariasis was 8.7% of the 610 subjects. When the subjects were grouped into 3 by their eosinophil counts, the antibody positive rates significantly differed by the groups; 5.9% (18/306) in the group<350/microL, 10.0% (11/110) in the group 350-500/microL, and 12.4% (24/194) in the group>500/microL (P=0.028). A total of 22 serum samples cross-reacted with other tissue-invading helminth antigens. A questionnaire analysis recognized drinking alcohol and smoking as significant risk factors of toxocariasis. In conclusion, toxocariasis antibody positive rate is correlated with eosinophil counts. It is recommended that healthy subjects with eosinophilia by routine health examination and risk factors undergo Toxocara serology by multiantigen ELISA to investigate etiology.
Age Distribution
;
Comorbidity
;
Eosinophilia/*diagnosis/*epidemiology
;
Female
;
Humans
;
Incidence
;
Male
;
Middle Aged
;
Reference Values
;
Republic of Korea/epidemiology
;
Risk Factors
;
Rural Population/*statistics & numerical data
;
Serologic Tests/statistics & numerical data
;
Sex Distribution
;
Toxocariasis/*diagnosis/*epidemiology
;
Urban Population/*statistics & numerical data
10.Open Bronchus Sign on CT: A Risk Factor for Hemoptysis after Percutaneous Transthoracic Biopsy.
Hyungjin KIM ; Chang Min PARK ; Soon Ho YOON ; Eui Jin HWANG ; Jong Hyuk LEE ; Su Yeon AHN ; Jin Mo GOO
Korean Journal of Radiology 2018;19(5):880-887
OBJECTIVE: We hypothesized that open bronchi within target pulmonary lesions are associated with percutaneous transthoracic needle biopsy (PTNB)-related hemoptysis. We sought to analyze and compare patient characteristics and target features as well as biopsy-related factors between patients with and without PTNB-related hemoptysis. MATERIALS AND METHODS: We retrospectively analyzed 1484 patients (870 males and 614 females; median age, 66 years) who had undergone 1569 cone-beam CT (CBCT)-guided PTNBs. Patient characteristics (sex, age, and pathologic diagnosis), nodule features (nodule type, size, location, and presence of an open bronchus in target nodules), and biopsy-related factors (biopsy needle size, pleura-to-target distance, blood test results, open bronchus unavoidability [OBU] index, etc.) were investigated. OBU index, which was assessed using the pre-procedural CBCT, was a subjective scoring system for the probability of needle penetration into the open bronchus. Univariate analysis and subsequent multivariate logistic regression analysis were conducted to reveal the independent risk factors for PTNB-related hemoptysis. For a subgroup of nodules with open bronchi, a trend analysis between the occurrence of hemoptysis and the OBU index was performed. RESULTS: The independent risk factors for hemoptysis were sex (female; odds ratio [OR], 1.918; p < 0.001), nodule size (OR, 0.837; p < 0.001), open bronchus (OR, 2.101; p < 0.001), and pleura-to-target distance (OR, 1.135; p = 0.003). For the target nodules with open bronchi, a significant trend between hemoptysis and OBU index (p < 0.001) was observed. CONCLUSION: An open bronchus in a biopsy target is an independent predictor of hemoptysis, and careful imaging review may potentially reduce PTNB-related hemoptysis.
Biopsy*
;
Biopsy, Needle
;
Bronchi*
;
Cone-Beam Computed Tomography
;
Female
;
Hematologic Tests
;
Hemoptysis*
;
Humans
;
Image-Guided Biopsy
;
Logistic Models
;
Lung Neoplasms
;
Male
;
Needles
;
Odds Ratio
;
Retrospective Studies
;
Risk Factors*