1.CRAKUT:integrating contrastive regional attention and clinical prior knowledge in U-transformer for radiology report generation.
Yedong LIANG ; Xiongfeng ZHU ; Meiyan HUANG ; Wencong ZHANG ; Hanyu GUO ; Qianjin FENG
Journal of Southern Medical University 2025;45(6):1343-1352
OBJECTIVES:
We propose a Contrastive Regional Attention and Prior Knowledge-Infused U-Transformer model (CRAKUT) to address the challenges of imbalanced text distribution, lack of contextual clinical knowledge, and cross-modal information transformation to enhance the quality of generated radiology reports.
METHODS:
The CRAKUT model comprises 3 key components, including an image encoder that utilizes common normal images from the dataset for extracting enhanced visual features, an external knowledge infuser that incorporates clinical prior knowledge, and a U-Transformer that facilitates cross-modal information conversion from vision to language. The contrastive regional attention in the image encoder was introduced to enhance the features of abnormal regions by emphasizing the difference between normal and abnormal semantic features. Additionally, the clinical prior knowledge infuser within the text encoder integrates clinical history and knowledge graphs generated by ChatGPT. Finally, the U-Transformer was utilized to connect the multi-modal encoder and the report decoder in a U-connection schema, and multiple types of information were used to fuse and obtain the final report.
RESULTS:
We evaluated the proposed CRAKUT model on two publicly available CXR datasets (IU-Xray and MIMIC-CXR). The experimental results showed that the CRAKUT model achieved a state-of-the-art performance on report generation with a BLEU-4 score of 0.159, a ROUGE-L score of 0.353, and a CIDEr score of 0.500 in MIMIC-CXR dataset; the model also had a METEOR score of 0.258 in IU-Xray dataset, outperforming all the comparison models.
CONCLUSIONS
The proposed method has great potential for application in clinical disease diagnoses and report generation.
Humans
;
Radiology Information Systems
;
Radiology
2.Design and validation of a multimodal model integrating text and imaging data for intelligent assessment of psychological stress in college students.
Huirong XIE ; Chaobin HU ; Guohua LIANG ; Hongzhe HAN ; Mu HUANG ; Qianjin FENG
Journal of Southern Medical University 2025;45(11):2504-2510
OBJECTIVES:
We propose a multimodal model integrating social media text and image data for automated assessment of psychological stress in college students to support the development of intelligent mental health services in higher education institutions.
METHODS:
Based on deep learning technology, we designed an evaluation framework comprising a text sentiment modeling module, an image sentiment modeling module, and a multimodal fusion prediction module. Text sentiment features were extracted using Bi-LSTM, and image semantic cues were extracted via U-Net. A feature concatenation strategy was used to enable cross-modal semantic collaboration to achieve automatic identification of 3 psychological stress levels: mild, moderate, and severe. We constructed a multimodal annotated dataset using social platform data from 1577 students across multiple universities in Guangdong Province. After data cleaning, 252 samples were randomly selected for model training and testing.
RESULTS:
In the 3-classification task, the model demonstrated outstanding performance on the test set, and achieved an accuracy of 92.86% and an F1 score of 0.9276, exhibiting excellent stability and consistency. Confusion matrix analysis further revealed the model's ability to effectively distinguish between different pressure levels.
CONCLUSIONS
The multimodal psychological stress assessment model developed in this study effectively integrates unstructured social behavior data to enhance the scientific rigor and practical applicability of psychological state recognition, and thus provides support for developing intelligent psychological service systems.
Humans
;
Stress, Psychological/diagnosis*
;
Students/psychology*
;
Universities
;
Social Media
;
Deep Learning
3.A multi-constraint optimal puncture path planning algorithm for percutaneous interventional radiofrequency thermal fusion of the L5/S1 segments
Hu LIU ; Zhihai SU ; Chengjie HUANG ; Lei ZHAO ; Yangfan CHEN ; Yujia ZHOU ; Hai LÜ ; Qianjin FENG
Journal of Southern Medical University 2024;44(9):1783-1795
Objective To minimize variations in treatment outcomes of L5/S1 percutaneous intervertebral radiofrequency thermocoagulation(PIRFT)arising from physician proficiency and achieve precise quantitative risk assessment of the puncture paths.Methods We used a self-developed deep neural network DWT-UNet for automatic segmentation of the magnetic resonance(MR)images of the L5/S1 segments into 7 key structures:L5,S1,Ilium,Disc,N5,Dura mater,and Skin,based on which a needle insertion path planning environment was modeled.Six hard constraints and 6 soft constraints were proposed based on clinical criteria for needle insertion,and the physician's experience was quantified into weights using the analytic hierarchy process and incorporated into the risk function for needle insertion paths to enhance individual case adaptability.By leveraging the proposed skin entry point sampling sub-algorithm and Kambin's triangle projection area sub-algorithm in conjunction with the analytic hierarchy process,and employing various technologies such as ray tracing,CPU multi-threading,and GPU parallel computing,a puncture path was calculated that not only met clinical hard constraints but also optimized the overall soft constraints.Results A surgical team conducted a subjective evaluation of the 21 needle puncture paths planned by the algorithm,and all the paths met the clinical requirements,with 95.24%of them rated excellent or good.Compared with the physician's planning results,the plans generated by the algorithm showed inferior DIlium,DS1,and Depth(P<0.05)but much better DDura,DL5,DN5,and AKambin(P<0.05).In the 21 cases,the planning time of the algorithm averaged 7.97±3.73 s,much shorter than that by the physicians(typically beyond 10 min).Conclusion The multi-constraint optimal puncture path planning algorithm offers an efficient automated solution for PIRFT of the L5/S1 segments with great potentials for clinical application.
4.A multi-constraint optimal puncture path planning algorithm for percutaneous interventional radiofrequency thermal fusion of the L5/S1 segments
Hu LIU ; Zhihai SU ; Chengjie HUANG ; Lei ZHAO ; Yangfan CHEN ; Yujia ZHOU ; Hai LÜ ; Qianjin FENG
Journal of Southern Medical University 2024;44(9):1783-1795
Objective To minimize variations in treatment outcomes of L5/S1 percutaneous intervertebral radiofrequency thermocoagulation(PIRFT)arising from physician proficiency and achieve precise quantitative risk assessment of the puncture paths.Methods We used a self-developed deep neural network DWT-UNet for automatic segmentation of the magnetic resonance(MR)images of the L5/S1 segments into 7 key structures:L5,S1,Ilium,Disc,N5,Dura mater,and Skin,based on which a needle insertion path planning environment was modeled.Six hard constraints and 6 soft constraints were proposed based on clinical criteria for needle insertion,and the physician's experience was quantified into weights using the analytic hierarchy process and incorporated into the risk function for needle insertion paths to enhance individual case adaptability.By leveraging the proposed skin entry point sampling sub-algorithm and Kambin's triangle projection area sub-algorithm in conjunction with the analytic hierarchy process,and employing various technologies such as ray tracing,CPU multi-threading,and GPU parallel computing,a puncture path was calculated that not only met clinical hard constraints but also optimized the overall soft constraints.Results A surgical team conducted a subjective evaluation of the 21 needle puncture paths planned by the algorithm,and all the paths met the clinical requirements,with 95.24%of them rated excellent or good.Compared with the physician's planning results,the plans generated by the algorithm showed inferior DIlium,DS1,and Depth(P<0.05)but much better DDura,DL5,DN5,and AKambin(P<0.05).In the 21 cases,the planning time of the algorithm averaged 7.97±3.73 s,much shorter than that by the physicians(typically beyond 10 min).Conclusion The multi-constraint optimal puncture path planning algorithm offers an efficient automated solution for PIRFT of the L5/S1 segments with great potentials for clinical application.
5.Identification of osteoid and chondroid matrix mineralization in primary bone tumors using a deep learning fusion model based on CT and clinical features: a multi-center retrospective study.
Caolin LIU ; Qingqing ZOU ; Menghong WANG ; Qinmei YANG ; Liwen SONG ; Zixiao LU ; Qianjin FENG ; Yinghua ZHAO
Journal of Southern Medical University 2024;44(12):2412-2420
METHODS:
We retrospectively collected CT scan data from 276 patients with pathologically confirmed primary bone tumors from 4 medical centers in Guangdong Province between January, 2010 and August, 2021. A convolutional neural network (CNN) was employed as the deep learning architecture. The optimal baseline deep learning model (R-Net) was determined through transfer learning, and an optimized model (S-Net) was obtained through algorithmic improvements. Multivariate logistic regression analysis was used to screen the clinical features such as sex, age, mineralization location, and pathological fractures, which were then connected with the imaging features to construct the deep learning fusion model (SC-Net). The diagnostic performance of the SC-Net model and machine learning models were compared with radiologists' diagnoses, and their classification performance was evaluated using the area under the receiver operating characteristic curve (AUC) and F1 score.
RESULTS:
In the external test set, the fusion model (SC-Net) achieved the best performance with an AUC of 0.901 (95% CI: 0.803-1.00), an accuracy of 83.7% (95% CI: 69.3%-93.2%) and an F1 score of 0.857, and outperformed the S-Net model with an AUC of 0.818 (95% CI: 0.694-0.942), an accuracy of 76.7% (95% CI: 61.4%-88.2%), and an F1 score of 0.828. The overall classification performance of the fusion model (SC-Net) exceeded that of radiologists' diagnoses.
CONCLUSIONS
The deep learning fusion model based on multi-center CT images and clinical features is capable of accurate classification of osseous and chondroid matrix mineralization and may potentially improve the accuracy of clinical diagnoses of osteogenic versus chondrogenic primary bone tumors.
Humans
;
Deep Learning
;
Bone Neoplasms/diagnostic imaging*
;
Retrospective Studies
;
Tomography, X-Ray Computed/methods*
;
Neural Networks, Computer
;
Male
;
Female
;
ROC Curve
;
Algorithms
6.Deep learning-based dose prediction in radiotherapy planning for head and neck cancer.
Lin TENG ; Bin WANG ; Qianjin FENG
Journal of Southern Medical University 2023;43(6):1010-1016
OBJECTIVE:
To propose an deep learning-based algorithm for automatic prediction of dose distribution in radiotherapy planning for head and neck cancer.
METHODS:
We propose a novel beam dose decomposition learning (BDDL) method designed on a cascade network. The delivery matter of beam through the planning target volume (PTV) was fitted with the pre-defined beam angles, which served as an input to the convolution neural network (CNN). The output of the network was decomposed into multiple sub-fractions of dose distribution along the beam directions to carry out a complex task by performing multiple simpler sub-tasks, thus allowing the model more focused on extracting the local features. The subfractions of dose distribution map were merged into a distribution map using the proposed multi-voting mechanism. We also introduced dose distribution features of the regions-of-interest (ROIs) and boundary map as the loss function during the training phase to serve as constraining factors of the network when extracting features of the ROIs and areas of dose boundary. Public datasets of radiotherapy planning for head and neck cancer were used for obtaining the accuracy of dose distribution of the BDDL method and for implementing the ablation study of the proposed method.
RESULTS:
The BDDL method achieved a Dose score of 2.166 and a DVH score of 1.178 (P < 0.05), demonstrating its superior prediction accuracy to that of current state-ofthe-art (SOTA) methods. Compared with the C3D method, which was in the first place in OpenKBP-2020 Challenge, the BDDL method improved the Dose score and DVH score by 26.3% and 30%, respectively. The results of the ablation study also demonstrated the effectiveness of each key component of the BDDL method.
CONCLUSION
The BDDL method utilizes the prior knowledge of the delivery matter of beam and dose distribution in the ROIs to establish a dose prediction model. Compared with the existing methods, the proposed method is interpretable and reliable and can be potentially applied in clinical radiotherapy.
Humans
;
Deep Learning
;
Head and Neck Neoplasms/radiotherapy*
;
Algorithms
;
Neural Networks, Computer
7.Quality analysis of Rosae Radix et Rhizoma.
Hai-Hui LIU ; Chen-Na LU ; Xuan-Xuan ZHU ; Lu BAI ; Li-Mei LIN ; Qian-Wen CHEN ; Wei-Hong FENG ; Duan-Fang LIAO ; Chun LI
China Journal of Chinese Materia Medica 2023;48(10):2781-2791
Rosae Radix et Rhizoma is a herbal medicine in a variety of famous Chinese patent medicines, while the quality standard for this medicine remains to be developed due to the insufficient research on the quality of Rosae Radix et Rhizoma from different sources. Therefore, this study comprehensively analyzed the components in Rosae Radix et Rhizoma of different sources from the aspects of extract, component category content, identification based on thin-lay chromatography, active component content determination, and fingerprint, so as to improve the quality control. The results showed that the content of chemical components varied in the samples of different sources, while there was little difference in the chemical composition among the samples. The content of components in the roots of Rosa laevigata was higher than that in the other two species, and the content of components in the roots was higher than that in the stems. The fingerprints of triterpenoids and non-triterpenoids were established, and the content of five main triterpenoids including multiflorin, rosamultin, myrianthic acid, rosolic acid, and tormentic acid in Rosae Radix et Rhizoma was determined. The results were consistent with those of major component categories. In conclusion, the quality of Rosae Radix et Rhizoma is associated with the plant species, producing area, and medicinal parts. The method established in this study lays a foundation for improving the quality standard of Rosae Radix et Rhizoma and provides data support for the rational use of the stem.
Drugs, Chinese Herbal/chemistry*
;
Rhizome/chemistry*
;
Plant Roots/chemistry*
;
Plants, Medicinal
;
Quality Control
8.Investigation on in Vitro Antioxidant Activity and Chemical Composition of Different Polar Parts of Extract of Rosa cymosa Roots
Xuan-xuan ZHU ; Xiao-qian LIU ; Yao-hua LIANG ; Li-mei LIN ; Lu BAI ; Wei-hong FENG ; Zhi-min WANG ; Chun LI ; Duan-fang LIAO
Chinese Journal of Experimental Traditional Medical Formulae 2021;27(8):117-125
Objective:To investigate the antioxidant activity and chemical composition of 75% ethanol extract of
9.Clinical analysis of 1 057 patients with critical illnesses in a dermatological ward
Hai LONG ; Li JIANG ; Yueqi QIU ; Nan YAO ; Licong LIU ; Yuming XIE ; Feng XIONG ; Siqi TAN ; Qiqi KUANG ; Ruixuan YOU ; Ke CHAI ; Xin LUO ; Haojun LONG ; Yue XIN ; Ziyu GUO ; Jiaqi WANG ; Yixin TAN ; Qing ZHANG ; Guiying ZHANG ; Yaping LI ; Yuwen SU ; Rong XIAO ; Qianjin LU
Chinese Journal of Dermatology 2021;54(9):790-797
Objective:To summarize clinical characteristics of and treatment experience with patients with critical illnesses in a dermatological ward.Methods:All patients with serious or life-threatening conditions, who were hospitalized at the dermatological ward of the Second Xiangya Hospital of Central South University from July 9, 2011 to December 31, 2020, were collected, and their clinical data were retrospectively analyzed. Demographic characteristics, disease types and proportions, main complications, causes of serious or life-threatening conditions, important treatment measures and outcomes were summarized, and causes of death were also analyzed and discussed.Results:A total of 1 057 patients with critical illnesses were collected, with a male-to-female ratio of 1∶1.11, and 64.81% of them aged 18 to 65 years. The types of diseases mainly included drug eruptions (332 cases) , connective tissue diseases (226 cases) , bullous skin diseases (104 cases) , psoriasis (57 cases) , erythroderma (45 cases) , infectious skin diseases (67 cases) , etc. Among them, psoriasis (39 cases) and erythroderma (32 cases) mostly occurred in males, and connective tissue diseases (168 cases) mostly occurred in females. Common complications mainly involved infections, important organ damage or dysfunction, hypoalbuminemia, and fluid, electrolyte and acid-base imbalances. A total of 94 patients were diagnosed with life-threatening conditions, which were found to be mainly caused by primary skin diseases, hematologic abnormalities, respiratory failure, nervous system abnormalities, renal failure, sepsis, fluid, electrolyte and acid-base imbalances, etc. During the management of critical illnesses, 43 patients were treated with high-dose glucocorticoid pulse therapy, 264 were treated with gamma-globulin pulse therapy, 355 were transfused with other blood products, and 34 received special therapies such as hemoperfusion/immunoadsorption therapy, plasma exchange, dialysis, artificial liver support therapy; 42 patients were transferred to the intensive care unit (ICU) , 12 were transferred to the department of surgery for operations, and 12 were transferred to the department of obstetrics and gynecology for delivery or induction of labor. After treatment, 989 patients (93.57%) achieved improvement and were discharged. A total of 14 patients (1.32%) died, of whom 7 died of secondary sepsis, 2 died of severe pulmonary infections, 2 died of asphyxia caused by respiratory mucosa shedding-induced airway obstruction, the other 3 died of gastrointestinal hemorrhage, cerebral hemorrhage and neuropsychiatric systemic lupus erythematosus, respectively.Conclusions:Critical cases in the dermatological ward mainly suffered from serious skin diseases such as severe drug eruptions, connective tissue diseases and bullous skin diseases, as well as complications such as severe underlying diseases, severe organ dysfunction, sepsis or severe fluid, electrolyte and acid-base imbalances. In terms of treatment, it is of critical significance to make a clear diagnosis and assess the severity of disease as early as possible, monitor and prevent possible complications, and to consult with specialists in relevant disciplines in time.
10.Research progress of tannins in traditional Chinese medicines in recent ten years.
Xuan-Xuan ZHU ; Lu BAI ; Xiao-Qian LIU ; Yao-Hua LIANG ; Li-Mei LIN ; Wei-Hong FENG ; Zhi-Min WANG ; Chun LI ; Duan-Fang LIAO
China Journal of Chinese Materia Medica 2021;46(24):6353-6365
In this paper, the newly isolated tannins were sorted after a review of the literature concerning tannins in recent 10 years, and their research progress was summarized in terms of extraction, isolation, pharmacological activity and metabolism. Hydrolysable tannins and condensed tannins are the main structural types. Modern research shows that tannins have many pharmacological effects, such as bacteriostasis, antioxidation, antitumor, antivirus and blood glucose reduction, and have broad development prospects. They are usually extracted by water, ethanol and acetone and isolated and purified by macroporous resin and gel column chromatography. The packings commonly adopted for the column chromatography mainly included Sephadex LH-20, Diaion HP-20, MCI-gel CHP-20 and Toyopearl HW-40. Modern analytical techniques such as nuclear magnetic resonance spectroscopy(NMR), fast atom bombardment mass spectrometry(FAB-MS) and circular dichroism(CD) are generally used for the structural identification of tannins. Howe-ver, their isolation, purification and structural identification are still challenging. It is necessary to use a variety of high-throughput screening methods to explore their pharmacological activities and to explore the material basis responsible for their functions through experiments in vivo.
China
;
Hydrolyzable Tannins
;
Medicine, Chinese Traditional
;
Proanthocyanidins
;
Tannins

Result Analysis
Print
Save
E-mail