1.Comparison of Automatic Evaluation Methods for Pattern Hallucinations of Large Language Models in Traditional Chinese Medicine Syndrome Differentiation
Qinwei WU ; Yuzhu GAO ; Xingyue GOU ; Junyu YAO ; Chuangan ZHOU ; Zhengchun XUE ; Zhirong XU ; Xinlin CHEN ; Dong CAO
Journal of Traditional Chinese Medicine 2026;67(17):1845-1852
ObjectiveTo compare the performance of different automatic evaluation methods for detecting pattern hallucinations generated by large language models (LLMs) in traditional Chinese medicine (TCM) syndrome differentiation and to identify a strategy with higher overall discriminative performance against expert manual judgment. MethodsA standardized TCM syndrome-differentiation dataset containing 598 cases was constructed from case reports published in Chinese core journals indexed by Peking University Core Journals or Chinese Science Citation Database. Each of the 598 cases was submitted to five Chinese LLMs, with each model generating one syndrome-pattern output per case, yielding 2990 outputs in total. The 598 outputs generated by Qwen3-Next-80B-A3B-Thinking were independently annotated for syndrome-pattern hallucinations by two licensed TCM physicians with intermediate or higher professional titles. Disagreements were adjudicated by a third licensed TCM physician with a senior associate professional title and more than 10 years of clinical experience. The resulting consensus annotations served as the reference labels for training and validating the automated evaluation method. Based on these reference labels, three categories of automated evaluation methods were developed and compared following a progressive strategy from text-level semantic similarity, to syndrome-element structural consistency, and finally multi-feature fusion. Method A employed sentence-embedding models based on semantic similarity between pattern texts. Method B adopted a rule-based threshold method using weighted Jaccard coefficients of disease-location and disease-nature syndrome elements. Method C utilized a machine-learning classifier integrating syndrome-element matching scores, missing and redundant syndrome-element counts, and multiple semantic-similarity features. Five-fold cross-validation was used to evaluate the discriminative performance of each method against manual judgment. The optimal method was then applied to uniformly assess the pattern hallucination rates of five LLMs. ResultsIn method A, text2vec-large-chinese showed best overall performance, with an area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (PR-AUC), F1-score, sensitivity, and specificity of 0.849, 0.884, 0.794, 0.801, and 0.691, respectively. In method B, the corresponding metrics of the rule-based syndrome-element Jaccard threshold method were 0.848, 0.851, 0.817, 0.845, and 0.684, respectively; those of the LLM-based syndrome-element Jaccard method were 0.872, 0.906, 0.795, 0.764, and 0.780, respectively. In method C, the multilayer perceptron (MLP) classifier showed the best overall performance, with AUROC, PR-AUC, F1-score, sensitivity, and specificity of 0.933, 0.951, 0.871, 0.857, and 0.846, respectively. Under the unified evaluation framework combining DeepSeek-R1 syndrome-element extraction and MLP classifier, the estimated pattern hallucination rates of the five LLMs ranged from 56.0% to 77.4%. ConclusionAmong the three automatic evaluation methods, Method C integrating syndrome-element discrepancy features and text semantic features using an MLP classifier showed the highest overall discriminative performance against manual judgment, and is more suitable for evaluating syndrome hallucinations in LLM-based TCM syndrome differentiation tasks.
2.Analysis of risk factors and their warning effectiveness for postoperative intestinal barrier dysfunction in patients with severe traumatic brain injury
Chunlong DING ; Junjie CHEN ; Shaodong XI ; Qinwei ZHOU ; Huijun WANG ; Jie QIU ; Huize LIU ; Yelei ZHANG ; Yunxu ZHENG ; Fukang DONG
Chinese Journal of Trauma 2024;40(2):127-132
Objective:To investigate the risk factors and their warning effectiveness for postoperative intestinal barrier dysfunction (IBD) in patients with severe traumatic brain injury (sTBI).Methods:A retrospective cohort study was conducted to analyze the clinical data of 101 patients with sTBI admitted to Wuxi Branch of Zhongda Hospital Affiliated to Southeast University from May 2020 to February 2023, including 63 males and 38 females, aged 21-81 years [(53.4±14.2)years]. All the patients underwent emergency surgery. The patients were divided into IBD group ( n=67) and non-IBD group ( n=34) according to whether or not they had IBD after surgery. The gender, age, basic diseases (hypertension and diabetes), types of intracranial hematoma (subdural, epidural, and intracerebral hematoma), preoperative Glasgow Coma Scale (GCS), cerebral hernia, intraoperative initial intracranial pressure (iICP), operation time, removal of bone flap, treatment time in ICU, initiation time of enteral nutrition, and use of broad-spectrum antibiotics were recorded in the two groups. Univariate and multivariate binary Logistic regression analyses were conducted to assess the correlations between above-mentioned indicators and incidence of postoperative IBD in sTBI patients and determine the independent risk factors for sTBI. Receiver operating characteristic (ROC) curve and area under the curve (AUC) were used to evaluate the warning effectiveness of each risk factor for IBD. Results:The results of the univariate analysis showed that preoperative GCS, cerebral hernia, intraoperative iICP, removal of bone flap, treatment time in ICU, initiation time of enteral nutrition, and use of broad-spectrum antibiotics were significantly correlated with the incidence of IBD in sTBI patients ( P<0.05 or 0.01), while there were no correlations of IBD with gender, age, basic diseases, types of intracranial hematoma and operation time ( P>0.05). The results of the multivariate binary Logistic regression analysis showed that preoperative GCS≤5 points ( OR=2.49, 95% CI 1.17, 5.32, P<0.05), intraoperative iICP>23 mmHg (1 mmHg=0.133 kPa)( OR=1.20, 95% CI 1.03, 1.39, P<0.05), and initiation time of enteral nutrition>24 hours ( OR=10.03, 95% CI 1.26, 80.21, P<0.05) were highly correlated with postoperative IBD in sTBI patients. The results of the ROC curve analysis showed that intraoperative iICP had the highest warning value (AUC=0.91, 95% CI 0.85, 0.96), followed by preoperative GCS (AUC=0.88, 95% CI 0.82, 0.95), and initiation time of enteral nutrition had the lowest warning value (AUC=0.78, 95% CI 0.69, 0.87). Conclusions:Preoperative GCS≤5 points, intraoperative iICP>23 mmHg, and initiation time of enteral nutrition>24 hours are independent risk factors for postoperative IBD in sTBI patients. The warning value of intraoperative iICP ranks the highest for postoperative IBD in sTBI patients, followed by preoperative GCS, with initiation time of enteral nutrition having the lowest warning value.
3.Clinical Effect of Shugan Hewei Prescription Combined with Vonoprazan on Refractory Gastroesophageal Reflux Disease Due to Qi Depression and Phlegm Obstruction
Mei QI ; Yue ZHOU ; Mengyuan ZHANG ; Yi WANG ; Hongwei WANG ; Qinwei ZHENG ; Shengquan FANG
Chinese Journal of Experimental Traditional Medical Formulae 2023;29(24):146-154
ObjectiveTo evaluate the clinical efficacy and adverse effects of Shugan Hewei prescription combined with vonoprazan in the treatment of refractory gastroesophageal reflux disease (RGERD) due to qi depression and phlegm obstruction. MethodEighty RGERD patients who met the inclusion criteria underwent 24-hour pH impedance and high-resolution esophageal manometry and electronic gastroscopy. The 80 patients were randomly assigned to an observation group (Shugan Hewei prescription, one bag each time, twice a day + vonoprazan, 20 mg each time, once a day) and a control group (vonoprazan, 20 mg each time, once a day) by the random number table method. The treatment in both groups lasted for 4 weeks. The clinical efficacy was examined. The scores of TCM symptoms (pharyngeal discomforts such as phlegm obstruction, retrosternal discomfort, and belching), somatic symptoms, quality of life, and improvement of esophageal mucosa under gastroscopy were observed in both groups before treatment and after treatment for 2 and 4 weeks. ResultSeventy-five patients completed the trial were included in this study, including 38 patients in the observation group and 37 patients in the control group. The total response rate in the observation group was 89.47%(34/38), which was higher than that (62.16%,23/37) in the control group (χ2=13.014, P<0.01). After treatment, the scores of esophageal mucous membrane, reflux disease symptoms, TCM symptoms, gastroesophageal reflux disease health-related quality of life scale (GERD-HRQL), and somatic self-rating scale (SSS) decreased in both groups(P<0.05). Moreover, the observation group outperformed the control group in alleviating heartburn, acid reflux, throat discomforts, midnight coughing, nausea and dry vomiting, mucousy mouth, and insomnia in the patients with GERD (P<0.05,P<0.01). However, the two groups showed no statistically significant differences in the improvement of esophageal mucosa after treatment. ConclusionThe combination of Shugan Hewei prescription with vonoprazan was superior to vonoprazan alone in treating RGERD regarding clinical symptoms, physical signs, quality of life, and somatic symptoms, without causing obvious adverse effects.
4.Neuroendoscopy in microsurgical clipping for ruptured posterior communicating artery aneurysms via keyhole approaches
Qinwei ZHOU ; Junjie CHEN ; Shaodong XI ; Ailin CHEN ; Chungang DAI ; Qing ZHU
Chinese Journal of Microsurgery 2022;45(3):304-309
Objective:To explore the value of application and manipulation technique of neuroendoscope in microsurgical clipping of ruptured posterior communicating artery(PCoA)aneurysms via keyhole approaches.Methods:From January 2018 to December 2020, the clinical data of 52 patients who received microsurgical clipping for ruptured via keyhole approach were retrospectively analysed. Forty-one patients had the intraoperative endoscopic monitoring. The supraorbital keyhole approach or pterional keyhole approach was applied based on the characteristics of the aneurysms. According to the in-surgery requirement, a 30° rigid neuroendoscope was used before and/or after clipping. All patients entered postoperative follow-up in outpatient clinic and were evaluated with the modified Rankin Scale(mRS).Results:All 52 patients had 52 ruptured PCoA aneurysms. Eighteen of the patients were treated via supraorbital keyhole approach and 34 via pterion keyhole approach. Pre-and post-clipping endoscopic observation were carried out in 12 cases and 29 only with post-clipping endoscopic observation. Residual aneurysmal neck was detected in 3 patients. Missed clipping of perforators was found in 2 patients and followed by proper adjustment of clips. All patients received follow-up angiographic examinations. Total obliteration of the aneurysm and an intact of internal carotid artery and PCoA were found in 41 patients by the intraoperative endoscopic observation. Two residual aneurysmal neck were detected in 11 patients without intraoperative endoscopic observation. After 11 to 45 months of follow-up, all patients had good recovery(mRS 0-1).Conclusion:It is a safe and effective method with endoscopic observation during microsurgical clipping procedure for ruptured PCoA aneurysms via keyhole approaches. It can effectively make up for the insufficient visual angle of microscope, realise the anatomical relationship between the aneurysm and adjacent structures, and avoid residual aneurysmal neck and an iatrogenic injury to the parent artery and perforators.
5.Analysis of the use of antidepressants and its influencing factors among depressive patients
Yu ZHOU ; Qinwei XU ; Chuandong WU
Chinese Journal of Behavioral Medicine and Brain Science 2018;27(9):830-833
Objective To analyze the use of antidepressants and its influencing factors in patients with depressive disorders in Haikou City,and to provide some reference for clinical rational drug use.Methods 310 patients with depressive disorder in Haikou were selected.All patients were assessed with the selfdesigned General Situation and Drug Use Questionnaire,General Self-efficacy Scale (GSES),Quality of Life Questionnaire for Psychiatric Patients (QOL-P) and Hamilton Depression Scale (HAMD-17).Results (1)The usage rate of serotonin reuptake inhibitors (SSRIs) was 53.22%,30.32% for tricyclic antidepressants (TCAs) and 16.45% for other new antidepressants.(2)Patients'self-efficacy (β=-0.473,OR=0.713,P=0.030),social function (β=0.371,OR=1.437,P=0.036),family income (β=-1.242,OR=0.423,P=0.001) were the factors influencing patients' choice of TCAs;family income (β=-1.762,OR=0.234,P=0.001),payment method (β =0.248,OR =1.157,P =0.030) were the factors influencing the choice of SSRIs drugs;self-efficacy (β=0.563,OR=1.913,P=0.041) and depression (β=0.543,OR=2.225,P=0.026) were the factors influencing the choice of other new antidepressants.Conclusion Traditional antidepressants such as serotonin reuptake inhibitors and tricyclic antidepressants are still widely used in patients with depressive disorders.Self-efficacy,social function,income,and payment methods can influence the drug selection of depressive patients.
6.Clinical outcomes of endoscopic and microsurgical treatments in patients with hypertensive basal ganglia hemorrhage
Qinwei ZHOU ; Min LIU ; Xiaodong XI ; Binquan XU ; Junjie CHEN ; Jirong DONG ; Jun DONG
Chinese Journal of Neuromedicine 2018;17(3):301-305
Objective To evaluate the clinical outcomes of endoscopic and microsurgical treatments in patients with hypertensive basal ganglia hemorrhage (HBGH).Methods A retrospective analysis of clinical features of 37 patients received microsurgical treatment via transsylvian-transinsular approach or 32 patients received endoscopic treatment for evacuation of HBGH in our hospital from January 2011 to January 2015 was performed.The operation time,hematoma clearance rate,re-bleeding rate and prognoses of the patients were investigated.Results As compared with patients accepted microsurgical treatment,patients accepted endoscopic treatment had significantly shorter operation time,smaller peroperative bleeding volume,and shorter hemostatic time (P<0.05).The preoperative Glasgow coma scale (GCS) scores in patients accepted endoscopic treatment and microsurgical treatment were 8.63±1.24 and 8.67±1.31,without significant difference (P>0.05);24 h after operation,GCS scores in patients accepted endoscopic treatment increased to 12.79±1.20,which had significant difference as compared with those in patients accepted microsurgical treatment (11.23±1.29,P<0.05).The cerebral edema volume in patients accepted endoscopic treatment and microsurgical treatment was (11.83±4.08) mL and (18.76±7.92) mL,with significant difference (t=6.460,P=0.000).The hematoma clearance and prognosis in patients accepted endoscopic treatment were better than those in patients accepted microsurgical treatment.Conclusion Endoscopic evacuation ofhematoma for HBGH is efficient and safe,enjoying better efficacy than microsurgery.
7.Influence of microsurgical operation through lateral fissure-insula lobe approach in the treatment of hyperten-sive cerebral hemorrhage in basal ganglia region
Jian ZHANG ; Min LIU ; Qinwei ZHOU ; Ming ZHOU
Journal of Regional Anatomy and Operative Surgery 2016;25(11):804-807
Objective To explore the impact of Microsurgical operation through lateral fissure-insula lobe approach in the treatment of hypertensive cerebral hemorrhage in basal ganglia region on the clinical curative effect.Methods Selected 90 cases of patients with hyper-tensive cerebral hemorrhage in basal ganglia region in our hospital from April 2013 to December 2015.According to the random number table method,they were divided into the observation group and the control group.Regarded the even numbers as the observation group while the odd numbers as the control group,with 45 cases in each group.Patients of the two groups were all performed general anaesthesia with tracheal in-tubation.And patients of the control group were given large trauma craniotomy,while patients of the observation group were treated by micro-surgical operation through lateral fissure-insula lobe approach.Recorded the situation of surgery and postoperative recovery of patients in the two groups.In addition,compared the postoperative complications and evaluated living conditions of the two groups through activity of daily living scale(ADL)half a year after surgery.Results The good rate of recovery in the observation group was 84.44% (38 /45),while it was 51.11%(23 /45)in the control group,and there was statistically significant difference between the two groups(χ2 =11.447,P =0.000).Be-sides,the mortality rate of the observation group was 0(0 /45)half a year after surgery,while it was 8.89%(4 /45)in the control group,and there was statistically significant difference between the two groups(χ2 =4.186,P =0.041).The operation time of observation group and the automatic opening time in the observation group were obviously shorter than that of the control group(P <0.05).Moreover,the hematoma clearance rate and GCS score of the observation group 1 week after operation were significantly higher than those of the control group (P <0.05).The incidence of postoperative complications of the observation group was 8.89%(4 /45),which was significantly lower than 31.11%(14 /45)in the control group,and there was significant difference(P <0.05).Conclusion It has many advantages such as shorter operative time,faster postoperative recovery,and higher hematoma clearance rate to apply microsurgical operation through lateral fissure-insula lobe approach in the treatment of hypertensive cerebral hemorrhage in basal ganglia region.Besides,it pays attention to protecting cerebral vascular of patients.The clinical treatment effect is ideal and safe.Therefore,it is worth popularizing in clinical application.

Result Analysis
Print
Save
E-mail