1.Development and validation of PhenoRAG: A visualization tool for automated human phenotype ontology term annotation based on large language models and retrieval-augmented generation technology.
Wei ZHONG ; Yousheng YAN ; Kai YANG ; Yan LIU ; Xinyu FU ; Zhengyang YAO ; Chenghong YIN
Chinese Journal of Medical Genetics 2026;43(1):36-43
OBJECTIVE:
To develop a user-friendly visualization application for the automatic annotation of Human Phenotype Ontology (HPO) terms based on large language models and retrieval-augmented generation (RAG) technology, and to validate its performance in an authoritative case dataset.
METHODS:
By integrating the domestic open-source large language model DeepSeek-V3 with RAG technology, an interactive web application was deployed on the Streamlit cloud platform. Using only the latest official HPO dataset as the data source, the lightweight sentence-embedding model BAAI/bge-small-en-v1.5 was employed to construct a FAISS vector index. During the online phase, a four-step closed-loop process is automatically completed: multilingual translation, phenotype phrase extraction, RAG candidate retrieval, term mapping, and official database validation. 121 English case reports publicly released by BMJ Case Reports and Oxford Medical Case Reports (with a gold-standard HPO set of 1 794 terms) were selected for application validation. Precision, recall, and F1 score were calculated and compared horizontally with traditional dictionary tools, standalone large language models, and the similar application "RAG-HPO". Finally, replace the model with the more advanced ChatGPT-5 and evaluate its performance on the newly extracted dataset.
RESULTS:
An HPO term automatic annotation visualization application named PhenoRAG, based on large language models and RAG technology, was successfully developed. Users can access it directly via a web link. Across the 112 cases, a total of 2 150 HPO terms were generated; 2,064 (96.0%) were fully validated by the official database, with a hallucination rate of 1.3% and an HPO ID-name mismatch rate of 2.7%. After deduplication, 1,906 terms remained for testing. The overall precision was 63.65%, recall was 67.34%, and F1 was 65.44%, significantly outperforming traditional annotation tools (F1: 0.45-0.49, P < 0.001). Although PhenoRAG's F1 was lower than that of RAG-HPO (F1 = 0.78, P < 0.001), which relies on a manually constructed synonym database of 54 000 entries plus the HPO dataset, it requires no additional dictionary maintenance and can be used without any background in computer programming. Moreover, after switching to the GPT-5 model, PhenoRAG exhibited no hallucination rate on the new dataset, and its F1 score significantly increased (P = 0.038).
CONCLUSION
Without constructing a synonym database, the PhenoRAG achieved high-accuracy automatic mapping from clinical text to standard HPO terms. It features a low usage threshold, free access, and a Chinese-language interface, and can directly serve rare disease diagnosis, genetic counseling, and research scenarios in China and worldwide, warranting further clinical promotion and multicenter validation.
Humans
;
Phenotype
;
Biological Ontologies
;
Language
;
Software
;
Large Language Models
2.Application value of MRI in evaluating the efficacy of anti-PD-1 combined with neoadjuvant therapy for microsatellite stability/proficient mismatch repair locally advanced rectal cancer
Jie ZHANG ; Lixue XU ; Zhengyang YANG ; Liting SUN ; Hongwei YAO ; Guangyong CHEN ; Zhenghan YANG
Chinese Journal of Digestive Surgery 2024;23(6):859-867
Objective:To investigate the application value of magnetic resonance imaging(MRI) in evaluating the efficacy of anti-PD-1 combined with neoadjuvant therapy for microsatellite stability (MSS)/proficient mismatch repair (pMMR) locally advanced rectal cancer (LARC).Methods:The prospective single-arm phase Ⅱ study was conducted. The clinicopathological data of 37 patients with MSS/pMMR LARC who were admitted to Beijing Friendship Hospital of Capital Medical University from April 2021 to September 2022 were collected. All patients underwent anti-PD-1 combined with neoadjuvant therapy and radical total mesorectal excision. Observation indicators: (1) enrolled pati-ents; (2) MRI and pathological examination; (3) concordance analysis of MRI examination reading; (4) evaluation of MRI examination. Measurement data with normal distribution were represented as Mean± SD. Count data were expressed as absolute numbers or percentages. Linear weighted κ value was used to evaluate the concordance of radiologist assessment. Sensitivity, negative predictive value, accuracy, overstaging rate and understaging rate were used to evaluate the predictive value. Results:(1) Enrolled patients. A total of 37 eligible patients were screened out, including 21 males and 16 females, aged (61±11)years. MRI examination was performed before and after combined therapy, and pathological examination was performed after radical resection. (2) MRI and pathological examination of patients. Among the 37 patients, MRI before combined therapy showed 0, 0, 5, 24 and 8 cases in stage T0, T1, T2, T3 and T4, 10, 17 and 10 cases in stage N0, N1 and N2, 28 and 9 cases of positive and negative extramural vascular invasion (EMVI), 4 and 33 cases of positive and negative mesorectal fascia (MRF), respectively. MRI examination after combined therapy showed 15, 4, 7, 10 and 1 cases in stage T0, T1, T2, T3 and T4, 34, 2 and 1 cases in stage N0, N1 and N2, 9 and 28 cases of positive and negative EMVI, 1 and 36 cases of positive and negative MRF. There were 16, 13, 8 and 0 cases of tumor regression grading (TRG) 0, 1, 2 and 3, respectively. Postoperative pathological examination showed 18, 4, 3, 11, 1 cases in stage T0, T1, T2, T3, T4, 33, 3, 1 cases in stage N0, N1, N2, positive and negative EMVI and unknown data in 1, 35, 1 cases, positive and negative circumferential margin in 0 and 37 cases, grade 0, grade 1, grade 2, grade 3 of American Joint Committee on Cancer TRG in 18, 9, 8, 2 cases, respectively. Pathological complete response rate was 48.6%(18/37) and approximate pathological complete response rate was 24.3%(9/37). (3)Concordance analysis of MRI examination reading. The κ value of T staging and N staging on MRI before combined therapy was 0.839 ( P<0.05) and 0.838 ( P<0.05), respectively. The κ value of T staging and N staging on MRI after combined therapy was 0.531 ( P<0.05) and 0.846 ( P<0.05), respectively. The κ value of EMVI and MRF was 0.708 ( P<0.05) and 0.680 ( P<0.05) before combined therapy, and they were 0.561 ( P<0.05) and 1.000 ( P<0.05) after combined therapy, respectively. The κ value of TRG 3-round reading for TRG was 0.448 ( P<0.05). (4) Evaluation of MRI examination. ① MRI evaluation of T and N staging. The accuracy of MRI examination after combined therapy for distinguishing stage T0 was 75.7%[28/37, 95% confidence interval ( CI) as 62.2%-89.2%], the understaging rate was 8.1%(3/37, 95% CI as 0-18.9%), the overstaging rate was 16.2%(6/37, 95% CI as 5.4%-29.7%). The accuracy of MRI examination for distinguishing stage T0-T2 was 86.5%(32/37, 95% CI as 73.0%-97.3%), its understaging rate and overstaging rate were 8.1%(3/37, 95% CI as 0-18.9%) and 5.4% (2/37, 95% CI as 0-13.5%), respectively. The accuracy of MRI examination for distinguishing N staging was 91.9%(34/37, 95% CI was 81.1%-100.0%), its understaging rate and overstaging rate were 5.4%(2/37, 95% CI as 0-13.5%) and 2.7%(1/37, 95% CI as 0-8.1%), respectively. Among 18 patients in pathological stage T0, the overstaging rate of MRI was 33.3%(6/18). All the 4 patients in pathological stage T1 and 3 pati-ents in pathological stage T2 had correct diagnosis. There were 3 cases with understaging among 12 patients in pathological stage T3-T4. Among the 37 patients in pathological stage N0-N2, 34 cases had correct diagnosis, 1 case was overstaged as stage N1 due to a round mesorectal lymph node with short diameter as 6 mm, and 2 cases were diagnosed as stage N0 due to the small lymph nodes with the maximum short diameter as 3 mm. ② MRI evaluation of EMVI and MRF. The accuracy, sensitivity and negative predictive value of MRI for evaluating EMVI were 86.5%(32/37, 95% CI as 75.0%-97.2%), 100.0% and 100.0%, respectively, and the overestimation rate of EMVI was 13.9%(5/36, 95% CI as 2.8%-25.0%), and no underestimation occurred. Of 35 pathologically negative EMVI patients, a rate of 14.3%(5/35) of patients were positive on MRI. The main reason for overestaging was that thickened fibrous tissue outside the rectal wall was mistaken for vascular invasion. The accuracy of MRI for evaluating MRF was 97.3%(36/37, 95% CI as 91.9%-100.0%), and 1 case (1/37, 2.7%, 95% CI as 0-8.1%) was overestimated as positive MRF due to misdiagnosis of pararectal MRF lymph nodes. The negative predictive value of MRI for assessing MRF was 100.0%. ③ MRI evaluation of TRG. The accuracy, understaging and overstaging rates of MRI for evaluating pathological TRG 0 were 78.4%(29/37, 95% CI as 64.9%-91.9%), 8.1%(3/37, 95% CI as 0-18.9%), 13.5%(5/37, 95% CI as 5.4%-27.0%), respectively. The accuracy, understaging and overstaging rates of MRI for evaluating pathological TRG 0-1 were 89.2%(33/37, 95% CI as 78.4%-97.3%), 8.1%(3/37, 95% CI as 0-18.9%), 2.7%(1/37, 95% CI as 0-8.1%), respectively. Of the 18 patients with pathologic complete response, 5 cases were diagnosed as pathological TRG 1 and 13 cases as pathological TRG 0. One near-pCR patient was assessed as pathological TRG 2. Two patients with pathological TRG 3 were incorrectly diagnosed on MRI. Conclusions:Anti-PD-1 combined with neoadjuvant therapy can downstage the LARC pati-ents with MSS/pMMR. MRI is effective in predicting T staging, N staging, EMVI, MRF and TRG. However, overstaging should be prevented.
3.A nomogram based on clinical, ultrasound and contrast-enhanced ultrasound features for preoperative differentiating intrahepatic cholangiocarcinoma from hepatocellular carcinoma
Chunrui LIU ; Haiyan XUE ; Han LIU ; Peng WAN ; Wentao KONG ; Zhengyang ZHOU ; Jing YAO
Chinese Journal of Ultrasonography 2024;33(5):369-377
Objective:To establish a nomogram for preoperative differentiating intrahepatic cholangiocarcinoma (ICC) from hepatocellular carcinoma (HCC) based on clinical, ultrasound, and contrast-enhanced ultrasound (CEUS) data.Methods:A retrospective analysis was conducted on ultrasound and CEUS data of 462 patients who underwent hepatectomy in Nanjing Drum Tower Hospital, the Affiliated Hospital of Nanjing University Medical School from January 2016 to December 2023, including 262 cases of HCC (56.7%) and 200 cases of ICC (43.3%). The data were randomly divided into training set ( n=324) and validation set ( n=138) in a 7∶3 ratio. Univariate analysis was used to initially screen for variables with statistically significant differences between HCC and ICC groups in the training set, and LASSO regression was performed to select the variables with higher coefficients. Logistic regression analyses were then used to predict independent risk factors for ICC. A nomogram was drawn using R software. The performance of the nomogram was then validated using ROC curve, calibration curve, and decision curve analysis (DCA). Results:Univariate analysis showed that there were significant differences in age, gender, liver cirrhosis, HBsAg (+ ), ALP >185 U/L, CA19-9 >27 kU/L, CA242>10 kU/L, irregular shape, border, cholangiectasis, portal vein tumor thrombus, enhanced pattern in arterial phase, clearance time <60 s, intra-tumoral vein between ICC and HCC groups (all P<0.05). The top 10 features were selected for LASSO regression analysis. Logistic regression analysis revealed that gender, cirrhosis, CA19-9>27 kU/L, CA242>10 kU/L, cholangiectasis, clearance time <60 s, intra-tumoral vein and enhanced pattern in arterial phase were risk factors for ICC (all P<0.05). The area under the ROC curve in the training and validation groups were 0.963 and 0.914, respectively. In the training group, the specificity and sensitivity of the nomogram were 0.926 and 0.917, respectively, and in the validation group, they were 0.875 and 0.871, respectively. The calibration curve showed that the prediction effect of the model was in good agreement with the actual situation. DCA showed that the nomogram could increase the net benefit to the different diagnosis of ICC in patients. Conclusions:The nomogram based on clinical, ultrasound and CEUS features has a good predictive value for preoperative identification of ICC and provides reliable evidence for clinical practice.
4.Development and evaluation of a clinical and ultrasound features-based nomogram for the preoperative diagnosis of intrahepatic cholangiocarcinoma
Chunrui LIU ; Haiyan XUE ; Han LIU ; Peng WAN ; Jing YAO ; Wentao KONG ; Zhengyang ZHOU
Chinese Journal of Hepatobiliary Surgery 2024;30(5):354-359
Objective:To establish and evaluate a clinical and ultrasound parameters-based nomogram for the preoperative differentiating diagnosis of intrahepatic cholangiocarcinoma (ICC).Methods:A total of 723 patients undergoing hepatectomy in Nanjing Drum Tower Hospital, Affiliated Hospital of Medical School, Nanjing University from January 2016 to August 2022 were retrospectively screened. A total of 399 patients with hepatocellular carcinoma (HCC, 198 cases) or ICC (201 cases) were enrolled in this study, including 284 males and 115 females, aged (60.5±10.5) years. Through random sampling using computer-generated random numbers, patients were divided into training ( n=279) and validation groups ( n=120) in a ratio of 7∶3. Univariate and multivariate logistic regression were performed to identify factors differentiating ICC, and a nomogram was established using R software based on independent risk factors for ICC. The accuracy of the nomogram was evaluated by receiver operating characteristic curve and calibration curves. Decision curve analysis was performed to assess the net benefit of the model. Results:Multivariate logistic regression analysis showed that irregular shape, cholangiectasis, female, cirrhosis, carbohydrate antigen 242 >10 U/ml, carbohydrate antigen 125 >30 U/ml and alpha-fetoprotein >10 μg/L were independent differentiating factors for ICC (all P<0.05). A nomogram was constructed based on those factors. The nomogram showed a better discrimination between ICC and HCC. The area under the curve of the training group and the validation group were 0.966 and 0.956, respectively. The calibration curve showed that the prediction effect of the model is in good agreement with the actual situation. Decision curve analysis showed that the nomogram was more effective than diagnosing all patients as either HCC or ICC, which yielded a net benefit at the most reasonable threshold probabilities. Conclusion:The nomogram for the preoperative diagnosis of ICC based on clinical and ultrasound features showed a good diagnostic performance.
5.Clinical value of magnetic resonance imaging in predicting pathological complete response after immunotherapy combined with neoadjuvant therapy for local advanced rectal cancer
Xiao ZHANG ; Jiale GAO ; Zhengyang YANG ; Jie ZHANG ; Hongwei YAO ; Zhongtao ZHANG
Chinese Journal of Digestive Surgery 2022;21(11):1467-1474
Objective:To investigate the clinical value of magnetic resonance imaging (MRI) in predicting pathological complete response (pCR) after immunotherapy combined with neo-adjuvant therapy for local advanced rectal cancer.Methods:The retrospective and descriptive study was conducted. The clinicopathological data of 48 patients with local advanced rectal cancer who were admitted to Beijing Friendship Hospital of Capital Medical University from January 2020 to March 2022 were collected. There were 35 males and 13 females, aged 62(32?77)years. Of 48 patients, 30 patients received neoadjuvant therapy, 18 patients received immunotherapy combined with neoadjuvant therapy. All patients underwent total mesorectal excision. Observation indicators: (1) T staging on MRI and postoperative pathological examination after neoadjuvant therapy and immunotherapy combined with neoadjuvant therapy; (2) changes of apparent diffusion coefficients (ADC) in pCR and non-pCR patients after neoadjuvant therapy and immunotherapy combined with neoadjuvant therapy; (3) evaluation of predictive performance of MRI for pCR after immunotherapy combined with neoadjuvant therapy. Measurement data with normal distribution were represented as Mean± SD, and t test was used for comparison between groups. Measurement data with skewed distribution were represented as M(range). Count data were expressed as absolute numbers or percentages. Sensitivity, specificity and accuracy were used to evaluate the predictive performance. Results:(1) T staging on MRI and postoperative pathological examination after neoadjuvant therapy and immunotherapy combined with neoadjuvant therapy. Of the 30 patients receiving neoadjuvant therapy, 1 patient in stage T2 showed stage T2 on both MRI and postoperative pathological examination after neoadjuvant therapy, 16 patients in stage T3 showed stage T0, T1, T2, T3, T4 of 0, 1, 6, 9, 0 cases and 3, 0, 8, 4, 1 cases on MRI and postoperative pathological examination respectively after neoadjuvant therapy, 13 patients in stage T4 showed stage T0, T1, T2, T3, T4 of 0, 0, 1, 2, 10 cases and 1, 0, 4, 7, 1 cases on MRI and postoperative pathological examination respectively after neoadjuvant therapy. The pCR rate was 13.3%(4/30) and the accuracy rate of MRI was 43.33% for patients with neoadjuvant therapy. Of the 18 patients receiving immunotherapy combined with neoadjuvant therapy, 12 patients in stage T3 showed stage T0, T1, T2, T3, T4 in 4, 2, 2, 4,0 cases and 5, 1, 1, 5, 0 cases on MRI and postoperative pathological examination respectively after immunotherapy combined with neoadjuvant therapy, 6 patients in stage T4 showed stage T0, T1, T2, T3, T4 in 0, 0, 1, 3, 2 cases and 4, 0, 0, 2, 0 cases on MRI and postoperative pathological examination respectively after immunotherapy combined with neoadjuvant therapy. The pCR rate was 50.0%(9/18) and the accuracy rate of MRI was 38.89% for patients with neoadjuvant therapy. (2) Changes of ADC in pCR and non-pCR patients after neoadjuvant therapy and immunotherapy combined with neoadjuvant chemoradiotherapy. Of the 30 patients receiving neoadjuvant therapy, the ADC differences were 0.30±0.04 and 0.21±0.17 for 4 pCR and 26 non-pCR patients, respectively, showing a significant difference ( t=2.36, P<0.05). Of the 18 patients receiving immunotherapy combined with neoadjuvant therapy, the ADC change rates and ADC differences were 40%±14% and 0.39±0.14 for 9 pCR patients, versus 22%±13% and 0.21±0.12 of 9 non-pCR patients, showing significant differences in the above indicators ( t=2.86, 2.79, P<0.05). Receiver operation charac-teristic curve analysis of ADC change rate and ADC difference associated with pCR for 18 patients receiving immunotherapy combined with neoadjuvant therapy suggested that the areas under the curve were 0.81 (95% confidence interval as 0.60?1.00, P<0.05) and 0.86 (95% confidence interval as 0.70?1.00, P<0.05), with cutoff values as 0.23 and 0.36, respectively. (3) Evaluation of predictive performance of MRI for pCR after immunotherapy combined with neoadjuvant therapy. For the 18 patients receiving immunotherapy combined with neoadjuvant therapy, the sensitivity, specificity, accuracy were 33.33%, 88.89%, 61.11% of stage T0 on MRI for predicting pCR, 88.89%, 55.56%, 72.22% of down-staging of T staging on MRI for predicting pCR, and all 77.78% of ADC difference greater than the cutoff value for predicting pCR. Conclusions:Patients with local advanced rectal cancer who received immunotherapy combined with neoadjuvant therapy achieve a higher pCR rate. ADC difference and down-staging of T staging on MRI can predict pCR effectively.
6.Actively participate in international high-quality clinical trials for improving the research level of colorectal surgery in China
Hongwei YAO ; Zhengyang YANG ; Jiale GAO ; Yishan LIU ; Jurriaan TUYNMAN ; Jaap BONJER ; Zhongtao ZHANG
Chinese Journal of Digestive Surgery 2022;21(6):749-752
The number of colorectal cancer patients in China ranks the top in the world, but there are few international guidelines for the diagnosis and treatment of colorectal cancer formulated by Chinese, nor high-level evidence-based medicine research of colorectal cancer from China. Transanal total mesorectal excision (taTME) is a new technology in the field of colorectal surgery in recent years. At present, clinical practice related to taTME has been carried out simul-taneously with clinical researches in the world. Based on the experience of participating in the top clinical trials in the field of international colorectal surgery, like the COLOR series prospective research, the authors introduce the organization and implementation of COLOR Ⅲ research in China. It is hoped that the COLOR series trials will become an example in the field of high-quality surgical clinical research, so as to improve the clinical research level of colorectal surgery in China.

Result Analysis
Print
Save
E-mail