1.Research on Lightweight Large Language Models for Ancient Traditional Chinese Medicine Texts Based on Lora Fine-Tuning
Jingxian CHAI ; Xufeng LANG ; Hongyan LI ; Zuojian ZHOU ; Yun LING ; Libin ZHAN ; Kongfa HU ; Xuebin QIAO
World Science and Technology-Modernization of Traditional Chinese Medicine 2025;27(3):823-831
Objective To address the challenges of constructing large language models for traditional Chinese medicine(TCM)classics,which are complex and expensive to fine-tune,this study explores a lightweight fine-tuning method for such models,aiming to develop a question-answering model centered on TCM classics,particularly various editions of Shang Han Lun through the ages.Methods Dataset construction involved designing prompts to guide GPT-4 in generating Q&A pairs based on Shang Han Lun and integrating them with the ShenNong_TCM_Dataset and cMedQA2 datasets.Five general-purpose large models were selected for Lora fine-tuning.The best model was chosen through evaluation,and the performance of multiple quantized versions was validated.Results After fine-tuning,the BLEU,ROUGE-1,ROUGE-2,and ROUGE-L metrics for the Qwen-7B-Chat model improved by 17.61,19.63,14.3,and 21.4,respectively,compared to the base model.Conclusion The selected model in this study is capable of effectively understanding and utilizing professional terms and concepts from TCM classics,such as Shang Han Lun,to provide accurate answers to user queries.Compared to similar models,it requires lower fine-tuning costs and computational power,contributing to the dissemination of TCM knowledge and the development of intelligent systems.
2.Research on Lightweight Large Language Models for Ancient Traditional Chinese Medicine Texts Based on Lora Fine-Tuning
Jingxian CHAI ; Xufeng LANG ; Hongyan LI ; Zuojian ZHOU ; Yun LING ; Libin ZHAN ; Kongfa HU ; Xuebin QIAO
World Science and Technology-Modernization of Traditional Chinese Medicine 2025;27(3):823-831
Objective To address the challenges of constructing large language models for traditional Chinese medicine(TCM)classics,which are complex and expensive to fine-tune,this study explores a lightweight fine-tuning method for such models,aiming to develop a question-answering model centered on TCM classics,particularly various editions of Shang Han Lun through the ages.Methods Dataset construction involved designing prompts to guide GPT-4 in generating Q&A pairs based on Shang Han Lun and integrating them with the ShenNong_TCM_Dataset and cMedQA2 datasets.Five general-purpose large models were selected for Lora fine-tuning.The best model was chosen through evaluation,and the performance of multiple quantized versions was validated.Results After fine-tuning,the BLEU,ROUGE-1,ROUGE-2,and ROUGE-L metrics for the Qwen-7B-Chat model improved by 17.61,19.63,14.3,and 21.4,respectively,compared to the base model.Conclusion The selected model in this study is capable of effectively understanding and utilizing professional terms and concepts from TCM classics,such as Shang Han Lun,to provide accurate answers to user queries.Compared to similar models,it requires lower fine-tuning costs and computational power,contributing to the dissemination of TCM knowledge and the development of intelligent systems.
3.Research on A TabNet-Based Predictive Model and Medication Patterns in the Diagnosis and Treatment of Hyperthyroidism by Professor Zhou Zhongying
Xiaona YANG ; Yao ZHU ; Xiangling XING ; Zuojian ZHOU ; Kankan SHE
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(5):534-542
OBJECTIVE Taking Professor Zhou Zhongying's clinical cases of treating hyperthyroidism as the research object,this article explored the use of the TabNet model based on neural networks to discover the diagnosis and treatment rules of hyperthyroid-ism,providing a method reference for inheriting the academic thoughts of famous veteran traditional Chinese medicine practitioners and assisting clinical diagnosis and treatment.METHODS Based on the clinical diagnosis and treatment cases of hyperthyroidism of Pro-fessor Zhou Zhongying and his team,standardized and structured training data were constructed;algorithms based on attention mecha-nism and sparse feature selection mechanism were studied;a pathogenesis prediction model was constructed by inputting standardized clinical manifestations,standardized tongue and pulse conditions;core symptoms,pathogenesis and medication were analyzed,as well as the relationship between the three.RESULTS The trained prediction model was used to predict the 6 pathogenesis of liver stagna-tion,liver fire,phlegm fluid,kidney deficiency,yin deficiency,and blood stasis.Compared with multi-label classification models constructed by classic algorithms such as decision trees and random forests,this model had better classification and prediction indica-tors.Mining was carried out through the decision tree algorithm,and 6 core pathogenesis corresponding Chinese medicine groups were summarized:vinegar-baked Bupleurum chinense,prunella vulgaris,oyster,processed Carapax trionycis,Scrophularia ningpoensis,Asparagus cochinchinensis,Ophiopogon japonicus,etc.CONCLUSION Using the TabNet algorithm on clinical medical record data to build a pathogenesis prediction model based on clinical manifestations,tongue and pulse conditions can effectively predict the core pathogenesis,and then discover the connection between symptoms,pathogenesis and medication,providing methodological references for the inheritance of academic ideas of famous veteran traditional Chinese medicine practitioners and clinical auxiliary diagnosis and treatment decision-making.
4.Entity Recognition in Treatise on Cold Damage Based on Relative Position Representation Self-Attention Mechanism
Hongmin XU ; Hongyan LI ; Xufeng LANG ; Zuojian ZHOU ; Yun LING ; Ziyan WANG
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(12):1357-1365
OBJECTIVE Treatise on Cold Damage is one of the"Four Classics of Traditional Chinese Medicine,"containing a wealth of medical practice experience and medication rules.However,there has been insufficient data mining in the ancient literature of Treatise on Cold Damage,particularly due to the complex contextual semantics,making it challenging to fully grasp the interrelation-ships.This study aims to conduct entity recognition in Treatise on Cold Damage to facilitate comprehensive knowledge extraction.METHODS A Bert-BiLSTM-RPRSA-CRF model was constructed based on the specialized terminology and concise sentence struc-ture of the ancient literature.By incorporating a relative position representation self-attention(RPRSA)layer,this named entity recog-nition model aimed to identify entities within the text while learning information at different levels,thereby enhancing accuracy.RE-SULTS Experimental verification demonstrated that our named entity recognition model achieved F1-Score,precision,and recall rates of 88.24%,88.48%,and 88.00%respectively on the Treatise on Cold Damage dataset,outperforming other commonly used models.CONCLUSION Our method outperforms other models in identifying entities within Treatise on Cold Damage,providing a foundation for information extraction from traditional Chinese medicine ancient texts such as Treatise on Cold Damage while offering ef-fective means for intelligent assisted diagnosis and treatment in traditional Chinese medicine.
5.Construction of Traditional Chinese Medicine Question-Answering Large Language Model Based on Retrieval-Augmented Generation Technology
Yuming ZHANG ; Hongyan LI ; Xufeng LANG ; Zuojian ZHOU ; Yun LING ; Ziyan WANG
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(12):1375-1382
OBJECTIVE To construct a large language model for TCM question-answering.METHODS TCM corpora were built by collecting TCM classics such as Treatise on Cold Damage,TCM textbooks,prescriptions from famous TCM doctors,and other manually annotated TCM datasets.A TCM knowledge vector library was constructed.The RAG technology was fused with the P-Tuning v2 fine-tuning method and the large language model(ChatGLM2-6B)to build the TCM question-answering large language model.RESULTS Recision,Recall,and F1 score were used as evaluation metrics for knowledge question-answering tasks.The model achieved over 90%accuracy in simple TCM question-answering,with the highest accuracy in component-type questions,reac-hing an F1 score of 0.928.The accuracy of medium to high difficulty questions ranged from 75.8%to 87.7%,with F1 scores all ex-ceeding 0.766.Expert ratings based on diversity and accuracy were used as evaluation metrics for TCM question generation tasks,and the model in this paper scored 9.5 points higher than the baseline model.CONCLUSION The model in this paper demonstrates good semantic understanding and high reliability,effectively alleviating model hallucinations and helping patients clarify their question intentions.It is of great significance for advancing research on TCM knowledge and providing personalized interactive answers.It also provides an innovative approach to promoting the inheritance and popularization of TCM experience and the intelligent construction of TCM diagnosis and treatment.
6.Issues and Challenges in the AI-Empowered High-Quality Development of Traditional Chinese Medicine
Tao YANG ; Haiyan REN ; Zuojian ZHOU ; Xuefang ZHU ; Kongfa HU
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(12):1285-1290
The high-quality development of Traditional Chinese Medicine(TCM)is a significant issue faced by the development of TCM in the new era.Artificial Intelligence(AI),as one of the representatives of new productive forces,is expected to provide strong momentum for the inheritance,innovation,and development of TCM.By focusing on three major directions—intelligent TCM early warning and diagnosis,intelligent TCM treatment and rehabilitation,and intelligent TCM research and teaching—the paper reviews the existing problems and challenges in AI-empowered TCM and proposes solutions.In terms of intelligent TCM early warning and diagno-sis,there are issues with the normalization and standardization of TCM theory itself,insufficient open and high-quality large-scale TCM annotation data resources,and lack of TCM theory and thinking guidance in the design of intelligent methods.In terms of intelli-gent TCM treatment and rehabilitation,there are issues such as the feedback and adjustment mechanism are not yet sound,the depth of multidisciplinary collaborative innovation is insufficient,and technical safety and laws and regulations are not perfect.In terms of intel-ligent TCM research and teaching,there are issues such as insufficient mining and interpretation of TCM implicit knowledge,and insuf-ficient learning and simulation of TCM expert thinking.Based on the above issues,corresponding solutions are proposed to explore ide-as for the research in the interdisciplinary field of"artificial intelligence+TCM"and to provide reference for the high-quality develop-ment of TCM inheritance and innovation.
7.Phlegm-Dampness and Yin-Deficiency Constitution Identification Model Based on Human Body 3D Reconstruction
Ziyan WANG ; Tao YANG ; Zuojian ZHOU ; Kongfa HU
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(12):1340-1347
OBJECTIVE To propose an approach based on monocular optical camera-captured full-body two-dimensional ima-ges.Through a three-dimensional reconstruction algorithm,to extract three-dimensional shape parameters and utilize them for intelli-gent identification of phlegm-dampness,yin-deficiency,and other constitutions.METHODS Standard static standing posture ima-ges of subjects in their natural state were collected,and subjects filled out constitution assessment forms or were assessed by the chief TCM physician to obtain constitution information.Constitution served as data labels.A parametric human body three-dimensional re-construction algorithm was employed to extract three-dimensional shape features.Sample distribution was improved using synthetic mi-nority oversampling technique(SMOTE),and a neural network was utilized to establish the connection between human body shape and constitution.RESULTS Experimental results indicate that the accuracy of the phlegm-dampness and yin-deficiency constitution iden-tification model based on human body three-dimensional reconstruction could reach 86.16%,with an F1 score of 79.35%.After using SMOTE to enhance sample distribution,the model accuracy increases to 89.91%,with an F1 score of 84.33%.This demonstrated the feasibility and accuracy of the identification model based on human body three-dimensional reconstruction.CONCLUSION The ex-traction of human body shape features based on three-dimensional reconstruction can effectively identify phlegm-dampness and yin de-ficiency constitutions.Compared to existing methods,this approach is more convenient and enables the rapid detection of potential bia-ses in individual constitutions.Early intervention and correction can be applied to achieve the goal of"preventing disease before it oc-curs".In outpatient clinics,health checkups,and other clinical scenarios,this method has high potential and value for clinical appli-cation.Additionally,this method provides new insights for the intelligent and objective identification of TCM constitution.
8.Entity Recognition in Treatise on Cold Damage Based on Relative Position Representation Self-Attention Mechanism
Hongmin XU ; Hongyan LI ; Xufeng LANG ; Zuojian ZHOU ; Yun LING ; Ziyan WANG
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(12):1357-1365
OBJECTIVE Treatise on Cold Damage is one of the"Four Classics of Traditional Chinese Medicine,"containing a wealth of medical practice experience and medication rules.However,there has been insufficient data mining in the ancient literature of Treatise on Cold Damage,particularly due to the complex contextual semantics,making it challenging to fully grasp the interrelation-ships.This study aims to conduct entity recognition in Treatise on Cold Damage to facilitate comprehensive knowledge extraction.METHODS A Bert-BiLSTM-RPRSA-CRF model was constructed based on the specialized terminology and concise sentence struc-ture of the ancient literature.By incorporating a relative position representation self-attention(RPRSA)layer,this named entity recog-nition model aimed to identify entities within the text while learning information at different levels,thereby enhancing accuracy.RE-SULTS Experimental verification demonstrated that our named entity recognition model achieved F1-Score,precision,and recall rates of 88.24%,88.48%,and 88.00%respectively on the Treatise on Cold Damage dataset,outperforming other commonly used models.CONCLUSION Our method outperforms other models in identifying entities within Treatise on Cold Damage,providing a foundation for information extraction from traditional Chinese medicine ancient texts such as Treatise on Cold Damage while offering ef-fective means for intelligent assisted diagnosis and treatment in traditional Chinese medicine.
9.Construction of Traditional Chinese Medicine Question-Answering Large Language Model Based on Retrieval-Augmented Generation Technology
Yuming ZHANG ; Hongyan LI ; Xufeng LANG ; Zuojian ZHOU ; Yun LING ; Ziyan WANG
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(12):1375-1382
OBJECTIVE To construct a large language model for TCM question-answering.METHODS TCM corpora were built by collecting TCM classics such as Treatise on Cold Damage,TCM textbooks,prescriptions from famous TCM doctors,and other manually annotated TCM datasets.A TCM knowledge vector library was constructed.The RAG technology was fused with the P-Tuning v2 fine-tuning method and the large language model(ChatGLM2-6B)to build the TCM question-answering large language model.RESULTS Recision,Recall,and F1 score were used as evaluation metrics for knowledge question-answering tasks.The model achieved over 90%accuracy in simple TCM question-answering,with the highest accuracy in component-type questions,reac-hing an F1 score of 0.928.The accuracy of medium to high difficulty questions ranged from 75.8%to 87.7%,with F1 scores all ex-ceeding 0.766.Expert ratings based on diversity and accuracy were used as evaluation metrics for TCM question generation tasks,and the model in this paper scored 9.5 points higher than the baseline model.CONCLUSION The model in this paper demonstrates good semantic understanding and high reliability,effectively alleviating model hallucinations and helping patients clarify their question intentions.It is of great significance for advancing research on TCM knowledge and providing personalized interactive answers.It also provides an innovative approach to promoting the inheritance and popularization of TCM experience and the intelligent construction of TCM diagnosis and treatment.
10.Issues and Challenges in the AI-Empowered High-Quality Development of Traditional Chinese Medicine
Tao YANG ; Haiyan REN ; Zuojian ZHOU ; Xuefang ZHU ; Kongfa HU
Journal of Nanjing University of Traditional Chinese Medicine 2024;40(12):1285-1290
The high-quality development of Traditional Chinese Medicine(TCM)is a significant issue faced by the development of TCM in the new era.Artificial Intelligence(AI),as one of the representatives of new productive forces,is expected to provide strong momentum for the inheritance,innovation,and development of TCM.By focusing on three major directions—intelligent TCM early warning and diagnosis,intelligent TCM treatment and rehabilitation,and intelligent TCM research and teaching—the paper reviews the existing problems and challenges in AI-empowered TCM and proposes solutions.In terms of intelligent TCM early warning and diagno-sis,there are issues with the normalization and standardization of TCM theory itself,insufficient open and high-quality large-scale TCM annotation data resources,and lack of TCM theory and thinking guidance in the design of intelligent methods.In terms of intelli-gent TCM treatment and rehabilitation,there are issues such as the feedback and adjustment mechanism are not yet sound,the depth of multidisciplinary collaborative innovation is insufficient,and technical safety and laws and regulations are not perfect.In terms of intel-ligent TCM research and teaching,there are issues such as insufficient mining and interpretation of TCM implicit knowledge,and insuf-ficient learning and simulation of TCM expert thinking.Based on the above issues,corresponding solutions are proposed to explore ide-as for the research in the interdisciplinary field of"artificial intelligence+TCM"and to provide reference for the high-quality develop-ment of TCM inheritance and innovation.

Result Analysis
Print
Save
E-mail