Research on Lightweight Large Language Models for Ancient Traditional Chinese Medicine Texts Based on Lora Fine-Tuning
- VernacularTitle:基于Lora微调的轻量化中医药古籍大语言模型研究
- Author:
Jingxian CHAI
1
;
Xufeng LANG
;
Hongyan LI
;
Zuojian ZHOU
;
Yun LING
;
Libin ZHAN
;
Kongfa HU
;
Xuebin QIAO
Author Information
- Publication Type:Journal Article
- Keywords: Large language models; Traditional Chinese Medicine ancient texts; Shang Han Lun; Low-rank adaptation; Model optimization
- From: World Science and Technology-Modernization of Traditional Chinese Medicine 2025;27(3):823-831
- CountryChina
- Language:Chinese
- Abstract: Objective To address the challenges of constructing large language models for traditional Chinese medicine(TCM)classics,which are complex and expensive to fine-tune,this study explores a lightweight fine-tuning method for such models,aiming to develop a question-answering model centered on TCM classics,particularly various editions of Shang Han Lun through the ages.Methods Dataset construction involved designing prompts to guide GPT-4 in generating Q&A pairs based on Shang Han Lun and integrating them with the ShenNong_TCM_Dataset and cMedQA2 datasets.Five general-purpose large models were selected for Lora fine-tuning.The best model was chosen through evaluation,and the performance of multiple quantized versions was validated.Results After fine-tuning,the BLEU,ROUGE-1,ROUGE-2,and ROUGE-L metrics for the Qwen-7B-Chat model improved by 17.61,19.63,14.3,and 21.4,respectively,compared to the base model.Conclusion The selected model in this study is capable of effectively understanding and utilizing professional terms and concepts from TCM classics,such as Shang Han Lun,to provide accurate answers to user queries.Compared to similar models,it requires lower fine-tuning costs and computational power,contributing to the dissemination of TCM knowledge and the development of intelligent systems.
