ObjectiveTo investigate the relationship between Jiaotai pill (JTP), its main component berberine (BBR), and the serotonin (5-HT) system in regulating islet hormone secretion and alleviating pancreatic β-cell dysfunction during type 2 diabetes mellitus (T2DM) progression.MethodsT2DM rat model was established using a high-fat diet and streptozotocin injection. JTP, BBR, and Metformin were intragastrically administered for 35 days. The analyzed indices included blood glucose, blood lipids, islet hormones, and proteins related to 5-HT synthesis, secretion, and transport. Additionally, an in vitro model of glucose injury in islet cells was established to study the effects of JTP and BBR on islet hormone secretion following tryptophan hydroxylase 1 (TPH1) inhibition.ResultsJTP and BBR significantly improved blood glucose and lipid levels and islet morphology in T2DM rats. Both models exhibited reduced islet 5-HT levels and impaired islet hormone secretion. However, the administration of JTP and BBR reversed these effects. Furthermore, JTP and BBR upregulated the expression of TPH1(P = .0194, P = .0413) transglutaminase 2 (TGase2; P = .0492, P = .0349), serotonin transporter (SERT, P = .0090), and 5- hydroxytryptamine 1F receptor (5-HT1FR) in the islet 5-HT pathway (P = .0194). In the cell model, the regulatory effects of JTP and BBR on islet hormone levels were significantly weakened after TPH1 inhibition (P = .001), suggesting that JTP and BBR influence islet hormone secretion through the pancreatic 5-HT system.ConclusionThe islet 5-HT system is correlated with islet hormone secretion dysfunction in T2DM. JTP and BBR can improve islet hormone secretion by activating the TPH1/TGase2/SERT/5-HT1FR pathway in the islet 5-HT system in T2DM rats.