Prostate cancer remains the second most common malignancy in men worldwide, is a global health issue, and poses a huge health burden. Precision medicine provides more treatment options for prostate cancer patients, but its popularity, drug resistance, and adverse reactions still need to be focused on. Chinese herbal medicines (CHMs) have been widely accepted as an alternative therapy for cancer, with the advantages of multiple targets, multiple pathways, and low toxicity. We searched the experimental research and clinical practice of CHMs for prostate cancer treatment published in PubMed, Embase, and Web of Science in the last five years. We found five CHM formulas and six single CHM extracts as well as 12 CHM-derived compounds, which showed induction of apoptosis, autophagy, and cell cycle arrest, suppression of angiogenesis, proliferation, and migration of prostate cancer cells, reversal of drug resistance, and enhancement of anti-tumor immunity. The mechanisms of action include the PI3K/Akt/mTOR, AR, EGFR and Wnt/β-catenin signaling pathways, which are commonly implicated in the development of prostate cancer. We also summarized the advantages of CHMs in patients with hormone-sensitive and castration-resistant prostate cancer and provided ideas for their further experimental design and application.