The mitogen-activated protein kinase (MAPK)/extracellular signal-regulated kinase 1/2 (ERK1/2) signaling pathway is widely activated by a variety of extracellular stimuli, and its dysregulation is associated with the proliferation, invasion, and migration of cancer cells. ERK1/2 is located at the distal end of this pathway and rarely undergoes mutations, making it an attractive target for anticancer drug development. Currently, an increasing number of ERK1/2 inhibitors have been designed and synthesized for antitumor therapy, among which representative compounds have entered clinical trials. When ERK1/2 signal transduction is eliminated, ERK5 may provide a bypass route to rescue proliferation, and weaken the potency of ERK1/2 inhibitors. Therefore, drug research targeting ERK5 or based on the compensatory mechanism of ERK5 for ERK1/2 opens up a new way for oncotherapy. This review provides an overview of the physiological and biological functions of ERKs, focuses on the structure-activity relationships of small molecule inhibitors targeting ERKs, with a view to providing guidance for future drug design and optimization, and discusses the potential therapeutic strategies to overcome drug resistance.