Objective: To examine the association between frailty and cognitive function in middle-aged and elderly populations using group-based trajectory model (GBTM), so as to provide insights into the prevention and intervention strategies for cognitive impairment in this population. Methods: From 2011 to 2018, data of demographic information and lifestyle behavior for individuals aged ≥45 years were collected from the China Health and Retirement Longitudinal Study. Frailty status was assessed using the frailty index, and cognitive function was evaluated using the Chinese version of the Mini-Mental State Examination. GBTM was applied to establish frailty trajectories based on the frailty index from 2011 to 2018 to analyze the trajectory of frailty changes. The association between frailty and cognitive function was analyzed using multiple linear regression model. Results: A total of 4 809 participants were included, with a mean age of (56.63±7.73) years. There were 2 621 males (54.50%) and 2 188 females (45.50%). The 2018 survey identified 574 individuals (11.94%) with cognitive function decline. GBTM identified four distinct frailty trajectories including the normal and stable group (69.27%), recovering frail group (9.32%), progressing pre-frail group (16.20%) and persistently frail group (5.22%). The mean cognitive funtion scores for these groups were (17.24±4.83) (16.38±5.53) (15.74±5.20) (15.02±5.11) points, respectively. The differences in cognitive funtion scores across groups were statistically significant (P<0.05). Multiple linear regression analysis showed, after adjusting for confounding factors such as gender, age, nighttime sleep duration, and smoking, compared with the normal and stable group, the cognitive function decline risk was higher in the frailty recovery group (β=-0.581), the pre-frail progression group (β=-0.583), and the persistently frail group (β=-0.662) compared with the non-frail stable group (all P<0.05). Conclusions There are four groups of changes in frailty among the middle-aged and elderly populations. Compared with the normal and stable group, the groups experiencing progression, persistence, and recovery of frailty exhibit a more pronounced risk of cognitive funtion decline.