Background: Maintaining good exercise behavior contributes to the prevention of adverse health outcomes in older inpatients with musculoskeletal disorders. This prospective cohort study aimed to identify the predictors of poor exercise behavior in older inpatients with musculoskeletal disorders after discharge from a rehabilitation hospital. Methods: The study included 117 older inpatients aged ≥60 years with musculoskeletal disorders at a rehabilitation hospital, which consisted of the convalescent rehabilitation ward and the community-based integrated care ward. Baseline assessments, including demographics, physical function, cognitive function, psychological status, sleep disorders, and nutritional status, were obtained at discharge from the hospital. Exercise behavior was assessed using the transtheoretical model at baseline and at the 6-month follow-up after discharge from the hospital. Based on behavioral changes at baseline and follow-up, participants were classified into those with poor exercise behavior and those without. Logistic regression analyses were performed to identify the predictors of poor exercise behavior. Results: Of the 117 participants, 53 (45.3%) had poor exercise behavior. Lower exercise self-efficacy and more depressive symptoms at baseline were associated with poor exercise behavior during the 6-month follow-up after discharge. These associations remained significant after adjusting for age, sex, body mass index, comorbidities, and physical function measures. Conclusion Lower exercise self-efficacy and more depressive symptoms are predictors of poor exercise behavior after hospital discharge among older inpatients with musculoskeletal disorders. Our findings underscore the importance of strategies to enhance mental well-being and maintain good exercise behavior following discharge in this population.