Background: A decrease in physical fitness with age can impact work ability. Exploring the role of physical fitness is important for identifying interventions to enhance work ability among aging workers. We sought to determine the association between physical fitness and physically demanding work ability in aging workers. Methods: We recruited workers aged 45–65 years from eight departments of a university hospital in Bangkok, Thailand. Work ability was assessed using the work ability index (WAI), and physical fitness components were evaluated through various tests. Associations between work ability and physical fitness were examined using a multivariable regression analysis. Results: The response rate was 46.4% (n = 216). The mean WAI score was 41.6. Participants with an overweight or obese body mass index (BMI) had 1.8 (95% CI –3.1, –0.4) points lower WAI scores than did those within a healthy BMI range. A handgrip strength test revealed that participants in the good/very good group had 2.5 (95% CI: 0.6, 4.3) higher WAI scores than did those in the very poor/poor group. For a single-leg stance test, participants in the above-average group had 2.1 (95% CI: 0.7, 3.5) points greater WAI scores than did those in the below-average group. No significant associations were found between work ability and 3-min step, chair stand, or sit-and-reach test scores. Conclusion The overall work ability of the participants was good. Body composition, muscle strength, and balance were associated with work ability. Promoting physical fitness is a viable strategy for enhancing work ability among aging workers.