Previous studies among middle-aged and older adults have shown that engagement in exercise with others is more strongly associated with better mental health than engagement in exercise alone. However, the applicability of such findings to workers remains unclear. This study aimed to examine whether 1) engagement in exercise with others and time spent exercising with others were associated with lower stress response and mental distress among workers, and 2) self-determined motivation toward exercise mediated these associations among workers. This was a cross-sectional study. A web-based questionnaire survey was conducted among 810 workers aged 20 to 59 years. The survey measured respondents’ engagement and time spent exercising alone and with others, self-determined motivation toward exercise, psychological and physical stress responses, mental distress, and basic factors. Basic factors were treated as covariates. The analyses of covariance showed that engagement in exercise with others was significantly associated with lower psychological and physical stress responses and mental distress, while engagement in exercise alone was not. Multiple regression analyses revealed that longer time spent exercising with others was not associated with lower psychological and physical stress responses or mental distress. Path analyses showed that mediation effect of self-determined motivation on these associations was not significant. Although dose-response associations and the mediating role of self-determined motivation were not confirmed, this study found that engagement in exercise with others was associated with lower stress responses and mental distress among workers.