We aimed to evaluate the associations of individual-level social capital (SC) and physical activity (PA) among Japanese elderly men and women. Individuals aged 65 or above were selected via population-based random sampling of 3000 adults, aged 20 years or above and living in Fujisawa city, who were asked to complete self-administered questionnaires. The sample consisted of 260 men [median (25-75% tile) age 73 (68-78) yrs] and 274 women [73 (68-78) yrs]. The daily duration of PA, SC [e.g., norms of reciprocity (mutual support in the community), social networks (relationship with community people, exchange with other generations), social participation (participation in community organization activities)], socio-economic status [educational attainment, work with income, perceived household economic status], and demographic characteristics [sex, age, living arrangements, BMI, physical limitations] were assessed. Multivariate logistic regression analyses were used to calculate odds ratios (OR) for associations of SC with PA, stratified by sex. High level of mutual support in the community, relationship with community people, and exchange with other generations were associated with longer PA time in men after adjusting for potential confounders in the final model (OR = 2.40 (95% CI: 1.24-4.67), 5.28 (1.42-19.66), and 4.24 (1.15-15.67), respectively). In women, relationship with community people and participation in community organization activities were associated with longer PA time (3.10 (1.32-7.26) and 2.50 (1.27-4.91), respectively). The association of individual-level SC and PA differed between men and women. Future studies can investigate whether similar results can be obtained in other areas or attempt to conduct a large-scale survey.