The significance of lifelong learning is increasing in Japan’s aging society. The purpose of this study is to clarify the factors that affect the participation in lifelong learning of the elderly people in relation to subjective well-being, physical activity/exercise, and social activity. We conducted a questionnaire survey among lifelong learning participants. The contents of the questionnaire items were scored and the influence of the number of years of participation in the lifetime learning was examined using confirmatory factor analysis. There were 207 participants with an average age of 73.2 years old. In terms of health and life, most of the participants answered positively. In terms of coping with stress, many respondents had that coping strategy. Their nutritional intake and exercise habits were good. They had a high rate of health check-ups, were highly interested in disease prevention and paid attention to their health management. From the exploratory factor analysis of the questionnaire results, the five common factors behind lifelong learning participants were “subjective well-being,” “physical activity/exercise,” “subjective view of health,” “nutrition/diet,” and “social activity.” There was a negative relationship between “years of participation in lifelong learning” and “subjective well-being,” and a positive relationship between “physical activity/exercise” and “social activity” of these latent variables. In conclusion, many of the lifelong learning participants were highly conscious of their health, had acquired appropriate habits in terms of exercise and nutrition, and had a high subjective well-being. The elderly’s continuous participation in lifelong learning is beneficial for “physical activity/exercise” and “social activity.”