Based on the data of China Longitudinal Healthy Longevity Survey (CLHLS) in 2018, the influence of social participation on the health of the elderly and its urban-rural differences were comprehensively analyzed by logistic regression model and propensity score matching method. The study shows that social participation can improve the physical and mental health of the elderly. From the perspective of urban-rural differences, social participation has a greater effect on the self-rated health and ability of daily living (ADL) but a weaker effect on instrumental activities of daily living (IADL) in urban older people than in rural older people. In terms of mental health, social participation plays a more important role in reducing depression in urban elderly people and has a more significant impact on cognitive functioning in rural elderly people. On the basis of encouraging the elderly to actively participate in social participation, we should focus on the differences between urban and rural social environment and strive to achieve the common development of urban and rural areas. On the one hand, to improve the rural social security system and strengthen the construction of public infrastructure, and to provide policy guarantee and material basis for the social participation of the rural elderly; on the other hand, we should improve the urban environment and build a good social environment and "semi-acquaintance" community for the elderly in urban areas based on the community.