Abstract: To enhance the fast response and safe passage ability of self-driving vehicles in urban congested intersections, we proposed a multi-agent reinforcement learning vehicle cooperative control strategy based on the MAPPO-RCNN algorithm. The MAPPO algorithm was used to achieve cooperative control between vehicles using raw, unprocessed RGB images collected by vehicle sensors as input. It outputs vehicle actions, which considering the influence of mutual positions of vehicles on the traffic flow of the passing task to optimize vehicle passing time and safety. We designed the generation algorithm for strategy and the optimization objective function. To prevent the strategy from falling into the local optimum, we used the Nash equilibrium to judge the strategy's convergence. We employed the CARLA simulation platform for experimental simulation. Experimental results demonstrate that the vehicle cooperative control strategy moderately enhances the traffic flow effect of self-driving vehicles at intersections and ensures the stability of the whole control system.