Abstract: To improve the recognition accuracy of weather images and achieve a good weather image classification result, a weather image recognition algorithm based on transfer learning is studied. The network architecture is implemented by using the Xception image classification algorithm. And compared the performance of other models on the same data set, the experimental results show that the improved Xception model based on transfer learning effectively solves the problem of insufficient training samples and low accuracy and has achieved good results in improving weather image recognition, and this model can achieve the classification of the six weather conditions of cloudy days, foggy days, rain days, dust days,snow days, sunny days, and the total recognition accuracy rate can reache 94.39%.