Abstract: The convolutional neural network (CNN) model based on support vector machine (SVM) combines the margin maximization principle and achieves excellent generalization ability in image recognition applications. However, this method ignores a key fact that the generalization ability of SVM depends not only on the margin between the different categories, but also on the radius of the minimum enclosing ball which contains all the samples. Aiming at this problem, a CNN model driven by radius margin bound (RMB) is proposed to extract and identify the image features. Compared with the traditional CNN models, the proposed method not only considers the margin between different categories of the image features, but also further considers the radius of the minimum enclosing ball. In essence, the proposed CNN model adopts a strategy, which based on SVM generalization error bound, to guide the learning of the CNN model and the construction of the corresponding classifier. The model can improve the generalization ability of the deep convolution model without adding additional network complexity, and can also be applied to different depth models without being limited to a particular network structure. The experimental results on multiple datasets show that compared with the CNN model based on Sofmax loss, the CNN model based on center loss and the CNN model based on SVM loss, the proposed method can extract image features that have more discriminative power and further obtain higher recognition rate.