To improve the performance and applicability of license plate recognition, a license plate recognition method based on foreground polarity detection and improved convolutional neural network (CNN) is proposed. The proposed method consists of two main modules: character segmentation module and character recognition module. In the character segmentation module, foreground polarity detection based on RGB color is used for binarization and ROI segmentation, and then character height estimation and skew correction are performed. In the character recognition module, the depth features are extracted by the multi-channel deep CNN framework including the aggregation module to improve the representation ability of the output features. The experimental results show that the proposed method has good recognition accuracy, and the recognition rate is 92.2% and 94.1% respectively on the more difficult SSIG test set and aolp data set, and it is superior to the commercial vehicle access control system in some extreme cases.