2D License Plate Recognition based on Automatic Perspective Rectification

License plate recognition (LPR) remains a challenging task in face of some difficulties such as image deformation and multi-line character distribution. Text rectification that is crucial to eliminate the effects of image deformation has attracted increasing attentions in scene text recognition. However, current text rectification methods are not designed specifically for LPR, which did not take the features of plate deformation into account. Considering the fact that a license plate (LP) can only generate perspective distortion in the image due to its rigid feature, in this paper we propose a novel perspective rectification network (PRN) to automatically estimate the perspective transformation and rectify the distorted LP accordingly. For recognition, we propose a location-aware 2D attention based recognition network that is capable of recognizing both single-line and double-line plates with perspective deformation. The rectification network and recognition network are connected for end-to-end training. Experiments on common datasets show that the proposed method achieves the state-of-the-art performance, demonstrating the effectiveness of the proposed approach.

[1]  Kaigui Bian,et al.  Symmetry-Constrained Rectification Network for Scene Text Recognition , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[2]  Zihan Zhou,et al.  Learning to Read Irregular Text with Attention Mechanisms , 2017, IJCAI.

[3]  Liusheng Huang,et al.  Towards End-to-End License Plate Detection and Recognition: A Large Dataset and Baseline , 2018, ECCV.

[4]  Fei Wu,et al.  Text Perceptron: Towards End-to-End Arbitrary-Shaped Text Spotting , 2020, AAAI.

[5]  Jing-Ming Guo,et al.  License Plate Localization and Character Segmentation with feedback self-learning and hybrid-binarization techniques , 2008, TENCON 2007 - 2007 IEEE Region 10 Conference.

[6]  Yuxin Zhang,et al.  A Robust Chinese License Plate Detection and Recognition Systemin Natural Scenes , 2019, 2019 IEEE 4th International Conference on Signal and Image Processing (ICSIP).

[7]  Adel M. Alimi,et al.  Deep Learning System for Automatic License Plate Detection and Recognition , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[8]  Wael Badawy,et al.  Automatic License Plate Recognition (ALPR): A State-of-the-Art Review , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[9]  Ganesh Ramakrishnan,et al.  OCR On-the-Go: Robust End-to-End Systems for Reading License Plates & Street Signs , 2019, 2019 International Conference on Document Analysis and Recognition (ICDAR).

[10]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[11]  Fred L. Bookstein,et al.  Principal Warps: Thin-Plate Splines and the Decomposition of Deformations , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Lianwen Jin,et al.  A Multi-Object Rectified Attention Network for Scene Text Recognition , 2019, Pattern Recognit..

[13]  Peng Wang,et al.  Show, Attend and Read: A Simple and Strong Baseline for Irregular Text Recognition , 2018, AAAI.

[14]  K. R. Ramakrishnan,et al.  Deep automatic license plate recognition system , 2016, ICVGIP '16.

[15]  Huiyun Mao,et al.  Robust License Plate Recognition With Shared Adversarial Training Network , 2020, IEEE Access.

[16]  Yong Haur Tay,et al.  Segmentation-free Vehicle License Plate Recognition using ConvNet-RNN , 2017, ArXiv.

[17]  Gee-Sern Hsu,et al.  Application-Oriented License Plate Recognition , 2013, IEEE Transactions on Vehicular Technology.

[18]  Chunhua Shen,et al.  Reading Car License Plates Using Deep Convolutional Neural Networks and LSTMs , 2016, ArXiv.

[19]  Orhan Bulan,et al.  Segmentation- and Annotation-Free License Plate Recognition With Deep Localization and Failure Identification , 2017, IEEE Transactions on Intelligent Transportation Systems.

[20]  Hui Li,et al.  Towards End-to-End Car License Plates Detection and Recognition with Deep Neural Networks , 2017 .

[21]  Dong Liu,et al.  A CNN-Based Approach for Automatic License Plate Recognition in the Wild , 2017, BMVC.

[22]  Alexey Gruzdev,et al.  LPRNet: License Plate Recognition via Deep Neural Networks , 2018, ArXiv.

[23]  Yanjie Yao,et al.  Vehicle License Plate Recognition Based on Extremal Regions and Restricted Boltzmann Machines , 2016, IEEE Transactions on Intelligent Transportation Systems.

[24]  Cláudio Rosito Jung,et al.  Real-Time Brazilian License Plate Detection and Recognition Using Deep Convolutional Neural Networks , 2017, 2017 30th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[25]  Xiang Bai,et al.  ASTER: An Attentional Scene Text Recognizer with Flexible Rectification , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[27]  Seok-Bum Ko,et al.  License plate segmentation and recognition system using deep learning and OpenVINO , 2020 .

[28]  Qingming Huang,et al.  A configurable method for multi-style license plate recognition , 2009, Pattern Recognit..

[29]  Mahmood Fathy,et al.  Ieee Transactions on Intelligent Transportation Systems 1 an Iranian License Plate Recognition System Based on Color Features , 2022 .

[30]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[31]  Jürgen Schmidhuber,et al.  Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks , 2006, ICML.

[32]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  William R. Schwartz,et al.  A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[34]  Yu Wang,et al.  License plate recognition based on SIFT feature , 2015 .

[35]  Andrew Zisserman,et al.  Spatial Transformer Networks , 2015, NIPS.

[36]  Kevin Murphy,et al.  Attention-Based Extraction of Structured Information from Street View Imagery , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[37]  Simon Osindero,et al.  Recursive Recurrent Nets with Attention Modeling for OCR in the Wild , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Yu Cao,et al.  An End-to-End Neural Network for Multi-line License Plate Recognition , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).