A Novel OCR-RCNN for Elevator Button Recognition

Autonomous elevator operation is considered an intelligent solution in handling the inter-floor navigation problem of service robots. As one of the most fundamental steps, elevator button recognition starts to receive more and more attention. However, due to the challenging image conditions and severe class imbalance problem, the performance of existing results is unsatisfying. In this paper, we propose to combine an optical character recognition (OCR) network and the Faster RCNN architecture into a single neural network, called OCR-RCNN to facilitate an end-to-end training and elevator button recognition procedure. To verify our method, we collect a large dataset of elevator panels and carry out extensive comparative experiments. The experiment results show that our method can greatly outperform the traditional recognition pipelines, yielding an accurate and robust performance on recognizing untrained elevator buttons.

[1]  Dae-Jin Kim,et al.  Robust Elevator Button Recognition in the Presence of Partial Occlusion and Clutter by Specular Reflections , 2012, IEEE Transactions on Industrial Electronics.

[2]  Mohd Razali Daud,et al.  Elevator‘s External Button Recognition and Detection for Vision-based System , 2014 .

[3]  Max Q.-H. Meng,et al.  An autonomous elevator button recognition system based on convolutional neural networks , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[4]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[5]  Andrew Y. Ng,et al.  Autonomous operation of novel elevators for robot navigation , 2010, 2010 IEEE International Conference on Robotics and Automation.

[6]  Max Q.-H. Meng,et al.  Deep Reinforcement Learning Supervised Autonomous Exploration in Office Environments , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[9]  Norbert Stoll,et al.  A robust method for elevator operation in semi-outdoor environment for mobile robot transportation system in life science laboratories , 2016, 2016 IEEE 20th Jubilee International Conference on Intelligent Engineering Systems (INES).

[10]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Geoffrey E. Hinton,et al.  Grammar as a Foreign Language , 2014, NIPS.

[12]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[13]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[14]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[15]  Kevin Murphy,et al.  Attention-Based Extraction of Structured Information from Street View Imagery , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[16]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[17]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Surya Ganguli,et al.  Exact solutions to the nonlinear dynamics of learning in deep linear neural networks , 2013, ICLR.

[20]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Ram Gopal Raj,et al.  Elevator button and floor number recognition through hybrid image classification approach for navigation of service robot in buildings , 2017, 2017 International Conference on Engineering Technology and Technopreneurship (ICE2T).