Image-Based Automatic Watermeter Reading under Challenging Environments

With the rapid development of artificial intelligence and fifth-generation mobile network technologies, automatic instrument reading has become an increasingly important topic for intelligent sensors in smart cities. We propose a full pipeline to automatically read watermeters based on a single image, using deep learning methods to provide new technical support for an intelligent water meter reading. To handle the various challenging environments where watermeters reside, our pipeline disentangled the task into individual subtasks based on the structures of typical watermeters. These subtasks include component localization, orientation alignment, spatial layout guidance reading, and regression-based pointer reading. The devised algorithms for orientation alignment and spatial layout guidance are tailored to improve the robustness of our neural network. We also collect images of watermeters in real scenes and build a dataset for training and evaluation. Experimental results demonstrate the effectiveness of the proposed method even under challenging environments with varying lighting, occlusions, and different orientations. Thanks to the lightweight algorithms adopted in our pipeline, the system can be easily deployed and fully automated.

[1]  Jian Huang,et al.  Automatic Reading System for Analog Instruments Based on Computer Vision and Inspection Robot for Power Plant , 2018, 2018 10th International Conference on Modelling, Identification and Control (ICMIC).

[2]  Ni Gan Pointer Instrument Image Recognition Based on Priori Characteristics of Instrument Structure , 2013 .

[3]  Wenyu Liu,et al.  TextBoxes: A Fast Text Detector with a Single Deep Neural Network , 2016, AAAI.

[4]  Harris Drucker,et al.  Comparison of learning algorithms for handwritten digit recognition , 1995 .

[5]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[6]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Abhishek Dutta,et al.  The VGG Image Annotator (VIA) , 2019, ArXiv.

[8]  Xue Jun Li,et al.  Design and Implementation of a Self-Powered Smart Water Meter , 2019, Sensors.

[9]  L. Deng,et al.  The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web] , 2012, IEEE Signal Processing Magazine.

[10]  Fuqiang Zhou,et al.  FSSD: Feature Fusion Single Shot Multibox Detector , 2017, ArXiv.

[11]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[12]  Kaushik Deb,et al.  Digital electric meter reading recognition based on horizontal and vertical binary pattern , 2017, 2017 3rd International Conference on Electrical Information and Communication Technology (EICT).

[13]  Lawrence D. Jackel,et al.  Backpropagation Applied to Handwritten Zip Code Recognition , 1989, Neural Computation.

[14]  Ching Y. Suen,et al.  A novel hybrid CNN-SVM classifier for recognizing handwritten digits , 2012, Pattern Recognit..

[15]  Ping Fu,et al.  Development of an automatic reading method and software for pointer instruments , 2017, 2017 First International Conference on Electronics Instrumentation & Information Systems (EIIS).

[16]  Lamiaa A. Elrefaei,et al.  Automatic electricity meter reading based on image processing , 2015, 2015 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT).

[17]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[18]  Victor A. N. Barroso,et al.  Convolutional neural networks for automatic meter reading , 2019, J. Electronic Imaging.

[19]  Xin He,et al.  TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapes , 2018, ECCV.

[20]  Gregory Cohen,et al.  EMNIST: Extending MNIST to handwritten letters , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[21]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Dongyoon Han,et al.  Character Region Awareness for Text Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Yonatan Wexler,et al.  Detecting text in natural scenes with stroke width transform , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Xiang Bai,et al.  Mask TextSpotter: An End-to-End Trainable Neural Network for Spotting Text with Arbitrary Shapes , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Adnan M. Abu-Mahfouz,et al.  Smart water meter system for user-centric consumption measurement , 2015, 2015 IEEE 13th International Conference on Industrial Informatics (INDIN).

[27]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[28]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[29]  Huajun Feng,et al.  Libra R-CNN: Towards Balanced Learning for Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[31]  Isabelle Guyon,et al.  Comparison of classifier methods: a case study in handwritten digit recognition , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[32]  Lianwen Jin,et al.  Deep Matching Prior Network: Toward Tighter Multi-oriented Text Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[34]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[37]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[39]  Lv Ming,et al.  Reading recognition method of analog measuring instruments based on improved hough transform , 2011, IEEE 2011 10th International Conference on Electronic Measurement & Instruments.

[40]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[42]  Cephas Paul Edward Support Vector Machine based automatic electric meter reading system , 2013, 2013 IEEE International Conference on Computational Intelligence and Computing Research.

[43]  Peilin He,et al.  A robust approach to reading recognition of pointer meters based on improved mask-RCNN , 2020, Neurocomputing.

[44]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[46]  Xiaolin Li,et al.  Single Shot Text Detector with Regional Attention , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).