Improved Deep Neural Network Object Tracking System for Applications in Home Robotics

Robotic navigation in GPS-denied environments requires case specific approaches for controlling a mobile robot to any desired destinations. In general, a nominal path is created in an environment described by a set of distinct objects, in other words such obstacles and landmarks. Intelligent voice assistants or digital assistance devices are increasing their importance in today’s smart home. Especially, by the help of fast-growing Internet of Things (IoT) applications. These devices are amassing an ever-growing list of features such as controlling states of connected smart devices, recording tasks, and responding to queries. Assistive robots are the perfect complement to smart voice assistants for providing physical manipulation. A request made by a person can be assigned to the assistive robot by the voice assistant. In this chapter, a new approach for autonomous navigation is presented using pattern recognition and machine learning techniques such as Convolutional Neural Networks to identify markers or objects from images and videos. Computational intelligence techniques are implemented along with Robot Operating System and object positioning to navigate towards these objects and markers by using RGB-depth camera. Multiple potential matching objects detected by the robot with deep neural network object detectors will be displayed on a screen installed on the assistive robot to improve and evaluate Human-Robot Interaction (HRI).

[1]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[2]  Patrick Benavidez,et al.  Mobile robot navigation and target tracking system , 2011, 2011 6th International Conference on System of Systems Engineering.

[3]  Berat A. Erol,et al.  Cloud-based control and vSLAM through cooperative Mapping and Localization , 2016, 2016 World Automation Congress (WAC).

[4]  Paul Rad,et al.  Deep learning control for complex and large scale cloud systems , 2017, Intell. Autom. Soft Comput..

[5]  Rui Caseiro,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence High-speed Tracking with Kernelized Correlation Filters , 2022 .

[6]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[7]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[8]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[9]  Roman P. Pflugfelder,et al.  Clustering of static-adaptive correspondences for deformable object tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Shai Avidan,et al.  Extended Lucas-Kanade Tracking , 2014, ECCV.

[11]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Daniel Pack,et al.  A Reactive Bearing Angle Only Obstacle Avoidance Technique for Unmanned Ground Vehicles , 2014 .

[13]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[14]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[15]  Yi Wu,et al.  Online Object Tracking: A Benchmark , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[17]  Mohan Kumar,et al.  Design of a home multi-robot system for the elderly and disabled , 2015, 2015 10th System of Systems Engineering Conference (SoSE).

[18]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Berat A. Erol,et al.  A deep vision landmark framework for robot navigation , 2017, 2017 12th System of Systems Engineering Conference (SoSE).

[20]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[22]  Sotiris B. Kotsiantis,et al.  Machine learning: a review of classification and combining techniques , 2006, Artificial Intelligence Review.

[23]  Abhineet Singh,et al.  Modular Tracking Framework: A Unified Approach to Registration based Tracking , 2016, ArXiv.

[24]  Rynson W. H. Lau,et al.  Visual Tracking via Locality Sensitive Histograms , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  John F. Kolen,et al.  Reducing the time complexity of the fuzzy c-means algorithm , 2002, IEEE Trans. Fuzzy Syst..

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  David Zhang,et al.  Fast Visual Tracking via Dense Spatio-temporal Context Learning , 2014, ECCV.

[28]  Zdenek Kalal,et al.  Tracking-Learning-Detection , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[30]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Xiao-Jun Zeng,et al.  Fuzzy C-means++: Fuzzy C-means with effective seeding initialization , 2015, Expert Syst. Appl..

[32]  Patrick Benavidez,et al.  Cloud-based realtime robotic Visual SLAM , 2015, 2015 Annual IEEE Systems Conference (SysCon) Proceedings.

[33]  Berat A. Erol,et al.  Proposed testbed for the modeling and control of a system of autonomous vehicles , 2016, 2016 11th System of Systems Engineering Conference (SoSE).

[34]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[35]  Paul Rad,et al.  Cloud robotics: A software architecture: For heterogeneous large-scale autonomous robots , 2016, 2016 World Automation Congress (WAC).

[36]  Zhongli Wang,et al.  A survey on multiple object tracking algorithm , 2016, 2016 IEEE International Conference on Information and Automation (ICIA).

[37]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Mohan Kumar,et al.  Software interface design for home-based assistive multi-robot system , 2015, 2015 10th System of Systems Engineering Conference (SoSE).

[40]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[41]  Kai Xie,et al.  Kernel Regression Based Online Boosting Tracking , 2015, J. Inf. Sci. Eng..