Reinforcement Learning for Computer Vision and Robot Navigation

Nowadays, machine learning has become one of the basic technologies used in solving various computer vision tasks such as feature detection, image segmentation, object recognition and tracking. In many applications, various complex systems such as robots are equipped with visual sensors from which they learn the state of a surrounding environment by solving corresponding computer vision tasks. Solutions of these tasks are used for making decisions about possible future actions. Reinforcement learning is one of the modern machine learning technologies in which learning is carried out through interaction with the environment. In recent years, reinforcement learning has been used both for solving robotic computer vision problems such as object detection, visual tracking and action recognition as well as robot navigation. The paper describes shortly the reinforcement learning technology and its use for computer vision and robot navigation problems.

[1]  Xin Wang,et al.  Deep Reinforcement Learning for Visual Object Tracking in Videos , 2017, ArXiv.

[2]  Michael I. Jordan,et al.  MASSACHUSETTS INSTITUTE OF TECHNOLOGY ARTIFICIAL INTELLIGENCE LABORATORY and CENTER FOR BIOLOGICAL AND COMPUTATIONAL LEARNING DEPARTMENT OF BRAIN AND COGNITIVE SCIENCES , 1996 .

[3]  Evgeny Burnaev,et al.  Adaptive Design of Experiments Based on Gaussian Processes , 2015, SLDS.

[4]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[6]  Andrew Howard,et al.  Design and use paradigms for Gazebo, an open-source multi-robot simulator , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[7]  Jürgen Schmidhuber,et al.  A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots , 2016, IEEE Robotics and Automation Letters.

[8]  R. Bellman.  Dynamic programming. , 1957, Science.

[9]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[10]  Li Fei-Fei,et al.  End-to-End Learning of Action Detection from Frame Glimpses in Videos , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yuxi Li,et al.  Deep Reinforcement Learning: An Overview , 2017, ArXiv.

[12]  Gregory R. Koch,et al.  Siamese Neural Networks for One-Shot Image Recognition , 2015 .

[13]  Ruslan Salakhutdinov,et al.  Active Neural Localization , 2018, ICLR.

[14]  Narciso García,et al.  Event-Based Vision Meets Deep Learning on Steering Prediction for Self-Driving Cars , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[16]  Evgeny Burnaev,et al.  Machine Learning in Appearance-Based Robot Self-Localization , 2017, 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA).

[17]  Ashish Kapoor,et al.  AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles , 2017, FSR.

[18]  Bruno Sudret,et al.  Efficient design of experiments for sensitivity analysis based on polynomial chaos expansions , 2017, Annals of Mathematics and Artificial Intelligence.

[19]  Evgeny Burnaev,et al.  Mobile Robot Localization via Machine Learning , 2017, MLDM.

[20]  Ali Farhadi,et al.  Target-driven visual navigation in indoor scenes using deep reinforcement learning , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[22]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[24]  David Hsu,et al.  QMDP-Net: Deep Learning for Planning under Partial Observability , 2017, NIPS.

[25]  Sen Wang,et al.  Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning , 2017, RSS 2017.

[26]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[27]  Evgeny Burnaev,et al.  The influence of parameter initialization on the training time and accuracy of a nonlinear regression model , 2016 .

[28]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[29]  Vijay Kumar,et al.  Memory Augmented Control Networks , 2017, ICLR.

[30]  David Silver,et al.  Deep Reinforcement Learning with Double Q-Learning , 2015, AAAI.

[31]  Razvan Pascanu,et al.  Learning to Navigate in Complex Environments , 2016, ICLR.

[32]  Xin Wang,et al.  Look Before You Leap: Bridging Model-Free and Model-Based Reinforcement Learning for Planned-Ahead Vision-and-Language Navigation , 2018, ECCV.

[33]  Tao Zhang,et al.  A Self-Adaptive Proposal Model for Temporal Action Detection based on Reinforcement Learning , 2017, ArXiv.

[34]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[35]  Evgeny V. Burnaev,et al.  On a method for constructing ensembles of regression models , 2013, Autom. Remote. Control..

[36]  Wolfram Burgard,et al.  VR-Goggles for Robots: Real-to-Sim Domain Adaptation for Visual Control , 2018, IEEE Robotics and Automation Letters.

[37]  Svetlana Lazebnik,et al.  Active Object Localization with Deep Reinforcement Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[38]  Sergey Levine,et al.  (CAD)$^2$RL: Real Single-Image Flight without a Single Real Image , 2016, Robotics: Science and Systems.

[39]  Thomas A. Funkhouser,et al.  Semantic Scene Completion from a Single Depth Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[41]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[42]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[43]  Alex Graves,et al.  Asynchronous Methods for Deep Reinforcement Learning , 2016, ICML.

[44]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[45]  Wojciech Jaskowski,et al.  ViZDoom: A Doom-based AI research platform for visual reinforcement learning , 2016, 2016 IEEE Conference on Computational Intelligence and Games (CIG).

[46]  Marcin Andrychowicz,et al.  Hindsight Experience Replay , 2017, NIPS.

[47]  Qi Wu,et al.  Vision-and-Language Navigation: Interpreting Visually-Grounded Navigation Instructions in Real Environments , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[48]  Ruslan Salakhutdinov,et al.  Neural Map: Structured Memory for Deep Reinforcement Learning , 2017, ICLR.

[49]  Wei Liu,et al.  End-to-end Active Object Tracking via Reinforcement Learning , 2017, ICML.

[50]  Yi Zhang,et al.  UnrealCV: Virtual Worlds for Computer Vision , 2017, ACM Multimedia.

[51]  Francisco Bonin-Font,et al.  Visual Navigation for Mobile Robots: A Survey , 2008, J. Intell. Robotic Syst..

[52]  Matthias Nießner,et al.  Matterport3D: Learning from RGB-D Data in Indoor Environments , 2017, 2017 International Conference on 3D Vision (3DV).

[53]  Cewu Lu,et al.  Virtual to Real Reinforcement Learning for Autonomous Driving , 2017, BMVC.