A Doorway Detection and Direction (3Ds) System for Social Robots via a Monocular Camera

In this paper, we propose a novel algorithm to detect a door and its orientation in indoor settings from the view of a social robot equipped with only a monocular camera. The challenge is to achieve this goal with only a 2D image from a monocular camera. The proposed system is designed through the integration of several modules, each of which serves a special purpose. The detection of the door is addressed by training a convolutional neural network (CNN) model on a new dataset for Social Robot Indoor Navigation (SRIN). The direction of the door (from the robot’s observation) is achieved by three other modules: Depth module, Pixel-Selection module, and Pixel2Angle module, respectively. We include simulation results and real-time experiments to demonstrate the performance of the algorithm. The outcome of this study could be beneficial in any robotic navigation system for indoor environments.

[1]  Matthew Derry,et al.  Automated doorway detection for assistive shared-control wheelchairs , 2013, 2013 IEEE International Conference on Robotics and Automation.

[2]  Simon Fuhrmann,et al.  MVE - An image-based reconstruction environment , 2015, Comput. Graph..

[3]  Jie Chen,et al.  Autonomous indoor exploration of mobile robots based on door-guidance and improved dynamic window approach , 2009, 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[4]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Xiang Gao,et al.  Detecting, locating and crossing a door for a wide indoor surveillance robot , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[6]  Xiaodong Yang,et al.  Computer Vision-Based Door Detection for Accessibility of Unfamiliar Environments to Blind Persons , 2010, ICCHP.

[7]  Gang Wang,et al.  Door recognition and deep learning algorithm for visual based robot navigation , 2014, 2014 IEEE International Conference on Robotics and Biomimetics (ROBIO 2014).

[8]  Ashutosh Saxena,et al.  3-D Depth Reconstruction from a Single Still Image , 2007, International Journal of Computer Vision.

[9]  Jianliang Wang,et al.  A framework based on deep learning and mathematical morphology for cabin door detection in an automated aerobridge docking system , 2017, 2017 11th Asian Control Conference (ASCC).

[10]  Derek Hoiem,et al.  Indoor Segmentation and Support Inference from RGBD Images , 2012, ECCV.

[11]  Senem Velipasalar,et al.  Doorway detection for autonomous indoor navigation of unmanned vehicles , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[12]  Andreas Geiger,et al.  Vision meets robotics: The KITTI dataset , 2013, Int. J. Robotics Res..

[13]  Ian D. Reid,et al.  Single View Metrology , 2000, International Journal of Computer Vision.

[14]  Oliver Bittel,et al.  Real-Time Door Detection Based on AdaBoost Learning Algorithm , 2009, Eurobot Conference.

[15]  Matthew Derry,et al.  Using machine learning to blend human and robot controls for assisted wheelchair navigation , 2013, 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR).

[16]  Wan Kyun Chung,et al.  Door Detection Algorithm of Mobile Robot in Hallway Using PC-Camera , 2004 .

[17]  Ahmad B. Rad,et al.  An Indoor Room Classification System for Social Robots via Integration of CNN and ECOC , 2019, Applied Sciences.

[18]  Jörg Stückler,et al.  Improving indoor navigation of autonomous robots by an explicit representation of doors , 2010, 2010 IEEE International Conference on Robotics and Automation.

[19]  Santanu Chaudhury,et al.  Perceptual Depth Preserving Saliency based Image Compression , 2015, PerMIn '15.

[20]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[21]  François Chaumette,et al.  A visual servoing approach for autonomous corridor following and doorway passing in a wheelchair , 2016, Robotics Auton. Syst..

[22]  Chunping Hou,et al.  Depth recovery and refinement from a single image using defocus cues , 2015 .

[23]  Kurt Konolige,et al.  Autonomous door opening and plugging in with a personal robot , 2010, 2010 IEEE International Conference on Robotics and Automation.

[24]  Tao Yu,et al.  3D Reconstruction from a Single Still Image Based on Monocular Vision of an Uncalibrated Camera , 2017 .

[25]  Simon Fuhrmann,et al.  MVE - A Multi-View Reconstruction Environment , 2014, GCH.

[26]  V Aslantas.  A depth estimation algorithm with a single image. , 2007, Optics express.

[27]  Alistair A. Young,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2017, MICCAI 2017.

[28]  Sebastian Thrun,et al.  Detecting and modeling doors with mobile robots , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[29]  Peter Wonka,et al.  High Quality Monocular Depth Estimation via Transfer Learning , 2018, ArXiv.

[30]  Belén Curto,et al.  A Real-time Door Detection System for Domestic Robotic Navigation , 2014, J. Intell. Robotic Syst..