Benchmark Dataset Based on Category Maps with Indoor-Outdoor Mixed Features for Positional Scene Recognition by a Mobile Robot

This study was conducted to develop original benchmark datasets that simultaneously include indoor–outdoor visual features. Indoor visual information related to images includes outdoor features to a degree that varies extremely by time, weather, and season. We obtained time-series scene images using a wide field of view (FOV) camera mounted on a mobile robot moving along a 392-m route in an indoor environment surrounded by transparent glass walls and windows for two directions in three seasons. For this study, we propose a unified method for extracting, characterizing, and recognizing visual landmarks that are robust to human occlusion in a real environment in which robots coexist with people. Using our method, we conducted an evaluation experiment to recognize scenes divided up to 64 zones with fixed intervals. The experimentally obtained results using the datasets revealed the performance and characteristics of meta-parameter optimization, mapping characteristics to category maps, and recognition accuracy. Moreover, we visualized similarities between scene images using category maps. We also identified cluster boundaries obtained from mapping weights.

[1]  Martin Glavin,et al.  Overcoming Occlusion in the Automotive Environment—A Review , 2021, IEEE Transactions on Intelligent Transportation Systems.

[2]  Allan W. Snyder,et al.  Information capacity of eyes , 1977, Vision Research.

[3]  Giancarmine Fasano,et al.  Differential GNSS and Vision-Based Tracking to Improve Navigation Performance in Cooperative Multi-UAV Systems , 2016, Sensors.

[4]  Gerhard P. Hancke,et al.  A Review on Challenges of Autonomous Mobile Robot and Sensor Fusion Methods , 2020, IEEE Access.

[5]  Luo,et al.  The Application of Improved YOLO V3 in Multi-Scale Target Detection , 2019, Applied Sciences.

[6]  Tiziana Catarci,et al.  Who is Willing to Help Robots? A User Study on Collaboration Attitude , 2019, International Journal of Social Robotics.

[7]  Jean-Claude Latombe,et al.  Landmark-Based Robot Navigation , 1992, Algorithmica.

[8]  Ou Ma,et al.  Survey on artificial intelligence based techniques for emerging robotic communication , 2019, Telecommun. Syst..

[9]  Yong Wang,et al.  VisDrone-MOT2019: The Vision Meets Drone Multiple Object Tracking Challenge Results , 2019, 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW).

[10]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Hirokazu Madokoro,et al.  Indoor Scene and Position Recognition Based on Visual Landmarks Obtained from Visual Saliency without Human Effect , 2019, Robotics.

[12]  Hirokazu Madokoro,et al.  Vision-Based Indoor Scene Recognition from Time-Series Aerial Images Obtained Using a MAV Mounted Monocular Camera , 2019, Drones.

[13]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[14]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[15]  Hendry,et al.  Automatic License Plate Recognition via sliding-window darknet-YOLO deep learning , 2019, Image Vis. Comput..

[16]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[17]  Xu Chen,et al.  In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning , 2018, IEEE Network.

[18]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[19]  Dan Yang,et al.  RiSH: A robot-integrated smart home for elderly care , 2018, Robotics Auton. Syst..

[20]  John J. Leonard,et al.  Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age , 2016, IEEE Transactions on Robotics.

[21]  Max A. Viergever,et al.  Efficient and reliable schemes for nonlinear diffusion filtering , 1998, IEEE Trans. Image Process..

[22]  Daxin Tian,et al.  An Automatic Car Accident Detection Method Based on Cooperative Vehicle Infrastructure Systems , 2019, IEEE Access.

[23]  Hema Swetha Koppula,et al.  Anticipating Human Activities Using Object Affordances for Reactive Robotic Response , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Yo-Ping Huang,et al.  Structure From Motion Technique for Scene Detection Using Autonomous Drone Navigation , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.