Development of a Highly Precise Place Recognition Module for Effective Human-robot Interactions in Changing Lighting and Viewpoint Conditions

We present a highly precise and robust module for indoor place recognition, extending the work by Lemaignan et al. and Robert Jr. by giving the robot the ability to recognize its environment context. We developed a full end-to-end convolutional neural network architecture, using a pre-trained deep convolutional neural network and the explicit inductive bias transfer learning strategy. Experimental results based on the York University and Rzeszów University dataset show excellent performance values (over 94.75 and 97.95 percent accuracy) and a high level of robustness over changes in camera viewpoint and lighting conditions, outperforming current benchmarks. Furthermore, our architecture is 82.46 percent smaller than the current benchmark, making our module suitable for embedding into mobile robots and easily adoptable to other datasets without the need for heavy adjustments.

[1]  Ricardo Buettner,et al.  Microsaccades as a Predictor of a User’s Level of Concentration , 2018, Information Systems and Neuroscience.

[2]  Ji Zhang,et al.  Visual-lidar odometry and mapping: low-drift, robust, and fast , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Ricardo Buettner,et al.  Investigation of the Relationship Between Visual Website Complexity and Users’ Mental Workload: A NeuroIS Perspective , 2015 .

[4]  Jürgen Landes,et al.  Argumentation-Based Negotiation? Negotiation-Based Argumentation! , 2012, EC-Web.

[5]  Conny H. Antoni,et al.  Analyzing the Effects of Role Configuration in Logistics Processes using Multiagent-Based Simulation: An Interdisciplinary Approach , 2019, HICSS.

[6]  Ricardo Buettner,et al.  A Highly Effective Deep Learning Based Escape Route Recognition Module for Autonomous Robots in Crisis and Emergency Situations , 2019, HICSS.

[7]  Ricardo Buettner Getting a job via career-oriented social networking markets , 2017, Electron. Mark..

[8]  N. Kohls,et al.  Mindful in a random forest: Assessing the validity of mindfulness items using random forests methods , 2015 .

[9]  Vineet R. Kamat,et al.  Enhancing Perceived Safety in Human–Robot Collaborative Construction Using Immersive Virtual Environments , 2018, Automation in Construction.

[10]  Barbara Caputo,et al.  Visual Servoing to Help Camera Operators Track Better , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  Jürgen Landes,et al.  Web Service-based Applications for Electronic Labor Markets: A Multi-dimensional Price VCG Auction with Individual Utilities , 2012, ICIW 2012.

[12]  Pietro Perona,et al.  A Bayesian hierarchical model for learning natural scene categories , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  Ricardo Buettner,et al.  User Acceptance in Different Electronic Negotiation Systems - A Comparative Approach , 2013, 2013 IEEE 10th International Conference on e-Business Engineering.

[14]  Sangseok You,et al.  Trusting Robots in Teams: Examining the Impacts of Trusting Robots on Team Performance and Satisfaction , 2019, HICSS.

[15]  Teng Ye,et al.  Team Potency and Ethnic Diversity in Embodied Physical Action (EPA) Robot- Supported Dyadic Teams , 2017, ICIS.

[16]  Ricardo Buettner,et al.  Machine Learning Based Diagnosis of Diseases Using the Unfolded EEG Spectra: Towards an Intelligent Software Sensor , 2019, Information Systems and Neuroscience.

[17]  Lionel P. Robert,et al.  Emotional Attachment, Performance, and Viability in Teams Collaborating with Embodied Physical Action (EPA) Robots , 2018, J. Assoc. Inf. Syst..

[18]  Christian Maier,et al.  Real-time Prediction of User Performance based on Pupillary Assessment via Eye Tracking , 2018, AIS Trans. Hum. Comput. Interact..

[19]  Lionel P. Robert,et al.  Human–Robot Similarity and Willingness to Work with a Robotic Co-worker , 2018, 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[20]  John K. Tsotsos,et al.  Histogram of Oriented Uniform Patterns for robust place recognition and categorization , 2012, Int. J. Robotics Res..

[21]  Ricardo Buettner,et al.  Cognitive Workload of Humans Using Artificial Intelligence Systems: Towards Objective Measurement Applying Eye-Tracking Technology , 2013, KI.

[22]  Ricardo Buettner,et al.  Asking both the User's Brain and its Owner using Subjective and Objective Psychophysiological NeuroIS Instruments , 2017, ICIS.

[23]  Ingo J. Timm,et al.  Stationarity of a User’s Pupil Size Signal as a Precondition of Pupillary-Based Mental Workload Evaluation , 2018 .

[24]  Taghi M. Khoshgoftaar,et al.  A survey of transfer learning , 2016, Journal of Big Data.

[25]  Björn Niehaves,et al.  Standing on the Shoulders of Giants: Challenges and Recommendations of Literature Search in Information Systems Research , 2015, Commun. Assoc. Inf. Syst..

[26]  Björn Niehaves,et al.  Reconstructing the giant: On the importance of rigour in documenting the literature search process , 2009, ECIS.

[27]  T. Heidenreich,et al.  Mindful Machine Learning: Using Machine Learning Algorithms to Predict the Practice of Mindfulness , 2018 .

[28]  Ricardo Buettner,et al.  The Relationship Between Visual Website Complexity and a User’s Mental Workload: A NeuroIS Perspective , 2017 .

[29]  Ricardo Buettner,et al.  Analyzing the Problem of Employee Internal Social Network Site Avoidance: Are Users Resistant due to Their Privacy Concerns? , 2015, 2015 48th Hawaii International Conference on System Sciences.

[30]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[31]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[32]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[33]  Ingo J. Timm,et al.  High-performance exclusion of schizophrenia using a novel machine learning method on EEG data , 2019, 2019 IEEE International Conference on E-health Networking, Application & Services (HealthCom).

[34]  Jian Cheng,et al.  Quantized Convolutional Neural Networks for Mobile Devices , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[36]  Séverin Lemaignan,et al.  Artificial cognition for social human-robot interaction: An implementation , 2017, Artif. Intell..

[37]  Robert Harle,et al.  Pedestrian localisation for indoor environments , 2008, UbiComp.

[38]  Krista A. Ehinger,et al.  SUN Database: Exploring a Large Collection of Scene Categories , 2014, International Journal of Computer Vision.

[39]  Christian Maier,et al.  Towards Ex Ante Prediction of User Performance: A Novel NeuroIS Methodology Based on Real-Time Measurement of Mental Effort , 2015, 2015 48th Hawaii International Conference on System Sciences.

[40]  Ricardo Buettner,et al.  The State of the Art in Automated Negotiation Models of the Behavior and Information Perspective , 2006, Int. Trans. Syst. Sci. Appl..

[41]  John K. Tsotsos,et al.  Indoor Place Recognition System for Localization of Mobile Robots , 2016, 2016 13th Conference on Computer and Robot Vision (CRV).

[42]  Lionel Robert,et al.  Personality in the Human Robot Interaction Literature: A Review and Brief Critique , 2018, AMCIS.

[43]  Barbara Caputo,et al.  Learning Deep NBNN Representations for Robust Place Categorization , 2017, IEEE Robotics and Automation Letters.

[44]  Ricardo Buettner,et al.  Towards high-performance differentiation between Narcolepsy and Idiopathic Hypersomnia in 10 minute EEG recordings using a Novel Machine Learning Approach , 2019, 2019 IEEE International Conference on E-health Networking, Application & Services (HealthCom).

[45]  Xuhong Li,et al.  Explicit Inductive Bias for Transfer Learning with Convolutional Networks , 2018, ICML.

[46]  Solvi Arnold,et al.  Real-time scene parsing by means of a convolutional neural network for mobile robots in disaster scenarios , 2017, 2017 IEEE International Conference on Information and Automation (ICIA).

[47]  Ricardo Buettner,et al.  Getting a Job via Career-Oriented Social Networking Sites: The Weakness of Ties , 2016, 2016 49th Hawaii International Conference on System Sciences (HICSS).

[48]  Ricardo Buettner,et al.  High-performance detection of epilepsy in seizure-free EEG recordings: A novel machine learning approach using very specific epileptic EEG sub-bands , 2019, ICIS.

[49]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[50]  Bolei Zhou,et al.  Places: A 10 Million Image Database for Scene Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Niko Sünderhauf,et al.  On the performance of ConvNet features for place recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[52]  Andreas Eckhardt,et al.  Cognitive Workload Induced by Information Systems: Introducing an Objective Way of Measuring based on Pupillary Diameter Responses , 2013 .

[53]  Stefan Kirn,et al.  Bargaining Power in Electronic Negotiations: A Bilateral Negotiation Mechanism , 2008, EC-Web.

[54]  Ricardo Buettner,et al.  A Classification Structure for Automated Negotiations , 2006, 2006 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology Workshops.

[55]  Jian Cheng,et al.  Quantized CNN: A Unified Approach to Accelerate and Compress Convolutional Networks , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[56]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[57]  Andrew Zisserman,et al.  Deep Fisher Networks for Large-Scale Image Classification , 2013, NIPS.

[58]  Luis Miguel Bergasa,et al.  Fusion and binarization of CNN features for robust topological localization across seasons , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[59]  Ricardo Buettner,et al.  Towards a New Personal Information Technology Acceptance Model: Conceptualization and Empirical Evidence from a Bring Your Own Device Dataset , 2015, AMCIS.

[60]  Ricardo Buettner,et al.  Asking Both the User’s Heart and Its Owner: Empirical Evidence for Substance Dualism , 2018, Information Systems and Neuroscience.

[61]  Ricardo Buettner,et al.  Development of a Machine Learning Based Algorithm To Accurately Detect Schizophrenia based on One-minute EEG Recordings , 2020, HICSS.

[62]  Ricardo Buettner,et al.  Cooperation in Hunting and Food-Sharing: A Two-Player Bio-inspired Trust Model , 2009, BIONETICS.

[63]  Ricardo Buettner,et al.  A Systematic Literature Review of Crowdsourcing Research from a Human Resource Management Perspective , 2015, 2015 48th Hawaii International Conference on System Sciences.

[64]  Ingo J. Timm,et al.  Colored micrographs significantly outperform grayscale ones in modern machine learning: Insights from a systematical analysis of lithium-ion battery micrographs using Convolutional Neural Networks , 2017 .

[65]  Eugenio Culurciello,et al.  Evaluation of neural network architectures for embedded systems , 2017, 2017 IEEE International Symposium on Circuits and Systems (ISCAS).

[66]  Christian Maier,et al.  Objective measures of IS usage behavior under conditions of experience and pressure using eye fixation data , 2013, ICIS.

[67]  Barbara Caputo,et al.  Incremental learning for place recognition in dynamic environments , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[68]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[69]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[70]  Ricardo Buettner,et al.  Trust as an Integral Part for Success of Cloud Computing , 2012, ICIW 2012.

[71]  Ricardo Buettner,et al.  Predicting user behavior in electronic markets based on personality-mining in large online social networks , 2017, Electron. Mark..

[72]  Ricardo Buettner,et al.  Social Inclusion in E-Participation and E-Government Solutions: A Systematic Laboratory-experimental Approach Using Objective Psychophysiological Measures , 2013, EGOV/ePart Ongoing Research.

[73]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[74]  Ricardo Buettner,et al.  High-performance Diagnosis of Sleep Disorders: A Novel, Accurate and Fast Machine Learning Approach Using Electroencephalographic Data , 2020, HICSS.

[75]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[76]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[77]  Kurt Konolige,et al.  Real-time Localization in Outdoor Environments using Stereo Vision and Inexpensive GPS , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[78]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[79]  Ricardo Buettner ELECTRONIC NEGOTIATIONS OF THE TRANSACTIONAL COSTS PERSPECTIVE , 2007 .

[80]  Yoshua Bengio,et al.  How transferable are features in deep neural networks? , 2014, NIPS.

[81]  Ricardo Buettner IMPERFECT INFORMATION IN ELECTRONIC NEGOTIATIONS : AN EMPIRICAL STUDY , 2008 .

[82]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[83]  Ricardo Buettner,et al.  High-performance detection of alcoholism by unfolding the amalgamated EEG spectra using the Random Forests method , 2019, Hawaii International Conference on System Sciences.

[84]  Antonio Torralba,et al.  Recognizing indoor scenes , 2009, CVPR.

[85]  Ricardo Buettner,et al.  Robust User Identification Based on Facial Action Units Unaffected by Users' Emotions , 2018, HICSS.

[86]  Ricardo Buettner,et al.  A User's Cognitive Workload Perspective in Negotiation Support Systems: an eye-tracking Experiment , 2016, PACIS.

[87]  Piotr Wozniak,et al.  Scene Recognition for Indoor Localization of Mobile Robots Using Deep CNN , 2018, ICCVG.

[88]  Juergen Landes,et al.  Job Allocation in a Temporary Employment Agency via Multi-dimensional Price VCG Auctions Using a Multi-agent System , 2011, 2011 10th Mexican International Conference on Artificial Intelligence.

[89]  Ricardo Buettner,et al.  Mental Workload States on the Basis of the Pupillary Hippus , 2022 .

[90]  Ricardo Buettner,et al.  Efficient machine learning based detection of heart disease , 2019, 2019 IEEE International Conference on E-health Networking, Application & Services (HealthCom).

[91]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[92]  Guillermo Glez-de-Rivera,et al.  Autonomous indoor ultrasonic positioning system based on a low-cost conditioning circuit , 2012 .

[93]  Christian Maier,et al.  The Influence of Pressure to Perform and Experience on Changing Perceptions and User Performance: A Multi-Method Experimental Analysis , 2012, ICIS.