A dual approach for machine-awareness in indoor environment combining pseudo-3D imaging and soft-computing techniques

Abstract3D spatial characterization of objects is a key-step for autonomous knowledge extraction leading to machine-awareness. In this paper we describe a dual-images-based approach for objects’ 3D characterization and localization for Machine-Awareness in indoor environment. The so-called dual images are provided by color and depth cameras of Kinect system, which presents an appealing potential for 3D objects modeling and localization. Placing the human–machine (including human-robot) interaction as a primary outcome of the intended visual Machine-Awareness in investigated system, we aspire proffering the machine spatial awareness about its surrounding environment with identification and semantically description of relative positions of detected items in that environment. In this research work, the aforementioned pseudo-3D imaging and Soft-computing techniques are combined in order to extract, to recognize and to spatially characterize objects and the distance between objects in 3D environment. In other words, on the one hand, a pseudo-3D object modeling and localization method based on the dual-images of Kinect system is proposed, in which the computation of object’s spatial characterization is presented. On the other hand, a computational semantically description algorithm by using Fuzzy Inference System is proposed, in which the numerical-to-semantic conversion for machine awareness is accomplished. Experimental results validating the investigated approach are reported and discussed.

[1]  Joachim Hertzberg,et al.  Towards semantic maps for mobile robots , 2008, Robotics Auton. Syst..

[2]  Pushmeet Kohli,et al.  When Can We Use KinectFusion for Ground Truth Acquisition , 2012 .

[3]  Scott McCloskey,et al.  Recognition of 3D package shapes for single camera metrology , 2014, IEEE Winter Conference on Applications of Computer Vision.

[4]  Wolfram Burgard,et al.  Robust place recognition for 3D range data based on point features , 2010, 2010 IEEE International Conference on Robotics and Automation.

[5]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[6]  Juho Kannala,et al.  Accurate and Practical Calibration of a Depth and Color Camera Pair , 2011, CAIP.

[7]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[8]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[9]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[10]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[11]  Fakhri Karray,et al.  Soft Computing and Tools of Intelligent Systems Design: Theory and Applications , 2004 .

[12]  Serge Guillaume,et al.  Designing fuzzy inference systems from data: An interpretability-oriented review , 2001, IEEE Trans. Fuzzy Syst..

[13]  Daniel O. Sales,et al.  Mobile Robots Navigation in Indoor Environments Using Kinect Sensor , 2012, 2012 Second Brazilian Conference on Critical Embedded Systems.

[14]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.

[15]  Vladimir G. Kim,et al.  Shape-based recognition of 3D point clouds in urban environments , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[17]  Javier Civera,et al.  Towards semantic SLAM using a monocular camera , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Marjorie Skubic,et al.  Fall Detection in Homes of Older Adults Using the Microsoft Kinect , 2015, IEEE Journal of Biomedical and Health Informatics.

[19]  George Baciu,et al.  Lightness biased cartoon-and-texture decomposition for textile image segmentation , 2015, Neurocomputing.

[20]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[21]  Marsette Vona,et al.  Moving Volume KinectFusion , 2012, BMVC.

[22]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[23]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[24]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[25]  Zhengyou Zhang,et al.  Calibration between depth and color sensors for commodity depth cameras , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[26]  Danica Kragic,et al.  Integrating Active Mobile Robot Object Recognition and SLAM in Natural Environments , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[28]  Cipriano Galindo,et al.  Multi-hierarchical semantic maps for mobile robotics , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[29]  Slawomir Zólkiewski,et al.  Robot Control and Online Programming by Human Gestures Using a Kinect Motion Sensor , 2014, WorldCIST.

[30]  Zhihui Lai,et al.  Manifold discriminant regression learning for image classification , 2015, Neurocomputing.

[31]  Silvio Savarese,et al.  Accurate Localization of 3D Objects from RGB-D Data Using Segmentation Hypotheses , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Tom Duckett,et al.  Probabilistic Semantic Mapping with a Virtual Sensor for Building/Nature detection , 2007, 2007 International Symposium on Computational Intelligence in Robotics and Automation.

[33]  Andrzej Skalski,et al.  Metrological Analysis Of Microsoft Kinect In The Context Of Object Localization , 2015 .

[34]  Roland Siegwart,et al.  Cognitive maps for mobile robots - an object based approach , 2007, Robotics Auton. Syst..

[35]  Pedro Arias,et al.  Application of Kinect Gaming Sensor in Forensic Science , 2015, Journal of forensic sciences.

[36]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[37]  Erik Maehle,et al.  Real-Time Visual SLAM Using FastSLAM and the Microsoft Kinect Camera , 2012, ROBOTIK.

[38]  Luis Salgado,et al.  Depth-Color Fusion Strategy for 3-D Scene Modeling With Kinect , 2013, IEEE Transactions on Cybernetics.

[39]  Vincent Lepetit,et al.  Dominant orientation templates for real-time detection of texture-less objects , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Xizhao Wang,et al.  Fuzziness based sample categorization for classifier performance improvement , 2015, J. Intell. Fuzzy Syst..

[41]  Ebrahim H. Mamdani,et al.  An Experiment in Linguistic Synthesis with a Fuzzy Logic Controller , 1999, Int. J. Hum. Comput. Stud..