Complex-Valued Representation for RGB-D Object Recognition

Object recognition methods usually tend to focus on single cues coming from traditional vision based systems but ignore to incorporate multi-modal data. With the advent of depth RGB-D sensors which provide synchronized multi-modal data with good quality, new opportunities have been emerged. In this paper, we make use of RGB and depth images to propose a new object recognition approach. Using a pixel-wise scheme, we propose a novel method to describe RGB-D images with a complex-valued representation. By means of neural network, we introduce a new CVNN (Complex-Valued Neural Network) with RBF neurons. Different from many RGB-D features, the proposed approach is able to jointly use RGB and depth data within a unified end-to-end learning framework. Category and instance object recognition tasks are evaluated through experiments carried out on a large scale RGB-D object dataset. Results show that our method can efficiently recognize objects in RGB-D images and outperforms state-of-the-art approaches.

[1]  Simone G. O. Fiori.  Learning by Criterion Optimization on a Unitary Unimodular Matrix Group , 2008, Int. J. Neural Syst..

[2]  Simone G. O. Fiori,et al.  Nonlinear Complex-Valued Extensions of Hebbian Learning: An Essay , 2005, Neural Computation.

[3]  Kazuyuki Murase,et al.  Single-layered complex-valued neural network for real-valued classification problems , 2009, Neurocomputing.

[4]  N. Sundararajan,et al.  A fully complex-valued radial basis function network and its learning algorithm. , 2009 .

[5]  Dieter Fox,et al.  Depth kernel descriptors for object recognition , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[6]  Dieter Fox,et al.  Unsupervised Feature Learning for RGB-D Based Object Recognition , 2012, ISER.

[7]  Akira Hirose,et al.  Complex-Valued Neural Networks: Theories and Applications , 2003 .

[8]  John K. Tsotsos,et al.  50 Years of object recognition: Directions forward , 2013, Comput. Vis. Image Underst..

[9]  Akira Hirose,et al.  Complex-Valued Neural Networks , 2006, Studies in Computational Intelligence.

[10]  Dieter Fox,et al.  A Scalable Tree-Based Approach for Joint Object and Pose Recognition , 2011, AAAI.

[11]  Tülay Adali,et al.  Fully Complex Multi-Layer Perceptron Network for Nonlinear Signal Processing , 2002, J. VLSI Signal Process..

[12]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Dieter Fox,et al.  Hierarchical Matching Pursuit for Image Classification: Architecture and Fast Algorithms , 2011, NIPS.

[14]  Akira Hirose,et al.  Dynamics of fully complex-valued neural networks , 1992 .

[15]  Wolfram Burgard,et al.  Multimodal deep learning for robust RGB-D object recognition , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Narasimhan Sundararajan,et al.  A fully complex-valued radial basis function classifier for real-valued classification problems , 2012, Neurocomputing.

[17]  Silvio Savarese,et al.  Robust single-view instance recognition , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Akira Hirose,et al.  Continuous complex-valued back-propagation learning , 1992 .

[19]  Jitendra Malik,et al.  Learning Rich Features from RGB-D Images for Object Detection and Segmentation , 2014, ECCV.

[20]  Rong Jin,et al.  Multiple Kernel Learning for Visual Object Recognition: A Review , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Xiao Li,et al.  Learning Coupled Classifiers with RGB images for RGB-D object recognition , 2017, Pattern Recognit..