Human gesture recognition using three-dimensional integral imaging.

Three-dimensional (3D) integral imaging allows one to reconstruct a 3D scene, including range information, and provides sectional refocused imaging of 3D objects at different ranges. This paper explores the potential use of 3D passive sensing integral imaging for human gesture recognition tasks from sequences of reconstructed 3D video scenes. As a preliminary testbed, the 3D integral imaging sensing is implemented using an array of cameras with the appropriate algorithms for 3D scene reconstruction. Recognition experiments are performed by acquiring 3D video scenes of multiple hand gestures performed by ten people. We analyze the capability and performance of gesture recognition using 3D integral imaging representations at given distances and compare its performance with the use of standard two-dimensional (2D) single-camera videos. To the best of our knowledge, this is the first report on using 3D integral imaging for human gesture recognition.

[1]  Bahram Javidi,et al.  Distortion-tolerant 3D recognition of occluded objects using computational integral imaging. , 2006, Optics express.

[2]  Herbert E. Ives,et al.  Optical Properties of a Lippmann Lenticulated Sheet , 1931 .

[3]  Bahram Javidi,et al.  Three-Dimensional Image Sensing, Visualization, and Processing Using Integral Imaging , 2006, Proceedings of the IEEE.

[4]  Bahram Javidi,et al.  Three-dimensional object-distortion-tolerant recognition for integral imaging using independent component analysis. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Luc Van Gool,et al.  Action snippets: How many frames does human action recognition require? , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Charles Hansen,et al.  Rectification of images for binocular and trinocular stereovision , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[8]  Myungjin Cho,et al.  Three-Dimensional Optical Sensing and Visualization Using Integral Imaging , 2011, Proceedings of the IEEE.

[9]  F. Okano,et al.  Analysis of resolution limitation of integral photography , 1998 .

[10]  Wei Liu,et al.  Double Fusion for Multimedia Event Detection , 2012, MMM.

[11]  Xiaogang Wang,et al.  Joint Deep Learning for Pedestrian Detection , 2013, 2013 IEEE International Conference on Computer Vision.

[12]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[14]  Hong Wei,et al.  A survey of human motion analysis using depth imagery , 2013, Pattern Recognit. Lett..

[15]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[16]  Bahram Javidi,et al.  Progress in 3-D Multiperspective Display by Integral Imaging , 2009, Proceedings of the IEEE.

[17]  Cordelia Schmid,et al.  Evaluation of Local Spatio-temporal Features for Action Recognition , 2009, BMVC.

[18]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[19]  Bahram Javidi,et al.  Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging. , 2003, Optics express.

[20]  Bahram Javidi,et al.  Profilometry and optical slicing by passive three-dimensional imaging. , 2009, Optics letters.

[21]  Nobuhiko Hata,et al.  Surgical navigation by autostereoscopic image overlay of integral videography , 2004, IEEE Transactions on Information Technology in Biomedicine.

[22]  Mario Cannataro,et al.  Protein-to-protein interactions: Technologies, databases, and algorithms , 2010, CSUR.

[23]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  F. Okano,et al.  Gradient-index lens-array method based on real-time integral photography for three-dimensional images. , 1998, Applied optics.

[25]  Bahram Javidi,et al.  Generalization of three-dimensional N-ocular imaging systems under fixed resource constraints. , 2012, Optics letters.

[26]  Reinhard Koch,et al.  Automated reconstruction of 3D scenes from sequences of images , 2000 .

[27]  M McCormick,et al.  Discussion of the optics of a new 3-D imaging system. , 1988, Applied optics.

[28]  A. P. Sokolov,et al.  Autostereoscopy and integral photography by Professor Lippmann's method , 1911 .

[29]  Makoto Okui,et al.  Real-Time Integral Imaging Based on Extremely High Resolution Video System , 2006, Proceedings of the IEEE.

[30]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[31]  Samuel Greengard.  Computational photography comes into focus , 2014, CACM.

[32]  Bahram Javidi,et al.  A 3D integral imaging optical see-through head-mounted display. , 2014, Optics express.

[33]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[34]  C. Wheatstone.  XVIII. Contributions to the physiology of vision. —Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision , 1962, Philosophical Transactions of the Royal Society of London.

[35]  Larry S. Davis,et al.  Recognizing Human Actions by Learning and Matching Shape-Motion Prototype Trees , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Bahram Javidi,et al.  Three-dimensional volumetric object reconstruction using computational integral imaging. , 2004, Optics express.

[37]  Bahram Javidi,et al.  Distortion-tolerant 3D recognition of underwater objects using neural networks. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[38]  Myungjin Cho,et al.  3D passive photon counting automatic target recognition using advanced correlation filters. , 2011, Optics letters.

[39]  Myungjin Cho,et al.  Three-Dimensional Visualization of Objects in Turbid Water Using Integral Imaging , 2010, Journal of Display Technology.

[40]  H. Murata,et al.  3-D Display System Using a Computer Generated Integral Photograph , 1978 .

[41]  Bahram Javidi,et al.  Three-Dimensional Imaging for Creating Real-World-Like Environments , 2013, Proceedings of the IEEE.

[42]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[43]  C. Burckhardt.  Optimum Parameters and Resolution Limitation of Integral Photography , 1968 .

[44]  T. Okoshi,et al.  Three-dimensional displays , 1980, Proceedings of the IEEE.

[45]  B. Javidi,et al.  Three-dimensional synthetic aperture integral imaging. , 2002, Optics letters.

[46]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[47]  B. Javidi,et al.  Integral imaging with improved depth of field by use of amplitude-modulated microlens arrays. , 2004, Applied optics.

[48]  Dimitrios I. Fotiadis,et al.  Multiple Kernel Learning Algorithms and Their Use in Biomedical Informatics , 2016 .

[49]  Makoto Okui,et al.  Integral Three-Dimensional Television Using a 33-Megapixel Imaging System , 2010, Journal of Display Technology.

[50]  Bahram Javidi,et al.  Three Dimensional Sensing, Visualization, and Display , 2008, 3DTV-CON 2008.

[51]  Jeong-A Lee,et al.  Fast 3D Computational Integral Imaging Using Graphics Processing Unit , 2012, Journal of Display Technology.

[52]  Osamu Matoba,et al.  Multidimensional optical sensor and imaging system. , 2006, Applied optics.

[53]  Faliu Yi,et al.  Simultaneous reconstruction of multiple depth images without off-focus points in integral imaging using a graphics processing unit. , 2014, Applied optics.