Multidimensional spatio-temporal filters for depth-velocity filtering in light field videos

A novel multidimensional spatio-temporal signal processing algorithm is proposed for depth-velocity filtering in 5-D light field videos along with a tutorial review of light field video spectra and the general architecture of associated 5-D depth-velocity filters. A 5-D FIR filter having a hyperfan-shaped passband which can selectively enhance the spectra of moving objects in light field videos based on their depth and motion trajectory is employed. A 5-D hyperfan-shaped passband is highly desired to achieve enhanced spatio-temporal selectivity over a wide range of lower frequencies. Experimental results are provided by using a light field videos captured using a Lytro light-field camera and mean structural similarity index is used to verify the effectiveness of the proposed 5-D FIR hyperfan depth-velocity filter.

[1]  Richard Szeliski,et al.  Refocusing plenoptic images using depth-adaptive splatting , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[2]  Stefan B. Williams,et al.  Linear Volumetric Focus for Light Field Cameras , 2015, TOGS.

[3]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[4]  Leonard T. Bruton,et al.  Velocity filtering for attenuating moving artifacts in videos using an ultra-low complexity 3-D linear-phase IIR filter , 2017, Multidimens. Syst. Signal Process..

[5]  David J. Kriegman,et al.  Synthetic Aperture Tracking: Tracking through Occlusions , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Donald G. Dansereau,et al.  Plenoptic Signal Processing for Robust Vision in Field Robotics , 2013 .

[7]  Ren Ng.  Fourier Slice Photography , 2005 .

[8]  Leonard T. Bruton,et al.  A 4D frequency-planar IIR filter and its application to light field processing , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[9]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[10]  Masayuki Tanimoto,et al.  3D-TV System with Depth-Image-Based Rendering , 2012 .

[11]  Leonard T. Bruton,et al.  Five-Dimensional Depth-Velocity Filtering for Enhancing Moving Objects in Light Field Videos , 2015, IEEE Transactions on Signal Processing.

[12]  Anton Kummert,et al.  A multidimensional wave digital filter bank for video-based motion analysis , 2014, Multidimens. Syst. Signal Process..

[13]  Leonard T. Bruton,et al.  A low-complexity 3D spatio-temporal FIR filter for enhancing linear trajectory signals , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Ryad Benosman,et al.  Plenoptic cameras in real-time robotics , 2013, Int. J. Robotics Res..

[15]  John W. Woods.  Multidimensional Signal, Image, and Video Processing and Coding, Second Edition , 2011 .

[16]  Yeung Sam Hung,et al.  On the Hardware/Software Design and Implementation of a High Definition Multiview Video Surveillance System , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[17]  Soo-Chang Pei,et al.  Two-dimensional general fan-type FIR digital filter design , 1994, Signal Process..

[18]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[19]  Chamira U. S. Edussooriya,et al.  Low-Complexity Multi-Dimensional Filters for Plenoptic Signal Processing , 2015 .

[20]  Panajotis Agathoklis,et al.  Enhancing Moving Objects in light field videos using 5-D IIR adaptive depth-velocity filters , 2015, 2015 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM).

[21]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[22]  Marc Levoy,et al.  Synthetic Aperture Focusing using a Shear-Warp Factorization of the Viewing Transform , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[23]  A. Wallace,et al.  Real-Time People Tracking in a Camera Network , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[24]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[25]  Leonard T. Bruton,et al.  A novel 5-D depth–velocity filter for enhancing noisy light field videos , 2017, Multidimens. Syst. Signal Process..

[26]  Leonard T. Bruton,et al.  A 5-D IIR depth-velocity filter for enhancing objects moving on linear-trajectories in light field videos , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[27]  Yuzo Iano,et al.  Employing Light Field Cameras in Surveillance: An Analysis of Light Field Cameras in a Surveillance Scenario , 2014 .

[28]  Amar Aggoun,et al.  Embedded FIR filter design for real-time refocusing using a standard plenoptic video camera , 2014, Electronic Imaging.

[29]  M. Landy,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .