Multi Depth-Velocity Filters for Enhancing Multiple Moving Objects in 5-D Light Field Videos

A moving object in a five-dimensional (5-D) light field video (LFV) can be selectively enhanced using the depth and the velocity of the object. In this paper, a 5-D depth-velocity (DV) filter is proposed to enhance multiple moving objects at different depths and with different velocities in an LFV. The 5-D DV filter is designed as a cascade of an infinite-extent impulse response multi-depth filter and a finite-extent impulse response multi-velocity filter. Experimental results obtained with numerically generated and real LFVs indicate that more than 15 dB improvement in signal-to-interference ratio can be achieved with the proposed 5-D multi DV filter compared to previously proposed multi depth-only filters.

[1]  Arjuna Madanayake,et al.  Multi-depth filtering and occlusion suppression in 4-D light fields: Algorithms and architectures , 2020, Signal Process..

[2]  Namalka Liyanage,et al.  Design and Implementation of 5-D IIR Depth Velocity Filters for Light Field Video Processing , 2019, IEEE Transactions on Circuits and Systems II: Express Briefs.

[3]  Leonard T. Bruton,et al.  A 4-D Sparse FIR Hyperfan Filter for Volumetric Refocusing of Light Fields by Hard Thresholding , 2018, 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP).

[4]  Hanumant Singh,et al.  Real-Time Light Field Processing for Autonomous Robotics , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Panajotis Agathoklis,et al.  Multidimensional spatio-temporal filters for depth-velocity filtering in light field videos , 2017, 2017 IEEE International Conference on Industrial and Information Systems (ICIIS).

[6]  Andreas Riener,et al.  Potential of Plenoptic Cameras in the Field of Automotive Safety , 2017 .

[7]  Simon Kriegel,et al.  Benefits of plenoptic cameras for robot vision during close range on-orbit servicing maneuvers , 2017, 2017 IEEE Aerospace Conference.

[8]  Leonard T. Bruton,et al.  A 5-D IIR depth-velocity filter for enhancing objects moving on linear-trajectories in light field videos , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[9]  Arjuna Madanayake,et al.  VLSI architecture for 4-D depth filtering , 2015, Signal Image Video Process..

[10]  Leonard T. Bruton,et al.  Five-Dimensional Depth-Velocity Filtering for Enhancing Moving Objects in Light Field Videos , 2015, IEEE Transactions on Signal Processing.

[11]  Stefan B. Williams,et al.  Linear Volumetric Focus for Light Field Cameras , 2015, TOGS.

[12]  Richard Szeliski,et al.  Refocusing plenoptic images using depth-adaptive splatting , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[13]  Yuzo Iano,et al.  Employing Light Field Cameras in Surveillance: An Analysis of Light Field Cameras in a Surveillance Scenario , 2014 .

[14]  Donald G. Dansereau,et al.  Plenoptic Signal Processing for Robust Vision in Field Robotics , 2013 .

[15]  A. Wallace,et al.  Real-Time People Tracking in a Camera Network , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[16]  M. Johnson-Roberson,et al.  Benthic monitoring with robotic platforms — The experience of Australia , 2013, 2013 IEEE International Underwater Technology Symposium (UT).

[17]  Stefan B. Williams,et al.  Light field image denoising using a linear 4D frequency-hyperfan all-in-focus filter , 2013, Electronic Imaging.

[18]  Ryad Benosman,et al.  Plenoptic cameras in real-time robotics , 2013, Int. J. Robotics Res..

[19]  Andrew Lumsdaine,et al.  The focused plenoptic camera , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[20]  Leonard T. Bruton,et al.  A 4-D Dual-Fan Filter Bank for Depth Filtering in Light Fields , 2007, IEEE Transactions on Signal Processing.

[21]  Ren Ng.  Fourier slice photography , 2005, ACM Trans. Graph..

[22]  Leonard T. Bruton,et al.  A 4D frequency-planar IIR filter and its application to light field processing , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[23]  Leonard McMillan,et al.  Dynamically reparameterized light fields , 2000, SIGGRAPH.

[24]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[25]  Leonard T. Bruton,et al.  Three-dimensional image processing using the concept of network resonance , 1985 .

[26]  Leonard T. Bruton,et al.  Practical-BIBO stability of n-dimensional discrete systems , 1983 .

[27]  Arjuna Madanayake,et al.  Low-Complexity 4-D IIR Filters for Multi-Depth Filtering and Occlusion Suppression in Light Fields , 2018, 2018 IEEE International Symposium on Circuits and Systems (ISCAS).

[28]  Leonard T. Bruton,et al.  A novel 5-D depth–velocity filter for enhancing noisy light field videos , 2017, Multidimens. Syst. Signal Process..

[29]  Chamira U. S. Edussooriya,et al.  Low-Complexity Multi-Dimensional Filters for Plenoptic Signal Processing , 2015 .

[30]  Arjuna Madanayake,et al.  Integral form 4-D light field filters using Xilinx FPGAs and 45 nm CMOS technology , 2015, Multidimens. Syst. Signal Process..

[31]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[32]  E. Adelson,et al.  The Plenoptic Function and the Elements of Early Vision , 1991 .