Design and Implementation of 5-D IIR Depth Velocity Filters for Light Field Video Processing

The design and hardware implementation of a low-complexity signal processing algorithm is proposed for real-time depth-velocity filtering in 5-D light field videos (LFVs). The proposed design is based on a stable 5-D infinite impulse response digital filter having three cascaded sections, each synthesized using the concept of multidimensional passive network resonance. A novel semi-systolic hardware implementation is proposed. Each section of the filter is implemented and tested on a Xilinx Virtex-7 FPGA platform using MATLAB-based hardware co-simulation with both synthetic and real LFV signals. A real-time processing throughput of 467 LFV frames/s is implied with each section of the filter operating at 204, 164, and 115 MHz for input LFV frames of size $9 \times 9 \times 220 \times 360$ .

[1]  Marc Levoy,et al.  Reconstructing Occluded Surfaces Using Synthetic Apertures: Stereo, Focus and Robust Measures , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  Leonard T. Bruton,et al.  A 5-D IIR depth-velocity filter for enhancing objects moving on linear-trajectories in light field videos , 2015, 2015 IEEE International Symposium on Circuits and Systems (ISCAS).

[3]  Leonard T. Bruton,et al.  Practical-BIBO stability of n-dimensional discrete systems , 1983 .

[4]  H.L.P. Madanayake,et al.  Low-complexity distributed parallel processor for 2D IIR broadband beam plane-wave filters , 2007, Canadian Journal of Electrical and Computer Engineering.

[5]  Stefan B. Williams,et al.  Linear Volumetric Focus for Light Field Cameras , 2015, TOGS.

[6]  Leonard T. Bruton,et al.  Five-Dimensional Depth-Velocity Filtering for Enhancing Moving Objects in Light Field Videos , 2015, IEEE Transactions on Signal Processing.

[7]  Leonard T. Bruton,et al.  A novel 5-D depth–velocity filter for enhancing noisy light field videos , 2017, Multidimens. Syst. Signal Process..

[8]  Leonard T. Bruton,et al.  Three-dimensional image processing using the concept of network resonance , 1985 .

[9]  Yuejin Zhang,et al.  Applications of 3-D LCR networks in the design of 3-D recursive filters for processing image sequences , 1994, IEEE Trans. Circuits Syst. Video Technol..

[10]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[11]  Marc Levoy,et al.  Using plane + parallax for calibrating dense camera arrays , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[12]  Ryad Benosman,et al.  Plenoptic cameras in real-time robotics , 2013, Int. J. Robotics Res..

[13]  Leonard T. Bruton,et al.  A 4D frequency-planar IIR filter and its application to light field processing , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..