Integral form 4-D light field filters using Xilinx FPGAs and 45 nm CMOS technology

Four-dimensional (4-D) infinite impulse response frequency hyper-planar filter and a digital VLSI architecture for real time light field based depth filtering applications is proposed. A signal flow graph based on discrete spatial integrators is introduced, which leads to improved sensitivity properties for perturbations in filter coefficients. First order sensitivity analysis of filter transfer function shows a 92.9 % reduction of maximum gain error in frequency response with 12 bits of fractional precision, when compared with a direct-form architecture. Prototype FPGA hardware-in-the-loop co-simulations are performed for two different light field geometries. Register transfer level design validation is carried out via FPGA hardware emulation with a host computer providing memory buffers, and the full-design emulation is carried out on a standalone Berkely Emulation Engine (BEE3), operating at 36.44 and 37.31 MHz for the two light field geometries, respectively. 45 nm CMOS implementation is carried out up to the synthesis level, yielding operating frequencies of 154.4 and 153.3 MHz (correspondingly frame rates of 1.15 and 18.286 Hz) for the two light field geometries, respectively.

[1]  Leonard T. Bruton,et al.  A 4D frequency-planar IIR filter and its application to light field processing , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[2]  David J. Kriegman,et al.  Synthetic Aperture Tracking: Tracking through Occlusions , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[3]  Arjuna Madanayake,et al.  A Fully Multiplexed First-Order Frequency-Planar Module for Fan, Beam, and Cone Plane-Wave Filters , 2006, IEEE Transactions on Circuits and Systems II: Express Briefs.

[4]  John G. Proakis,et al.  Digital Signal Processing: Principles, Algorithms, and Applications , 1992 .

[5]  Leonard T. Bruton,et al.  Practical-BIBO stability of n-dimensional discrete systems , 1983 .

[6]  Donald G. Dansereau 4D light field processing and its application to computer vision , 2003 .

[7]  Hari C. Reddy,et al.  Symmetry Study for Delta- Operator-Based 2-D Digital Filters , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[8]  Stefan B. Williams,et al.  Plenoptic flow: Closed-form visual odometry for light field cameras , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Meikang Qiu,et al.  Grounding high efficiency cloud computing architecture: HW-SW co-design and implementation of a stand-alone web server on FPGA , 2011, Fourth International Conference on the Applications of Digital Information and Web Technologies (ICADIWT 2011).

[10]  Leonard T. Bruton,et al.  Three-dimensional image processing using the concept of network resonance , 1985 .

[11]  John G. Proakis,et al.  Digital signal processing (3rd ed.): principles, algorithms, and applications , 1996 .

[12]  E. Rogers,et al.  Positive real control of two-dimensional systems: Roesser models and linear repetitive processes , 2003 .

[13]  A.R. Stubberud,et al.  Study of various symmetries in the frequency response of two-dimensional delta operator formulated discrete-time systems , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[14]  Arjuna Madanayake,et al.  A Systolic Array 2-D IIR Broadband RF Beamformer , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[15]  Mark A. Horowitz,et al.  Light field video camera , 2000, IS&T/SPIE Electronic Imaging.

[16]  Leonard T. Bruton RC-Active Circuits. Theory and Design , 1980 .

[17]  Arjuna Madanayake,et al.  A systolic-array architecture for first-order 4-D IIR frequency-planar digital filters , 2012, 2012 IEEE International Symposium on Circuits and Systems.

[18]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[19]  Leonard T. Bruton,et al.  Two-dimensional discrete filters using spatial integrators , 1983 .

[20]  Hari C. Reddy,et al.  Delta operator based design of 1-D and 2-D filters: An overview , 2008, APCCAS 2008 - 2008 IEEE Asia Pacific Conference on Circuits and Systems.

[21]  Chen Chang,et al.  BEE3: Revitalizing Computer Architecture Research , 2009 .

[22]  Leonard T. Bruton,et al.  A 4-D Dual-Fan Filter Bank for Depth Filtering in Light Fields , 2007, IEEE Transactions on Signal Processing.

[23]  Andreas Koch,et al.  An FPGA-based scalable platform for high-speed malware collection in large IP networks , 2010, 2010 International Conference on Field-Programmable Technology.

[24]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[25]  Marc Levoy,et al.  High-speed videography using a dense camera array , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[26]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[27]  Leonard T. Bruton,et al.  BIBO stability of inverse 2-D digital filters in the presence of nonessential singularities of the second kind , 1989 .

[28]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[29]  Stefan B. Williams,et al.  Seabed modeling and distractor extraction for mobile AUVs using light field filtering , 2011, 2011 IEEE International Conference on Robotics and Automation.

[30]  Arjuna Madanayake,et al.  Design and FPGA-implementation of 1st-order 4D IIR frequency-hyperplanar digital filters , 2011, 2011 IEEE 54th International Midwest Symposium on Circuits and Systems (MWSCAS).