An FPGA-Based Fully Pipelined Bilateral Grid for Real-Time Image Denoising

The bilateral filter (BF) is widely used in image processing because it can perform denoising while preserving edges. It has disadvantages in that it is nonlinear, and its computational complexity and hardware resources are directly proportional to its window size. Thus far, several approximation methods and hardware implementations have been proposed to solve these problems. However, processing large-scale and high-resolution images in real time under severe hardware resource constraints remains a challenge. This paper proposes a real-time image denoising system that uses an FPGA based on the bilateral grid (BG). In the BG, a 2D image consisting of x- and y-axes is projected onto a 3D space called a “grid,” which consists of axes that correlate to the x-component, y-component, and intensity value of the input image. This grid is then blurred using the Gaussian filter, and the output image is generated by interpolating the grid. Although it is possible to change the window size in the BF, it is impossible to change it on the input image in the BG. This makes it difficult to associate the BG with the BF and to obtain the property of suppressing the increase in hardware resources when the window radius is enlarged. This study demonstrates that a BG with a variable-sized window can be realized by introducing the window radius parameter wherein the window radius on the grid is always 1. We then implement this BG on an FPGA in a fully pipelined manner. Further, we verify that our design suppresses the increase in hardware resources even when the window size is enlarged and outperforms the existing designs in terms of computation speed and hardware resources.

[1]  Matthias Kuba,et al.  An FPGA-Based Fully Synchronized Design of a Bilateral Filter for Real-Time Image Denoising , 2014, IEEE Transactions on Industrial Electronics.

[2]  Michael Unser,et al.  Fast $O(1)$ Bilateral Filtering Using Trigonometric Range Kernels , 2011, IEEE Transactions on Image Processing.

[3]  Ben Weiss.  Fast median and bilateral filtering , 2006, SIGGRAPH 2006.

[4]  Narendra Ahuja,et al.  Constant Time Median and Bilateral Filtering , 2014, International Journal of Computer Vision.

[5]  Jürgen Teich,et al.  A Design Methodology for Hardware Acceleration of Adaptive Filter Algorithms in Image Processing , 2006, IEEE 17th International Conference on Application-specific Systems, Architectures and Processors (ASAP'06).

[6]  Jürgen Teich,et al.  A deeply pipelined and parallel architecture for denoising medical images , 2010, 2010 International Conference on Field-Programmable Technology.

[7]  R. Deriche.  Recursively implementating the Gaussian and its derivatives , 1993 .

[8]  Masayuki Ikebe,et al.  O(1) bilateral filtering with low memory usage , 2010, 2010 IEEE International Conference on Image Processing.

[9]  Michael F. Cohen,et al.  Capturing and viewing gigapixel images , 2007, ACM Trans. Graph..

[10]  Kunal N. Chaudhury,et al.  Acceleration of the shiftable O(1) algorithm for bilateral filtering and non-local means , 2012, ArXiv.

[11]  Fatih Porikli,et al.  Constant time O(1) bilateral filtering , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[13]  Hui Cheng,et al.  Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection , 2006, ECCV.

[14]  Kunal N. Chaudhury,et al.  Fast and Provably Accurate Bilateral Filtering , 2016, IEEE Transactions on Image Processing.

[15]  B. Gooch,et al.  Real-time video abstraction , 2006, ACM Trans. Graph..

[16]  G. N. Rathna,et al.  A Reconfigurable and Scalable FPGA Architecture for Bilateral Filtering , 2018, IEEE Transactions on Industrial Electronics.

[17]  Manuel Menezes de Oliveira Neto,et al.  Adaptive manifolds for real-time high-dimensional filtering , 2012, ACM Trans. Graph..

[18]  Kunal Narayan Chaudhury,et al.  Fast high-dimensional filtering using clustering , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[19]  Kunal Narayan Chaudhury,et al.  A Fast Approximation of the Bilateral Filter using the Discrete Fourier Transform , 2017, Image Process. Line.

[20]  Lucas J. van Vliet,et al.  Recursive implementation of the Gaussian filter , 1995, Signal Process..

[21]  B. Erickson,et al.  Comparative study of two image space noise reduction methods for computed tomography: Bilateral filter and nonlocal means , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[22]  Sylvain Paris,et al.  Real-time edge-aware image processing with the bilateral grid , 2007, ACM Trans. Graph..

[23]  Amlan Chakrabarti,et al.  Hardware software co-design of a fast bilateral filter in FPGA , 2013, 2013 Annual IEEE India Conference (INDICON).

[24]  N. Ahuja,et al.  Real-time O(1) bilateral filtering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Truong Quang Vinh,et al.  FPGA Implementation of Real-Time Edge-Preserving Filter for Video Noise Reduction , 2008, 2008 International Conference on Computer and Electrical Engineering.