Background Registration-Based Adaptive Noise Filtering of LWIR/MWIR Imaging Sensors for UAV Applications

Unmanned aerial vehicles (UAVs) are equipped with optical systems including an infrared (IR) camera such as electro-optical IR (EO/IR), target acquisition and designation sights (TADS), or forward looking IR (FLIR). However, images obtained from IR cameras are subject to noise such as dead pixels, lines, and fixed pattern noise. Nonuniformity correction (NUC) is a widely employed method to reduce noise in IR images, but it has limitations in removing noise that occurs during operation. Methods have been proposed to overcome the limitations of the NUC method, such as two-point correction (TPC) and scene-based NUC (SBNUC). However, these methods still suffer from unfixed pattern noise. In this paper, a background registration-based adaptive noise filtering (BRANF) method is proposed to overcome the limitations of conventional methods. The proposed BRANF method utilizes background registration processing and robust principle component analysis (RPCA). In addition, image quality verification methods are proposed that can measure the noise filtering performance quantitatively without ground truth images. Experiments were performed for performance verification with middle wave infrared (MWIR) and long wave infrared (LWIR) images obtained from practical military optical systems. As a result, it is found that the image quality improvement rate of BRANF is 30% higher than that of conventional NUC.

[1]  Francois P. J. le Roux,et al.  Non-uniformity correction and bad pixel replacement on LWIR and MWIR images , 2011, 2011 Saudi International Electronics, Communications and Photonics Conference (SIECPC).

[2]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[3]  J. Scott Tyo,et al.  Feedback-integrated scene cancellation scene-based nonuniformity correction algorithm , 2014, J. Electronic Imaging.

[4]  E Armstrong,et al.  Scene-based nonuniformity correction with video sequences and registration. , 2000, Applied optics.

[5]  Soon-Geul Lee,et al.  Lateral Scanning Linnik Interferometry for Large Field of View and Fast Scanning: Wafer Bump Inspection , 2011 .

[6]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization and Its Applications to Low Level Vision , 2016, International Journal of Computer Vision.

[7]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[8]  Liangpei Zhang,et al.  Total-Variation-Regularized Low-Rank Matrix Factorization for Hyperspectral Image Restoration , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Sabine Süsstrunk,et al.  Non-Parametric Blur Map Regression for Depth of Field Extension , 2016, IEEE Transactions on Image Processing.

[10]  R. Dinesh,et al.  Non-parametric adaptive region of support useful for corner detection: a novel approach , 2004, Pattern Recognit..

[11]  Liangpei Zhang,et al.  Hyperspectral Image Denoising Employing a Spectral–Spatial Adaptive Total Variation Model , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Liangpei Zhang,et al.  Stripe Noise Separation and Removal in Remote Sensing Images by Consideration of the Global Sparsity and Local Variational Properties , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Bradley M. Ratliff,et al.  Scene-based correction of fixed pattern noise in hyperspectral image data using temporal reordering , 2015 .

[14]  Jérôme Gilles,et al.  Non-uniformity Correction of Infrared Images by Midway Equalization , 2012, Image Process. Line.

[15]  Soon Ki Jung,et al.  Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset , 2015, Comput. Sci. Rev..

[16]  Fuan Tsai,et al.  Striping Noise Detection and Correction of Remote Sensing Images , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Loong Fah Cheong,et al.  Block-Sparse RPCA for Consistent Foreground Detection , 2012, ECCV.

[18]  Baojun Zhao,et al.  Robust Approach for Nonuniformity Correction in Infrared Focal Plane Array , 2016, Sensors.

[19]  Liu Zheng,et al.  ON METHODS OF NOISE REDUCTION IN A STRIPPED IMAGE , 2008 .

[20]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[21]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[22]  Michael Ying Yang,et al.  Effective Strip Noise Removal for Low-Textured Infrared Images Based on 1-D Guided Filtering , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Sungho Kim,et al.  Two-point correction and minimum filter-based nonuniformity correction for scan-based aerial infrared cameras , 2012 .

[24]  Xiliang Lu,et al.  A Universal Destriping Framework Combining 1-D and 2-D Variational Optimization Methods , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Liangpei Zhang,et al.  Hyperspectral Image Restoration Using Low-Rank Matrix Recovery , 2014, IEEE Transactions on Geoscience and Remote Sensing.