Temporal-Spatial Nonlinear Filtering for Infrared Focal Plane Array Stripe Nonuniformity Correction

In this work, we introduce a temporal-spatial approach for infrared focal plane array (IRFPA) stripe nonuniformity correction in infrared images that generates visually appealing results. We posit that the nonuniformity appears as a striped structure in the spatial domain and that the pixel values change slowly in the temporal domain. Based on this, we formulate our correction method in two steps. In the first step, weighted guided image filtering with our adaptive weight is utilized to predict the stripe nonuniformity using a single frame. In the second step, the temporal profile of each pixel can be formed using a few frames of successive nonuniformity images. Further, we present a temporal nonlinear diffusion equation to remove scene residuals from the temporal profile of nonuniformity images in order to estimate a more accurate value of the stripe nonuniformity. The results of extensive experiments demonstrate that the proposed nonuniformity correction algorithm substantially outperforms many state-of-the-art approaches, including both traditional and deep convolution-neural-network-based methods, on four popular infrared videos. In addition, the proposed method only requires a fraction (less than ten) of the video frames.

[1]  Xiubao Sui,et al.  Single Infrared Image Stripe Noise Removal Using Deep Convolutional Networks , 2017, IEEE Photonics Journal.

[2]  Melvin R. Kruer,et al.  Nonuniformity correction for staring IR focal plane arrays using scene-based techniques , 1990, Defense, Security, and Sensing.

[3]  Zhengguo Li,et al.  Detail-Enhanced Multi-Scale Exposure Fusion , 2017, IEEE Transactions on Image Processing.

[4]  Zhang Tian PDE-based deghosting algorithm for correction of nonuniformity in infrared focal plane array , 2012 .

[5]  Ju-feng Zhao,et al.  Single image stripe nonuniformity correction with gradient-constrained optimization model for infrared focal plane arrays , 2013 .

[6]  Tianxu Zhang,et al.  PDE-based deghosting algorithm for correction of nonuniformity in infrared focal plane array: PDE-based deghosting algorithm for correction of nonuniformity in infrared focal plane array , 2012 .

[7]  Hanlin Qin,et al.  Single Infrared Image-Based Stripe Nonuniformity Correction via a Two-Stage Filtering Method , 2018, Sensors.

[8]  Yintang Yang,et al.  Spatiotemporal Adaptive Nonuniformity Correction Based on BTV Regularization , 2019, IEEE Access.

[9]  Zhengguo Li,et al.  Edge-Preserving Decomposition-Based Single Image Haze Removal , 2015, IEEE Transactions on Image Processing.

[10]  Yanlong Cao,et al.  Single-image-based nonuniformity correction of uncooled long-wave infrared detectors: a deep-learning approach. , 2018, Applied optics.

[11]  Xiubao Sui,et al.  Adaptive Image-Registration-Based Nonuniformity Correction Algorithm With Ghost Artifacts Eliminating for Infrared Focal Plane Arrays , 2015, IEEE Photonics Journal.

[12]  Russell C Hardie,et al.  Scene-based nonuniformity correction with reduced ghosting using a gated LMS algorithm. , 2009, Optics express.

[13]  Qiang Guo,et al.  Single image non-uniformity correction using compressive sensing , 2016 .

[14]  E Armstrong,et al.  Scene-based nonuniformity correction with video sequences and registration. , 2000, Applied optics.

[15]  Guohua Gu,et al.  New temporal high-pass filter nonuniformity correction based on bilateral filter , 2011 .

[16]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  M. Kruer,et al.  Infrared focal plane array technology , 1991, Proc. IEEE.

[18]  John G. Harris,et al.  Nonuniformity correction of infrared image sequences using the constant-statistics constraint , 1999, IEEE Trans. Image Process..

[19]  Melvin R. Kruer,et al.  Adaptive nonuniformity correction for IR focal-plane arrays using neural networks , 1991, Optics & Photonics.

[20]  Lai Rui,et al.  Nonuniformity correction for an infrared focal plane array based on diamond search block matching. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[21]  J. Scott Tyo,et al.  Feedback-integrated scene cancellation scene-based nonuniformity correction algorithm , 2014, J. Electronic Imaging.

[22]  Zhengguo Li,et al.  Gradient Domain Guided Image Filtering , 2015, IEEE Transactions on Image Processing.

[23]  Shiqian Wu,et al.  Weighted Guided Image Filtering , 2016, IEEE Transactions on Image Processing.

[24]  Xiubao Sui,et al.  FPN estimation based nonuniformity correction for infrared imaging system , 2019, Infrared Physics & Technology.

[25]  Ping Jia,et al.  Scene-based nonuniformity correction for airborne point target detection systems. , 2017, Optics express.

[26]  Guohua Gu,et al.  Space low-pass and temporal high-pass nonuniformity correction algorithm , 2010 .

[27]  Sungho Kim,et al.  Two-point correction and minimum filter-based nonuniformity correction for scan-based aerial infrared cameras , 2012 .

[28]  Rui Lai,et al.  Total Variation Based Neural Network Regression for Nonuniformity Correction of Infrared Images , 2018, Symmetry.

[29]  Tongsheng Shen,et al.  Scene-based nonuniformity correction based on bilateral filter with reduced ghosting , 2016 .

[30]  Rui Lai,et al.  Wavelet Deep Neural Network for Stripe Noise Removal , 2019, IEEE Access.

[31]  Abraham Friedenberg,et al.  Nonuniformity two-point linear correction errors in infrared focal plane arrays , 1998 .

[32]  Jérôme Gilles,et al.  Non-uniformity Correction of Infrared Images by Midway Equalization , 2012, Image Process. Line.