Target attention deep neural network for infrared image enhancement

Abstract The inherent high background radiation and low contrast of infrared images severely cripple the precision of target detection and recognition. However, existing infrared image enhancement methods still struggle to balance the target enhancement and background suppression. To further address this issue, we present an innovative target attention deep neural network (TADNN) to realize a discriminative enhancement in an end-to-end manner. In this framework, a joint convolution unit (JCU) is constructed to comprehensively excavate the complementary multi-scale spatial features, followed that, a target attention unit (TAU) is designed to further refine the features from JCU for particularly enhancing the targets. Besides, an improved S-curve response model is put forward to generate the task-oriented training set for pursuing a superior fitting solution. Extensive experiments validate that the proposed method outperforms the competitive approaches on both of subjective visual effect and quantitative assessments.

[1]  Meng Zhang,et al.  Recent Advances in Convolutional Neural Network Acceleration , 2018, Neurocomputing.

[2]  Eren Erdal Aksoy,et al.  Deep Neural Network Compression for Image Classification and Object Detection , 2019, 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA).

[3]  Pong C. Yuen,et al.  Learning Modality-Consistency Feature Templates: A Robust RGB-Infrared Tracking System , 2019, IEEE Transactions on Industrial Electronics.

[4]  Rui Lai,et al.  A quantitative measure based infrared image enhancement algorithm using plateau histogram , 2010 .

[5]  Ashish Kumar Bhandari,et al.  Cuckoo search algorithm-based brightness preserving histogram scheme for low-contrast image enhancement , 2019, Soft Computing.

[6]  Pankaj Kandhway,et al.  Modified clipping based image enhancement scheme using difference of histogram bins , 2019, IET Image Process..

[7]  Muhammad Arsalan,et al.  Investigating the Performance of a Few-Mode Fiber for Distributed Acoustic Sensing , 2019, IEEE Photonics Journal.

[8]  Xiaofang Wang,et al.  Learnable Embedding Space for Efficient Neural Architecture Compression , 2019, ICLR.

[9]  Rui Lai,et al.  Multi-Scale Visual Attention Deep Convolutional Neural Network for Multi-Focus Image Fusion , 2019, IEEE Access.

[10]  M. Ali Akber Dewan,et al.  A Dynamic Histogram Equalization for Image Contrast Enhancement , 2007, IEEE Transactions on Consumer Electronics.

[11]  Feng Xue,et al.  Remote Sensing Image Enhancement Via Edge-Preserving Multiscale Retinex , 2019, IEEE Photonics Journal.

[12]  Michael G. Wing,et al.  A low-cost near-infrared digital camera for fire detection and monitoring , 2018 .

[13]  Fathi E. Abd El-Samie,et al.  Resolution and quality enhancement of images using interpolation and contrast limited adaptive histogram equalization , 2018, Multimedia Tools and Applications.

[14]  Kok-Swee Sim,et al.  Infrared image enhancement using adaptive trilateral contrast enhancement , 2015, Pattern Recognit. Lett..

[15]  Rui Lai,et al.  Wavelet Deep Neural Network for Stripe Noise Removal , 2019, IEEE Access.

[16]  Xavier Maldague,et al.  Infrared Image Enhancement Using Adaptive Histogram Partition and Brightness Correction , 2018, Remote. Sens..

[17]  Basavaraj N Jagadale,et al.  Standard Intensity Deviation Approach based Clipped Sub Image Histogram Equalization Algorithm for Image Enhancement , 2018 .

[18]  Pankaj Kandhway,et al.  A novel reformed histogram equalization based medical image contrast enhancement using krill herd optimization , 2020, Biomed. Signal Process. Control..

[19]  Prerna Singh,et al.  Feature Enhancement in Medical Ultrasound Videos Using Contrast-Limited Adaptive Histogram Equalization , 2019, Journal of Digital Imaging.

[20]  Sos S. Agaian,et al.  Transform Coefficient Histogram-Based Image Enhancement Algorithms Using Contrast Entropy , 2007, IEEE Transactions on Image Processing.

[21]  Yun Liang,et al.  Fune: An FPGA Tuning Framework for CNN Acceleration , 2020, IEEE Design & Test.

[22]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[23]  Qian Chen,et al.  Single infrared image enhancement using a deep convolutional neural network , 2019, Neurocomputing.

[24]  Ke Xu,et al.  DSP-Efficient Hardware Acceleration of Convolutional Neural Network Inference on FPGAs , 2020, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[25]  Yuhan Liu,et al.  Infrared small-target detection based on multi-directional multi-scale high-boost response , 2019, Optical Review.

[26]  Xiangzhi Bai,et al.  Infrared image enhancement through contrast enhancement by using multiscale new top-hat transform , 2011 .

[27]  Jinseong Jang,et al.  Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System , 2018, IEEE Access.

[28]  Xuejin Liu,et al.  A dynamic foveated infrared imager for surveillance , 2020 .

[29]  Gabriel Eilertsen,et al.  HDR image reconstruction from a single exposure using deep CNNs , 2017, ACM Trans. Graph..

[30]  S. Pizer,et al.  Adaptive grey level assignment in CT scan display. , 1984, Journal of computer assisted tomography.

[31]  Yifan Gong,et al.  SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency , 2020, ArXiv.

[32]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[33]  Ling-Feng Shi,et al.  A Fast Image Contrast Enhancement Algorithm Using Entropy-Preserving Mapping Prior , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Chen Chen,et al.  Deep Learning and Superpixel Feature Extraction Based on Contractive Autoencoder for Change Detection in SAR Images , 2018, IEEE Transactions on Industrial Informatics.

[35]  Zunlin Fan,et al.  Infrared image enhancement with learned features , 2017 .

[36]  Yang Yintang,et al.  Improvement in adaptive nonuniformity correction method with nonlinear model for infrared focal plane arrays , 2009 .

[37]  Yintang Yang,et al.  Spatiotemporal Adaptive Nonuniformity Correction Based on BTV Regularization , 2019, IEEE Access.

[38]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[39]  Ezzatollah Salari,et al.  Small dim object tracking using frequency and spatial domain information , 2016, Pattern Recognit..

[40]  Thomas Bashford-Rogers,et al.  ExpandNet: A Deep Convolutional Neural Network for High Dynamic Range Expansion from Low Dynamic Range Content , 2018, Comput. Graph. Forum.

[41]  Nam Ik Cho,et al.  A New Convolutional Network-in-Network Structure and Its Applications in Skin Detection, Semantic Segmentation, and Artifact Reduction , 2017, ArXiv.

[42]  Li Tao,et al.  LLCNN: A convolutional neural network for low-light image enhancement , 2017, 2017 IEEE Visual Communications and Image Processing (VCIP).

[43]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[44]  Juntao Guan,et al.  Fixed Pattern Noise Reduction for Infrared Images Based on Cascade Residual Attention CNN , 2019, Neurocomputing.

[45]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Yanzhi Wang,et al.  PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning , 2020, ASPLOS.