Detail Restoration and Tone Mapping Networks for X-Ray Security Inspection

X-ray imaging is one of the most widely used security measures for maintaining airport and transportation security. Conventional X-ray imaging systems typically apply tone-mapping (TM) algorithms to visualize high-dynamic-range (HDR) X-ray images on a standard 8-bit display device. However, X-ray images obtained through traditional TM algorithms often suffer from halo artifacts or detail loss in inter-object overlapping regions, which makes it difficult for an inspector to detect unsafe or hazardous objects. To alleviate these problems, this article proposes a deep learning-based TM method for X-ray inspection. The proposed method consists of two networks called detail-recovery network (DR-Net) and TM network (TM-Net). The goal of DR-Net is to restore the details in the input HDR image, whereas TM-Net aims to compress the dynamic range while preserving the restored details and preventing halo artifacts. Since there are no standard ground-truth images available for the TM of X-ray images, we propose a novel loss function for unsupervised learning of TM-Net. We also introduce a dataset synthesis technique using the Beer-Lambert law for supervised learning of DR-Net. Extensive experiments comparing the performance of our proposed method with state-of-the-art TM methods demonstrate that the proposed method not only achieves visually compelling results but also improves the quantitative performance measures such as FSITM and HDR-VDP-2.2.

[1]  Bernd Weissmuller,et al.  Radiological Imaging The Theory Of Image Formation Detection And Processing , 2016 .

[2]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[3]  Greg Ward,et al.  A Contrast-Based Scalefactor for Luminance Display , 1994, Graphics Gems.

[4]  Mongi A. Abidi,et al.  Gray-level grouping (GLG): an automatic method for optimized image contrast Enhancement-part I: the basic method , 2006, IEEE Transactions on Image Processing.

[5]  Lewis D. Griffin,et al.  Threat Image Projection (TIP) into X-ray images of cargo containers for training humans and machines , 2016, 2016 IEEE International Carnahan Conference on Security Technology (ICCST).

[6]  Lei Zhang,et al.  Clustering based content and color adaptive tone mapping , 2017, Comput. Vis. Image Underst..

[7]  Xianming Liu,et al.  When Image Denoising Meets High-Level Vision Tasks: A Deep Learning Approach , 2017, IJCAI.

[8]  Zhichao Yin,et al.  GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Toby P. Breckon,et al.  Using Deep Convolutional Neural Network Architectures for Object Classification and Detection Within X-Ray Baggage Security Imagery , 2018, IEEE Transactions on Information Forensics and Security.

[10]  Nikos Komodakis,et al.  Deep Tone Mapping Operator for High Dynamic Range Images , 2019, IEEE Transactions on Image Processing.

[11]  Christine D. Piatko,et al.  A visibility matching tone reproduction operator for high dynamic range scenes , 1997, SIGGRAPH '97.

[12]  Graham D. Finlayson,et al.  Learning to display high dynamic range images , 2004, Pattern Recognit..

[13]  Jack L Glover,et al.  Improved Threat Identification Using Tonemapping of High-Dynamic-Range X-ray Images. , 2018, Journal of testing and evaluation.

[14]  Christophe De Vleeschouwer,et al.  Bilateral Histogram Equalization for X-Ray Image Tone Mapping , 2019, 2019 IEEE International Conference on Image Processing (ICIP).

[15]  Qingjie Liu,et al.  Road Extraction by Deep Residual U-Net , 2017, IEEE Geoscience and Remote Sensing Letters.

[16]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[17]  Hans-Peter Seidel,et al.  Dynamic range independent image quality assessment , 2008, ACM Trans. Graph..

[18]  Alexia Briassouli,et al.  Deep Learning for Robust end-to-end Tone Mapping , 2019, BMVC.

[19]  Gangyi Jiang,et al.  Novel Tone Mapping Method via Macro-Micro Modeling of Human Visual System , 2019, IEEE Access.

[20]  David Zhang,et al.  A Hybrid l1-l0 Layer Decomposition Model for Tone Mapping , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Domingo Mery,et al.  GDXray: The Database of X-ray Images for Nondestructive Testing , 2015, Journal of Nondestructive Evaluation.

[22]  Tadashi Wadayama,et al.  Trainable Projected Gradient Detector for Massive Overloaded MIMO Channels: Data-Driven Tuning Approach , 2018, IEEE Access.

[23]  Jia Xu,et al.  Fast Image Processing with Fully-Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[24]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[25]  Ishtiaq Rasool Khan,et al.  Tone-Mapping Using Perceptual-Quantizer and Image Histogram , 2020, IEEE Access.

[26]  Sung-Ho Park,et al.  Evaluating tone mapping algorithms for rendering non-pictorial (scientific) high-dynamic-range images , 2007, J. Vis. Commun. Image Represent..

[27]  Rafal Mantiuk,et al.  A comparative review of tone‐mapping algorithms for high dynamic range video , 2017, Comput. Graph. Forum.

[28]  Robert Stanton,et al.  Radiological Imaging: The Theory of Image Formation, Detection, and Processing , 1983 .

[29]  Valero Laparra,et al.  Perceptually Optimized Image Rendering , 2017, Journal of the Optical Society of America. A, Optics, image science, and vision.

[30]  Laurence Meylan,et al.  High dynamic range image rendering with a retinex-based adaptive filter , 2006, IEEE Transactions on Image Processing.

[31]  Erik Reinhard,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Dynamic Range Reduction Inspired by Photoreceptor Physiology , 2022 .

[32]  Erik Reinhard,et al.  Photographic tone reproduction for digital images , 2002, ACM Trans. Graph..

[33]  Sos S. Agaian,et al.  Transform Coefficient Histogram-Based Image Enhancement Algorithms Using Contrast Entropy , 2007, IEEE Transactions on Image Processing.

[34]  Ronggang Wang,et al.  Deep tone mapping network in HSV color space , 2019, 2019 IEEE Visual Communications and Image Processing (VCIP).

[35]  Zeev Farbman,et al.  Edge-preserving decompositions for multi-scale tone and detail manipulation , 2008, ACM Trans. Graph..

[36]  Olaf Ronneberger,et al.  Invited Talk: U-Net Convolutional Networks for Biomedical Image Segmentation , 2017, Bildverarbeitung für die Medizin.

[37]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Kai Zeng,et al.  High Dynamic Range Image Compression by Optimizing Tone Mapped Image Quality Index , 2015, IEEE Transactions on Image Processing.

[39]  Pascal Fua,et al.  A domain-adaptive two-stream U-Net for electron microscopy image segmentation , 2018, 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018).

[40]  Zhou Wang,et al.  Objective Quality Assessment of Tone-Mapped Images , 2013, IEEE Transactions on Image Processing.

[41]  Y. Salih,et al.  Tone mapping of HDR images: A review , 2012, 2012 4th International Conference on Intelligent and Advanced Systems (ICIAS2012).

[42]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[43]  Young Hwan Kim,et al.  Non-Iterative Tone Mapping With High Efficiency and Robustness , 2018, IEEE Access.

[44]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[45]  Susanto Rahardja,et al.  A Tone-Mapping Technique Based on Histogram Using a Sensitivity Model of the Human Visual System , 2018, IEEE Transactions on Industrial Electronics.

[46]  Mohamed Cheriet,et al.  FSITM: A Feature Similarity Index For Tone-Mapped Images , 2015, IEEE Signal Processing Letters.

[47]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[48]  Patrick Le Callet,et al.  HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images , 2014, J. Electronic Imaging.

[49]  Donald P. Greenberg,et al.  A model of visual adaptation for realistic image synthesis , 1996, SIGGRAPH.

[50]  Sung-Jea Ko,et al.  Unsupervised Deep Contrast Enhancement With Power Constraint for OLED Displays , 2020, IEEE Transactions on Image Processing.

[51]  Kai Zeng,et al.  Objective quality assessment of tone-mapped videos , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[52]  Fang Wan,et al.  SIXray: A Large-Scale Security Inspection X-Ray Benchmark for Prohibited Item Discovery in Overlapping Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Anelia Angelova,et al.  Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.