Multi-view Object Detection Using Epipolar Constraints within Cluttered X-ray Security Imagery

Automatic detection for threat object items is an increasing emerging area of future application in X-ray security imagery. Although modern X-ray security scanners can provide two or more views, the integration of such object detectors across the views has not been widely explored with rigour. Therefore, we investigate the application of geometric constraints using the epipolar nature of multi-view imagery to improve object detection performance. Furthermore, we assume that images come from uncalibrated views, such that a method to estimate the fundamental matrix using ground truth bounding box centroids from multiple view object labels is proposed. In addition, detections are given a confidence probability based on its similarity with respect to the distribution of the distance to the epipolar line. This probability is used as confidence weights for merging duplicated predictions using non-maximum suppression. Using a standard object detector (YOLOv3), our technique increases the average precision of detection by 2.8% on a dataset composed of firearms, laptops, knives and cameras. These results indicate that the integration of images at different views significantly improves the detection performance of threat items of cluttered X-ray security images.

[1]  Toby P. Breckon,et al.  An evaluation of region based object detection strategies within X-ray baggage security imagery , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[2]  S. Akcay,et al.  On using deep Convolutional Neural Network architectures for automated object detection and classification within X-ray baggage security imagery. , 2018 .

[3]  He Ma,et al.  SeFM: A Sequential Feature Point Matching Algorithm for Object 3D Reconstruction , 2018, ArXiv.

[4]  Samet Akcay,et al.  Towards Automatic Threat Detection: A Survey of Advances of Deep Learning within X-ray Security Imaging , 2020, Pattern Recognit..

[5]  Vignesh Prasad,et al.  Epipolar Geometry based Learning of Multi-view Depth and Ego-Motion from Monocular Sequences , 2018, ICVGIP.

[6]  Lawrence Carin,et al.  Toward Automatic Threat Recognition for Airport X-ray Baggage Screening with Deep Convolutional Object Detection , 2019, ArXiv.

[7]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Joachim Hornegger,et al.  On Feature Tracking in X-Ray Images , 2014, Bildverarbeitung für die Medizin.

[9]  Domingo Mery,et al.  Threat Objects Detection in X-ray Images Using an Active Vision Approach , 2017 .

[10]  Mohamed F. Tolba,et al.  Fundamental matrix estimation: A study of error criteria , 2017, Pattern Recognit. Lett..

[11]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[12]  O. Wetter,et al.  Imaging in airport security: Past, present, future, and the link to forensic and clinical radiology , 2013 .

[13]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[14]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[16]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Muhammet Bastan,et al.  Object Recognition in Multi-View Dual Energy X-ray Images , 2013, BMVC.

[18]  Domingo Mery,et al.  Automated detection in complex objects using a tracking algorithm in multiple X-ray views , 2011, CVPR 2011 WORKSHOPS.

[19]  Zhen Liu,et al.  Efficient image features selection and weighting for fundamental matrix estimation , 2016, IET Comput. Vis..

[20]  Katerina Fragkiadaki,et al.  Epipolar Transformers , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Toby P. Breckon,et al.  Transfer learning using convolutional neural networks for object classification within X-ray baggage security imagery , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[24]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[26]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[27]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Jan-Martin O. Steitz,et al.  Multi-view X-ray R-CNN , 2018, GCPR.

[29]  Domingo Mery,et al.  Automated X-Ray Object Recognition Using an Efficient Search Algorithm in Multiple Views , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[30]  Toby P. Breckon,et al.  The Good, the Bad and the Ugly: Evaluating Convolutional Neural Networks for Prohibited Item Detection Using Real and Synthetically Composited X-ray Imagery , 2019, ArXiv.