Assessment and Estimation of Face Detection Performance Based on Deep Learning for Forensic Applications †

Face recognition is a valuable forensic tool for criminal investigators since it certainly helps in identifying individuals in scenarios of criminal activity like fugitives or child sexual abuse. It is, however, a very challenging task as it must be able to handle low-quality images of real world settings and fulfill real time requirements. Deep learning approaches for face detection have proven to be very successful but they require large computation power and processing time. In this work, we evaluate the speed–accuracy tradeoff of three popular deep-learning-based face detectors on the WIDER Face and UFDD data sets in several CPUs and GPUs. We also develop a regression model capable to estimate the performance, both in terms of processing time and accuracy. We expect this to become a very useful tool for the end user in forensic laboratories in order to estimate the performance for different face detection options. Experimental results showed that the best speed–accuracy tradeoff is achieved with images resized to 50% of the original size in GPUs and images resized to 25% of the original size in CPUs. Moreover, performance can be estimated using multiple linear regression models with a Mean Absolute Error (MAE) of 0.113, which is very promising for the forensic field.

[1]  Xiaoqing Yu,et al.  Eye landmarks detection via two-level cascaded CNNs with multi-task learning , 2018, Signal Process. Image Commun..

[2]  Jinze Li,et al.  Face Detection Based on YOLOv3 , 2019, Recent Trends in Intelligent Computing, Communication and Devices.

[3]  Sridha Sridharan,et al.  Using Synthetic Data to Improve Facial Expression Analysis with 3D Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[4]  Jian Yang,et al.  DSFD: Dual Shot Face Detector , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Elena Carlotta Olivetti,et al.  Engagement Evaluation in a Virtual Learning Environment via Facial Expression Recognition and Self-Reports: A Preliminary Approach , 2019, Applied Sciences.

[6]  Sotiris Skevoulis,et al.  Comparing TensorFlow Deep Learning Performance and Experiences Using CPUs via Local PCs and Cloud Solutions , 2019 .

[7]  Nhien-An Le-Khac,et al.  Improving Borderline Adulthood Facial Age Estimation through Ensemble Learning , 2019, ARES.

[8]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Klemen Grm,et al.  Strengths and weaknesses of deep learning models for face recognition against image degradations , 2017, IET Biom..

[10]  Steven C. H. Hoi,et al.  Feature Agglomeration Networks for Single Stage Face Detection , 2017, Neurocomputing.

[11]  Qiang Wang,et al.  Benchmarking State-of-the-Art Deep Learning Software Tools , 2016, 2016 7th International Conference on Cloud Computing and Big Data (CCBD).

[12]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[13]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[14]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[15]  Fatih Kurugollu,et al.  A Survey of Deep Learning Solutions for Multimedia Visual Content Analysis , 2019, IEEE Access.

[16]  Heike Hofmann,et al.  Machine learning in forensic applications , 2019, Significance.

[17]  Eduardo Fidalgo,et al.  Query Based Object Retrieval Using Neural Codes , 2017, SOCO-CISIS-ICEUTE.

[18]  Steven C. H. Hoi,et al.  Face Detection using Deep Learning: An Improved Faster RCNN Approach , 2017, Neurocomputing.

[19]  Shifeng Zhang,et al.  S^3FD: Single Shot Scale-Invariant Face Detector , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Adrian Barbu,et al.  Face Detection with a 3D Model , 2016 .

[21]  Shuo Yang,et al.  WIDER FACE: A Face Detection Benchmark , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Eduardo Fidalgo,et al.  Textile Retrieval Based on Image Content from CDC and Webcam Cameras in Indoor Environments , 2018, Sensors.

[24]  Shilpi Singh,et al.  Techniques and Challenges of Face Recognition: A Critical Review , 2018 .

[25]  Amandeep Kaur,et al.  Face detection techniques: a review , 2018, Artificial Intelligence Review.

[26]  M. Swapna,et al.  A Survey on Face Recognition Using Convolutional Neural Network , 2020 .

[27]  Abdulrazak Yahya Saleh,et al.  Facial recognition using deep learning , 2018 .

[28]  Adnan Yassine,et al.  3D face detection based on salient features extraction and skin colour detection using data mining , 2017 .

[29]  Thomas S. Huang,et al.  Survey of Face Detection on Low-Quality Images , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[30]  Stefanos Zafeiriou,et al.  RetinaFace: Single-stage Dense Face Localisation in the Wild , 2019, ArXiv.

[31]  Mingyu You,et al.  Systematic evaluation of deep face recognition methods , 2020, Neurocomputing.

[32]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[33]  Eduardo Fidalgo,et al.  Object Detection for Crime Scene Evidence Analysis Using Deep Learning , 2017, ICIAP.

[34]  Alexei Botchkarev,et al.  A New Typology Design of Performance Metrics to Measure Errors in Machine Learning Regression Algorithms , 2019, Interdisciplinary Journal of Information, Knowledge, and Management.

[35]  Mark S. Nixon,et al.  Feature extraction & image processing for computer vision , 2012 .

[36]  Vishal M. Patel,et al.  Pushing the Limits of Unconstrained Face Detection: a Challenge Dataset and Baseline Results , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[37]  C. Lawrence Zitnick,et al.  Edge Boxes: Locating Object Proposals from Edges , 2014, ECCV.

[38]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[39]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[40]  Damir Demirovic,et al.  Performance of Some Image Processing Algorithms in Tensorflow , 2018, 2018 25th International Conference on Systems, Signals and Image Processing (IWSSIP).

[41]  Ran He,et al.  PyramidBox++: High Performance Detector for Finding Tiny Face , 2019, ArXiv.

[42]  Eduardo Fidalgo,et al.  Pornography and child sexual abuse detection in image and video: A comparative evaluation , 2017, ICDP.

[43]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[44]  Mei Wang,et al.  Deep Face Recognition: A Survey , 2018, Neurocomputing.

[45]  Kaizhu Huang,et al.  Triple loss for hard face detection , 2020, Neurocomputing.

[46]  Rama Chellappa,et al.  HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Xu Tang,et al.  PyramidBox: A Context-assisted Single Shot Face Detector , 2018, ECCV.

[48]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[49]  Qingshan Liu,et al.  FaceHunter: A multi-task convolutional neural network based face detector , 2016, Signal Process. Image Commun..

[50]  Deva Ramanan,et al.  Face detection, pose estimation, and landmark localization in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[51]  Enrique Alegre,et al.  Una Revisión Sistemática de Métodos para Localizar Automáticamente Objetos en Imágenes , 2018, Revista Iberoamericana de Automática e Informática industrial.

[52]  Parag H Rughani,et al.  MACHINE LEARNING FORENSICS:A NEW BRANCH OF DIGITAL FORENSICS , 2017 .

[53]  Deisy Chaves,et al.  Improving speed-accuracy trade-off in face detectors for forensic tools by image resizing , 2019 .

[54]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[55]  ZhangZhengyou,et al.  A survey on face detection in the wild , 2015 .

[56]  Shifeng Zhang,et al.  Faceboxes: A CPU real-time and accurate unconstrained face detector , 2019, Neurocomputing.

[57]  Weihong Deng,et al.  Deep face recognition with clustering based domain adaptation , 2020, Neurocomputing.

[58]  Ravindra S. Hegadi,et al.  Unconstrained face detection: a Deep learning and Machine learning combined approach , 2017, CSI Transactions on ICT.

[59]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[60]  Jiahong Wu,et al.  Accurate Face Detection for High Performance , 2019, ArXiv.

[61]  Rubel Biswas,et al.  Boosting child abuse victim identification in forensic tools with hashing techniques , 2019 .

[62]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).