How to Correctly Detect Face-Masks for COVID-19 from Visual Information?

The new Coronavirus disease (COVID-19) has seriously affected the world By the end of November 2020, the global number of new coronavirus cases had already exceeded 60 million and the number of deaths 1,410,378 according to information from the World Health Organization (WHO) To limit the spread of the disease, mandatory face-mask rules are now becoming common in public settings around the world Additionally, many public service providers require customers to wear face-masks in accordance with predefined rules (e g , covering both mouth and nose) when using public services These developments inspired research into automatic (computer-vision-based) techniques for face-mask detection that can help monitor public behavior and contribute towards constraining the COVID-19 pandemic Although existing research in this area resulted in efficient techniques for face-mask detection, these usually operate under the assumption that modern face detectors provide perfect detection performance (even for masked faces) and that the main goal of the techniques is to detect the presence of face-masks only In this study, we revisit these common assumptions and explore the following research questions: (i) How well do existing face detectors perform with masked-face images? (ii) Is it possible to detect a proper (regulation-compliant) placement of facial masks? and iii) How useful are existing face-mask detection techniques for monitoring applications during the COVID-19 pandemic? To answer these and related questions we conduct a comprehensive experimental evaluation of several recent face detectors for their performance with masked-face images Furthermore, we investigate the usefulness of multiple off-the-shelf deep-learning models for recognizing correct face-mask placement Finally, we design a complete pipeline for recognizing whether face-masks are worn correctly or not and compare the performance of the pipeline with standard face-mask detection models from the literature To facilitate the study, we compile a large dataset of facial images from the publicly available MAFA and Wider Face datasets and annotate it with compliant and non-compliant labels The annotation dataset, called Face-Mask-Label Dataset (FMLD), is made publicly available to the research community

[1]  Alex Krizhevsky,et al.  One weird trick for parallelizing convolutional neural networks , 2014, ArXiv.

[2]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[3]  Jian Yang,et al.  DSFD: Dual Shot Face Detector , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Sorensen Tc,et al.  Humans in the Loop , 2016 .

[5]  Trisiladevi C. Nagavi,et al.  COVID-19 Face Mask Detection Using Deep Learning , 2021, 2021 6th International Conference on Signal Processing, Computing and Control (ISPCC).

[6]  Gang Yu,et al.  Face Attention Network: An Effective Face Detector for the Occluded Faces , 2017, ArXiv.

[7]  Yu Qiao,et al.  Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks , 2016, IEEE Signal Processing Letters.

[8]  Narinder Singh Punn,et al.  Face Mask Detection using Transfer Learning of InceptionV3 , 2020, BDA.

[9]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[10]  Prateek Khandelwal,et al.  Using Computer Vision to enhance Safety of Workforce in Manufacturing in a Post COVID World , 2020, ArXiv.

[11]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[12]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[13]  Mingjie Jiang,et al.  RetinaMask: A Face Mask detector , 2020, ArXiv.

[14]  Irene Kotsia,et al.  RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Shuo Yang,et al.  WIDER FACE: A Face Detection Benchmark , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Rodrigo Ventura,et al.  Robust Object Recognition Through Symbiotic Deep Learning In Mobile Robots , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Hao Wu,et al.  Masked Face Recognition Dataset and Application , 2020, ArXiv.

[18]  Gunasekaran Manogaran,et al.  A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic , 2020, Measurement.

[19]  Luc Van Gool,et al.  The Pascal Visual Object Classes Challenge: A Retrospective , 2014, International Journal of Computer Vision.

[20]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[21]  Stefanos Zafeiriou,et al.  A survey on face detection in the wild: Past, present and future , 2015, Comput. Vis. Image Underst..

[22]  Alexandros Iosifidis,et al.  Forecasting Stock Prices from the Limit Order Book Using Convolutional Neural Networks , 2017, 2017 IEEE 19th Conference on Business Informatics (CBI).

[23]  Dongxiao Li,et al.  Identifying Facemask-Wearing Condition Using Image Super-Resolution with Classification Network to Prevent COVID-19 , 2020, Sensors.

[24]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[25]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  V. Štruc.  Deep Face Recognition for Surveillance Applications , 2016 .

[27]  B. Cowling,et al.  Rational use of face masks in the COVID-19 pandemic , 2020, The Lancet Respiratory Medicine.

[28]  Alberto Fernández Villán,et al.  Mastering OpenCV 4 with Python: A practical guide covering topics from image processing, augmented reality to deep learning with OpenCV 4 and Python 3.7 , 2019 .

[29]  Naser Damer,et al.  The Effect of Wearing a Mask on Face Recognition Performance: an Exploratory Study , 2020, 2020 International Conference of the Biometrics Special Interest Group (BIOSIG).

[30]  Rohini Basak,et al.  Covid-19 Face Mask Detection Using TensorFlow, Keras and OpenCV , 2020, 2020 IEEE 17th India Council International Conference (INDICON).

[31]  Benjamin Schrauwen,et al.  Deep content-based music recommendation , 2013, NIPS.

[32]  Xu Tang,et al.  PyramidBox: A Context-assisted Single Shot Face Detector , 2018, ECCV.

[33]  Peter Peer,et al.  A comprehensive investigation into sclera biometrics: a novel dataset and performance study , 2020, Neural Computing and Applications.

[34]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[35]  Klemen Grm,et al.  Strengths and weaknesses of deep learning models for face recognition against image degradations , 2017, IET Biom..

[36]  Peter Peer,et al.  Constellation-Based Deep Ear Recognition , 2020, Deep Biometrics.

[37]  Patrick Grother,et al.  Ongoing Face Recognition Vendor Test (FRVT) Part 6B: Face recognition accuracy with face masks using post-COVID-19 algorithms , 2020 .

[38]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[39]  Shiming Ge,et al.  Detecting Masked Faces in the Wild with LLE-CNNs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Seong-Whan Lee,et al.  Subject-Independent Brain–Computer Interfaces Based on Deep Convolutional Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[41]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[42]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[43]  Naser Damer,et al.  Learning privacy-enhancing face representations through feature disentanglement , 2020, 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020).

[44]  Benjamin J. Cowling,et al.  Respiratory virus shedding in exhaled breath and efficacy of face masks , 2020, Nature Medicine.

[45]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[46]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[47]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  P. A. Lyakhov,et al.  Application of the residue number system to reduce hardware costs of the convolutional neural network implementation , 2020, Math. Comput. Simul..

[49]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).