BinaryCoP: Binary Neural Network-based COVID-19 Face-Mask Wear and Positioning Predictor on Edge Devices

Face masks have long been used in many areas of everyday life to protect against the inhalation of hazardous fumes and particles. They also offer an effective solution in healthcare for bi-directional protection against air-borne diseases. Wearing and positioning the mask correctly is essential for its function. Convolutional neural networks (CNNs) offer an excellent solution for face recognition and classification of correct mask wearing and positioning. In the context of the ongoing COVID-19 pandemic, such algorithms can be used at entrances to corporate buildings, airports, shopping areas, and other indoor locations, to mitigate the spread of the virus. These application scenarios impose major challenges to the underlying compute platform. The inference hardware must be cheap, small and energy efficient, while providing sufficient memory and compute power to execute accurate CNNs at a reasonably low latency. To maintain data privacy of the public, all processing must remain on the edge-device, without any communication with cloud servers. To address these challenges, we present BinaryCoP, a low-power binary neural network classifier for correct facial-mask wear and positioning. The classification task is implemented on an embedded FPGA accelerator, performing high-throughput binary operations. Classification can take place at up to ∼6400 frames-per-second and 2W power consumption, easily enabling multi-camera and speed-gate settings. When deployed on a single entrance or gate, the idle power consumption is reduced to 1.65W, improving the battery-life of the device. We achieve an accuracy of up to 98% for four wearing positions of the MaskedFace-Net dataset. To maintain equivalent classification accuracy for all face structures, skin-tones, hair types, and mask types, the algorithms are tested for their ability to generalize the relevant features over a diverse set of examples using the Grad-CAM approach.

[1]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Asif Iqbal Khan,et al.  CoroNet: A deep neural network for detection and diagnosis of COVID-19 from chest x-ray images , 2020, Computer Methods and Programs in Biomedicine.

[3]  Walter Stechele,et al.  OrthrusPE: Runtime Reconfigurable Processing Elements for Binary Neural Networks , 2020, 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[4]  Alexander Wong,et al.  COVID-Net: a tailored deep convolutional neural network design for detection of COVID-19 cases from chest X-ray images , 2020, Scientific reports.

[5]  M. Melkemi,et al.  MaskedFace-Net – A dataset of correctly/incorrectly masked face images in the context of COVID-19 , 2020, Smart Health.

[6]  Wei Pan,et al.  Towards Accurate Binary Convolutional Neural Network , 2017, NIPS.

[7]  Niall O' Mahony,et al.  Deep Learning vs. Traditional Computer Vision , 2019, CVC.

[8]  Eriko Nurvitadhi,et al.  Accelerating Binarized Neural Networks: Comparison of FPGA, CPU, GPU, and ASIC , 2016, 2016 International Conference on Field-Programmable Technology (FPT).

[9]  Mahmoud Melkemi,et al.  Validating the correct wearing of protection mask by taking a selfie: design of a mobile application "CheckYourMask'' to limit the spread of COVID-19 , 2020 .

[10]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Jishen Zhao,et al.  Towards Fast and Energy-Efficient Binarized Neural Network Inference on FPGA , 2018, FPGA.

[12]  T. Mitze,et al.  Face masks considerably reduce COVID-19 cases in Germany , 2020, medRxiv.

[13]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[14]  Ran El-Yaniv,et al.  Binarized Neural Networks , 2016, ArXiv.

[15]  Yoshua Bengio,et al.  Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation , 2013, ArXiv.

[16]  Philip Heng Wai Leong,et al.  FINN: A Framework for Fast, Scalable Binarized Neural Network Inference , 2016, FPGA.

[17]  Hao Wu,et al.  Masked Face Recognition Dataset and Application , 2020, ArXiv.

[18]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[19]  Shiming Ge,et al.  Detecting Masked Faces in the Wild with LLE-CNNs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Luca Benini,et al.  YodaNN: An Architecture for Ultralow Power Binary-Weight CNN Acceleration , 2016, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[21]  Ali Farhadi,et al.  XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks , 2016, ECCV.

[22]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[23]  Pengwei Wang,et al.  WearMask: Fast In-browser Face Mask Detection with Serverless Edge Computing for COVID-19 , 2021, ArXiv.

[24]  Tadahiro Kuroda,et al.  BRein Memory: A Single-Chip Binary/Ternary Reconfigurable in-Memory Deep Neural Network Accelerator Achieving 1.4 TOPS at 0.6 W , 2018, IEEE Journal of Solid-State Circuits.

[25]  Arijit Raychowdhury,et al.  Masked Face Recognition for Secure Authentication , 2020, ArXiv.

[26]  Yoshua Bengio,et al.  BinaryConnect: Training Deep Neural Networks with binary weights during propagations , 2015, NIPS.

[27]  Federico Tombari,et al.  Binary DAD-Net: Binarized Driveable Area Detection Network for Autonomous Driving , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Ruizhi Chen,et al.  FBNA: A Fully Binarized Neural Network Accelerator , 2018, 2018 28th International Conference on Field Programmable Logic and Applications (FPL).

[29]  Mouloud Belbahri,et al.  BNN+: Improved Binary Network Training , 2018, ArXiv.