Trustworthy Method for Person Identification in IIoT Environments by Means of Facial Dynamics

In industrial Internet of Things (IIoT) environments, dependability of a complex manufacturing process in which human operators play a key role can be improved by identity recognition/authentication of whoever is involved in various stages of a production process, according to where and when he/she is supposed to be. To this aim, we propose an approach that exploits the dynamic appearance and the time-dependent local features characterizing the face of an individual during speech utterance with regard to their spatial and temporal components. The proposed method models these dynamic facial patterns captured from edge Internet of Things devices by means of the Local Binary Pattern on Three Orthogonal Planes descriptor, which effectively extract both face's local features and movement at the fog level of the architecture. A deep feedforward network available in the cloud is trained and optimized to match the extracted features to a reference database. The achieved results highlight state-of-the-art performances of the proposed method with regard to robustness and trustworthiness of identification, especially for challenging IIoT scenarios.

[1]  Tie Qiu,et al.  Fog Computing Based Face Identification and Resolution Scheme in Internet of Things , 2017, IEEE Transactions on Industrial Informatics.

[2]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Chi Harold Liu,et al.  The Emerging Internet of Things Marketplace From an Industrial Perspective: A Survey , 2015, IEEE Transactions on Emerging Topics in Computing.

[4]  Navdeep Jaitly,et al.  Towards End-To-End Speech Recognition with Recurrent Neural Networks , 2014, ICML.

[5]  Kashif Munir,et al.  Biometric Smartcard Authentication for Fog Computing , 2018, International Journal of Network Security & Its Applications.

[6]  Paul A. Wortman,et al.  Evolving authentication design considerations for the Internet of biometric things (IoBT) , 2016, 2016 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS).

[7]  Athanasios V. Vasilakos,et al.  Biometrics-Based Privacy-Preserving User Authentication Scheme for Cloud-Based Industrial Internet of Things Deployment , 2018, IEEE Internet of Things Journal.

[8]  Vaishali Kulkarni,et al.  Lip Tracking Using Deformable Models and Geometric Approaches , 2019 .

[9]  Wu He,et al.  Internet of Things in Industries: A Survey , 2014, IEEE Transactions on Industrial Informatics.

[10]  Information and Communication Technology for Intelligent Systems , 2019, Smart Innovation, Systems and Technologies.

[11]  Naixue Xiong,et al.  A Privacy-Preserving Outsourcing Scheme for Image Local Binary Pattern in Secure Industrial Internet of Things , 2020, IEEE Transactions on Industrial Informatics.

[12]  Michele Nappi,et al.  Deceiving faces: When plastic surgery challenges face recognition , 2016, Image Vis. Comput..

[13]  Sherali Zeadally,et al.  Certificateless Public Key Authenticated Encryption With Keyword Search for Industrial Internet of Things , 2018, IEEE Transactions on Industrial Informatics.

[14]  David Windridge,et al.  Detection of Face Spoofing Using Visual Dynamics , 2015, IEEE Transactions on Information Forensics and Security.

[15]  Michael Wagner,et al.  "liveness" Verification in Audio-video Authentication , 2004, INTERSPEECH.

[16]  Xin Liu,et al.  Learning Multi-Boosted HMMs for Lip-Password Based Speaker Verification , 2014, IEEE Transactions on Information Forensics and Security.

[17]  Erich Elsen,et al.  Deep Speech: Scaling up end-to-end speech recognition , 2014, ArXiv.

[18]  Wei Xi,et al.  SALM: Smartphone-Based Identity Authentication Using Lip Motion Characteristics , 2017, 2017 IEEE International Conference on Smart Computing (SMARTCOMP).

[19]  M. Shamim Hossain,et al.  Cloud-assisted Industrial Internet of Things (IIoT) - Enabled framework for health monitoring , 2016, Comput. Networks.

[20]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Ioannis Pitas,et al.  Visual Lip Activity Detection and Speaker Detection Using Mouth Region Intensities , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[22]  Fan Wu,et al.  A Robust ECC-Based Provable Secure Authentication Protocol With Privacy Preserving for Industrial Internet of Things , 2018, IEEE Transactions on Industrial Informatics.

[23]  Shan He,et al.  Research on Visual Speech Recognition Based on Local Binary Pattern and Stacked Sparse Autoencoder , 2018, IHSED.

[24]  WADOOD ABDUL,et al.  Biometric Security Through Visual Encryption for Fog Edge Computing , 2017, IEEE Access.

[25]  Tie Qiu,et al.  Security and Privacy Preservation Scheme of Face Identification and Resolution Framework Using Fog Computing in Internet of Things , 2017, IEEE Internet of Things Journal.

[26]  Yoonmee Doh,et al.  Guaranteeing Real-Time Services for Industrial Wireless Sensor Networks With IEEE 802.15.4 , 2010, IEEE Transactions on Industrial Electronics.

[27]  Lingjuan Lyu,et al.  Fog-Embedded Deep Learning for the Internet of Things , 2019, IEEE Transactions on Industrial Informatics.

[28]  Kim-Kwang Raymond Choo,et al.  Context Aware Ubiquitous Biometrics in Edge of Military Things , 2017, IEEE Cloud Computing.

[29]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  M. Shamim Hossain,et al.  Toward end-to-end biomet rics-based security for IoT infrastructure , 2016, IEEE Wireless Communications.

[31]  Andreas Willig,et al.  Recent and Emerging Topics in Wireless Industrial Communications: A Selection , 2008, IEEE Transactions on Industrial Informatics.

[32]  Mohamed Amine Ferrag,et al.  Authentication Protocols for Internet of Things: A Comprehensive Survey , 2016, Secur. Commun. Networks.

[33]  Gennaro Boggia,et al.  A Markov Model for Characterizing IEEE 802.15.4 MAC Layer in Noisy Environments , 2015, IEEE Transactions on Industrial Electronics.

[34]  A. Murat Tekalp,et al.  Discriminative Analysis of Lip Motion Features for Speaker Identification and Speech-Reading , 2006, IEEE Transactions on Image Processing.

[35]  Laurence T. Yang,et al.  Privacy-Preserving Tensor-Based Multiple Clusterings on Cloud for Industrial IoT , 2019, IEEE Transactions on Industrial Informatics.

[36]  Tiberiu Popa,et al.  Time series matching for biometric visual passwords , 2017, SIGGRAPH Posters.

[37]  Per Lindgren,et al.  End-to-End Response Time of IEC 61499 Distributed Applications Over Switched Ethernet , 2017, IEEE Transactions on Industrial Informatics.

[38]  Mark Mohammad Tehranipoor,et al.  Hardware security meets biometrics for the age of IoT , 2016, 2016 IEEE International Symposium on Circuits and Systems (ISCAS).

[39]  Stefanos Gritzalis,et al.  Cryptographic Solutions for Industrial Internet-of-Things: Research Challenges and Opportunities , 2018, IEEE Transactions on Industrial Informatics.

[40]  Hodjat Hamidi,et al.  An approach to develop the smart health using Internet of Things and authentication based on biometric technology , 2019, Future Gener. Comput. Syst..

[41]  Yue Xu,et al.  A unified face identification and resolution scheme using cloud computing in Internet of Things , 2018, Future Gener. Comput. Syst..

[42]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[43]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[44]  Matti Pietikäinen,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON MULTIMEDIA 1 Lipreading with Local Spatiotemporal Descriptors , 2022 .

[45]  Josef Bigün,et al.  Motion Features from Lip Movement for Person Authentication , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[46]  Josef Bigün,et al.  Audio-visual person authentication using lip-motion from orientation maps , 2007, Pattern Recognit. Lett..