Robust multimodal biometric authentication on IoT device through ear shape and arm gesture

Nowadays, authentication is required for both physical access to buildings and internal access to computers and systems. Biometrics are one of the emerging technologies used to protect these highly sensitive structures. However, biometric systems based on a single trait enclose several problems such as noise sensitivity and vulnerability to spoof attacks. In this regard, we present in this paper a fully unobtrusive and robust multimodal authentication system that automatically authenticates a user by the way he/she answers his/her phone, after extracting ear and arm gesture biometric modalities from this single action. To deal the challenges facing ear and arm gesture authentication systems in real-world applications, we propose a new method based on image fragmentation that makes the ear recognition more robust in relation to occlusion. The ear feature extraction process has been made locally using Local Phase Quantization (LPQ) in order to get robustness with respect to pose and illumination variation. We also propose a set of four statistical metrics to extract features from arm gesture signals. The two modalities are combined on score-level using a weighted sum. In order to evaluate our contribution, we conducted a set of experiments to demonstrate the contribution of each of the two biometrics and the advantage of their fusion on the overall performance of the system. The multimodal biometric system achieves an equal error rate (EER) of 5.15%.

[1]  Dhvani Shah,et al.  IoT Based Biometrics Implementation on Raspberry Pi , 2016 .

[2]  Juan Arteaga-Falconi,et al.  ECG and fingerprint bimodal authentication , 2018, Sustainable Cities and Society.

[3]  Ana F. Sequeira,et al.  MobBIO: A multimodal database captured with a portable handheld device , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[4]  Mikhail I. Gofman,et al.  Multimodal biometrics for enhanced mobile device security , 2016, Commun. ACM.

[5]  Venu Govindaraju,et al.  Evaluation of biometric spoofing in a multimodal system , 2010, 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[6]  Qing Yang,et al.  HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users , 2015, IEEE Transactions on Information Forensics and Security.

[7]  Yu Bai,et al.  Multimodal Biometrics for Enhanced IoT Security , 2019, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC).

[8]  Matti Pietikäinen,et al.  Face Description with Local Binary Patterns: Application to Face Recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Marina L. Gavrilova,et al.  Decision Fusion for Multimodal Biometrics Using Social Network Analysis , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[10]  Tieniu Tan,et al.  Deep Feature Fusion for Iris and Periocular Biometrics on Mobile Devices , 2018, IEEE Transactions on Information Forensics and Security.

[11]  Venu Govindaraju,et al.  Robustness of multimodal biometric fusion methods against spoof attacks , 2009, J. Vis. Lang. Comput..

[12]  Subrajeet Mohapatra,et al.  Development of Multimodal Biometric Framework for Smartphone Authentication System , 2014 .

[13]  Kevin Cheng,et al.  Quality-Based Score-level Fusion for Secure and Robust Multimodal Biometrics-based Authentication on Consumer Mobile Devices , 2015, ICSEA 2015.

[14]  B. S. Manjunath,et al.  Texture Features for Browsing and Retrieval of Image Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Richa Singh,et al.  Fingerphoto Authentication Using Smartphone Camera Captured Under Varying Environmental Conditions , 2017 .

[16]  Matti Pietikäinen,et al.  A comparative study of texture measures with classification based on featured distributions , 1996, Pattern Recognit..

[17]  Fadi Dornaika,et al.  A Comparative Study of Image Segmentation Algorithms and Descriptors for Building Detection , 2017 .

[18]  Chenye Wu,et al.  Automated human identification using ear imaging , 2012, Pattern Recognit..

[19]  Mazen M. Selim,et al.  Fusion Time Reduction of a Feature Level Based Multimodal Biometric Authentication System , 2020, Int. J. Sociotechnology Knowl. Dev..

[20]  Yanjiao Chen,et al.  LVID: A Multimodal Biometrics Authentication System on Smartphones , 2020, IEEE Transactions on Information Forensics and Security.

[21]  Mikhail I. Gofman,et al.  Hidden Markov Models for feature-level fusion of biometrics on mobile devices , 2016, 2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA).

[22]  Sharath Pankanti,et al.  Multi-modal biometrics for mobile authentication , 2014, IEEE International Joint Conference on Biometrics.

[23]  Vitomir Struc,et al.  Adaptation of SIFT Features for Robust Face Recognition , 2010, ICIAR.

[24]  Andrea F. Abate,et al.  I-Am: Implicitly Authenticate Me—Person Authentication on Mobile Devices Through Ear Shape and Arm Gesture , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[25]  Esa Rahtu,et al.  Rotation invariant local phase quantization for blur insensitive texture analysis , 2008, 2008 19th International Conference on Pattern Recognition.

[26]  Bruno Crispo,et al.  Please hold on: Unobtrusive user authentication using smartphone's built-in sensors , 2017, 2017 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA).

[27]  Enrico Magli,et al.  Secrecy Analysis of Finite-Precision Compressive Cryptosystems , 2020, IEEE Transactions on Information Forensics and Security.

[28]  Christoph Busch,et al.  A comparative study on texture and surface descriptors for ear biometrics , 2014, 2014 International Carnahan Conference on Security Technology (ICCST).

[29]  V. M. Thakare,et al.  Survey of Fusion Techniques for Design of Efficient Multimodal Systems , 2013, 2013 International Conference on Machine Intelligence and Research Advancement.

[30]  Mitko Bogdanoski,et al.  Multimodal Biometric Authentication in IoT: Single Camera Case Study , 2016 .

[31]  Michele Nappi,et al.  Have you permission to answer this phone? , 2018, 2018 International Workshop on Biometrics and Forensics (IWBF).

[32]  Arun Ross,et al.  2D ear classification based on unsupervised clustering , 2014, IEEE International Joint Conference on Biometrics.

[33]  Bruno Crispo,et al.  Multimodal smartphone user authentication using touchstroke, phone-movement and face patterns , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[34]  Ayman Haggag,et al.  Multimodal biometric scheme for human authentication technique based on voice and face recognition fusion , 2018, Multimedia Tools and Applications.

[35]  Shriram D. Raut,et al.  Ear Biometrics: A Survey on Ear Image Databases and Techniques for Ear Detection and Recognition , 2015 .

[36]  Esa Rahtu,et al.  BSIF: Binarized statistical image features , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[37]  Alice Caplier,et al.  Face Recognition with Patterns of Oriented Edge Magnitudes , 2010, ECCV.

[38]  Yi Zhang,et al.  Ear verification under uncontrolled conditions with convolutional neural networks , 2018, IET Biom..

[39]  Ville Ojansivu,et al.  Blur Insensitive Texture Classification Using Local Phase Quantization , 2008, ICISP.

[40]  Wolfgang Leister,et al.  A Novel Authentication Framework Based on Biometric and Radio Fingerprinting for the IoT in eHealth , 2014 .

[41]  Zahid Akhtar,et al.  Spoof Attacks on Multimodal Biometric Systems , 2011 .

[42]  Peter Peer,et al.  Ear recognition: More than a survey , 2016, Neurocomputing.