Multi-Channel Biometrics for eHealth Combining Acoustic and Machine Vision Analysis of Speech, Lip Movement and Face: a Case Study

The purpose of this work is to present a solution combining user-friendliness and cost-effectiveness use of audio (speech) & visual (video/image) biometrics, for eHealth, able to achieve better accuracy and increase the ability to avoid counterfeiting. This work shows the evaluation results for an eHealth pilot study that tested the security, privacy, usability and cost-effective features of a user authentication platform for the management of sensitive heterogeneous multi-scale medical data (i.e. medical imaging such as MRI/CT scans, physical reports, and laboratory results), through easy acquisition of biometric data via laptops, and tablets equipped with cameras and microphones. Regarding the user enrollment and verification, audio-visual biometric information from an individual is captured, processed and stored as a biometric template. In subsequent uses, biometric information is captured and compared with the biometric templates. If the comparison is successful the verified user could be allowed to sign in to a medical collaboration platform of the hospitals infrastructure. In this work we present the biometric platform developed, the testing methodology and the administrative framework and legal processes, related to GDPR, for the eHealth pilot study and the results from our quantitative and qualitative analysis that was performed.

[1]  Josephine Sullivan,et al.  One millisecond face alignment with an ensemble of regression trees , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Manolis Tsiknakis,et al.  Designing a digital patient avatar in the context of the MyHealthAvatar project initiative , 2013, 13th IEEE International Conference on BioInformatics and BioEngineering.

[3]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Dijana Petrovska-Delacrétaz,et al.  State-of-the-art face recognition performance using publicly available software and datasets , 2018, 2018 4th International Conference on Advanced Technologies for Signal and Image Processing (ATSIP).

[5]  Margherita Antona,et al.  Multi-modal User Interface Design for a Face and Voice Recognition Biometric Authentication System , 2017, MobiHealth.

[6]  T. Stamkopoulos,et al.  Home monitoring and personal health management services in a regional health telematics network , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Dana Marohn,et al.  Biometrics in healthcare , 2006 .

[8]  A. Cocioceanu,et al.  Statistical description of phonetic balance for greek utterances , 2019 .

[9]  Sourav Mukhopadhyay,et al.  An improved biometric-based remote user authentication scheme for connected healthcare , 2015, Int. J. Ad Hoc Ubiquitous Comput..

[10]  Georgios C. Manikis,et al.  Personalized Mobile eHealth Services for Secure User Access Through a Multi Feature Biometric Framework , 2019 .

[11]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[13]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[14]  Mahadev Satyanarayanan,et al.  OpenFace: A general-purpose face recognition library with mobile applications , 2016 .

[15]  A.C.J. Sprokkereef,et al.  ETHICAL PRACTICE IN THE USE OF BIOMETRIC IDENTIFIERS WITHIN THE EU , 2007 .