BeCAPTCHA: Bot Detection in Smartphone Interaction using Touchscreen Biometrics and Mobile Sensors

In this paper we study the suitability of a new generation of CAPTCHA methods based on smartphone interactions. The heterogeneous flow of data generated during the interaction with the smartphones can be used to model human behaviour when interacting with the technology and improve bot detection algorithms. For this, we propose a CAPTCHA method based on the analysis of the information obtained during a single drag and drop task in combination with the accelerometer data. We evaluate the method by generating fake samples synthesized with Generative Adversarial Neural Networks and handcrafted methods. Our results suggest the potential of mobile sensors to characterize the human behaviour and develop a new generation of CAPTCHAs. The experiments are evaluated with HuMIdb (Human Mobile Interaction database), a novel multimodal mobile database collected for this work that comprises 14 mobile sensors acquired from 600 users. HuMIdb is freely available to the research community.

[1]  Mohamed Akrout,et al.  Hacking Google reCAPTCHA v3 using Reinforcement Learning , 2019, ArXiv.

[2]  Abdenour Hadid,et al.  Learning to Detect Genuine versus Posed Pain from Facial Expressions using Residual Generative Adversarial Networks , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[3]  Julian Fiérrez,et al.  MultiLock: Mobile Active Authentication based on Multiple Biometric and Behavioral Patterns , 2019, Advanced Sciences and Technologies for Security Applications.

[4]  Julian Fiérrez,et al.  Multiple classifiers in biometrics. Part 2: Trends and challenges , 2018, Inf. Fusion.

[5]  Kirsi Helkala,et al.  Biometric Gait Authentication Using Accelerometer Sensor , 2006, J. Comput..

[6]  Dawn Xiaodong Song,et al.  Touchalytics: On the Applicability of Touchscreen Input as a Behavioral Biometric for Continuous Authentication , 2012, IEEE Transactions on Information Forensics and Security.

[7]  John V. Monaco,et al.  The partially observable hidden Markov model and its application to keystroke dynamics , 2016, Pattern Recognit..

[8]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[9]  Michele Nappi,et al.  Have you permission to answer this phone? , 2018, 2018 International Workshop on Biometrics and Forensics (IWBF).

[10]  Sushil Jajodia,et al.  Who is tweeting on Twitter: human, bot, or cyborg? , 2010, ACSAC '10.

[11]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[12]  Patrick Bours,et al.  Studying WiFi and Accelerometer Data Based Authentication Method on Mobile Phones , 2018, ICBEA '18.

[13]  Lin Sun,et al.  Activity Recognition on an Accelerometer Embedded Mobile Phone with Varying Positions and Orientations , 2010, UIC.

[14]  Steven Gianvecchio,et al.  Bot or Human? A Behavior-Based Online Bot Detection System , 2018, From Database to Cyber Security.

[15]  Rama Chellappa,et al.  Continuous User Authentication on Mobile Devices: Recent progress and remaining challenges , 2016, IEEE Signal Processing Magazine.

[16]  Jun Yang,et al.  SenGuard: Passive user identification on smartphones using multiple sensors , 2011, 2011 IEEE 7th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob).

[17]  Ian Butterworth,et al.  Detection of Motor Impairment in Parkinson's Disease Via Mobile Touchscreen Typing , 2017, IEEE Transactions on Biomedical Engineering.

[18]  Victor C. M. Leung,et al.  EMC: Emotion-aware mobile cloud computing in 5G , 2015, IEEE Network.

[19]  Joo Han Oh,et al.  Within-day reliability of shoulder range of motion measurement with a smartphone. , 2012, Manual therapy.

[20]  Javier Hernandez-Ortega,et al.  Active detection of age groups based on touch interaction , 2018, IET Biom..

[21]  Dimitrios Hristu-Varsakelis,et al.  Smartphone-based evaluation of parkinsonian hand tremor: Quantitative measurements vs clinical assessment scores , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[22]  Konrad Paul Kording,et al.  Using Mobile Phones for Activity Recognition in Parkinson’s Patients , 2012, Front. Neur..

[23]  Katharina Krombholz,et al.  Sensor Captchas: On the Usability of Instrumenting Hardware Sensors to Prove Liveliness , 2016, TRUST.

[24]  Aythami Morales,et al.  Be-CAPTCHA: Detecting Human Behavior in Smartphone Interaction using Multiple Inbuilt Sensors , 2020, ArXiv.

[25]  Aythami Morales,et al.  Smartphone Sensors for Modeling Human-Computer Interaction: General Outlook and Research Datasets for User Authentication , 2020, 2020 IEEE 44th Annual Computers, Software, and Applications Conference (COMPSAC).

[26]  Julian Fierrez,et al.  Benchmarking Touchscreen Biometrics for Mobile Authentication , 2018, IEEE Transactions on Information Forensics and Security.

[27]  John C. Mitchell,et al.  Text-based CAPTCHA strengths and weaknesses , 2011, CCS '11.

[28]  Julian Fiérrez,et al.  MobileTouchDB: Mobile Touch Character Database in the Wild and Biometric Benchmark , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[29]  C. V. Jawahar,et al.  AutoRate: How attentive is the driver? , 2019, 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019).

[30]  Arun Ross,et al.  Actions Speak Louder Than (Pass)words: Passive Authentication of Smartphone* Users via Deep Temporal Features , 2019, 2019 International Conference on Biometrics (ICB).

[31]  Nan Jiang,et al.  A gesture-based CAPTCHA design supporting mobile devices , 2015, BCS HCI.

[32]  Yufei Chen,et al.  Multi-source Interactive Behavior Analysis for Continuous User Authentication on Smartphones , 2018, CCBR.

[33]  Susanne Fuchs-Winkelmann,et al.  Objective Assessment of shoulder mobility with a new 3D gyroscope - a validation study , 2011, BMC musculoskeletal disorders.

[34]  Dave Levin,et al.  unCaptcha: A Low-Resource Defeat of reCaptcha's Audio Challenge , 2017, WOOT.

[35]  Rama Chellappa,et al.  Active user authentication for smartphones: A challenge data set and benchmark results , 2016, 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[36]  Ruizhi Chen,et al.  Inquiry-Based Bluetooth Indoor Positioning via RSSI Probability Distributions , 2010, 2010 Second International Conference on Advances in Satellite and Space Communications.