Deploying authentication in the wild: towards greater ecological validity in security usability studies

Pico is a token-based login method that claims to be simultaneously more usable and more secure than passwords. It does not ask users to remember any secrets, nor to type one-time passwords. We evaluate Pico’s claim with two deployments and user studies, one on a web-based service and another within an organisation. Our main aim is to collect actionable intelligence on how to improve the usability and deployability of Pico. In our first study we team up with an established website, Gyazo, to offer this alternative login mechanism to users intent on performing a real task of image sharing. From the lessons of this first study we retarget Pico’s focus from replacing web passwords to replacing desktop login passwords; and thus in our second study we engage with a government organisation, Innovate UK, to offer employees the ability to lock and unlock their computer automatically based on proximity. We focus particularly on the ecological validity of the trials and we thereby gain valuable insights into the viability of Pico, not just through the actual responses from the participants but also through the many practical challenges we had to face and overcome. Reflecting on the bigger picture, from our experience we believe the security usability community would greatly benefit from pushing towards greater ecological validity in published work, despite the considerable difficulties and costs involved. ‡Authors listed in alphabetical order. All authors contributed in some form to the Gyazo study while only authors Aebischer, Dettoni, Krol, Llewellyn-Jones and Stajano contributed to the Innovate UK study. Author Jenkinson designed and coded the reverse proxy used in the Gyazo study but left the Pico project before the study started. Author Masui, inventor and CTO of Gyazo, visited the other authors in Cambridge several times for a total of 6 months in connection with the Gyazo study. Authors Aebischer, Dettoni, Krol and Llewellyn-Jones are no longer at Cambridge but were while they carried out the research described in this paper. The Principal Investigator was Stajano. Corresponding author: Frank Stajano, University of Cambridge, Department of Computer Science and Technology, 15 JJ Thomson Avenue, Cambridge, CB3 0FD, United Kingdom. +44-1223-763-500. Invited submission to the special issue on Usable Security and Privacy of the Journal of Cybersecurity. Original manuscript submitted on 2019-02-19. Major revision requested on 2019-10-19. Revised manuscript submitted on 2020-01-14 and accepted on 2020-05-19. This is the authors’ preprint, dated 2020-05-27, before copyediting and typesetting by the publisher.

[1]  Frank Stajano,et al.  Responsibility and Tangible Security: Towards a Theory of User Acceptance of Security Tokens , 2016, ArXiv.

[2]  B. Everitt,et al.  Statistical methods for rates and proportions , 1973 .

[3]  Kat Krol,et al.  Towards Robust Experimental Design for User Studies in Security and Privacy , 2016 .

[4]  Mikael Linden,et al.  An Empirical Study on the Usability of Logout in a Single Sign-on System , 2005, ISPEC.

[5]  Carl E. Landwehr,et al.  Protecting unattended computers without software , 1997, Proceedings 13th Annual Computer Security Applications Conference.

[6]  Scott Ruoti,et al.  Authentication Melee: A Usability Analysis of Seven Web Authentication Systems , 2015, WWW.

[7]  Brian D. Noble,et al.  Mobile Device Security Using Transient Authentication , 2006, IEEE Transactions on Mobile Computing.

[8]  Brian D. Noble,et al.  Zero-interaction authentication , 2002, MobiCom '02.

[9]  Frank Stajano,et al.  Pico in the Wild: Replacing Passwords, One Site at a Time , 2017 .

[10]  Frank Stajano,et al.  I Bought a New Security Token and All I Got Was This Lousy Phish - Relay Attacks on Visual Code Authentication Schemes , 2014, Security Protocols Workshop.

[11]  Dennis Strouble,et al.  PRODUCTIVITY AND USABILITY EFFECTS OF USING A TWO-FACTOR SECURITY SYSTEM , 2009 .

[12]  Kat Krol,et al.  Better the Devil You Know: A User Study of Two CAPTCHAs and a Possible Replacement , 2016 .

[13]  Kevin Crowston,et al.  Amazon Mechanical Turk: A Research Tool for Organizations and Information Systems Scholars , 2012, Shaping the Future of ICT Research.

[14]  M. Angela Sasse,et al.  Evaluating the usability and security of a graphical one-time PIN system , 2010, BCS HCI.

[15]  Bruce Christianson,et al.  Pico Without Public Keys , 2015, Security Protocols Workshop.

[16]  M. Angela Sasse,et al.  Are Passfaces More Usable Than Passwords? A Field Trial Investigation , 2000, BCS HCI.

[17]  Kat Krol,et al.  Report: Authentication Diary Study , 2014 .

[18]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[19]  Andy Hopper,et al.  Active badges and personal interactive computing objects , 1992 .

[20]  Hsing Ping Fu,et al.  Pico: No More Passwords! , 2013 .

[21]  Lujo Bauer,et al.  “It's not actually that horrible”: Exploring Adoption of Two-Factor Authentication at a University , 2018, CHI.

[22]  Robert Biddle,et al.  Graphical passwords: Learning from the first twelve years , 2012, CSUR.

[23]  Frank Stajano,et al.  Password-Manager Friendly (PMF): Semantic Annotations to Improve the Effectiveness of Password Managers , 2014, PASSWORDS.

[24]  Chris J. Mitchell,et al.  A Taxonomy of Single Sign-On Systems , 2003, ACISP.

[25]  Cormac Herley,et al.  Where do security policies come from? , 2010, SOUPS.

[26]  Michael K. Reiter,et al.  On User Choice in Graphical Password Schemes , 2004, USENIX Security Symposium.

[27]  Frank Stajano,et al.  The Quest to Replace Passwords: A Framework for Comparative Evaluation of Web Authentication Schemes , 2012, 2012 IEEE Symposium on Security and Privacy.

[28]  Sunny Consolvo,et al.  Experimenting at scale with google chrome's SSL warning , 2014, CHI.

[29]  尚弘 島影.  National Institute of Standards and Technologyにおける超伝導研究及び生活 , 2001 .

[30]  Frank Stajano,et al.  Bootstrapping Adoption of the Pico Password Replacement System , 2014, Security Protocols Workshop.

[31]  M. Angela Sasse,et al.  The true cost of unusable password policies: password use in the wild , 2010, CHI.

[32]  Min Wu,et al.  Do security toolbars actually prevent phishing attacks? , 2006, CHI.

[33]  Joseph Bonneau,et al.  Towards Reliable Storage of 56-bit Secrets in Human Memory , 2014, USENIX Security Symposium.

[34]  Kat Krol,et al.  "Too Taxing on the Mind!" Authentication Grids are not for Everyone , 2015, HCI.

[35]  Ma Sasse,et al.  Gathering realistic authentication performance data through field trials , 2010, SOUPS 2010.

[36]  Ken Thompson,et al.  Password security: a case history , 1979, CACM.

[37]  Hugo Krawczyk,et al.  SIGMA: The 'SIGn-and-MAc' Approach to Authenticated Diffie-Hellman and Its Use in the IKE-Protocols , 2003, CRYPTO.

[38]  Emiliano De Cristofaro,et al.  "They brought in the horrible key ring thing!" Analysing the Usability of Two-Factor Authentication in UK Online Banking , 2015, ArXiv.

[39]  Claudio Carpineto,et al.  A Survey of Automatic Query Expansion in Information Retrieval , 2012, CSUR.

[40]  Frank Stajano,et al.  To have and have not: variations on secret sharing to model user presence , 2014, UbiComp Adjunct.