Applying Psychometrics to Measure User Comfort when Constructing a Strong Password

As mobile devices become increasingly common for accessing services online, the security of these services in turn depends more on password entry on these devices. Unfortunately, users are not comfortable with existing textual password entry mechanisms on mobile phone handsets. In this study, we investigate this issue of user comfort from the viewpoint of psychometrics. By applying standard techniques of psychometrics, we develop a questionnaire (known as a scale in psychometrics) that measures the comfort of constructing a strong password when using a particular interface. We establish the essential psychometric properties (reliability and validity) of this scale and demonstrate how the scale can be used to profile password construction interfaces of popular smartphone handsets. We also theoretically conceptualize user comfort across different dimensions and use confirmatory factor analysis to verify our theory. Finally, we highlight several issues related to scale development and discuss how psychometric approaches may be useful in general for measuring various subjective concepts that are related to usable security.

[1]  M. Jakobsson.  Rethinking Passwords to Adapt to Constrained Keyboards , 2011 .

[2]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[3]  Sudhir Aggarwal,et al.  Testing metrics for password creation policies by attacking large sets of revealed passwords , 2010, CCS '10.

[4]  Lujo Bauer,et al.  Guess Again (and Again and Again): Measuring Password Strength by Simulating Password-Cracking Algorithms , 2012, 2012 IEEE Symposium on Security and Privacy.

[5]  Lujo Bauer,et al.  Of passwords and people: measuring the effect of password-composition policies , 2011, CHI.

[6]  John Hattie,et al.  Procedures for Assessing the Validities of Tests Using the "Known-Groups" Method , 1984 .

[7]  Gilbert A. Churchill.  A Paradigm for Developing Better Measures of Marketing Constructs , 1979 .

[8]  H. Kraemer,et al.  Summated Rating Scale Construction An Introduction , 2009 .

[9]  John R. Rossiter,et al.  Marketing measurement revolution: the C-OAR-SE method and why it must replace psychometrics , 2011 .

[10]  Kent L. Norman,et al.  Development of an instrument measuring user satisfaction of the human-computer interface , 1988, CHI '88.

[11]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[12]  L. Davis,et al.  Selection and use of content experts for instrument development. , 1997, Research in nursing & health.

[13]  N. McNamara,et al.  Defining usability: quality of use or quality of experience? , 2005, IPCC 2005. Proceedings. International Professional Communication Conference, 2005..

[14]  Florian Jentsch,et al.  The Play Experience Scale , 2012, Hum. Factors.

[15]  Elizabeth Schroeder.  Activity , 2005 .

[16]  Tonya L. Smith-Jackson,et al.  Reliability and validity of the mobile phone usability questionnaire (MPUQ) , 2006 .

[17]  Harry N. Boone,et al.  Analyzing Likert Data , 2012, Journal of Extension.

[18]  Darren George,et al.  SPSS for Windows Step by Step: A Simple Guide and Reference , 1998 .

[19]  Kirstie Hawkey,et al.  Heuristics for evaluating IT security management tools , 2011, SOUPS.

[20]  Mary Corbett,et al.  SUMI: the Software Usability Measurement Inventory , 1993, Br. J. Educ. Technol..

[21]  C. H. Lawshe.  A QUANTITATIVE APPROACH TO CONTENT VALIDITY , 1975 .

[22]  James Dean Brown,et al.  Testing in language programs , 1996 .

[23]  R. Guion,et al.  On Trinitarian doctrines of validity. , 1980 .

[24]  A. Parasuraman,et al.  SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality. , 1988 .

[25]  Markus Jakobsson,et al.  Implicit authentication for mobile devices , 2009 .

[26]  David C. Howell,et al.  Unidimensional Scaling , 2004 .

[27]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[28]  Melenie J. Lankau,et al.  Improving Construct Measurement In Management Research: Comments and a Quantitative Approach for Assessing the Theoretical Content Adequacy of Paper-and-Pencil Survey-Type Instruments , 1993 .

[29]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[30]  J. Doug Tygar,et al.  Why Johnny Can't Encrypt: A Usability Evaluation of PGP 5.0 , 1999, USENIX Security Symposium.

[31]  H. Kaiser.  The varimax criterion for analytic rotation in factor analysis , 1958 .

[32]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[33]  高田哲司,et al.  "Exploring the Design Space of Graphical Passwords on Smartphones"の紹介 , 2013 .

[34]  E. Ghiselli.  Theory of psychological measurement , 1964 .

[35]  Matthew K. Wright,et al.  Passwords and interfaces: towards creating stronger passwords by using mobile phone handsets , 2013, SPSM '13.

[36]  C. Spearman,et al.  "THE ABILITIES OF MAN". , 1928, Science.

[37]  J. Nunnally.  Psychometric Theory (2nd ed), New York: McGraw-Hill. , 1978 .

[38]  Sonia Chiasson,et al.  Improving user authentication on mobile devices: a touchscreen graphical password , 2013, MobileHCI '13.

[39]  Jeff Sauro,et al.  Correlations among prototypical usability metrics: evidence for the construct of usability , 2009, CHI.

[40]  A. N. Leont’ev,et al.  Activity, consciousness, and personality , 1978 .

[41]  J. Gliem,et al.  Calculating, Interpreting, And Reporting Cronbach’s Alpha Reliability Coefficient For Likert-Type Scales , 2003 .

[42]  Matthew Smith,et al.  On the ecological validity of a password study , 2013, SOUPS.

[43]  H. Kaiser.  An index of factorial simplicity , 1974 .