A usability test of whitelist and blacklist-based anti-phishing application

Anti-phishing tools on a web browser warn about spoofing pages or/and prompt to essential and necessary information that assists users to identify spoofing and potentially harmful pages. In order to discover how well users can identify phishing pages with these tools after they understand how the tools work, we designed and conducted usability tests for two detection mechanisms of anti-phishing tools: the blacklist-based and whitelist-based anti-phishing toolbars. As a result, we report that no significant performance differences between the blacklist-based and whitelist-based applications were found; but some other interesting findings and observations were collected. The most valuable observation is that due to the deficiency of existing web identities on the web pages and web browsers, e.g. abstract and professional web page security certificate information, anti-phishing toolbars need to be more illustrative and instructional in order to assist users to find reliable information for identifying the authenticity of the content on the web pages.

[1]  Min Wu,et al.  Web wallet: preventing phishing attacks by revealing user intentions , 2006, SOUPS '06.

[2]  José Carlos Brustoloni,et al.  Using reinforcement to strengthen users' secure behaviors , 2010, CHI.

[3]  Carol M. Barnum The ‘magic number 5’: Is it enough for web-testing? , 2003 .

[4]  Gitte Lindgaard,et al.  Usability testing: what have we overlooked? , 2007, CHI.

[5]  Markus Jakobsson,et al.  Designing ethical phishing experiments , 2007, IEEE Technology and Society Magazine.

[6]  Markus Jakobsson,et al.  Designing ethical phishing experiments: a study of (ROT13) rOnl query features , 2006, WWW '06.

[7]  David Ma,et al.  Does domain highlighting help people identify phishing sites? , 2011, CHI.

[8]  Ponnurangam Kumaraguru,et al.  Who falls for phish?: a demographic analysis of phishing susceptibility and effectiveness of interventions , 2010, CHI.

[9]  Heinrich Hußmann,et al.  Does MoodyBoard make internet use more secure?: evaluating an ambient security visualization tool , 2011, CHI.

[10]  Min Wu,et al.  Do security toolbars actually prevent phishing attacks? , 2006, CHI.

[11]  Lorrie Faith Cranor,et al.  You've been warned: an empirical study of the effectiveness of web browser phishing warnings , 2008, CHI.

[12]  L. Faulkner Beyond the five-user assumption: Benefits of increased sample sizes in usability testing , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[13]  Linfeng Li,et al.  Usability evaluation of anti-phishing toolbars , 2007, Journal in Computer Virology.

[14]  Jakob Nielsen,et al.  The "magic number 5": is it enough for web testing? , 2002, CHI Extended Abstracts.

[15]  John A. Clark,et al.  F for fake: four studies on how we fall for phish , 2011, CHI.

[16]  Lorrie Faith Cranor,et al.  Phinding Phish: An Evaluation of Anti-Phishing Toolbars , 2007, NDSS.

[17]  Jason Hong,et al.  The state of phishing attacks , 2012, Commun. ACM.

[18]  Avivah Litan Phishing Attack Victims Likely Targets for Identity Theft , 2005 .