Put Your Warning Where Your Link Is: Improving and Evaluating Email Phishing Warnings

Phishing emails often disguise a link's actual URL. Thus, common anti-phishing advice is to check a link's URL before clicking, but email clients do not support this well. Automated phishing detection enables email clients to warn users that an email is suspicious, but current warnings are often not specific. We evaluated the effects on phishing susceptibility of (1) moving phishing warnings close to the suspicious link in the email, (2) displaying the warning on hover interactions with the link, and (3) forcing attention to the warning by deactivating the original link, forcing users to click the URL in the warning. We assessed the effectiveness of such link-focused phishing warning designs in a between-subjects online experiment (n=701). We found that link-focused phishing warnings reduced phishing click-through rates compared to email banner warnings; forced attention warnings were most effective. We discuss the implications of our findings for phishing warning design.

[1]  Adrienne Porter Felt,et al.  Alice in Warningland: A Large-Scale Field Study of Browser Security Warning Effectiveness , 2013, USENIX Security Symposium.

[2]  Lorrie Faith Cranor,et al.  Necessary But Not Sufficient: Standardized Mechanisms for Privacy Notice and Choice , 2012, J. Telecommun. High Technol. Law.

[3]  Ninghui Li,et al.  Effectiveness of a phishing warning in field settings , 2015, HotSoS.

[4]  Gang Liu,et al.  Smartening the crowds: computational techniques for improving human verification to fight phishing scams , 2011, SOUPS.

[5]  Melanie Volkamer,et al.  Spot the phish by checking the pruned URL , 2016, Inf. Comput. Secur..

[6]  Lorrie Faith Cranor,et al.  Harder to Ignore? Revisiting Pop-Up Fatigue and Approaches to Prevent It , 2014, SOUPS.

[7]  Alexandra Kunz,et al.  User experiences of TORPEDO: TOoltip-poweRed Phishing Email DetectiOn , 2017, Comput. Secur..

[8]  Lujo Bauer,et al.  Warning Design Guidelines (CMU-CyLab-13-002) , 2013 .

[9]  Marti A. Hearst,et al.  Why phishing works , 2006, CHI.

[10]  Lorrie Faith Cranor,et al.  Anti-Phishing Phil: the design and evaluation of a game that teaches people not to fall for phish , 2007, SOUPS '07.

[11]  Jason Hong,et al.  The state of phishing attacks , 2012, Commun. ACM.

[12]  David A. Hoffman,et al.  Law and Psychology Grows Up, Goes Online, and Replicates , 2017 .

[13]  Stuart E. Schechter,et al.  The Emperor's New Security Indicators , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[14]  Marie-Francine Moens,et al.  New filtering approaches for phishing email , 2010, J. Comput. Secur..

[15]  Sunny Consolvo,et al.  Improving SSL Warnings: Comprehension and Adherence , 2015, CHI.

[16]  Alessandro Acquisti,et al.  The Effect of Online Privacy Information on Purchasing Behavior: An Experimental Study , 2011, WEIS.

[17]  Norman M. Sadeh,et al.  Learning to detect phishing emails , 2007, WWW '07.

[18]  Samuel Marchal,et al.  Know Your Phish: Novel Techniques for Detecting Phishing Sites and Their Targets , 2015, 2016 IEEE 36th International Conference on Distributed Computing Systems (ICDCS).

[19]  Lorrie Faith Cranor,et al.  Your attention please: designing security-decision UIs to make genuine risks harder to ignore , 2013, SOUPS.

[20]  Lorrie Faith Cranor,et al.  Your Location has been Shared 5,398 Times!: A Field Study on Mobile App Privacy Nudging , 2015, CHI.

[21]  Tonya L Smith-Jackson,et al.  Research-based guidelines for warning design and evaluation. , 2002, Applied ergonomics.

[22]  Michael S. Wogalter,et al.  Habituation, Dishabituation, and Recovery Effects in Visual Warnings , 2009 .

[23]  Lorrie Faith Cranor,et al.  School of phish: a real-world evaluation of anti-phishing training , 2009, SOUPS.

[24]  Pieter H. Hartel,et al.  How Effective is Anti-Phishing Training for Children? , 2017, SOUPS.

[25]  Randolph G. Bias,et al.  Research Methods for Human-Computer Interaction , 2010, J. Assoc. Inf. Sci. Technol..

[26]  Lorrie Faith Cranor,et al.  Behavioral response to phishing risk , 2007, eCrime '07.

[27]  David Ma,et al.  Does domain highlighting help people identify phishing sites? , 2011, CHI.

[28]  Brian Ryner,et al.  Large-Scale Automatic Classification of Phishing Pages , 2010, NDSS.

[29]  Simson L. Garfinkel,et al.  Usable Security: History, Themes, and Challenges , 2014, Usable Security: History, Themes, and Challenges.

[30]  José Carlos Brustoloni,et al.  Improving security decisions with polymorphic and audited dialogs , 2007, SOUPS '07.

[31]  Lorrie Faith Cranor,et al.  Teaching Johnny not to fall for phish , 2010, TOIT.

[32]  Sunny Consolvo,et al.  Experimenting at scale with google chrome's SSL warning , 2014, CHI.

[33]  Gang Liu,et al.  Discovering phishing target based on semantic link network , 2010 .

[34]  Max L. Wilson,et al.  Brain activity and mental workload associated with artistic practice , 2018 .

[35]  Lorrie Faith Cranor,et al.  A Design Space for Effective Privacy Notices , 2015, SOUPS.

[36]  Sameer Patil,et al.  Reflection or action?: how feedback and control affect location sharing decisions , 2014, CHI.

[37]  Carolyn Penstein Rosé,et al.  CANTINA+: A Feature-Rich Machine Learning Framework for Detecting Phishing Web Sites , 2011, TSEC.

[38]  Rick Wash,et al.  Who Provides Phishing Training?: Facts, Stories, and People Like Me , 2018, CHI.

[39]  Lorrie Faith Cranor,et al.  You've been warned: an empirical study of the effectiveness of web browser phishing warnings , 2008, CHI.

[40]  Lorrie Faith Cranor,et al.  Bridging the Gap in Computer Security Warnings: A Mental Model Approach , 2011, IEEE Security & Privacy.

[41]  Sameer Patil,et al.  Interrupt Now or Inform Later?: Comparing Immediate and Delayed Privacy Feedback , 2015, CHI.

[42]  Matthew Smith,et al.  Using personal examples to improve risk communication for security & privacy decisions , 2014, CHI.

[43]  Cristian Bravo-Lillo.  Improving Computer Security Dialogs: An Exploration of Attention and Habituation , 2014 .

[44]  Samuel Marchal,et al.  Off-the-Hook: An Efficient and Usable Client-Side Phishing Prevention Application , 2017, IEEE Transactions on Computers.

[45]  Lorrie Faith Cranor,et al.  Crying Wolf: An Empirical Study of SSL Warning Effectiveness , 2009, USENIX Security Symposium.

[46]  Lorrie Faith Cranor,et al.  A "nutrition label" for privacy , 2009, SOUPS.

[47]  Lorrie Faith Cranor,et al.  Cantina: a content-based approach to detecting phishing web sites , 2007, WWW '07.

[48]  Yang Wang,et al.  Nudges for Privacy and Security , 2017, ACM Comput. Surv..

[49]  Bonnie Brinton Anderson,et al.  How Polymorphic Warnings Reduce Habituation in the Brain: Insights from an fMRI Study , 2015, CHI.

[50]  Robert E. Kraut,et al.  Should I open this email?: inbox-level cues, curiosity and attention to email , 2011, CHI.