Measuring the Emergence of Consent Management on the Web

Privacy laws like the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) have pushed internet firms processing personal data to obtain user consent. Uncertainty around sanctions for non-compliance led many websites to embed a Consent Management Provider (CMP), which collects users' consent and shares it with third-party vendors and other websites. Our paper maps the formation of this ecosystem using longitudinal measurements. Primary and secondary data sources are used to measure each actor within the ecosystem. Using 161 million browser crawls, we estimate that CMP adoption doubled from June 2018 to June 2019 and then doubled again until June 2020. Sampling 4.2 million unique domains, we observe that CMP adoption is most prevalent among moderately popular websites (Tranco top 50-10k) but a long tail exists. Using APIs from the ad-tech industry, we quantify the purposes and lawful bases used to justify processing personal data. A controlled experiment on a public website provides novel insights into how the time-to-complete of two leading CMPs' consent dialogues varies with the preferences expressed, showing how privacy aware users incur a significant time cost.

[1]  J. Barlow.  A Declaration of the Independence of Cyberspace , 2021, Commonplace.

[2]  Jorge Lobo,et al.  A Survey of Privacy Policy Languages , 2007 .

[3]  Stefan Savage,et al.  Priceless: the role of payments in abuse-advertised goods , 2012, CCS.

[4]  Kai-Lung Hui,et al.  Internet opt-in and opt-out: investigating the roles of frames, defaults and privacy concerns , 2006, SIGMIS CPR '06.

[5]  Jonathan Grudin,et al.  A study of preferences for sharing and privacy , 2005, CHI Extended Abstracts.

[6]  Martino Trevisan,et al.  Uncovering the Flop of the EU Cookie Law , 2017, ArXiv.

[7]  Denis Regaud.  Commission Nationale de l'Informatique et des Libertés , 2016 .

[8]  David M. Kristol,et al.  HTTP Cookies: Standards, privacy, and politics , 2001, TOIT.

[9]  Balachander Krishnamurthy,et al.  Best paper -- Follow the money: understanding economics of online aggregation and advertising , 2013, Internet Measurement Conference.

[10]  Lorrie Faith Cranor,et al.  You've been warned: an empirical study of the effectiveness of web browser phishing warnings , 2008, CHI.

[11]  Cristiana Santos,et al.  Do Cookie Banners Respect my Choice? : Measuring Legal Compliance of Banners from IAB Europe’s Transparency and Consent Framework , 2020, 2020 IEEE Symposium on Security and Privacy (SP).

[12]  Pablo Rodriguez,et al.  If you are not paying for it, you are the product: how much do advertisers pay to reach you? , 2017, Internet Measurement Conference.

[13]  Alessandro Acquisti,et al.  The challenges of personal data markets and privacy , 2015, Electronic Markets.

[14]  Gianluca Stringhini,et al.  The web centipede: understanding how web communities influence each other through the lens of mainstream and alternative news sources , 2017, Internet Measurement Conference.

[15]  Bonnie Brinton Anderson,et al.  The Fog of Warnings: How Non-essential Notifications Blur with Security Warnings , 2019, SOUPS @ USENIX Security Symposium.

[16]  Wouter Joosen,et al.  Herding Vulnerable Cats: A Statistical Approach to Disentangle Joint Responsibility for Web Security in Shared Hosting , 2017, CCS.

[17]  Philipp Winter,et al.  The Impact of User Location on Cookie Notices (Inside and Outside of the European Union) , 2019 .

[18]  Guang-xin Xie,et al.  Perceived Privacy Violation: Exploring the Malleability of Privacy Expectations , 2019 .

[19]  Martino Trevisan,et al.  4 Years of EU Cookie Law: Results and Lessons Learned , 2017, Proc. Priv. Enhancing Technol..

[20]  Narseo Vallina-Rodriguez,et al.  An Analysis of the Privacy and Security Risks of Android VPN Permission-enabled Apps , 2016, Internet Measurement Conference.

[21]  Wouter Joosen,et al.  Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation , 2018, NDSS.

[22]  Gianluca Stringhini,et al.  Screenshot Classifier annotated images pHashes of non-screenshot annotated images Know Your Meme Generic Annotation Sites Meme Annotation Sites Generic Web Communities , 2018 .

[23]  Martin Degeling,et al.  (Un)informed Consent: Studying GDPR Consent Notices in the Field , 2019, CCS.

[24]  Lorrie Faith Cranor,et al.  An Empirical Analysis of Data Deletion and Opt-Out Choices on 150 Websites , 2019, SOUPS @ USENIX Security Symposium.

[25]  Aleecia M. McDonald,et al.  The Cost of Reading Privacy Policies , 2009 .

[26]  Narseo Vallina-Rodriguez,et al.  An Empirical Analysis of the Commercial VPN Ecosystem , 2018, Internet Measurement Conference.

[27]  Anja Feldmann,et al.  Annoyed Users: Ads and Ad-Block Usage in the Wild , 2015, Internet Measurement Conference.

[28]  Martin Degeling,et al.  We Value Your Privacy ... Now Take Some Cookies , 2018, Informatik Spektrum.

[29]  Rainer Böhme,et al.  Multiple Purposes, Multiple Problems: A User Study of Consent Dialogs after GDPR , 2019, Proc. Priv. Enhancing Technol..

[30]  Batya Friedman,et al.  Cookies and Web browser design: toward realizing informed consent online , 2001, CHI.

[31]  John C. Mitchell,et al.  Third-Party Web Tracking: Policy and Technology , 2012, 2012 IEEE Symposium on Security and Privacy.

[32]  Daniel Zappala,et al.  Condensing Steam: Distilling the Diversity of Gamer Behavior , 2016, Internet Measurement Conference.

[33]  Thorsten Holz,et al.  We Value Your Privacy ... Now Take Some Cookies: Measuring the GDPR's Impact on Web Privacy , 2019, NDSS.

[34]  Andrew C. Simpson,et al.  Rethinking the Proposition of Privacy Engineering , 2018, NSPW '18.

[35]  Daniel W. Woods,et al.  The commodification of consent , 2022, Comput. Secur..

[36]  Elizabeth Harding,et al.  Understanding the scope and impact of the California Consumer Privacy Act of 2018 , 2019, Journal of Data Protection & Privacy.

[37]  Yang Wang,et al.  Smart, useful, scary, creepy: perceptions of online behavioral advertising , 2012, SOUPS.

[38]  Midas Nouwens,et al.  Dark Patterns after the GDPR: Scraping Consent Pop-ups and Demonstrating their Influence , 2020, CHI.

[39]  William Enck,et al.  Cardpliance: PCI DSS Compliance of Android Applications , 2020, USENIX Security Symposium.

[40]  Lorrie Faith Cranor,et al.  P3P: Making Privacy Policies More Useful , 2003, IEEE Secur. Priv..

[41]  Florian Cech,et al.  A Human-Centric Perspective on Digital Consenting: The Case of GAFAM , 2020, KES-HCIS.

[42]  Evangelos P. Markatos,et al.  No More Chasing Waterfalls: A Measurement Study of the Header Bidding Ad-Ecosystem , 2019, Internet Measurement Conference.

[43]  Narseo Vallina-Rodriguez,et al.  Tales from the Porn: A Comprehensive Privacy Analysis of the Web Porn Ecosystem , 2019, Internet Measurement Conference.

[44]  Leyla Bilge,et al.  Can I Opt Out Yet?: GDPR and the Global Illusion of Cookie Control , 2019, AsiaCCS.

[45]  Rainer Böhme,et al.  Trained to accept?: a field experiment on consent dialogs , 2010, CHI.

[46]  Tyler Moore,et al.  Concentrating Correctly on Cybercrime Concentration , 2015, WEIS.

[47]  Tarleton Gillespie,et al.  The politics of ‘platforms’ , 2010, New Media Soc..

[48]  Arvind Narayanan,et al.  The Web Never Forgets: Persistent Tracking Mechanisms in the Wild , 2014, CCS.

[49]  Norbert Pohlmann,et al.  Beyond the Front Page:Measuring Third Party Dynamics in the Field , 2020, WWW.

[50]  Narseo Vallina-Rodriguez,et al.  A Long Way to the Top: Significance, Structure, and Stability of Internet Top Lists , 2018, Internet Measurement Conference.

[51]  Gildas Avoine,et al.  Browser Fingerprinting , 2020, ACM Trans. Web.

[52]  Sokol Kosta,et al.  Before and After GDPR: The Changes in Third Party Presence at Public and Private European Websites , 2019, WWW.

[53]  Cristiana Santos,et al.  Purposes in IAB Europe's TCF: Which Legal Basis and How Are They Used by Advertisers? , 2020, APF.

[54]  Danfeng Yao,et al.  Security Certification in Payment Card Industry: Testbeds, Measurements, and Recommendations , 2019, CCS.

[55]  Nikolaos Laoutaris,et al.  Tracing Cross Border Web Tracking , 2018, Internet Measurement Conference.

[56]  Joseph Bonneau,et al.  The Privacy Jungle: On the Market for Data Protection in Social Networks , 2009, WEIS.

[57]  Laura A. Dabbish,et al.  "My Data Just Goes Everywhere: " User Mental Models of the Internet and Implications for Privacy and Security , 2015, SOUPS.

[58]  John Sören Pettersson,et al.  The Dilemma of User Engagement in Privacy Notices , 2020, ACM Trans. Priv. Secur..

[59]  Wouter Joosen,et al.  Cookieless Monster: Exploring the Ecosystem of Web-Based Device Fingerprinting , 2013, 2013 IEEE Symposium on Security and Privacy.

[60]  Hana Ross.  Data subject consent: How will the General Data Protection Regulation affect this? , 2017 .

[61]  Alessandro Acquisti,et al.  Sleights of privacy: framing, disclosures, and the limits of transparency , 2013, SOUPS.

[62]  Oliver Günther,et al.  Privacy in e-commerce: stated preferences vs. actual behavior , 2005, CACM.