Designing for the better by taking users into account: a qualitative evaluation of user control mechanisms in (news) recommender systems

Recommender systems (RS) are on the rise in many domains. While they offer great promises, they also raise concerns: lack of transparency, reduction of diversity, little to no user control. In this paper, we align with the normative turn in computer science which scrutinizes the ethical and societal implications of RS. We focus and elaborate on the concept of user control because that mitigates multiple problems at once. Taking the news industry as our domain, we conducted four focus groups, or moderated think-aloud sessions, with Dutch news readers (N=21) to systematically study how people evaluate different control mechanisms (at the input, process, and output phase) in a News Recommender Prototype (NRP). While these mechanisms are sometimes met with distrust about the actual control they offer, we found that an intelligible user profile (including reading history and flexible preferences settings), coupled with possibilities to influence the recommendation algorithms is highly valued, especially when these control mechanisms can be operated in relation to achieving personal goals. By bringing (future) users' perspectives to the fore, this paper contributes to a richer understanding of why and how to design for user control in recommender systems.

[1]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[2]  Bart P. Knijnenburg,et al.  Each to his own: how different users call for different interaction methods in recommender systems , 2011, RecSys '11.

[3]  Li Chen,et al.  Human Decision Making and Recommender Systems , 2015, Recommender Systems Handbook.

[4]  Gediminas Adomavicius,et al.  Context-aware recommender systems , 2008, RecSys '08.

[5]  Peter Brusilovsky,et al.  User-controllable personalization: A case study with SetFusion , 2015, Int. J. Hum. Comput. Stud..

[6]  Li Chen,et al.  Evaluating recommender systems from the user’s perspective: survey of the state of the art , 2012, User Modeling and User-Adapted Interaction.

[7]  Judith Masthoff,et al.  Explaining Recommendations: Design and Evaluation , 2015, Recommender Systems Handbook.

[8]  Bart P. Knijnenburg,et al.  Recommender Systems for Self-Actualization , 2016, RecSys.

[9]  Tobias Höllerer,et al.  TasteWeights: a visual interactive hybrid recommender system , 2012, RecSys.

[10]  Katrien Verbert,et al.  Effects of personal characteristics on music recommender systems with different levels of controllability , 2018, RecSys.

[11]  Jenna Burrell,et al.  How the machine ‘thinks’: Understanding opacity in machine learning algorithms , 2016 .

[12]  Tobias Höllerer,et al.  Hypothetical Recommendation: A Study of Interactive Profile Manipulation Behavior for Recommender Systems , 2015, FLAIRS.

[13]  Nava Tintarev,et al.  Reading News with a Purpose: Explaining User Profiles for Self-Actualization , 2019, UMAP.

[14]  Douglas Schuler,et al.  Participatory Design: Principles and Practices , 1993 .

[15]  Sean M. McNee,et al.  Making recommendations better: an analytic model for human-recommender interaction , 2006, CHI Extended Abstracts.

[16]  Kartik Hosanagar,et al.  Blockbuster Culture's Next Rise or Fall: The Impact of Recommender Systems on Sales Diversity , 2007, Manag. Sci..

[17]  Cory Maloney.  Review of "Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy" by Cathy O’Neil , 2017 .

[18]  Dietmar Jannach,et al.  News recommender systems - Survey and roads ahead , 2018, Inf. Process. Manag..

[19]  N. Helberger.  On the Democratic Role of News Recommenders , 2019, Algorithms, Automation, and News.

[20]  Saul Vargas,et al.  Novelty and Diversity in Recommender Systems , 2015, Recommender Systems Handbook.

[21]  Eli Pariser.  FILTER BUBBLE: Wie wir im Internet entmündigt werden , 2012 .

[22]  Nicholas Diakopoulos,et al.  Accountability in algorithmic decision making , 2016, Commun. ACM.

[23]  Eli Pariser,et al.  The Filter Bubble: What the Internet Is Hiding from You , 2011 .

[24]  Dietmar Jannach,et al.  User Control in Recommender Systems: Overview and Interaction Challenges , 2016, EC-Web.

[25]  Natali Helberger,et al.  Democratizing algorithmic news recommenders: how to materialize voice in a technologically saturated media ecosystem , 2018, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[26]  Shuaiqiang Wang,et al.  A survey of serendipity in recommender systems , 2016, Knowl. Based Syst..

[27]  Nigel Shadbolt,et al.  "So, Tell Me What Users Want, What They Really, Really Want!" , 2018, CHI Extended Abstracts.

[28]  Rashmi R. Sinha,et al.  The role of transparency in recommender systems , 2002, CHI Extended Abstracts.

[29]  John Riedl,et al.  Recommender systems: from algorithms to user experience , 2012, User Modeling and User-Adapted Interaction.

[30]  Andreas Graefe,et al.  Burst of the Filter Bubble? , 2018 .

[31]  R. Kitchin,et al.  Thinking critically about and researching algorithms , 2014, The Social Power of Algorithms.

[32]  Natali Helberger,et al.  Challenged by news personalisation: five perspectives on the right to receive information , 2017 .

[33]  Tony Doyle,et al.  Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy , 2017, Inf. Soc..

[34]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[35]  Lucia D'Acunto,et al.  Exposure diversity as a design principle for recommender systems , 2018 .

[36]  Jaron Lanier,et al.  You Are Not a Gadget: A Manifesto , 2010 .

[37]  Dennis R. Wixon,et al.  CHI '02 Extended Abstracts on Human Factors in Computing Systems , 2002, CHI 2002.

[38]  Harmanpreet Kaur,et al.  Putting Users in Control of their Recommendations , 2015, RecSys.

[39]  Jin Ha Lee,et al.  Understanding Users of Commercial Music Services through Personas: Design Implications , 2015, ISMIR.

[40]  Glen Weyl,et al.  Radical Markets: Uprooting Capitalism and Democracy for a Just Society , 2018 .

[41]  J. Christman,et al.  Relational Autonomy, Liberal Individualism, and the Social Constitution of Selves , 2004 .

[42]  Katrien Verbert,et al.  IntersectionExplorer, a multi-perspective approach for exploring recommendations , 2019, Int. J. Hum. Comput. Stud..

[43]  Dietmar Jannach,et al.  Interacting with Recommenders—Overview and Research Directions , 2017, ACM Trans. Interact. Intell. Syst..

[44]  Sara Leckner.  Presentation factors affecting reading behaviour in readers of newspaper media: an eye-tracking perspective , 2012 .

[45]  Katrien Verbert,et al.  Interactive recommender systems: A survey of the state of the art and future research challenges and opportunities , 2016, Expert Syst. Appl..

[46]  Malte Ziewitz.  Governing Algorithms , 2016 .

[47]  D. Boyd,et al.  CRITICAL QUESTIONS FOR BIG DATA , 2012 .

[48]  Martijn C. Willemsen,et al.  Behaviorism is Not Enough: Better Recommendations through Listening to Users , 2016, RecSys.

[49]  F. Maxwell Harper,et al.  Letting Users Choose Recommender Algorithms: An Experimental Study , 2015, RecSys.

[50]  E. Brink,et al.  Constructing grounded theory : A practical guide through qualitative analysis , 2006 .

[51]  D. Morgan.  Focus groups for qualitative research. , 1988, Hospital guest relations report.

[52]  Derek Bridge,et al.  Diversity, Serendipity, Novelty, and Coverage , 2016, ACM Trans. Interact. Intell. Syst..