The ability of personas: An empirical evaluation of altering incorrect preconceptions about users

Abstract False preconceptions about users can result in poor design, product development, and marketing decisions, so rectifying these preconceptions is essential for organizations. This research quantitatively evaluates the ability of data-driven personas to alter decision makers’ preconceptions about their online social media users. We conduct a within-participant experiment of 31 professionals carrying out a workplace task scenario. The participants’ conceptions of user attributes are recorded both before and after interacting with personas created from real user analytics data. Using statistical analysis to compare the responses, we find that personas had a significant effect on changing preconceptions about the audience segments. After interacting with personas, 81% of the participants changed their preconceptions of the audience, and 94% of the participants maintained or increased the accuracy of their perceptions of the audience after engaging with the personas. Moreover, the confidence of the participants in their responses increased. However, two participants did not change their preconceptions even when faced with contradictory factual information, which highlights the need for accompanying initiatives to align user preconceptions with user data fully.

[1]  Tamara Sumner,et al.  A latent semantic analysis methodology for the identification and creation of personas , 2008, CHI.

[2]  Mohamed Aboelmaged,et al.  Influencing models and determinants in big data analytics research: A bibliometric analysis , 2020, Inf. Process. Manag..

[3]  Chris Chapman,et al.  The Personas' New Clothes: Methodological and Practical Arguments against a Popular Method , 2006 .

[4]  Pam J. Mayhew,et al.  To Intervene or Not to Intervene:An Investigation of Three Think-Aloud Protocols in Usability Testing , 2017 .

[5]  Jonathan Grudin,et al.  Personas: practice and theory , 2003, DUX '03.

[6]  Lee Sproull,et al.  When the Interface Is a Face , 1996, Hum. Comput. Interact..

[7]  Elena Paslaru Bontas Simperl,et al.  Everything you always wanted to know about a dataset: studies in data summarisation , 2018, Int. J. Hum. Comput. Stud..

[8]  Tamara Sumner,et al.  Using Personas to Understand the Needs and Goals of Institutional Repositories , 2008, D Lib Mag..

[9]  Bernard J. Jansen,et al.  Are personas done? Evaluating their usefulness in the age of digital analytics , 2018, Persona Studies.

[10]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[11]  Wendell R. Smith.  Product Differentiation and Market Segmentation as Alternative Marketing Strategies , 1956 .

[12]  Robert Dingwall,et al.  The SAGE handbook of research management , 2015 .

[13]  Nicola Marsden,et al.  Personas and Identity: Looking at Multiple Identities to Inform the Construction of Personas , 2019, CHI.

[14]  George Demiris,et al.  Using scenarios and personas to enhance the effectiveness of heuristic usability evaluations for older adults and their care team , 2017, J. Biomed. Informatics.

[15]  Gianni Fenu,et al.  Connecting User and Item Perspectives in Popularity Debiasing for Collaborative Recommendation , 2021, Inf. Process. Manag..

[16]  Walter Schneider,et al.  Controlled and Automatic Human Information Processing: 1. Detection, Search, and Attention. , 1977 .

[17]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[18]  Cindy LeRouge,et al.  Introducing User Profiles and Personas into Information Systems Development , 2007, AMCIS.

[19]  John S. Pruitt,et al.  The Persona Lifecycle: Keeping People in Mind Throughout Product Design , 2006 .

[20]  Joseph Giacomin,et al.  What Is Human Centred Design? , 2014 .

[21]  Nicola Marsden,et al.  Cognitive styles and personas: designing for users who are different from me , 2017, OZCHI.

[22]  Chris Chapman,et al.  Quantitative Evaluation of Personas as Information , 2008 .

[23]  Keiichi Sato,et al.  Describing multiple aspects of use situation: applications of Design Information Framework (DIF) to scenario development , 2006 .

[24]  Bernard J. Jansen,et al.  Personas and Analytics: A Comparative User Study of Efficiency and Effectiveness for a User Identification Task , 2020, CHI.

[25]  Bernard J. Jansen,et al.  Who are your users?: comparing media professionals' preconception of users to data-driven personas , 2017, OZCHI.

[26]  Xiang Zhang,et al.  Data-driven Personas: Constructing Archetypal Users with Clickstreams and User Telemetry , 2016, CHI.

[27]  Kari Rönkkö,et al.  An Empirical Study Demonstrating How Different Design Constraints, Project Organization and Contexts Limited the Utility of Personas , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[28]  Asbjørn Følstad,et al.  My Chatbot Companion - a Study of Human-Chatbot Relationships , 2021, Int. J. Hum. Comput. Stud..

[29]  Charles Bellemare,et al.  Statistical Power of within and Between-Subjects Designs in Economic Experiments , 2014, SSRN Electronic Journal.

[30]  Tara Matthews,et al.  Comparing collaboration and individual personas for the design and evaluation of collaboration software , 2012, CHI.

[31]  Tomasz Miaskiewicz,et al.  Designing the Introductory IS Course Using Student Personas: Lessons Learned from Product Design , 2009, AMCIS.

[32]  Bruce Manciagli,et al.  Human Centered Design , 2011, Lecture Notes in Computer Science.

[33]  Bernard J. Jansen,et al.  Evaluating the performance of demographic targeting using gender in sponsored search , 2013, Inf. Process. Manag..

[34]  Alan Cooper,et al.  The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity (2nd Edition) , 1999 .

[35]  Tomasz Miaskiewicz,et al.  The Use of Data-Driven Personas to Facilitate Organizational Adoption – A Case Study , 2017 .

[36]  Bernard J. Jansen,et al.  Data-Driven Personas for Enhanced User Understanding: Combining Empathy with Rationality for Better Insights to Analytics , 2020, Data Inf. Manag..

[37]  Margaret M. Burnett,et al.  GenderMag: A Method for Evaluating Software's Gender Inclusiveness , 2016, Interact. Comput..

[38]  Stewart A. Birrell,et al.  A new perspective on personas and customer journey maps: Proposing systemic UX , 2021, Int. J. Hum. Comput. Stud..

[39]  Steve Whittaker,et al.  How do designers and user experience professionals actually perceive and use personas? , 2012, CHI.

[40]  G. Casella,et al.  Explaining the Gibbs Sampler , 1992 .

[41]  Lene Nielsen,et al.  Personas - User Focused Design , 2012, Human–Computer Interaction Series.

[42]  Tomasz Miaskiewicz,et al.  A Preliminary Examination of Using Personas to Enhance User-Centered Design , 2009, AMCIS.

[43]  Jonathan Grudin,et al.  Personas, Participatory Design and Product Development: An Infrastructure for Engagement , 2002 .

[44]  Bernard J. Jansen,et al.  View, Like, Comment, Post: Analyzing User Engagement by Topic at 4 Levels across 5 Social Media Platforms for 53 News Organizations , 2019, ICWSM.

[45]  Bill Sullivan,et al.  THE INMATES ARE RUNNING THE ASYLUM , 2000 .

[46]  Erin Friess,et al.  Personas and decision making in the design process: an ethnographic case study , 2012, CHI.

[47]  Rashmi R. Sinha,et al.  Persona development for information-rich domains , 2003, CHI Extended Abstracts.

[48]  Bernard J. Jansen,et al.  “Is More Better?”: Impact of Multiple Photos on Perception of Persona Profiles , 2018, CHI.

[49]  Eric J. Johnson,et al.  The validity of verbal protocols , 1989, Memory & cognition.

[50]  Patrick Langdon,et al.  Formats for User Data in Inclusive Design , 2007, HCI.

[51]  Jaewoong Choi,et al.  Social media analytics and business intelligence research: A systematic review , 2020, Inf. Process. Manag..

[52]  Bernard J. Jansen,et al.  From 2, 772 segments to five personas: Summarizing a diverse online audience by generating culturally adapted personas , 2018, First Monday.

[53]  Bernard J. Jansen,et al.  Persona Transparency: Analyzing the Impact of Explanations on Perceptions of Data-Driven Personas , 2019, Int. J. Hum. Comput. Interact..

[54]  Elina Eriksson,et al.  The secret life of a persona: when the personal becomes private , 2013, CHI.

[55]  Bernard J. Jansen,et al.  Persona Generation from Aggregated Social Media Data , 2017, CHI Extended Abstracts.

[56]  Michael F. Clarke,et al.  The Work of Mad Men that Makes the Methods of Math Men Work: Practically Occasioned Segment Design , 2015, International Conference on Human Factors in Computing Systems.

[57]  Bernard J. Jansen,et al.  Detecting Demographic Bias in Automatically Generated Personas , 2019, CHI Extended Abstracts.

[58]  Jan Stage,et al.  A Template for Design Personas: Analysis of 47 Persona Descriptions from Danish Industries and Organizations , 2015, Int. J. Sociotechnology Knowl. Dev..

[59]  J. Grudin.  12 – WHY PERSONAS WORK: THE PSYCHOLOGICAL EVIDENCE , 2006 .

[60]  Bernard J. Jansen,et al.  Imaginary People Representing Real Numbers , 2018, ACM Trans. Web.

[61]  K. A. Ericsson,et al.  Verbal reports as data. , 1980 .

[62]  Nicola Marsden,et al.  Stereotypes and Politics: Reflections on Personas , 2016, CHI.

[63]  Tomasz Miaskiewicz,et al.  Personas and user-centered design: How can personas benefit product design processes? , 2011 .

[64]  E. Gummesson.  Quality Research in Management: addressing Complexity, Context and Persona , 2006 .

[65]  Bernard J. Jansen,et al.  Confusion and information triggered by photos in persona profiles , 2019, Int. J. Hum. Comput. Stud..

[66]  Bernard J. Jansen,et al.  Customer segmentation using online platforms: isolating behavioral and demographic segments for persona creation via aggregated user data , 2018, Social Network Analysis and Mining.

[67]  Bernard J. Jansen,et al.  Data-Driven Personas , 2021, Synthesis Lectures on Human-Centered Informatics.