From Gender Biases to Gender-Inclusive Design: An Empirical Investigation

In recent years, research has revealed gender biases in numerous software products. But although some researchers have found ways to improve gender participation in specific software projects, general methods focus mainly on detecting gender biases -- not fixing them. To help fill this gap, we investigated whether the GenderMag bias detection method can lead directly to designs with fewer gender biases. In our 3-step investigation, two HCI researchers analyzed an industrial software product using GenderMag; we derived design changes to the product using the biases they found; and ran an empirical study of participants using the original product versus the new version. The results showed that using the method in this way did improve the software's inclusiveness: women succeeded more often in the new version than in the original; men's success rates improved too; and the gender gap entirely disappeared.

[1]  Margaret M. Burnett,et al.  A strategy-centric approach to the design of end-user debugging tools , 2010, CHI.

[2]  Ann Hergatt Huffman,et al.  Using technology in higher education: The influence of gender roles on technology self-efficacy , 2013, Comput. Hum. Behav..

[3]  G. Charness,et al.  Strong Evidence for Gender Differences in Risk Taking , 2012 .

[4]  A. Bandura.  Social Foundations of Thought and Action , 1986 .

[5]  David M. Nichols,et al.  Supporting Gender-Neutral Digital Library Creation: A Case Study Using the GenderMag Toolkit , 2016, ICADL.

[6]  Erika Shehan Poole,et al.  Putting the gender back in digital housekeeping , 2018, GenderIT.

[7]  Margaret M. Burnett,et al.  Open Source Barriers to Entry, Revisited: A Sociotechnical Perspective , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[8]  Margaret M. Burnett,et al.  Principles of a debugging-first puzzle game for computing education , 2014, 2014 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[9]  Yuriy Brun,et al.  Fairness testing: testing software for discrimination , 2017, ESEC/SIGSOFT FSE.

[10]  Margaret M. Burnett,et al.  Gender-Inclusiveness Personas vs. Stereotyping: Can We Have it Both Ways? , 2017, CHI.

[11]  Krzysztof Z. Gajos,et al.  Evaluation of Filesystem Provenance Visualization Tools , 2013, IEEE Transactions on Visualization and Computer Graphics.

[12]  Joanna Goode,et al.  Beyond access: broadening participation in high school computer science , 2012, INROADS.

[13]  Jane Margolis,et al.  Unlocking the clubhouse: a decade later and now what? , 2013, SIGCSE '13.

[14]  Gayna Williams,et al.  Are you sure your software is gender-neutral? , 2014, INTR.

[15]  M. Sheelagh T. Carpendale,et al.  Gendered or neutral?: considering the language of HCI , 2015, Graphics Interface.

[16]  Christophe Kolski,et al.  State of the Art on the Cognitive Walkthrough Method, Its Variants and Evolutions , 2010, Int. J. Hum. Comput. Interact..

[17]  Engin Bozdag,et al.  Bias in algorithmic filtering and personalization , 2013, Ethics and Information Technology.

[18]  Yasmin B. Kafai,et al.  Beyond Game Design for Broadening Participation: Building New Clubhouses of Computing for Girls , 2014 .

[19]  Leah Buechley,et al.  LilyPad in the wild: how hardware's long tail is supporting new engineering and design communities , 2010, Conference on Designing Interactive Systems.

[20]  Weng-Keen Wong,et al.  Why-oriented end-user debugging of naive Bayes text classification , 2011, ACM Trans. Interact. Intell. Syst..

[21]  A. Falk,et al.  Individual Risk Attitudes: Measurement, Determinants and Behavioral Consequences , 2009 .

[22]  Nicola Marsden,et al.  Cognitive walkthrough of a learning management system with gendered personas , 2018, GenderIT.

[23]  Margaret M. Burnett,et al.  A principled evaluation for a principled idea garden , 2015, 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[24]  Mary Beth Rosson,et al.  Design Planning in End-User Web Development: Gender, Feature Exploration and Feelings of Success , 2010, 2010 IEEE Symposium on Visual Languages and Human-Centric Computing.

[25]  Keith Kirkpatrick,et al.  Battling algorithmic bias , 2016, Commun. ACM.

[26]  Margaret M. Burnett,et al.  Effectiveness of end-user debugging software features: are there gender issues? , 2005, CHI.

[27]  Todd Kulesza,et al.  Can feature design reduce the gender gap in end-user software development environments? , 2008, 2008 IEEE Symposium on Visual Languages and Human-Centric Computing.

[28]  Margaret M. Burnett,et al.  Testing vs. code inspection vs. what else?: male and female end users' debugging strategies , 2008, CHI.

[29]  Michael Eisenberg,et al.  The LilyPad Arduino: Toward Wearable Engineering for Everyone , 2008, IEEE Pervasive Computing.

[30]  Allan Fisher,et al.  Unlocking the Clubhouse : Women in Computing by Allan Fisher , 2015 .

[31]  J. F. Kelley,et al.  An iterative design methodology for user-friendly natural language office information applications , 1984, TOIS.

[32]  Laura Beckwith,et al.  On to the Real World: Gender and Self-Efficacy in Excel , 2007 .

[33]  Manpreet Kaur,et al.  "Girls don't waste time": pre-adolescent attitudes toward ICT , 2006, CHI Extended Abstracts.

[34]  Margaret M. Burnett,et al.  End-user debugging strategies: A sensemaking perspective , 2012, TCHI.

[35]  Cathleen Wharton,et al.  The cognitive walkthrough method: a practitioner's guide , 1994 .

[36]  Cindy Grimm,et al.  Identifying Gender Differences in Information Processing Style, Self-efficacy, and Tinkering for Robot Tele-operation , 2018, 2018 15th International Conference on Ubiquitous Robots (UR).

[37]  Jacques Nantel,et al.  Uncovering the Nature of Information Processing of Men and Women Online: The Comparison of Two Models Using the Think-Aloud Method , 2012, J. Theor. Appl. Electron. Commer. Res..

[38]  Eric Gilbert,et al.  Specialization, homophily, and gender in a social curation site: findings from pinterest , 2014, CSCW.

[39]  Mary Czerwinski,et al.  Gender differences and programming environments: across programming populations , 2010, ESEM '10.

[40]  Margaret M. Burnett,et al.  Gender pluralism in problem-solving software , 2011, Interact. Comput..

[41]  Margaret M. Burnett,et al.  Finding Gender-Inclusiveness Software Issues with GenderMag: A Field Investigation , 2016, CHI.

[42]  Anil Singh,et al.  Role of gender, self-efficacy, anxiety and testing formats in learning spreadsheets , 2013, Comput. Hum. Behav..

[43]  Shaowen Bardzell,et al.  Feminist HCI: taking stock and outlining an agenda for design , 2010, CHI.

[44]  René Riedl,et al.  Are There Neural Gender Differences in Online Trust? An fMRI Study on the Perceived Trustworthiness of eBay Offers , 2010, MIS Q..

[45]  J. Butler.  Gender Trouble: Feminism and the Subversion of Identity , 1990 .

[46]  Karrie Karahalios,et al.  Auditing Algorithms : Research Methods for Detecting Discrimination on Internet Platforms , 2014 .

[47]  L. Suchman.  Agencies in Technology Design: Feminist Reconfigurations* , 2020, Machine Ethics and Robot Ethics.

[48]  Jonas Hallström,et al.  Gender and technology in free play in Swedish early childhood education , 2015 .

[49]  Margaret M. Burnett,et al.  A Debugging Perspective on End-User Mashup Programming , 2010, 2010 IEEE Symposium on Visual Languages and Human-Centric Computing.

[50]  Margaret M. Burnett,et al.  Gender HCl and microsoft: Highlights from a longitudinal study , 2017, 2017 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[51]  Caitlin Kelleher,et al.  Storytelling alice motivates middle school girls to learn computer programming , 2007, CHI.

[52]  Margaret M. Burnett,et al.  Gender HCI: What About the Software? , 2006, Computer.

[53]  David García,et al.  Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr , 2017, CSCW.

[54]  Amy J. Ko,et al.  Personifying programming tool feedback improves novice programmers' learning , 2011, ICER.

[55]  Jonathan Bean,et al.  Learning from IKEA hacking: i'm not one to decoupage a tabletop and call it a day. , 2009, CHI.

[56]  Joan Meyers-Levy,et al.  Revisiting gender differences: What we know and what lies ahead☆ , 2015 .

[57]  Margaret M. Burnett,et al.  GenderMag: A Method for Evaluating Software's Gender Inclusiveness , 2016, Interact. Comput..

[58]  Paul Dourish,et al.  An ethnographic examination of the relationship of gender & end-user programming , 2008 .

[59]  Desney S. Tan,et al.  Women go with the (optical) flow , 2003, CHI '03.