Usability problem identification in culturally diverse settings

There are indications that established methods for evaluating information system usability that have been developed for use in, e.g. Europe or the USA, fail to give reliable results in countries such as India, China or Malaysia. This paper presents the theoretical background, related work and a definition of culture that should be useful for studies of multiple‐country usability testing. This includes a discussion of cultural fit and the consequences of cultural (in)consistencies between stakeholders in system development and use. As an illustrative example of the kind of academic research that needs to be done, a pilot study is described. The pilot study exemplifies themes to explore, who should be participants and where should the study be done, how to find examples of multiple‐country usability testing, how to collect data and how to analyse that data and what kind of results and discussion of results that may be expected. The conclusion summarises the paper, discusses the limitations of the pilot study and gives detailed suggestions to future research.

[1]  G. Hofstede.  Culture′s Consequences: Comparing Values, Behaviors, Institutions and Organizations Across Nations , 2001 .

[2]  Jo Hanisch,et al.  Impediments to requirements engineering during global software development , 2007, Eur. J. Inf. Syst..

[3]  Jijie Wang,et al.  Information technology (IT) in Saudi Arabia: Culture and the acceptance and use of IT , 2007, Inf. Manag..

[4]  Ebba Þóra Hvannberg,et al.  Analysis of combinatorial user effect in international usability tests , 2004, CHI '04.

[5]  Oscar Mauricio Serrano Jaimes,et al.  EVALUACION DE LA USABILIDAD EN SITIOS WEB, BASADA EN EL ESTANDAR ISO 9241-11 (International Standard (1998) Ergonomic requirements For office work with visual display terminals (VDTs)-Parts II: Guidance on usability , 2012 .

[6]  Torkil Clemmensen,et al.  Templates for Cross-Cultural and Culturally Specific Usability Testing: Results From Field Studies and Ethnographic Interviewing in Three Countries , 2011, Int. J. Hum. Comput. Interact..

[7]  Ergonomic requirements for office work with visual display terminals ( VDTs ) — Part 11 : Guidance on usability , 1998 .

[8]  Torkil Clemmensen,et al.  Towards a Theory of Cultural Usability: A Comparison of ADA and CM-U Theory , 2009, HCI.

[9]  Morten Hertzum,et al.  Personal Usability Constructs: How People Construe Usability Across Nationalities and Stakeholder Groups , 2011, Int. J. Hum. Comput. Interact..

[10]  Qingxin Shi,et al.  A field study of the relationship and communication between Chinese evaluators and users in thinking aloud usability tests , 2008, NordiCHI.

[11]  Apala Lahiri Chavan,et al.  COVER STORYThe washing machine that ate my sari---mistakes in cross-cultural design , 2009, INTR.

[12]  Tone Bratteteig,et al.  Proceedings of the 11th Biennial Participatory Design Conference , 2010, PDC 2010.

[13]  Helene Gelderblom,et al.  The effects of culture on performance achieved through the use of human computer interaction , 2003 .

[14]  Gilbert Cockton,et al.  Sale must end: should discount methods be cleared off HCI's shelves? , 2002, INTR.

[15]  Andy Smith,et al.  Issues in Adapting Usability Testing for Global Usability , 2011, Global Usability.

[16]  Aaron Marcus,et al.  Culture class vs. culture clash , 2002, INTR.

[17]  Peter Dahler-Larsen.  At fremstille kvalitative data , 2008 .

[18]  Suzanne Rivard,et al.  A Three-Perspective Model of Culture, Information Systems, and Their Development and Use , 2008, MIS Q..

[19]  Qingxin Shi,et al.  Communication patterns and usability problem finding in cross-cultural thinking aloud usability testing , 2008, CHI Extended Abstracts.

[20]  R. Nisbett,et al.  Opinion TRENDS in Cognitive Sciences Vol.9 No.10 October 2005 The influence of culture: holistic versus analytic perception , 2022 .

[21]  Anthony F. Norcio,et al.  Cultural differences on attention and perceived usability: Investigating color combinations of animated graphics , 2006, Int. J. Hum. Comput. Stud..

[22]  Shinobu,et al.  Cultural Psychology of Workways , 2013 .

[23]  Ravikiran Vatrapu,et al.  Comparative informatics , 2011, INTR.

[24]  Heike Winschiers-Theophilus,et al.  Assumptions Considered Harmful , 2007, HCI.

[25]  Morten Hertzum,et al.  The Evaluator Effect: A Chilling Fact About Usability Evaluation Methods , 2001, Int. J. Hum. Comput. Interact..

[26]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[27]  Weina Qu,et al.  Cultural differences between Chinese and Dane in card sorting , 2007 .

[28]  Susan M. Dray,et al.  Tips and tricks for a better international usability test , 2004, CHI EA '04.

[29]  Takahiko Masuda,et al.  Culture and point of view. , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[30]  Edwin H. Blake,et al.  Being participated: a community approach , 2010, PDC '10.

[31]  Heike Winschiers-Theophilus,et al.  Interpretation of a cross-cultural usability evaluation: A case study based on a hypermedia system for rare species management in Namibia , 2011, Interact. Comput..

[32]  Mark Keil,et al.  Reporting bad news on software projects: the effects of culturally constituted views of face‐saving , 2007, Inf. Syst. J..

[33]  Huatong Sun,et al.  Cultural usability: a localization study of mobile text messaging use , 2003, CHI Extended Abstracts.

[34]  N. Hoffart.  Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory , 2000 .

[35]  Robert M. Davison,et al.  Cultural Bias in Information Systems Research and Practice: Are You Coming from the Same Place I Am? , 2005, ICIS.

[36]  Alvin Yeo,et al.  Cultural effects in usability assessment , 1998, CHI Conference Summary.

[37]  R. Nisbett,et al.  Is it culture or is it language? Examination of language effects in cross-cultural research on categorization. , 2004, Journal of personality and social psychology.

[38]  Kasper Hornbæk,et al.  Non-universal usability?: a survey of how usability is understood by Chinese and Danish users , 2009, CHI.

[39]  Daisuke Sato,et al.  Trends in Usability Research and Activities in Japan , 2004, Int. J. Hum. Comput. Interact..

[40]  Mie Nørgaard,et al.  Evaluating usability: using models of argumentation to improve persuasiveness of usability feedback , 2008, DIS '08.

[41]  Kun-Pyo Lee,et al.  Cultural differences and design methods for user experience research: Dutch and Korean participants compared , 2007, DPPI.

[42]  Morten Hertzum,et al.  AIS Electronic , 2022 .

[43]  Norhayati Zakaria,et al.  Designing and implementing culturally-sensitive IT applications: The interaction of culture values and privacy issues in the Middle East , 2003, Inf. Technol. People.

[44]  John W. Berry,et al.  Psychological Differentiation in Cross-Cultural Perspective , 1975 .

[45]  Morten Hertzum,et al.  Problem Prioritization in Usability Evaluation: From Severity Assessments Toward Impact on Design , 2006, Int. J. Hum. Comput. Interact..

[46]  J Tritten,et al.  The Influence of Culture , 2019, Advancing Sexual Health for the Christian Client.

[47]  Jakob Nielsen,et al.  Designing for international use (panel) , 1990, CHI '90.

[48]  R. Nisbett.  The Geography of Thought , 2003 .

[49]  Daniel D. Suthers,et al.  Culture and Computers: A Review of the Concept of Culture and Implications for Intercultural Collaborative Online Learning , 2007, IWIC.

[50]  Effie Lai-Chong Law.  Evaluating the Downstream Utility of User Tests and Examining the Developer Effect: A Case Study , 2006, Int. J. Hum. Comput. Interact..

[51]  J. Gosby.  MEDIA REVIEWS: Basics of Qualitative Research - Techniques and Procedures for Developing Grounded Theory 2nd Edition by A. Strauss and J. Corbin. Sage Publications, , 2000 .

[52]  David A. Caulton.  Relaxing the homogeneity assumption in usability testing , 2001 .

[53]  Ying-yi Hong,et al.  A dynamic constructivist approach to culture: Lessons learned from personality psychology , 2004 .

[54]  L. Herman.  Towards effective usability evaluation in Asia: cross-cultural differences , 1996, Proceedings Sixth Australian Conference on Computer-Human Interaction.

[55]  Manuel A. Pérez-Quiñones,et al.  Culture and usability evaluation: the effects of culture in structured interviews , 2006 .

[56]  Stephen A. R. Scrivener,et al.  Meaning, The Central Issue in Cross-Cultural HCI Design , 1998, Interact. Comput..

[57]  Alvin W. Yeo,et al.  Global-software development lifecycle: an exploratory study , 2001, CHI.

[58]  Dorothy E. Leidner,et al.  Review: A Review of Culture in Information Systems Research: Toward a Theory of Information Technology Culture Conflict , 2006, MIS Q..

[59]  H. A. Witkin,et al.  Psychological Differentiation in Cross-Cultural Perspective , 1975 .

[60]  Andy Smith,et al.  Evaluation methods and cultural differences: studies across three continents , 2008, NordiCHI.

[61]  Catherine E. Connelly,et al.  Information systems research and Hofstede's culture's consequences: an uneasy and incomplete partnership , 2003, IEEE Trans. Engineering Management.

[62]  Torkil Clemmensen,et al.  Culturel difference in structure of categories in Denmark and China , 2007 .

[63]  Albert N. Badre,et al.  Culturability: the merging of culture and usability , 1998 .

[64]  Kakoli Bandyopadhyay,et al.  The Effect of Culture on User Acceptance of Information Technology , 2007, Commun. Assoc. Inf. Syst..

[65]  Dennis F. Galletta,et al.  Applying TAM across cultures: the need for caution , 2007, Eur. J. Inf. Syst..

[66]  Lilly Irani.  HCI on the move: methods, culture, values , 2010, CHI EA '10.

[67]  Michael Harris Bond,et al.  The Psychology of the Chinese people , 1986 .