Evaluation methods and cultural differences: studies across three continents

This paper reviews issues and problems that arise in cross-cultural usability evaluations. It reports two separate empirical studies of a number of well-known techniques with UK, African and Indian users. The studies examine the effectiveness of methods based on think-aloud protocols, including the DUCE method, to elicit users' views. The results from all the studies show that these established Western methods are less effective with users from other cultures. It suggests that the reasons for this are the consequences of deep-rooted differences in personal interactions in different cultures. This paper provides evidence to guide choices for applications involving users from India and Africa.

[1]  G. Hofstede.  Culture′s Consequences: Comparing Values, Behaviors, Institutions and Organizations Across Nations , 2001 .

[2]  Manuel A. Pérez-Quiñones,et al.  Culture and usability evaluation: the effects of culture in structured interviews , 2006 .

[3]  Andy Smith,et al.  Prototype evaluation and redesign: structuring the design space through contextual techniques , 2002, Interact. Comput..

[4]  Gavriel Salvendy,et al.  A proposed index of usability: A method for comparing the relative usability of different software systems , 1997, Behav. Inf. Technol..

[5]  L. Herman.  Towards effective usability evaluation in Asia: cross-cultural differences , 1996, Proceedings Sixth Australian Conference on Computer-Human Interaction.

[6]  P. Falzon.  Human-computer interaction: lessons from human-human communication , 1990 .

[7]  Peter C. Wright,et al.  A Cost-Effective Evaluation Method for Use by Designers , 1991, Int. J. Man Mach. Stud..

[8]  Joseph A. Goguen,et al.  Formality and Informality in Requirements Engineering , 1996, ICRE.

[9]  D. Norman.  The psychology of everyday things", Basic Books Inc , 1988 .

[10]  Shailey Minocha,et al.  A process model for developing usable cross-cultural websites , 2004, Interact. Comput..

[11]  Alvin Yeo Wee.  Global-software development lifecycle: an exploratory study , 2001 .

[12]  B. Spolsky.  Conditions for Second Language Learning , 1989 .

[13]  Ron Henderson,et al.  A comparison of the four prominent user-based methods for evaluating the usability of computer software , 1995 .

[14]  Helen Sharp,et al.  An Approach to the Evaluation of Usefulness as a Social Construct Using Technological Frames , 2007, Int. J. Hum. Comput. Interact..

[15]  Gillian Khoo,et al.  The Suinn-Lew Asian Self-Identity Acculturation Scale: Concurrent and Factorial Validation , 1992 .

[16]  Elisa M. del Galdo,et al.  Culture and design , 1996 .

[17]  Vanessa Evers,et al.  Cultural aspects of user interface understanding : an empirical evaluation of an e-learning website by international user groups , 2001 .

[18]  Andy Smith,et al.  Designing for Shared Interfaces with Diverse User Groups , 1999, INTERACT.

[19]  Susan M. Dray.  Usable for the World: A Practical Guide to International User Studies , 2003, IWIPS.

[20]  Jan Gulliksen,et al.  Building Usability in India: Reflections from the Indo-European Systems Usability Partnership , 2005, BCS HCI.