A dialogue-based approach for evaluating educational software

Abstract This article offers a perspective on evaluating educational software based on users' dialogue as they engage in interaction with a particular program, rather than on features of the software per se. Building on analytical constructs and techniques of Conversation Analysis, we suggest an evaluative approach that identifies gaps or breakdowns in users' dialogues and maps the mismatches between users' actions and software behavior. The article presents two case studies to illustrate this dialogue-based approach, and discusses ways of integrating this perspective with more traditional guidelines for software evaluation based on features such as interface design and feedback types.

[1]  Clarisse Sieckenius de Souza,et al.  The Semiotic Engineering of User Interface Languages , 1993, Int. J. Man Mach. Stud..

[2]  Richard Noss.  Meaning mathematically with computers , 1997 .

[3]  Luciano Meira,et al.  Making Sense of Instructional Devices: The Emergence of Transparency in Mathematical Activity. , 1998 .

[4]  B. Nardi.  Context and consciousness: activity theory and human-computer interaction , 1995 .

[5]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[6]  Paul Cobb,et al.  Learning mathematics : constructivist and interactionist theories of mathematical development , 1994 .

[7]  U. Staudinger,et al.  Interactive minds : life-span perspectives on the social foundation of cognition , 1996 .

[8]  Tom Rodden,et al.  Proceedings of the Fifth European Conference on Computer Supported Cooperative Work , 1997, Springer Netherlands.

[9]  Ivana Marková,et al.  The dynamics of dialogue , 1990 .

[10]  Anita M. Pomerantz,et al.  Conversation analysis: an approach to the study of social action as sense making practices , 1997 .

[11]  Timothy Koschmann,et al.  Cscl : Theory and Practice of An Emerging Paradigm , 1996 .

[12]  Allison Woodruff,et al.  Practical strategies for integrating a conversation analyst in an iterative design process , 2002, DIS '02.

[13]  Susanne Bødker,et al.  Through the Interface: A Human Activity Approach To User Interface Design , 1990 .

[14]  Bertram C. Bruce.  Computers and the collaborative experience of learning , 1995 .

[15]  Michael A. Norman,et al.  Informing HCI Design through Conversation Analysis , 1991, Int. J. Man Mach. Stud..

[16]  Jorg Voigt,et al.  Negotiation of mathematical meaning and learning mathematics , 1994 .

[17]  Jan Wyndhamn,et al.  Understanding practice: Solving everyday problems in the formal setting: An empirical study of the school as context for thought , 1993 .

[18]  L. Vygotsky.  Mind in Society: The Development of Higher Psychological Processes: Harvard University Press , 1978 .

[19]  J. M. Atkinson.  Structures of Social Action: Contents , 1985 .

[20]  Jaan Valsiner,et al.  Culture and Human Development , 2000 .

[21]  Harvey Sacks.  Structures of Social Action: Notes on methodology , 1985 .

[22]  C. Crook.  Computers and the Collaborative Experience of Learning , 1996 .

[23]  James V. Wertsch,et al.  Culture, communication, and cognition : Vygotskian perspectives , 1987 .

[24]  Diane K. Kovacs,et al.  Computers, Communication and Mental Models , 1996 .

[25]  Deborah Schiffrin,et al.  Theory and method in discourse analysis: what context for what unit? , 1997 .

[26]  B. Latour.  Science in action : how to follow scientists and engineers through society , 1989 .

[27]  Yrjö Engeström,et al.  Mundane tool or object of affection?: the rise and fall of the Postal Buddy , 1995 .

[28]  ERNEST L. STECH,et al.  The analysis of conversational topic sequence structures , 1982 .

[29]  Caryl Emerson,et al.  Problems of Dostoevsky's Poetics , 1985 .

[30]  Luiz Antônio Marcuschi.  Análise da conversação , 2001 .

[31]  Daniel Povey,et al.  Large scale discriminative training of hidden Markov models for speech recognition , 2002, Comput. Speech Lang..

[32]  M. Bakhtin,et al.  Problems of Dostoevsky's Poetics , 1985 .

[33]  Jean Vanderdonckt,et al.  Development milestones towards a tool for working with guidelines , 1999, Interact. Comput..

[34]  James J. Kaput,et al.  Creating Cybernetic and Psychological Ramps from the Concrete to the Abstract: Examples from Multiplicative Structures , 1997 .

[35]  Theo Mandel,et al.  The Elements of User Interface Design , 1997 .

[36]  T. V. Dijk.  Discourse as Social Interaction , 1997 .

[37]  L. Vygotsky,et al.  Thought and Language , 1963 .

[38]  Luciano Meira,et al.  The Microevolution of Mathematical Representations in Children's Activity , 1995 .

[39]  David N. Perkins,et al.  Software Goes to School: Teaching for Understanding with New Technologies. , 1997 .

[40]  J. Roschelle.  Learning by Collaborating: Convergent Conceptual Change , 1992 .

[41]  S. Engel.  Thought and Language , 1964 .

[42]  Susanne Bødker,et al.  Cognition and communication at work: Users and designers in mutual activity: An analysis of cooperative activities in systems design , 1996 .

[43]  S. L. Star,et al.  Working together: Symbolic interactionism, activity theory, and information systems , 1996 .

[44]  Yvonne Rogers,et al.  What Are Workplace Studies For? , 1995, ECSCW.

[45]  J. Porac,et al.  Cognition and Communication at Work , 1999 .

[46]  J. Roschelle.  Designing for cognitive communication: epistemic fidelity or mediating collaborative inquiry? , 1997, Computers, Communication and Mental Models.

[47]  Terezinha Nunes,et al.  Learning and teaching mathematics : an international perspective , 1997 .

[48]  D. Norman.  The psychology of everyday things , 1990 .

[49]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[50]  Elaine W. Vine,et al.  Multiple Discourse Analyses of a Workplace Interaction , 2003 .