Advancing Information Systems Evaluation (Research): A Critical Realist Approach

The aim of Information Systems (IS) evaluation research is to produce ever more detailed answers to the question of why an IS initiative works (better) for whom and in what circumstances. This paper presents an alternative to traditional IS evaluation research approaches. The alternative, realistic IS evaluation research, is based on the philosophy of critical realism. Realistic IS evaluation research attends to how and why an IS initiative has the potential to cause (desired) changes and seeks to understand for whom and in what circumstances (contexts) an IS initiative works through the study of contextual conditioning. The paper also discusses problems with traditional IS evaluation research approaches and how realistic IS evaluation overcomes the problems.

[1]  James A. Senn,et al.  Research in Management Information Systems: The Minnesota Experiments , 1977 .

[2]  M. Patton.  Qualitative research and evaluation methods , 1980 .

[3]  P. Pettit.  The Possibility of Naturalism. , 1981 .

[4]  Cliff Hooker,et al.  A Realistic Theory of Science. , 1987 .

[5]  Henk G. Sol,et al.  Decision support systems: A decade in perspective , 1987, Decis. Support Syst..

[6]  Gordon B. Davis,et al.  Information Systems Assessment: Issues and Challenges: Proceedings of the IFIP Wg8.2 Working Conference on Information Systems Assessment, Noordwiijkerhout, the Netherlands, 27-29 Aug., 1986 , 1988 .

[7]  H. Tsoukas.  The Validity of Idiographic Research Explanations , 1989 .

[8]  J. Lovering,et al.  Reclaiming Reality , 1989 .

[9]  J. Forge.  A Realistic Theory of Science? , 1989 .

[10]  Izak Benbasat,et al.  The Information Systems Research Challenge: Experimental Research Methods , 1990 .

[11]  Tom Carney.  Fourth Generation Evaluation , 1991 .

[12]  M. Archer.  Culture and Agency: The Place of Culture in Social Theory , 1996 .

[13]  M. Reed,et al.  In Praise of Duality and Dualism: Rethinking Agency and Structure in Organizational Analysis , 1997 .

[14]  R. Bhaskar,et al.  Critical Realism , 2011 .

[15]  Eric W. K. Tsang,et al.  Replication and Theory Development in Organizational Science: A Critical Realist Perspective , 1999 .

[16]  Stephen Ackroyd,et al.  Realism in contemporary organisation and management studies , 2000 .

[17]  John Mingers,et al.  Combining IS Research Methods: Towards a Pluralist Methodology , 2001, Inf. Syst. Res..

[18]  Daniel L. Stufflebeam,et al.  Evaluation Models: New Directions for Evaluation , 2001 .

[19]  Michael Ivor Reed,et al.  Organization, Trust and Control: A Realist Analysis , 2001 .

[20]  Philip J. Dobson.  The Philosophy of Critical Realism—An Opportunity for Information Systems Research , 2001, Inf. Syst. Frontiers.

[21]  Alistair Mutch,et al.  Actors and Networks or Agents and Structures: Towards a Realist View of Information Systems , 2002 .

[22]  S Hassfeld,et al.  EVALUATION OF MODELS , 2002, Biomedizinische Technik. Biomedical engineering.

[23]  Stephen Ackroyd,et al.  Realist Perspectives on Management and Organisations , 2003 .

[24]  Sven A. Carlsson.  Using Critical Realism in IS Research , 2004 .

[25]  Fadhel Kaboub.  Realistic Evaluation , 2004 .

[26]  J. López,et al.  After Postmodernism: An Introduction to Critical Realism , 2005 .