A critique of statistical modelling in management science from a critical realist perspective: its role within multimethodology

Management science was historically dominated by an empiricist philosophy that saw quantitative modelling and statistical analysis as the only legitimate research method. More recently interpretive or constructivist philosophies have also developed employing a range of non-quantitative methods. This has sometimes led to divisive debates. ‘Critical realism’ has been proposed as a philosophy of science that can potentially provide a synthesis in recognizing both the value and limitations of these approaches. This paper explores the critical realist critique of quantitative modelling, as exemplified by multivariate statistics, and argues that its grounds must be re-conceptualized within a multimethodological framework.

[1]  Robert M. Kunst.  Econometric Forecasting , 2007 .

[2]  Jonathan Rosenhead,et al.  Soft Systems Methodology in Action , 1991 .

[3]  John Mingers,et al.  Multimethodology: Towards a framework for mixing methodologies , 1997 .

[4]  Geoff Walsham,et al.  The Emergence of Interpretivism in IS Research , 1995, Inf. Syst. Res..

[5]  A. King.  The Accidental Derogation of the Lay Actor , 2000 .

[6]  John Mingers,et al.  Towards Critical Pluralism , 1997 .

[7]  Arnold Zellner,et al.  Bayesian analysis in econometrics and statistics , 1997 .

[8]  Eric W. K. Tsang,et al.  Replication and Theory Development in Organizational Science: A Critical Realist Perspective , 1999 .

[9]  Deirdre N. McCloskey,et al.  The Standard Error of Regressions , 1996 .

[10]  Diederik Aerts,et al.  World Views and the Problem of Synthesis , 1999 .

[11]  D. Hendry,et al.  A Conversation on Econometric Methodology , 1990, Econometric Theory.

[12]  W. Dunlap,et al.  On the Logic and Purpose of Significance Testing , 1997 .

[13]  Andrew Ehrenberg,et al.  A Primer In Data Reduction , 2007 .

[14]  Kevin D. Hoover,et al.  The New Classical Macroeconomics: A Sceptical Inquiry , 1988 .

[15]  Robert Sandy,et al.  Statistics for Business and Economics , 1989 .

[16]  John Mingers,et al.  Recent Developments in Critical Management Science , 1992 .

[17]  Edward E. Leamer,et al.  Let's Take the Con Out of Econometrics , 1983 .

[18]  Roger E. Backhouse,et al.  New Directions in Economic Methodology , 1994 .

[19]  M. Uschold,et al.  Methods and applications , 1953 .

[20]  John Mingers,et al.  Critical Heuristics of Social Planning: A New Approach to Practical Philosophy , 1983 .

[21]  David J. Will.  A realist theory of science , 1981, Medical History.

[22]  Varun Grover,et al.  The evolution of empirical research in IS: A study in IS maturity , 1993, Inf. Manag..

[23]  John Mingers,et al.  The paucity of multimethod research: a review of the information systems literature , 2003, Inf. Syst. J..

[24]  R. Fildes,et al.  The Impact of Empirical Accuracy Studies On Time Series Analysis and Forecasting , 1995 .

[25]  Joe Nandhakumar,et al.  Too close for comfort? Distance and engagement in interpretive information systems research , 1997, Inf. Syst. J..

[26]  W. Whitham,et al.  Problems with “measurement” , 1991 .

[27]  J. Scott Armstrong,et al.  Principles of forecasting : a handbook for researchers and practitioners , 2001 .

[28]  R. D. Rosenkrantz,et al.  The significance test controversy , 1972, Synthese.

[29]  S. Fleetwood.  Critical Realism in Economics: Development and Debate , 1998 .

[30]  William N. Parker,et al.  Economic history and the modern economist , 1988 .

[31]  Jürgen Habermas,et al.  Reason and the rationalization of society , 1984 .

[32]  Malcolm Williams.  New strategies in social research , 2006 .

[33]  John Mingers,et al.  Multimethodology : the theory and practice of combining management science methodologies , 1997 .

[34]  Stephen F. LeRoy,et al.  Atheoretical macroeconometrics: A critique , 1985 .

[35]  J. Habermas.  Theory of Communicative Action , 1981 .

[36]  A. Sayer,et al.  Realism and Social Science , 1999 .

[37]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[38]  Poulton Ec,et al.  Unwanted asymmetrical transfer effects with balanced experimental designs. , 1966 .

[39]  Philip J. Dobson.  Longitudinal Case Research: A Critical Realist Perspective , 2001 .

[40]  David V. Pritchett.  Econometric policy evaluation: A critique , 1976 .

[41]  Alexander Rosenberg,et al.  Economics--Mathematical Politics or Science of Diminishing Returns? , 1994 .

[42]  D. Bakan,et al.  The test of significance in psychological research. , 1966, Psychological bulletin.

[43]  M. Gardner,et al.  Confidence intervals rather than P values: estimation rather than hypothesis testing. , 1986, British medical journal.

[44]  R. M. Oliver,et al.  Influence diagrams, belief nets and decision analysis , 1992 .

[45]  G. Midgley.  Pluralism and the legitimation of systems science , 1992 .

[46]  C. Sims.  MACROECONOMICS AND REALITY , 1977 .

[47]  Michael Pidd,et al.  Tools for Thinking—Modelling in Management Science , 1997 .

[48]  Humberto R. Maturana,et al.  Science and Daily Life: The Ontology of Scientific Explanations , 1990 .

[49]  R. Bhaskar,et al.  Critical Realism , 2011 .

[50]  Robert J. Genetski,et al.  Long-Range Forecasting: From Crystal Ball to Computer , 1981 .

[51]  T. Liu,et al.  Underidentification, Structural Estimation, and Forecasting , 1960 .

[52]  J. Mingers.  Criticizing the phenomenological critique—Autopoiesis and critical realism , 1992 .

[53]  L. Harlow,et al.  What if there were no significance tests , 1997 .

[54]  R. Bhaskar.  From Science to Emancipation , 2002 .

[55]  Edgar A. Whitley,et al.  The Construction of Social Reality , 1999 .

[56]  Richard Marsden.  The Politics of Organizational Analysis , 1993 .

[57]  David E. Avison,et al.  Information systems and anthropology: and anthropological perspective on IT and organizational culture , 1995, Inf. Technol. People.

[58]  H. Klein,et al.  Information systems research: contemporary approaches and emergent traditions , 1991 .

[59]  A. Mutch.  Critical Realism, Managers and Information , 1999 .

[60]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[61]  J. Mingers.  Subjectivism and Soft Systems Methodology - A Critique , 1984 .

[62]  R. Nickerson,et al.  Null hypothesis significance testing: a review of an old and continuing controversy. , 2000, Psychological methods.

[63]  Ojelanki K. Ngwenyama,et al.  Communication Richness in Electronic Mail: Critical Social Theory and the Contextuality of Meaning , 1997, MIS Q..

[64]  John Wilson.  The Practice of Operational Research , 1993 .

[65]  Robert Fildes,et al.  Generalising about univariate forecasting methods: further empirical , 1998 .

[66]  Alistair Mutch,et al.  Actors and Networks or Agents and Structures: Towards a Realist View of Information Systems , 2002 .

[67]  J. Habermas.  The theory of communicative action: Lifeworld and system - A critique of functionalist reason , 1990 .

[68]  Wanda J. Orlikowski,et al.  Studying Information Technology in Organizations: Research Approaches and Assumptions , 1991, Inf. Syst. Res..

[69]  Philip J. Dobson.  The Philosophy of Critical Realism—An Opportunity for Information Systems Research , 2001, Inf. Syst. Frontiers.

[70]  Jan R. Magnus,et al.  Methodology and Tacit Knowledge: Two Experiments in Econometrics , 1999 .

[71]  Peter Checkland,et al.  Systems Thinking, Systems Practice , 1981 .

[72]  J. Mingers,et al.  Rational analysis for a problematic world revisited : problem structuring methods for complexity, uncertainty and conflict , 1989 .

[73]  P. Nidditch,et al.  Enquiries: Concerning Human Understanding and Concerning the Principles of Morals , 1966 .

[74]  Robert Fildes,et al.  Generalising about univariate forecasting methods: Further empirical evidence , 1998 .

[75]  Jeff Harrison,et al.  Applied Bayesian Forecasting and Time Series Analysis , 1994 .

[76]  A. D. Ritchie.  The Open Society and its Enemies , 1946, Nature.

[77]  Stephen Ackroyd,et al.  Realist Perspectives on Management and Organisations , 2003 .

[78]  R. Fildes.  Quantitative Forecasting—the State of the Art: Econometric Models , 1985 .

[79]  Michael Elstob,et al.  Critical Heuristics of Social Planning , 1998 .

[80]  Ken Hoyle,et al.  AN INTRODUCTION TO BUSINESS STATISTICS , 1979 .

[81]  Spyros Makridakis,et al.  The M3-Competition: results, conclusions and implications , 2000 .

[82]  W. Ulrich,et al.  Beyond methodology choice: critical systems thinking as critically systemic discourse , 2003, J. Oper. Res. Soc..

[83]  R. Lawrence,et al.  Operational Research and the Social Sciences. , 1966 .

[84]  R. Fildes,et al.  The State of Macroeconomic Forecasting , 2002 .

[85]  Abbas Tashakkori,et al.  Mixed Methodology: Combining Qualitative and Quantitative Approaches , 1998 .

[86]  Stephen T. Ziliak,et al.  Size Matters: The Standard Error of Regressions in the American Economic Review , 2004 .

[87]  John Mingers,et al.  Synthesising Constructivism and Critical Realism: Towards Critical Pluralism , 1999 .

[88]  K. Ramachandran,et al.  Mathematical Statistics with Applications. , 1992 .

[89]  A. Sayer.  Method in Social Science , 1992 .

[90]  W. A. Sherden.  The Fortune Sellers: The Big Business of Buying and Selling Predictions , 1997 .

[91]  Anthony King,et al.  Against Structure: A Critique of Morphogenetic Social Theory , 1999 .

[92]  J. Gill.  The Insignificance of Null Hypothesis Significance Testing , 1999 .

[93]  Günter Küppers,et al.  Selforganization : portrait of a scientific revolution , 1990 .

[94]  Ronald Giles,et al.  The Fortune Sellers , 1998 .

[95]  S. Fleetwood.  Conceptualizing Un employment in a Period of Atypical Em ployment: A Critical Realist Perspective , 2001 .

[96]  C.J.H. Mann,et al.  Systems approaches to management , 2002 .

[97]  T. Nayak.  Statistical Significance: Rationale, Validity and Utility , 1997 .

[98]  Sue Holwell,et al.  Information, Systems and Information Systems: Making Sense of the Field , 1998 .

[99]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[100]  M. D. Myers,et al.  Dialectical hermeneutics: a theoretical framework for the implementation of information systems , 1995, Inf. Syst. J..

[101]  D. V. Lindley,et al.  The Craft of Decision Modelling , 1994 .

[102]  P. Pettit.  The Possibility of Naturalism. , 1981 .

[103]  Terence C. Mills,et al.  Assessing the forecasters: an analysis of the forecasting records of the Treasury, the London Business School and the National Institute , 1999 .

[104]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[105]  Roy Bhaskar.  On the Ontological Status of Ideas , 2008 .

[106]  John Mingers,et al.  Variety is the spice of life: combining soft and hard OR/MS methods , 2000 .

[107]  C. Whitbeck,et al.  A Realist Theory of Science. , 1977 .

[108]  Margaret S. Archer,et al.  For Structure: Its Reality, Properties and Powers: A Reply to Anthony King , 2000 .

[109]  Denton E. Morrison,et al.  The Significance Test Controversy , 1972 .

[110]  Francisco J. Samaniego,et al.  The Practice of Bayesian Analysis , 1999, Technometrics.

[111]  Tjalling C. Koopmans,et al.  Measurement Without Theory , 1947 .

[112]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[113]  Michael D. Myers,et al.  Scholarship and practice: the contribution of ethnographic research methods to bridging the gap , 1994, Business Process Re-Engineering.

[114]  D L Mann,et al.  Introductory Statistics , 1991 .

[115]  Stephen Ackroyd,et al.  Critical Realist Applications in Organisation and Management Studies , 2004 .

[116]  W. Mendenhall,et al.  A second course in business statistics: Regression analysis , 1981 .

[117]  J. Scott Armstrong,et al.  Long-Range Forecasting. , 1979 .

[118]  R. Bhaskar.  Dialectic: The Pulse of Freedom , 1993 .

[119]  D. Rindskopf.  Null-hypothesis tests are not completely stupid, but Bayesian statistics are better , 1998, Behavioral and Brain Sciences.

[120]  J. Habermas.  The Theory of Communicative Action: Reason and the Rationalization of Society , 1986 .

[121]  John Mingers,et al.  Embodying information systems: the contribution of phenomenology , 2001, Inf. Organ..

[122]  J. H. Steiger,et al.  The Problem Is Epistemology, Not Statistics: Replace Significance Tests by Confidence Intervals and Quantify Accuracy of Risky Numerical Predictions , 2002 .

[123]  W. W. Rozeboom.  The fallacy of the null-hypothesis significance test. , 1960, Psychological bulletin.

[124]  Colin Eden,et al.  From the playpen to the bombsite: the changing nature of management science , 1993 .

[125]  John Mingers,et al.  Re-Establishing the Real: Critical Realism and Information Systems , 2004 .

[126]  D. Hendry,et al.  The ET Dialogue: A Conversation on Econometric Methodology. , 1990 .

[127]  James Tobin,et al.  A Statistical Demand Function for Food in the U.S.A , 1950 .

[128]  Steve Fleetwood,et al.  Causal Laws, Functional Relations and Tendencies , 2001 .

[129]  R. M. Dudley,et al.  Influence Diagrams, Belief Nets and Decision Analysis. , 1991 .

[130]  Michael Jackson.  Towards coherent pluralism in management science , 1999, J. Oper. Res. Soc..

[131]  A. Ron,et al.  Regression Analysis and the Philosophy of Social Science , 2002 .

[132]  T. Lawson.  Economics and Reality , 1997 .

[133]  J. I.  The Design of Experiments , 1936, Nature.

[134]  R. Hirschheim,et al.  The paradigm is dead, the paradigm is dead ... long live the paradigm: the legacy of Burrell and Morgan , 2000 .

[135]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[136]  John Mingers,et al.  The contribution of critical realism as an underpinning philosophy for OR/MS and systems , 2000, J. Oper. Res. Soc..

[137]  John Mingers,et al.  Combining IS Research Methods: Towards a Pluralist Methodology , 2001, Inf. Syst. Res..

[138]  Andrew Abbott,et al.  The Causal Devolution , 1998 .

[139]  R. L. Hagen.  In praise of the null hypothesis statistical test. , 1997 .

[140]  Michael Jackson,et al.  Signposts to Critical Systems Thinking and Practice: An Invited Article , 1993 .