Agent-based modelling as a foundation for big data

In this article, we propose a process-based definition of big data, as opposed to the size- and technology-based definitions. We argue that big data should be perceived as a continuous, unstructured and unprocessed dynamics of primitives, rather than as points (snapshots) or summaries (aggregates) of an underlying phenomenon. Given this, we show that big data can be generated through agent-based models but not by equation-based models. Though statistical and machine learning tools can be used to analyse big data, they do not constitute a big data-generation mechanism. Furthermore, agent-based models can aid in evaluating the quality (interpreted as information aggregation efficiency) of big data. Based on this, we argue that agent-based modelling can serve as a possible foundation for big data. We substantiate this interpretation through some pioneering studies from the 1980s on swarm intelligence and several prototypical agent-based models developed around the 2000s.

[1]  G. N. Gilbert.  Computational Social Science , 2010 .

[2]  Susan Athey,et al.  Recursive partitioning for heterogeneous causal effects , 2015, Proceedings of the National Academy of Sciences.

[3]  C. Hommes.  Heterogeneous Agent Models in Economics and Finance , 2005 .

[4]  Barbara Webb,et al.  Swarm Intelligence: From Natural to Artificial Systems , 2002, Connect. Sci..

[5]  Tuomas Sandholm,et al.  When do markets with simple agents fail? , 2010, AAMAS.

[7]  M. Huberty.  Can we vote with our tweet? On the perennial difficulty of election forecasting with social media , 2015 .

[8]  G. Parisi,et al.  Empirical investigation of starling flocks: a benchmark study in collective animal behaviour , 2008, Animal Behaviour.

[9]  Mark P. Taylor,et al.  Covered Interest Parity: A High-Frequency, High-Quality Data Study , 1987 .

[10]  Giovanni Polverino,et al.  Collective behaviour across animal species , 2014, Scientific Reports.

[11]  Mark D. Reid,et al.  Interpreting prediction markets: a stochastic approach , 2012, NIPS.

[12]  Shu-Heng Chen,et al.  Information aggregation and computational intelligence , 2017 .

[13]  Prithviraj Dasgupta,et al.  Partially observable stochastic game-based multi-agent prediction markets , 2011, AAMAS.

[14]  Nicolaas J. Vriend,et al.  Was Hayek an Ace? , 1999 .

[15]  Leigh Tesfatsion,et al.  Handbook of Computational Economics, Volume 2: Agent-Based Computational Economics (Handbook of Computational Economics) , 2006 .

[16]  R. Kitchin,et al.  Big Data, new epistemologies and paradigm shifts , 2014, Big Data Soc..

[17]  T. Vicsek,et al.  Hierarchical group dynamics in pigeon flocks , 2010, Nature.

[18]  Gregoris Mentzas,et al.  Agent based information aggregation markets , 2010, AAMAS.

[19]  Robert S. Erikson,et al.  Are Political Markets Really Superior to Polls as Election Predictors , 2008 .

[20]  Hal R. Varian,et al.  Big Data: New Tricks for Econometrics , 2014 .

[21]  B. LeBaron.  Agent-based Computational Finance , 2006 .

[22]  Justin Wolfers,et al.  Using Prediction Markets to Track Information Flows: Evidence from Google , 2009, AMMA.

[23]  Cass R. Sunstein,et al.  Neither Hayek nor Habermas , 2007 .

[24]  Kenneth A. Froot,et al.  Chartists, Fundamentalists, and Trading in the Foreign Exchange Market , 1989 .

[25]  Francis X. Diebold,et al.  On the Origin(s) and Development of the Term 'Big Data' , 2012 .

[26]  B L Partridge,et al.  The structure and function of fish schools. , 1982, Scientific American.

[27]  Herbert Gintis,et al.  Handbook of Computational Economics: Agent-Based Computational Economics (Handbook of Computational Economics S.) by K. L. Judd, L. Tesfatsion, M. D. Intriligator and Kenneth J. Arrow (eds.) , 2007, J. Artif. Soc. Soc. Simul..

[28]  Jaroslav Bukovina,et al.  Social media big data and capital markets—An overview , 2016 .

[29]  T. Lawson.  Economics and Reality , 1997 .

[30]  D. Ruths,et al.  Social media for large studies of behavior , 2014, Science.

[31]  Brian Pinkstone.  Reorienting Economics , 2003 .

[32]  Thomas C. Schelling,et al.  Dynamic models of segregation , 1971 .

[33]  R. Mabsout.  Abduction and economics: the contributions of Charles Peirce and Herbert Simon , 2015 .

[34]  Vernon L. Smith,et al.  Papers in Experimental Economics: Markets as Economizers of Information: Experimental Examination of the “Hayek Hypothesis” , 1991 .

[35]  Bin-Tzong Chie,et al.  Spatial Modeling of Agent-Based Prediction Markets: Role of Individuals , 2014, MABS.

[36]  Paul C. Tetlock,et al.  The Promise of Prediction Markets , 2008, Science.

[37]  Gerhard Schurz,et al.  Patterns of abduction , 2008, Synthese.

[38]  Matus Halas,et al.  Abductive Reasoning As The Logic Of Agent-Based Modelling , 2011 .

[39]  Antonios Siganos,et al.  Facebook's daily sentiment and international stock markets , 2014 .

[40]  Thomas A. Rietz,et al.  Results from a Dozen Years of Election Futures Markets Research , 2008 .

[41]  Bin-Tzong Chie,et al.  The Use of Knowledge in Prediction Markets: How Much of Them Need He Know? , 2015, J. Inf. Sci. Eng..

[42]  C. Hartshorne,et al.  Collected Papers of Charles Sanders Peirce , 1935, Nature.

[43]  Ravi Bansal,et al.  Nonparametric estimation of structural models for high-frequency currency market data , 1995 .

[44]  Winston A Hide,et al.  Big data: The future of biocuration , 2008, Nature.

[45]  R. Michael Alvarez,et al.  Computational Social Science: Discovery and Prediction , 2016, Analytical Methods for Social Research.

[46]  C. Sunstein.  Infotopia: How Many Minds Produce Knowledge , 2006 .

[47]  Abraham Othman,et al.  Zero-intelligence agents in prediction markets , 2008, AAMAS.

[48]  Mark P. Taylor,et al.  Charts, Noise and Fundamentals in the London Foreign Exchange Market , 1990 .

[49]  I. Couzin,et al.  Inferring the structure and dynamics of interactions in schooling fish , 2011, Proceedings of the National Academy of Sciences.

[50]  Jonathan Levin,et al.  Economics in the age of big data , 2014, Science.

[51]  Adam Barker,et al.  Undefined By Data: A Survey of Big Data Definitions , 2013, ArXiv.

[52]  G. Parisi,et al.  Interaction ruling animal collective behavior depends on topological rather than metric distance: Evidence from a field study , 2007, Proceedings of the National Academy of Sciences.

[53]  Shu-Heng Chen,et al.  Computational Behavioural Economics , 2017 .

[54]  Shu-Heng Chen,et al.  Agents learned, but do we? Knowledge discovery using the agent-based double auction markets , 2011 .

[55]  C. Manski.  Interpreting the Predictions of Prediction Markets , 2004 .

[56]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[57]  G. Gigerenzer.  Gut Feelings: The Intelligence of the Unconscious , 2007 .

[58]  Steven Durlauf,et al.  Growing artificial societies , 1997, Complex.

[59]  Daniel G. Campos,et al.  On the distinction between Peirce’s abduction and Lipton’s Inference to the best explanation , 2011, Synthese.

[60]  Graham K. Rand,et al.  Quantitative Applications in the Social Sciences , 1983 .

[61]  L. V. Williams,et al.  Prediction Markets , 2003 .

[62]  D. Edwards.  Data Mining: Concepts, Models, Methods, and Algorithms , 2003 .

[63]  Ying-Fang Kao,et al.  Herbert Simon and Agent-Based Computational Economics , 2016 .

[64]  Wolfgang Ziegler,et al.  Swarm Intelligence From Natural To Artificial Systems , 2016 .

[65]  M. Batty.  Generative social science: Studies in agent-based computational modeling , 2008 .

[66]  F. Hayek.  The economic nature of the firm: The use of knowledge in society , 1945 .

[67]  C. Peirce,et al.  Collected Papers of Charles Sanders Peirce , 1936, Nature.

[68]  B. Partridge.  Internal dynamics and the interrelations of fish in schools , 1981, Journal of comparative physiology.

[69]  Claudio Cioffi-Revilla,et al.  Introduction to Computational Social Science: Principles and Applications , 2017 .

[70]  A. Pentland,et al.  Life in the network: The coming age of computational social science: Science , 2009 .

[71]  Rob Kitchin,et al.  What makes Big Data, Big Data? Exploring the ontological characteristics of 26 datasets , 2016, Big Data Soc..

[72]  David Rothschild,et al.  Combining forecasts for elections: Accurate, relevant, and timely , 2015 .

[73]  Dongcheol Kim,et al.  Investor Sentiment from Internet Message Postings and Predictability of Stock Returns , 2014 .

[74]  Shu-Heng Chen,et al.  Varieties of agents in agent-based computational economics: A historical and an interdisciplinary perspective , 2012 .

[75]  John B. Kidd,et al.  Models of Discovery , 1978 .

[76]  W ReynoldsCraig.  Flocks, herds and schools: A distributed behavioral model , 1987 .

[77]  Murtaza Haider,et al.  Beyond the hype: Big data concepts, methods, and analytics , 2015, Int. J. Inf. Manag..