A Method for Agent-Based Models Validation

This paper proposes a new method for empirically validate simulation models that generate artificial time series data comparable with real-world data. The approach is based on comparing structures of vector autoregression models which are estimated from both artificial and real-world data by means of causal search algorithms. This relatively simple procedure is able to tackle both the problem of confronting theoretical simulation models with the data and the problem of comparing different models in terms of their empirical reliability. Moreover the paper provides an application of the validation procedure to the Dosi et al. (2015) macro-model.

[1]  Patrik O. Hoyer,et al.  Estimation of causal effects using linear non-Gaussian causal models with hidden variables , 2008, Int. J. Approx. Reason..

[2]  Hans-Werner Wohltmann,et al.  Agent-based financial markets and New Keynesian macroeconomics: a synthesis , 2013 .

[3]  Giorgio Fagiolo,et al.  Fiscal and Monetary Policies in Complex Evolving Economies , 2014 .

[4]  Mauro Gallegati,et al.  Validation in agent-based models: an investigation on the CATS model , 2008 .

[5]  Mauro Gallegati,et al.  Validating and Calibrating Agent-Based Models: A Case Study , 2007 .

[6]  Ian T. Jolliffe,et al.  Introduction to Multiple Time Series Analysis , 1993 .

[7]  Mattia Guerini,et al.  No Man is an Island: The Impact of Heterogeneity and Local Interactions on Macroeconomic Dynamics , 2016 .

[8]  R. Lucas.  Econometric policy evaluation: A critique , 1976 .

[9]  K. Hoover,et al.  Searching for the Causal Structure of a Vector Autoregression , 2003 .

[10]  Patrik O. Hoyer,et al.  Causal Search in Structural Vector Autoregressive Models , 2009, NIPS Mini-Symposium on Causality in Time Series.

[11]  Norman R. Swanson,et al.  Impulse Response Functions Based on a Causal Approach to Residual Orthogonalization in Vector Autoregressions , 1997 .

[12]  Jacobo Torán,et al.  The graph isomorphism problem , 2020, Commun. ACM.

[13]  S. Johansen,et al.  MAXIMUM LIKELIHOOD ESTIMATION AND INFERENCE ON COINTEGRATION — WITH APPLICATIONS TO THE DEMAND FOR MONEY , 2009 .

[14]  Tjalling C. Koopmans,et al.  Statistical Inference in Dynamic Economic Models , 1951 .

[15]  C. Sims.  MACROECONOMICS AND REALITY , 1977 .

[16]  G. Fagiolo,et al.  Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead , 2012, J. Artif. Soc. Soc. Simul..

[17]  Francesco Lamperti,et al.  Empirical validation of simulated models through the GSL-div: an illustrative application , 2016 .

[18]  G. Dosi,et al.  Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles , 2010 .

[19]  Thomas Lux,et al.  Estimation of an agent-based model of investor sentiment formation in financial markets , 2012 .

[20]  T. Lux,et al.  Estimation of Agent-Based Models: The Case of an Asymmetric Herding Model , 2005 .

[21]  Frank Schorfheide,et al.  Priors from General Equilibrium Models for Vars , 2002 .

[22]  M. Watson,et al.  Sources of Business Cycle Fluctuations , 1988, NBER Macroeconomics Annual.

[23]  Peter Winker,et al.  A global optimization heuristic for estimating agent based models , 2003, Comput. Stat. Data Anal..

[24]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[25]  Frank Schorfheide,et al.  Staff Reports Forming Priors for DSGE Models ( and How It Affects the Assessment of Nominal Rigidities ) , 2008 .

[26]  G. Dosi,et al.  Income Distribution, Credit and Fiscal Policies in an Agent-Based Keynesian Model , 2012 .

[27]  F. Lamperti.  An Information Theoretic Criterion for Empirical Validation of Time Series Models , 2015 .

[28]  Thomas Lux,et al.  Estimation of a simple agent-based model of financial markets: An application to Australian stock and foreign exchange data , 2006 .

[29]  Fabio Canova,et al.  Back to Square One: Identification Issues in DSGE Models , 2009, SSRN Electronic Journal.

[30]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[31]  Phyllis Illari,et al.  Why look at causality in the sciences : A manifesto , 2011 .

[32]  A. Kirman.  ANTS AND NONOPTIMAL SELF-ORGANIZATION: LESSONS FOR MACROECONOMICS , 2014, Macroeconomic Dynamics.

[33]  Robert E. Marks,et al.  Validation and model selection: Three similarity measures compared , 2013 .

[34]  J. Köbler,et al.  The Graph Isomorphism Problem: Its Structural Complexity , 1993 .

[35]  Helmut Ltkepohl,et al.  New Introduction to Multiple Time Series Analysis , 2007 .

[36]  Finn E. Kydland,et al.  Time to Build and Aggregate Fluctuations , 1982 .

[37]  Patrik O. Hoyer,et al.  Discovering Cyclic Causal Models by Independent Components Analysis , 2008, UAI.

[38]  A. Pagan,et al.  ISSUES IN ADOPTING DSGE MODELS FOR USE IN THE POLICY PROCESS , 2006 .

[39]  A. Kirman.  Whom Or What Does the Representative Individual Represent , 1992 .

[40]  K. Juselius.  On the Role of Theory and Evidence in Macroeconomics , 2010 .

[41]  Paul Windrum,et al.  A Critical Guide to Empirical Validation of Agent-Based Models in Economics: Methodologies, Procedures, and Open Problems , 2007 .

[42]  Isabelle Salle,et al.  Efficient Sampling and Metamodeling for Computational Economic Models , 2012 .

[43]  Matteo G. Richiardi,et al.  Estimation of ergodic agent-based models by simulated minimum distance , 2015 .

[44]  George C. Verghese,et al.  Graph similarity scoring and matching , 2008, Appl. Math. Lett..

[45]  Jennifer Widom,et al.  SimRank: a measure of structural-context similarity , 2002, KDD.

[46]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[47]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[48]  Alessio Moneta,et al.  Can Graphical Causal Inference Be Extended to Nonlinear Settings , 2009 .

[49]  C. Granger.  Testing for causality: a personal viewpoint , 1980 .

[50]  Sylvain Barde,et al.  Direct comparison of agent-based models of herding in financial markets , 2016 .

[51]  Peter N. Ireland,et al.  A Method for Taking Models to the Data , 2004 .

[52]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[53]  Alessio Moneta,et al.  Graphical causal models and VARs: an empirical assessment of the real business cycles hypothesis , 2008 .

[54]  Jakob Grazzini,et al.  Analysis of the Emergent Properties: Stationarity and Ergodicity , 2012, J. Artif. Soc. Soc. Simul..

[55]  D. Foley,et al.  The economy needs agent-based modelling , 2009, Nature.

[56]  Alan Kirman,et al.  The Intrinsic Limits of Modern Economic-Theory - the Emperor Has No Clothes , 1989 .

[57]  Aapo Hyvärinen,et al.  Estimation of a Structural Vector Autoregression Model Using Non-Gaussianity , 2010, J. Mach. Learn. Res..

[58]  Alex Coad,et al.  Causal Inference by Independent Component Analysis: Theory and Applications , 2013 .

[59]  A. Haldane,et al.  The dog and the frisbee , 2012 .

[60]  Edward C. Prescott,et al.  The Computational Experiment: an Econometric Tool , 1996 .

[61]  David A. Bessler,et al.  Money and prices: U.S. Data 1869–1914 (A study with directed graphs) , 2002 .

[62]  Oyer,et al.  Causal Inference by Independent Component Analysis: Theory and Applications∗ , 2012 .

[63]  Improving the toolbox. New advances in Agent-based and Computational Models , 2012 .

[64]  T. Haavelmo,et al.  The Probability Approach in Econometrics , 1944 .

[65]  Thomas Lux,et al.  Empirical validation of stochastic models of interacting agents , 2007 .

[66]  Blake LeBaron,et al.  Modeling Macroeconomies As Open-Ended Dynamic Systems of Interacting Agents , 2008 .

[67]  O. Blanchard.  A Traditional Interpretation of Macroeconomic Fluctuations , 1989 .

[68]  Aapo Hyvärinen,et al.  A Linear Non-Gaussian Acyclic Model for Causal Discovery , 2006, J. Mach. Learn. Res..

[69]  Olivier J. Blanchard,et al.  The Dynamic Effects of Aggregate Demand and Supply Disturbances , 1988 .

[70]  Tommaso Ciarli.  Structural interactions and long run growth: An application of Experimental Design to Agent Based Models , 2012 .

[71]  Marco Del Negro,et al.  How Good Is What You ’ ve Got ? DGSE-VAR as a Toolkit for Evaluating DSGE Models , 2006 .

[72]  Sylvain Barde.  A Practical, Accurate, Information Criterion for Nth Order Markov Processes , 2017 .

[73]  Giorgio Fagiolo,et al.  Macroeconomic Policy in DSGE and Agent-Based Models Redux: New Developments and Challenges Ahead , 2017 .