Understanding the Predictive Power of Computational Mechanics and Echo State Networks in Social Media

There is a large amount of interest in understanding users of social media in order to predict their behavior in this space. Despite this interest, user predictability in social media is not well-understood. To examine this question, we consider a network of fifteen thousand users on Twitter over a seven week period. We apply two contrasting modeling paradigms: computational mechanics and echo state networks. Both methods attempt to model the behavior of users on the basis of their past behavior. We demonstrate that the behavior of users on Twitter can be well-modeled as processes with self-feedback. We find that the two modeling approaches perform very similarly for most users, but that they differ in performance on a small subset of the users. By exploring the properties of these performance-differentiated users, we highlight the challenges faced in applying predictive models to dynamic social data.

[1]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[2]  Robert Haslinger,et al.  The Computational Structure of Spike Trains , 2009, Neural Computation.

[3]  Simon Dedeo,et al.  Evidence for Non-Finite-State Computation in a Human Social System , 2012, ArXiv.

[4]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[5]  Colin Campbell,et al.  Kernel methods: a survey of current techniques , 2002, Neurocomputing.

[6]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[7]  C. Shalizi,et al.  Causal architecture, complexity and self-organization in time series and cellular automata , 2001 .

[8]  Simon DeDeo,et al.  Collective Phenomena and Non-Finite State Computation in a Human Social System , 2012, PloS one.

[9]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[10]  Cosma Rohilla Shalizi.  Optimal Nonlinear Prediction of Random Fields on Networks , 2003, DMCS.

[11]  Lluís Padró Cirera,et al.  A named entity recognition system based on a finite automata acquisition algorithm , 2005 .

[12]  Camille Roth,et al.  Intertemporal topic correlations in online media , 2007 .

[13]  Aram Galstyan,et al.  Information transfer in social media , 2011, WWW.

[14]  James P. Crutchfield,et al.  Computational Mechanics: Pattern and Prediction, Structure and Simplicity , 1999, ArXiv.

[15]  K. Marton,et al.  Entropy and the Consistent Estimation of Joint Distributions , 1993, Proceedings. IEEE International Symposium on Information Theory.

[16]  Peter Michael Young,et al.  A tighter bound for the echo state property , 2006, IEEE Transactions on Neural Networks.

[17]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[18]  Garrison W. Cottrell,et al.  2007 Special Issue: Learning grammatical structure with Echo State Networks , 2007 .

[19]  Cosma Rohilla Shalizi,et al.  Blind Construction of Optimal Nonlinear Recursive Predictors for Discrete Sequences , 2004, UAI.

[20]  Benjamin Schrauwen,et al.  An overview of reservoir computing: theory, applications and implementations , 2007, ESANN.

[21]  H. Jaeger,et al.  Overview of Reservoir Recipes A survey of new RNN training methods that follow the Reservoir paradigm , 2007 .

[22]  Aram Galstyan,et al.  Latent Point Process Models for Spatial-Temporal Networks , 2013, ArXiv.

[23]  Paul-Gerhard Plöger,et al.  Echo State Networks used for Motor Control , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[24]  S. Caires,et al.  On the Non-parametric Prediction of Conditionally Stationary Sequences , 2005 .

[25]  José Carlos Príncipe,et al.  Analysis and Design of Echo State Networks , 2007, Neural Computation.

[26]  Patrick J. Wolfe,et al.  Point process modelling for directed interaction networks , 2010, ArXiv.

[27]  Harald Haas,et al.  Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication , 2004, Science.

[28]  Herbert Jaeger,et al.  Reservoir computing approaches to recurrent neural network training , 2009, Comput. Sci. Rev..

[29]  Aram Galstyan,et al.  Latent self-exciting point process model for spatial-temporal networks , 2014 .