Supervised Aggregation of Classifiers using Artificial Prediction Markets

Prediction markets are used in real life to predict outcomes of interest such as presidential elections. In this work we introduce a mathematical theory for Artificial Prediction Markets for supervised classifier aggregation and probability estimation. We introduce the artificial prediction market as a novel way to aggregate classifiers. We derive the market equations to enforce total budget conservation, show the market price uniqueness and give efficient algorithms for computing it. We show how to train the market participants by updating their budgets using training examples. We introduce classifier specialization as a new differentiating characteristic between classifiers. Finally, we present experiments using random decision rules as specialized classifiers and show that the prediction market consistently outperforms Random Forest on real and synthetic data of varying degrees of difficulty.

[1]  Francesco Tortorella,et al.  Reducing the classification cost of support vector classifiers through an ROC-based reject rule , 2004, Pattern Analysis and Applications.

[2]  Bogdan E. Popescu,et al.  PREDICTIVE LEARNING VIA RULE ENSEMBLES , 2008, 0811.1679.

[3]  J. Friedman.  Special Invited Paper-Additive logistic regression: A statistical view of boosting , 2000 .

[4]  Steven Gjerstad.  Risk Aversion, Beliefs, and Prediction Market Equilibrium , 2004 .

[5]  J. Gross,et al.  Graph Theory and Its Applications , 1998 .

[6]  Robin Hanson,et al.  The policy analysis market: an electronic commerce application of a combinatorial information market , 2003, EC '03.

[7]  Robert E. Schapire,et al.  The Boosting Approach to Machine Learning An Overview , 2003 .

[8]  Song-Chun Zhu,et al.  Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling , 1998, International Journal of Computer Vision.

[9]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[10]  Lawrence K. Saul,et al.  Think Globally, Fit Locally: Unsupervised Learning of Low Dimensional Manifold , 2003, J. Mach. Learn. Res..

[11]  David Schlossberg,et al.  Clinical Infectious Disease: Clinical Syndromes – Respiratory Tract , 2008 .

[12]  Justin Wolfers,et al.  Using Prediction Markets to Track Information Flows: Evidence from Google , 2009, AMMA.

[13]  Trevor Darrell,et al.  Learning appearance manifolds from video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Winston C. Yang,et al.  Parimutuel betting markets as information aggregation devices: experimental results , 2003 .

[15]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[16]  Mikhail Belkin,et al.  Problems of learning on manifolds , 2003 .

[17]  Manish Agrawal,et al.  Information Market Based Decision Fusion , 2009, ICIS.

[18]  Andrew B. Nobel,et al.  Sequential Procedures for Aggregating Arbitrary Estimators of a Conditional Mean , 2008, IEEE Transactions on Information Theory.

[19]  Peter A. Flach,et al.  Delegating classifiers , 2004, ICML.

[20]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[21]  C. Manski.  Interpreting the Predictions of Prediction Markets , 2004 .

[22]  Mikhail Belkin,et al.  Semi-Supervised Learning on Riemannian Manifolds , 2004, Machine Learning.

[23]  Paul C. Tetlock,et al.  The Promise of Prediction Markets , 2008, Science.

[24]  Huafeng Xu,et al.  A self-organizing principle for learning nonlinear manifolds , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[25]  J. Wolfers,et al.  Prediction Markets , 2003 .

[26]  F. Nelson,et al.  Use of prediction markets to forecast infectious disease activity. , 2007, Clinical infectious diseases : an official publication of the Infectious Diseases Society of America.

[27]  Ahmed M. Elgammal,et al.  Inferring 3D body pose from silhouettes using activity manifold learning , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..