Active Information Acquisition

We propose a general framework for sequential and dynamic acquisition of useful information in order to solve a particular task. While our goal could in principle be tackled by general reinforcement learning, our particular setting is constrained enough to allow more efficient algorithms. In this paper, we work under the Learning to Search framework and show how to formulate the goal of finding a dynamic information acquisition policy in that framework. We apply our formulation on two tasks, sentiment analysis and image recognition, and show that the learned policies exhibit good statistical performance. As an emergent byproduct, the learned policies show a tendency to focus on the most prominent parts of each instance and give harder instances more attention without explicitly being trained to do so.

[1]  Kilian Q. Weinberger,et al.  Classifier Cascade for Minimizing Feature Evaluation Cost , 2012, AISTATS.

[2]  Daphne Koller,et al.  Active Classification based on Value of Classifier , 2011, NIPS.

[3]  Brendan J. Frey,et al.  Learning Wake-Sleep Recurrent Attention Models , 2015, NIPS.

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[6]  He He,et al.  Imitation Learning by Coaching , 2012, NIPS.

[7]  John Langford,et al.  Learning to Search Better than Your Teacher , 2015, ICML.

[8]  Eileen Kowler,et al.  Eye movements during visual search: the costs of choosing the optimal path , 2001, Vision Research.

[9]  Antoine Cornuéjols,et al.  Early Classification of Time Series as a Non Myopic Sequential Decision Making Problem , 2015, ECML/PKDD.

[10]  Enrico Blanzieri,et al.  A survey of learning-based techniques of email spam filtering , 2008, Artificial Intelligence Review.

[11]  Matthieu Cord,et al.  Sequentially Generated Instance-Dependent Image Representations for Classification , 2014, ICLR.

[12]  Andrew McCallum,et al.  Selecting actions for resource-bounded information extraction using reinforcement learning , 2012, WSDM '12.

[13]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[14]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[15]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[16]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[17]  Nathan R. Sturtevant,et al.  Learning when to stop thinking and do something! , 2009, ICML '09.

[18]  John Langford,et al.  Efficient programmable learning to search , 2014, ArXiv.

[19]  Patrick Gallinari,et al.  Text Classification: A Sequential Reading Approach , 2011, ECIR.