Information-Based Objective Functions for Active Data Selection

Learning can be made more efficient if we can actively select particularly salient data points. Within a Bayesian learning framework, objective functions are discussed that measure the expected informativeness of candidate measurements. Three alternative specifications of what we want to gain information about lead to three different criteria for data selection. All these criteria depend on the assumption that the hypothesis space is correct, which may prove to be their main weakness.

[1]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[2]  K. Abromeit.  Music Received , 2023, Notes.

[3]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[4]  S. Luttrell.  The use of transinformation in the design of data sampling schemes for inverse problems , 1985 .

[5]  J. Justice.  Maximum entropy and bayesian methods in applied statistics , 1986 .

[6]  E. T. Jaynes,et al.  BAYESIAN METHODS: GENERAL BACKGROUND ? An Introductory Tutorial , 1986 .

[7]  J. Berger.  Statistical Decision Theory and Bayesian Analysis , 1988 .

[8]  T. Loredo.  From Laplace to Supernova SN 1987A: Bayesian Inference in Astrophysics , 1990 .

[9]  Mahmoud A. El-Gamal.  The Role of Priors in Active Bayesian Learning in the Sequential Statistical Decision Framework , 1991 .

[10]  Eric B. Baum,et al.  Neural net algorithms that learn in polynomial time from examples and queries , 1991, IEEE Trans. Neural Networks.

[11]  D. Mackay,et al.  A Practical Bayesian Framework for Backprop Networks , 1991 .

[12]  Jenq-Neng Hwang,et al.  Query-based learning applied to partially trained multilayer perceptrons , 1991, IEEE Trans. Neural Networks.

[13]  J. Skilling.  Bayesian Solution of Ordinary Differential Equations , 1992 .

[14]  David J. C. MacKay,et al.  The Evidence Framework Applied to Classification Networks , 1992, Neural Computation.

[15]  David J. C. MacKay,et al.  Bayesian Interpolation , 1992, Neural Computation.

[16]  David J. C. MacKay,et al.  A Practical Bayesian Framework for Backpropagation Networks , 1992, Neural Computation.