Exploratory Analysis of Speedup Learning Data Using Epectation Maximization

Abstract Experimental evaluations of speedup learning methods have in the past used non-parametric hypothesis testing to determine whether or not learning is beneficial. We show here how to obtain deeper insight into the comparative performance of learning methods through a complementary parametric approach to data analysis. In this approach experimental data is used to estimate values for the parameters of a statistical model of the performance of a problem solver. To model problem solvers that use speedup learning methods, we propose a two-component linear model that captures how learned knowledge may accelerate the solution of some problems while leaving the solution of others relatively unchanged. We show how to apply expectation maximization (EM), a statistical technique, to fit this kind of multi-component model. EM allows us to fit the model in the presence of censored data, a methodological difficulty common to experiments involving speedup learning.

[1]  D. Rubin,et al.  Estimation and Hypothesis Testing in Finite Mixture Models , 1985 .

[2]  Steven Minton,et al.  Quantitative Results Concerning the Utility of Explanation-based Learning , 1988, Artif. Intell..

[3]  John N. Hooker,et al.  Needed: An Empirical Science of Algorithms , 1994, Oper. Res..

[4]  Raymond J. Mooney,et al.  The Effect of Rule Use on the Utility of Explanation-Based Learning , 1989, IJCAI.

[5]  E. Feigenbaum,et al.  Computers and Thought , 1963 .

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  Alberto Maria Segre,et al.  Using Hundreds of Workstations to Solve First-Order Logic Problems , 1994, AAAI.

[8]  Charles Elkan,et al.  Unsupervised learning of multiple motifs in biopolymers using expectation maximization , 1995, Mach. Learn..

[9]  Kamel Jedidi,et al.  A maximum likelihood method for latent class regression involving a censored dependent variable , 1993 .

[10]  Allen Newell,et al.  Empirical explorations with the logic theory machine: a case study in heuristics , 1995 .

[11]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[12]  Alberto Maria Segre On Combining Multiple Speedup Techniques , 1992, ML.

[13]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[14]  Charles Elkan,et al.  A High-Performance Explanation-Based Learning Algorithm , 1994, Artif. Intell..

[15]  Susan Chipman,et al.  Foundations of Knowledge Acquisition , 1993 .