Induction as model selection

Overview of hierarchical Bayesian approach to learning structural form proposed by Kemp and Tenenbaum (3), using examples of similarities among a set of animals. (A) The data at the bottom, in the form of a feature vector for each animal, can potentially be produced by alternative forms (ring, partition, tree, order, hierarchy) that can take on many different structures (defined by nodes and edges in graph). Likelihoods constrain the possible structural forms to those consistent with the data of feature vectors (blue background), but the set of possibilities may remain large. (B) The set of possible structural forms is further constrained by the prior probability of each form and by the prior conditional probability of each structure given a form. The priors for structures conditional on forms favor simpler structures (those with fewer nodes). Bayesian inference identifies the specific structure (hierarchy in green) that has maximal probability as determined by the product of the likelihood and prior knowledge: P(S, F|D) ∝ …