A Mathematical Theory of Generalization: Part II

A b st r a ct. The problem of how best to generalize from a given learning set of input-output examples is cent ral to the fields of neur al nets, statistics, approximation theory, and artificial intelligence. This series of pap ers inve stigates this problem from within an abst ract and mod elindepend ent fr amework and then test s some of the resultant concept s in real-world situations. In thi s abstrac t framework a generalizer is comple tely specified by a certain countably infinite set of functions, so the mathemati cs of generali zati on becomes an investigat ion into can dida te sets of criteria governing the behavior of that infinite set of funct ions . In the first pap er of thi s series the found ati ons of this mathematics are spelled out and some relatively simple generalization criteria are investigated . Elsewhere the real-world generalizing of systems constructed wit h these generalization criteria in mind have been favorably compared to neural nets for several real gene ralization problems, including Sejnowski 's problem of reading aloud. Th is leads to the conclusion that (current) neural nets in fact constitute a poo r means of general izing . In t he second of this pair of papers other sets of criteria, more sophisticated than those criteria embo died in this first series of pap ers, are investigated . Generalizers meeting these more sophis tica te d criteria can readil y be app roximated on computers. Some of these approximations employ network st ructures built via an evolutionary pro cess. A pr elimin ary and favo rable investigation in to the gener alization beh avior of these approxi mations finishes the second paper of this series.