Singularities in Learning Theory(Recent Topics on Real and Complex Singularities)

This paper shows the problems of singularities in learning theory. We introduce two important observables in learning theory, stochastic complexity and generalization error. A lot of learning machines used in information science have singularities in their parameter space, resulting that their mathematical properties have been left unknown. In this paper we show that the asymptotic behaviors of stochastic complexity and generalization error can be identified by the largest pole and its order of the zeta function of the learning machine.