A novel theory for studying the learning behavior of a neural network which is formed by interconnecting neurons is presented. This learning theory constitutes a new approach to the Boltzmann machine. The central idea is to minimize one of the two cross-entropy-like criteria, the cross-entropy and the reversed cross-entropy; the latter is used by Ackley et al. (Cognitive Sci., vol.9, p.147-59, 1985) in deriving the Boltzmann machine. The results derived by the present approach are closely related to those obtained by Ackley et al., with several significant modifications in the algorithm. A detailed discussion of the new algorithm, which is shown to be a probability-weighted version of the algorithm by Ackley et al., is presented.<<ETX>>
[1]
Rodney W. Johnson,et al.
Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy
,
1980,
IEEE Trans. Inf. Theory.
[2]
Geoffrey E. Hinton,et al.
A Learning Algorithm for Boltzmann Machines
,
1985,
Cogn. Sci..
[3]
Teuvo Kohonen,et al.
Self-Organization and Associative Memory
,
1988
.
[4]
Alvin W. Drake,et al.
Fundamentals of Applied Probability Theory
,
1967
.
[5]
Terrence J. Sejnowski,et al.
NETtalk: a parallel network that learns to read aloud
,
1988
.