The Storage Capacity of a Fully-Connected Committee Machine

We study the storage capacity of a fully-connected committee machine with a large number K of hidden nodes. The storage capacity is obtained by analyzing the geometrical structure of the weight space related to the internal representation. By examining the asymptotic behavior of order parameters in the limit of large K, the storage capacity αc is found to be proportional to K√ln K up to the leading order. This result satisfies the mathematical bound given by Mitchison and Durbin, whereas the replica-symmetric solution in a conventional Gardner's approach violates this bound.