Scaling Relationships in Back-Propagation Learning: Dependence on Training Set Size

We st ud y th e amount of ti me needed to learn a fixed t rain­ ing se t in the "back-pro pagation" proced ure for learning in multi-layer ne ural network models. The task chosen was 32-bit parity, a hi gh­ order fu nct ion for wh ich memor iza ti on o f specific in p u t- out put pairs is necessary. For small t raining sets, the learning time is consistent with a ~-power law depen dence on the nu mber of patterns in the t ra ining set. For lar ger training set s, t he learn ing t ime dive rges at a critical t ra ining set size which appears to be related to the st orage capacity of t he network.