Temporal generalization capability of simple recurrent networks

Simple recurrent networks have been widely used in temporal processing applications. In this study we investigate temporal generalization of simple recurrent networks, drawing comparisons between network capabilities and human characteristics. Elman networks were trained to regenerate temporal trajectories sampled at different rates, and then tested with trajectories at both the trained sampling rates and at other sampling rates. The networks were also tested with trajectories representing mixtures of different sampling rates. It was found that for simple trajectories, the networks show interval invariance, but not rate invariance. However, for complex trajectories which contain greater contextual information, these networks do not seem to show any temporal generalization. Similar results were also obtained employing measured speech data. Thus, these results suggest that this class of networks exhibits severe limitations in temporal generalization.