Using Bottlenecks in Feedforward Networks as a Dimension Reduction Technique: An Application to Optimization Tasks

For over 10 years now, connectionist networks have been applied to optimization tasks; most famously, to the Traveling Salesperson Problem (TSP; Hopfield and Tank 1985). The most popular approach is to express the task to be optimized in terms of an energy function, and then use network training algorithms to find weight configurations that minimize that function (Aarts and Stehouwer 1993). In this note, we investigate a novel application of connectionist networks to the TSP that exploits, instead, their ability to find compact, feature-preserving representations on hidden layers that have a restricted number of hidden units (Cottrell et al. 1987). In this formulation, the TSP can be conceptualized as a dimension reduction task: to find a path on an M-dimensional map of cities, one must reduce that map to a one-dimensional ordered list in which neighboring cities are listed close together. This list constitutes a path that visits every city exactly once. Here, a network with a one-unit bottleneck was trained on maps of cities to determine if suitable paths could be extracted from the featurepreserving representation formed on the bottleneck layer. Encouraging results are reported on 10and 30-city maps.