Neural Networks for the Segmentation of Teleoperation Tasks

In recent years, research in telemanipulation has been concerned primarily with problems related to the control of teleoperators and their man-machine aspects. As the need for extensive teleoperation for the deployment and maintenance of space systems increases, research needs to focus on techniques to improve operator performance and reduce operator fatigue during telemanipulation. This paper describes the development of an approach to teleoperation task supervision that can be used in advanced man-machine interfaces to monitor teleoperation performance and to help operators with task-level feedback. A Segmentation Program has been designed to identify teleoperation task phases, independently of variations in working conditions and in phase duration and sensor values. The program uses neural networks to segment the force data of a peg-in-hole task into the task phase sequence. The network connections were trained with an off-line Hidden Markov Model of the peg-in-hole task providing specific transition times and task phase sequences. Two network architectures have been tested during simulation with real teleoperation data: the first is based on turning temporal sequences into spatial patterns; the second extends the first model by including network output in the input array. We found that the first architecture needed a higher number of iterations to learn the associations, while the latter had a higher convergency speed and recognition rate. The architecture with the best performance was used in the realtime implementation of the segmentation program on a teleoperation system. In experiments of peg-in-hole tasks, the network had a lower recognition rate than in simulation, but it showed unexpected generalization capabilities by correctly segmenting tasks whose phase sequence was significantly different from those sequences in the training data.