A computational theory for movement pattern recognition based on optimal movement pattern generation

We have previously proposed an optimal trajectory and control theory for continuous movements, such as reaching or cursive handwriting. According to Marr's three-level description of brain function, our theory can be summarized as follows: (1) The computational theory is the minimum torque-change model; (2) the intermediate representation of a pattern is given as a set of via-points extracted from an example pattern; and (3) algorithm and hardware are provided by FIRM, a neural network that can generate and control minimum torque-change trajectories. In this paper, we propose a computational theory for movement pattern recognition that is based on our theory for optimal movement pattern generation. The three levels of the description of brain function in the recognition theory are tightly coupled with those for pattern generation. In recognition, the generation process and the recognition process are actually two flows of information in opposite directions within a single functional unit. In our theory, if the input movement trajectory data are identical to the optimal movement pattern reconstructed from an intermediate representation of some symbol, the input data are recognized as that symbol. If an error exists between the movement trajectory data and the generated trajectory, the putative symbol is corrected, and the generation is repeated. In particular, we present concrete computational procedures for the recognition of connected cursive handwritten characters, as well as for the estimation of phonemic timing in natural speech. Our most important contribution is to demonstrate the computational realizability for the ‘motor theory of movement pattern perception’: the movement-pattern recognition process can be realized by actively recruiting the movementpattern formation process. The way in which the formation process is utilized in pattern recognition in our theory suggests a duality between movement pattern formation and movement pattern perception.