This paper presents a complex-valued version of the back-propagation algorithm (called 'Complex-BP'), which can be applied to multi-layered neural networks whose weights, threshold values, input and output signals are all complex numbers. Some inherent properties of this new algorithm are studied. The results may be summarized as follows. The updating rule of the Complex-BP is such that the probability for a "standstill in learning" is reduced. The average convergence speed is superior to that of the real-valued back-propagation, whereas the generalization performance remains unchanged. In addition, the number of weights and thresholds needed is only about the half of real-valued back-propagation, where a complex-valued parameter z=x+iy (where i=-1) is counted as two because it consists of a real part x and an imaginary part y. The Complex-BP can transform geometric figures, e.g. rotation, similarity transformation and parallel displacement of straight lines, circles, etc., whereas the real-valued back-propagation cannot. Mathematical analysis indicates that a Complex-BP network which has learned a transformation, has the ability to generalize that transformation with an error which is represented by the sine. It is interesting that the above characteristics appear only by extending neural networks to complex numbers.
[1]
Geoffrey E. Hinton,et al.
Learning internal representations by error propagation
,
1986
.
[2]
Akira Watanabe,et al.
Interpretation of Optical Flow Through Complex Neural Network
,
1993,
IWANN.
[3]
Akira Watanabe,et al.
A Method to Interpret 3D Motions Using Neural Networks (Special Section on Information Theory and Its Applications)
,
1994
.
[4]
Akira Watanabe,et al.
Interpretation of optical flow through neural network learning
,
1992,
[Proceedings] Singapore ICCS/ISITA `92.
[5]
Clark C. Guest,et al.
Modification of backpropagation networks for complex-valued signal processing in frequency domain
,
1990,
1990 IJCNN International Joint Conference on Neural Networks.
[6]
M. Hart,et al.
Complex analysis and applications
,
1985
.
[7]
Shun-ichi Amari,et al.
A Theory of Adaptive Pattern Classifiers
,
1967,
IEEE Trans. Electron. Comput..
[8]
B. Widrow,et al.
The complex LMS algorithm
,
1975,
Proceedings of the IEEE.