Incorporation of liquid-crystal light valve nonlinearities in optical multilayer neural networks.

Sigmoidlike activation functions, as available in analog hardware, differ in various ways from the standard sigmoidal function because they are usually asymmetric, truncated, and have a nonstandard gain. We present an adaptation of the backpropagation learning rule to compensate for these nonstandard sigmoids. This method is applied to multilayer neural networks with all-optical forward propagation and liquid-crystal light valves (LCLV) as optical thresholding devices. The results of simulations of a backpropagation neural network with five different LCLV response curves as activation functions are presented. Although LCLV's perform poorly with the standard backpropagation algorithm, it is shown that our adapted learning rule performs well with these LCLV curves.

[1]  N. Collings,et al.  Liquid-crystal light valves as thresholding elements in neural networks: basic device requirements. , 1994, Applied optics.

[2]  Terrence J. Sejnowski,et al.  Analysis of hidden units in a layered network trained to classify sonar targets , 1988, Neural Networks.

[3]  R W Boyd,et al.  Passive one-way aberration correction using four-wave mixing. , 1988, Optics letters.

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  A A Friesem,et al.  All-optical neural network with inhibitory neurons. , 1989, Optics letters.

[6]  Emile Fiesler,et al.  Neural network classification and formalization , 1994 .

[7]  Demetri Psaltis,et al.  IV Adaptive Multilayer Optical Networks , 1993 .

[8]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[9]  K M Johnson,et al.  Noise analysis of polarization-based optoelectronic connectionist machines. , 1992, Applied optics.

[10]  H. J. Caulfield,et al.  Optical neural networks , 1989 .

[11]  Indu Saxena,et al.  Adaptive Multilayer Optical Neural Network with Optical Thresholding , 1995 .

[12]  Emile Fiesler,et al.  The Interchangeability of Learning Rate and Gain in Backpropagation Neural Networks , 1996, Neural Computation.

[13]  X Yang,et al.  Optical neural network with pocket-sized liquid-crystal televisions. , 1990, Optics letters.

[14]  H. B. Brown,et al.  Application Of The Liquid Crystal Light Valve To Real-Time Optical Data Processing , 1978 .

[15]  Soo-Young Lee,et al.  Dynamic optical interconnections using holographic lenslet arrays for adaptive neural networks , 1993 .

[16]  J. A. Hegt,et al.  Analog VLSI implementation of a feed-forward neural network , 1994 .

[17]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .