Missing Data with Recurrent Networks Handling Asynchronous or Missing Data with Recurrent Networks

An important issue with many sequential data analysis problems such as those encoun tered in nancial data sets is that di erent variables are known at di erent frequencies at di erent times asynchronicity or are sometimes missing To address this issue we propose to use recurrent networks with feedback into the input units based on two fun damental ideas The rst motivation is that the lled in value of the missing variable may not only depend in complicated ways on the value of this variable in the past of the sequence but also on the current and past values of other variables The second motivation is that for the purpose of making predictions or taking decisions it is not always necessary to ll in the best possible value of the missing variables In fact it is su cient to ll in a value which helps the system make better predictions or decisions The advantages of this approach are demonstrated through experiments on several tasks Missing Data with Recurrent Networks

[1]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[2]  Yoshua Bengio,et al.  Using a Financial Training Criterion Rather than a Prediction Criterion , 1997, Int. J. Neural Syst..

[3]  Yoshua Bengio,et al.  Neural networks for speech and sequence recognition , 1996 .

[4]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[5]  L. B. Almeida A learning rule for asynchronous perceptrons with feedback in a combinatorial environment , 1990 .

[6]  Fernando J. Pineda,et al.  Recurrent Backpropagation and the Dynamical Approach to Adaptive Neural Computation , 1989, Neural Computation.

[7]  Hervé Bourlard,et al.  Generalization and Parameter Estimation in Feedforward Netws: Some Experiments , 1989, NIPS.

[8]  Yves Chauvin Dynamic Behavior of Constained Back-Propagation Networks , 1989, NIPS.

[9]  Yann LeCun,et al.  Reverse TDNN: An Architecture For Trajectory Generation , 1991, NIPS.

[10]  Volker Tresp,et al.  Some Solutions to the Missing Feature Problem in Vision , 1992, NIPS.

[11]  Patrick Gallinari,et al.  A Framework for the Cooperation of Learning Algorithms , 1990, NIPS.

[12]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[13]  Yann LeCun,et al.  Second Order Properties of Error Surfaces , 1990, NIPS.

[14]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[15]  Michael I. Jordan,et al.  Supervised learning from incomplete data via an EM approach , 1993, NIPS.

[16]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[17]  Isabelle Guyon,et al.  Automatic Capacity Tuning of Very Large VC-Dimension Classifiers , 1992, NIPS.

[18]  B. V. K. Vijaya Kumar,et al.  Shooting Craps in Search of an Optimal Strategy for Training Connectionist Pattern Classifiers , 1991, NIPS.

[19]  Yann LeCun,et al.  Improving the convergence of back-propagation learning with second-order methods , 1989 .

[20]  E. R. Bareiss,et al.  Protos: An Exemplar-Based Learning Apprentice , 1988, Int. J. Man Mach. Stud..

[21]  Pineda,et al.  Generalization of back-propagation to recurrent neural networks. , 1987, Physical review letters.