Independent Factor Reinforcement Learning for Portfolio Management

In this paper we propose to do portfolio management using reinforcement learning (RL) and independent factor model. Factors in independent factor model are mutually independent and exhibit better predictability. RL is applied to each factor to capture temporal dependence and provide investment suggestion on factor. Optimal weights on factors are found by portfolio optimization method subject to the investment suggestions and general portfolio constraints. Experimental results and analysis are given to show that the proposed method has better performance when compare to two alternative portfolio management systems.

[1]  Lei Xu,et al.  An extended ASLD trading system to enhance portfolio management , 2003, IEEE Trans. Neural Networks.

[2]  M A H Dempster,et al.  An automated FX trading system using adaptive reinforcement learning , 2006, Expert Syst. Appl..

[3]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[4]  Klaus-Robert Müller,et al.  Prediction of Mixtures , 1996, ICANN.

[5]  Ralph Neuneier,et al.  Optimal Asset Allocation using Adaptive Dynamic Programming , 1995, NIPS.

[6]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[7]  Andrew W. Lo,et al.  Computational finance , 1999, Comput. Sci. Eng..

[8]  Peter W. Glynn,et al.  Kernel-Based Reinforcement Learning in Average-Cost Problems: An Application to Optimal Portfolio Choice , 2000, NIPS.

[9]  Andrew D. Back,et al.  A First Application of Independent Component Analysis to Extracting Structure from Stock Returns , 1997, Int. J. Neural Syst..

[10]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[11]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[12]  Ralph Neuneier,et al.  Enhancing Q-Learning for Optimal Asset Allocation , 1997, NIPS.

[13]  Lai-Wan Chan,et al.  Applying Independent Component Analysis to Factor Model in Finance , 2000, IDEAL.

[14]  Lai-Wan Chan,et al.  SELECTION OF INDEPENDENT FACTOR MODEL IN FINANCE , 2001 .

[15]  Erkki Oja,et al.  Independent Component Analysis for Parallel Financial Time Series , 1998, International Conference on Neural Information Processing.

[16]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .