Performance evaluation of neural network architectures: the case of predicting foreign exchange correlations

In the last decade, neural networks have emerged from an esoteric instrument in academic research to a rather common tool assisting auditors, investors, portfolio managers and investment advisors in making critical financial decisions. It is apparent that a better understanding of the network's performance and limitations would help both researchers and practitioners in analysing real-world problems. Unlike many existing studies which focus on a single type of network architecture, this study evaluates and compares the performance of models based on two competing neural network architectures, the multi-layered feedforward neural network (MLFN) and general regression neural network (GRNN). Our empirical evaluation measures the network models' strength on the prediction of currency exchange correlation with respect to a variety of statistical tests including RMSE, MAE, U statistic, Theil's decomposition test, Henriksson-Merton market timing test and Fair-Shiller informational content test. Results of experiments suggest that the selection of proper architectural design may contribute directly to the success in neural network forecasting. In addition, market timing tests indicate that both MLFN and GRNN models have economically significant values in predicting the exchange rate correlation. On the other hand, informational content tests discover that the neural network models based on different architectures capture useful information not found in each other and the information sets captured by the two network designs are independent of one another. An auxiliary experiment is developed and confirms the possible synergetic effect from combining forecasts made by the two different network architectures and from incorporating information from an implied correlation model into the neural network forecasts. Implied correlation and random walk models are also included in our empirical experiment for benchmark comparison. Copyright © 2005 John Wiley & Sons, Ltd.

[1]  Bharat A. Jain,et al.  Artificial Neural Network Models for Pricing Initial Public Offerings , 1995 .

[2]  Lawrence C. Rose,et al.  European Diversification Opportunities in the 1990's , 1992 .

[3]  F. Longin,et al.  Is the Correlation in International Equity Returns Constant: 1960-90? , 1995 .

[4]  Milton S. Boyd,et al.  Forecasting futures trading volume using neural networks , 1995 .

[5]  Michael Galler,et al.  Analysis of Small-Business Financial Statements Using Neural Nets , 1995 .

[6]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[7]  S. Ghosh,et al.  An application of a multiple neural network learning system to emulation of mortgage underwriting judgements , 1988, IEEE 1988 International Conference on Neural Networks.

[8]  An-Sing Chen,et al.  Regression neural network for error correction in foreign exchange forecasting and trading , 2004, Comput. Oper. Res..

[9]  David Mayers,et al.  The value line enigma (1965-1978) : A case study of performance evaluation issues , 1982 .

[10]  Jose A. Lopez,et al.  Is Implied Correlation Worth Calculating? , 2000 .

[11]  Michael S. Gibson,et al.  Evaluating Forecasts of Correlation Using Option Pricing , 1997 .

[12]  Duane DeSieno,et al.  Trading Equity Index Futures With a Neural Network , 1992 .

[13]  Louis Wehenkel.  Contingency severity assessment for voltage security using non-parametric regression techniques , 1996 .

[14]  Hans-Georg Wittkemper,et al.  Using neural networks to forecast the systematic risk of stocks , 1996 .

[15]  Campbell R. Harvey,et al.  Forecasting International Equity Correlations , 1994 .

[16]  Frank de Jong,et al.  Price discovery in the foreign exchange market: an empirical analysis of the yen/dmark rate , 1998 .

[17]  Melody Y. Kiang,et al.  Managerial Applications of Neural Networks: The Case of Bank Failure Predictions , 1992 .

[18]  Junsei Tsukuda,et al.  Predicting Japanese corporate bankruptcy in terms of financial data using neural network , 1994 .

[19]  Kit Pong Wong,et al.  Hedging with mismatched currencies , 1999 .

[20]  E. Kaplanis,et al.  Stability and forecasting of the comovement measures of international stock market returns , 1988 .

[21]  Edward I. Altman,et al.  Corporate distress diagnosis: Comparisons using linear discriminant analysis and neural networks (the Italian experience) , 1994 .

[22]  Gregory R. Madey,et al.  The Application of Neural Networks and a Qualitative Response Model to the Auditor's Going Concern Uncertainty Decision* , 1995 .

[23]  D. S. Acton,et al.  Phase-diversity wave-front sensor for imaging systems. , 1994, Applied optics.

[24]  D. Mukesh,et al.  Applications of Neural Computing for Process Chemists: I. Introduction to Neural Network , 1996 .

[25]  Ram S. Sriram,et al.  A Comparison of Selected Artificial Neural Networks that Help Auditors Evaluate Client Financial Viability , 2000, Decis. Sci..

[26]  José Manuel Campa,et al.  The Forecasting Ability of Correlations Implied in Foreign Exchange Options , 1997 .

[27]  C. B. Tilanus,et al.  Applied Economic Forecasting , 1966 .

[28]  Hian Chye Koh,et al.  A neural network approach to the prediction of going concern status , 1999 .

[29]  Gordon Leitch,et al.  Economic Forecast Evaluation: Profits versus the Conventional Error Measures , 1991 .

[30]  Trefor P. Williams,et al.  NEURAL NETWORKS FOR BACKCALCULATION OF MODULI FROM SASW TEST , 1995 .

[31]  An-Sing Chen,et al.  Forecasting Exchange Rates Using General Regression Neural Networks , 1999, Comput. Oper. Res..

[32]  Ray C. Fair,et al.  Comparing Information in Forecasts from Econometric Models , 1990 .

[33]  Ethem Alpaydin,et al.  Parametric distance functions vs. nonparametric neural networks for estimating road travel distances , 1996 .

[34]  S. Dutta,et al.  Bond rating: a nonconservative application of neural networks , 1988, IEEE 1988 International Conference on Neural Networks.

[35]  E. Mine Cinar,et al.  Neural Networks: A New Tool for Predicting Thrift Failures , 1992 .

[36]  Herbert L. Jensen,et al.  Using Neural Networks for Credit Scoring , 1992 .