Incremental constructive ridgelet neural network

In this paper, a new kind of neural network is proposed by combining ridgelet with feedforward neural network (FNN). The network adopts ridgelet as the activation function in the hidden layer, and an incremental constructive method is employed to determine the structure of the network. Since ridgelets are efficient in describing linear, curvilinear, and hyperplane like structures in high dimensions, accordingly the network can approximate quite a wide range of multivariate functions in a more stable and efficient way, especially those with certain kinds of spatial inhomogeneities. Moreover, the incremental extreme learning machine makes adding the hidden nodes one by one possible, and it only needs to adjust the output weights linking the hidden layer and the output layer when more hidden neurons are added. By defining the cost function as the difference between the previously approximated function and the currently approximating one, a genetic algorithm is used to determine the optimal directions of ridgelet neurons. The construction and learning of the network are presented in detail. The superiority of the proposed model is demonstrated by simulation experiments in function learning and image compression.

[1]  Jooyoung Park,et al.  Universal Approximation Using Radial-Basis-Function Networks , 1991, Neural Computation.

[2]  Shuyuan Yang,et al.  A New Adaptive Ridgelet Neural Network , 2005, ISNN.

[3]  Chee Kheong Siew,et al.  Universal Approximation using Incremental Constructive Feedforward Networks with Random Hidden Nodes , 2006, IEEE Transactions on Neural Networks.

[4]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[5]  Qinghua Zhang,et al.  Wavelet networks , 1992, IEEE Trans. Neural Networks.

[6]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[7]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[8]  O. Abdel-Wahhab,et al.  Image compression using multilayer neural networks , 1997 .

[9]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[10]  Moustafa M. Fahmy,et al.  Image compression using multi-layer neural networks , 1997, Proceedings Second IEEE Symposium on Computer and Communications.

[11]  E. Candès,et al.  Ridgelets: a key to higher-dimensional intermittency? , 1999, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[12]  Yang Shu.  HIGH-RATIO COMPRESSION OF REMOTE SENSING IMAGE BASED ON RIDGELET AND NEURAL NETWORK , 2007 .

[13]  Guang-Bin Huang,et al.  Learning capability and storage capacity of two-hidden-layer feedforward networks , 2003, IEEE Trans. Neural Networks.

[14]  Minh N. Do,et al.  The finite ridgelet transform for image representation , 2003, IEEE Trans. Image Process..