CollabNet - Collaborative Deep Learning Network

The goal is an improvement on learning of deep neural networks. This improvement is here called the CollabNet network, which consists of a new method of insertion of new layers hidden in deep feedforward neural networks, changing the traditional way of stacking autoencoders. The new form of insertion is considered collaborative and seeks to improve the training against approaches based on stacked autoencoders. In this new approach, the addition of a new layer is carried out in a coordinated and gradual way, keeping under the control of the designer the influence of this new layer in training and no longer in a random and stochastic way as in the traditional stacking. The collaboration proposed in this work consists of making the learning of newly inserted layer continuing the learning obtained from previous layers, without prejudice to the global learning of the network. In this way, the freshly added layer collaborates with the previous layers and the set works in a way more aligned to the learning. CollabNet has been tested in the Wisconsin Breast Cancer Dataset database, obtaining a satisfactory and promising result.

[1]  Pascal Vincent,et al.  Deep Learning using Robust Interdependent Codes , 2009, AISTATS.

[2]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[3]  John E. Hopcroft,et al.  Stacked Generative Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  P. Mahadevan,et al.  An overview , 2007, Journal of Biosciences.

[5]  Thomas Hofmann,et al.  Greedy Layer-Wise Training of Deep Networks , 2007 .

[6]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[7]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[8]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[9]  Nathan S. Netanyahu,et al.  DeepPainter: Painter Classification Using Deep Convolutional Autoencoders , 2016, ICANN.

[10]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[11]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[12]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.