Activity-dependent neuron model for noise resistance

Abstract Activity-dependent plasticity plays an important role in biological neural network learning. Unlike the backpropagation-based learning in artificial neural networks that depends on supervised signals, biological neurons adjust themselves using their own historical behaviors as a clue to benefit information processing. Inspired by this biological neural mechanism, this paper proposes a novel neuron model, named Activity-Dependent Neuron (ADN). The basic idea of ADN is to add a gate in the neuron model to control its signal conduction capability, that is, the gate will facilitate transmission of important signals while suppress trivial signals. This idea can be achieved by self-tuning of activation behaviors. We also develop an iterative training algorithm, so that the ADNs can be smoothly incorporated into deep neural networks to jointly learn with the network weights. Experimental results found that the ADNs can efficiently improve the noise-resistant capability. Compared with the state-of-the-art, it is more robust to unforeseen noises. It does not need noise data for training.