Performance Optimization of Wildlife Recognition with Distributed Learning in Ecological Surveillance

Recently, deep learning technology has been widely used in ecological environment monitoring, but there are still huge challenges in specific tasks. For example, data sets have increased over the past few years, and in a CPU environment, training is quite time consuming. In order to solve these problems, we bring up a novel and effective method to improve training speed. In this paper, we make use of multi-GPUs to achieve distributed parallel acceleration. Small and shallow networks are not suitable for distributed training, because the computation of each parameter in this network is much higher than that of multi-layer perception or automatic encoder architecture, we use convolutional neural network with parameter sharing as the training network. In this paper, the main adopted method is to compare the performance of training wildlife recognition in single GPU and multi-GPUs environments by using distributed deep learning framework TensorFlow. The experimental results show that multi-GPUs which adopt distributed architecture can significantly accelerate training time consumption than single GPU. The results of this experiment also provides strong support for our follow-up work.

[1]  Luca Maria Gambardella,et al.  Deep Big Simple Neural Nets Excel on Handwritten Digit Recognition , 2010, ArXiv.

[2]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[3]  Tara N. Sainath,et al.  Optimization Techniques to Improve Training Speed of Deep Neural Networks for Large Speech Tasks , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[4]  John Langford,et al.  Slow Learners are Fast , 2009, NIPS.

[5]  Horacio Franco,et al.  Context-dependent connectionist probability estimation in a hybrid hidden Markov model-neural net speech recognition system , 1994, Comput. Speech Lang..

[6]  Luca Maria Gambardella,et al.  Deep, Big, Simple Neural Nets for Handwritten Digit Recognition , 2010, Neural Computation.

[7]  Lukás Burget,et al.  Parallel training of neural networks for speech recognition , 2010, INTERSPEECH.

[8]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[9]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[10]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[11]  Jhony-Heriberto Giraldo-Zuluaga,et al.  Recognition of Mammal Genera on Camera-Trap Images Using Multi-layer Robust Principal Component Analysis and Mixture Neural Networks , 2017, 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI).

[12]  Xiaohui Zhang,et al.  Improving deep neural network acoustic models using generalized maxout networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Tara N. Sainath,et al.  Scalable Minimum Bayes Risk Training of Deep Neural Network Acoustic Models Using Distributed Hessian-free Optimization , 2012, INTERSPEECH.

[14]  Samy Bengio,et al.  Understanding deep learning requires rethinking generalization , 2016, ICLR.

[15]  Yifan Gong,et al.  Restructuring of deep neural network acoustic models with singular value decomposition , 2013, INTERSPEECH.

[16]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[17]  Dong Yu,et al.  Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[18]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups , 2012, IEEE Signal Processing Magazine.

[19]  John C. Duchi,et al.  Distributed delayed stochastic optimization , 2011, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).