A MapReduce Cortical Algorithms Implementation for Unsupervised Learning of Big Data

In the big data era, the need for fast robust machine learning techniques is rapidly increasing. Deep network architectures such as cortical algorithms are challenged by big data problems which result in lengthy and complex training. In this paper, we present a distributed cortical algorithm implementation for the unsupervised learning of big data based on a combined node-data parallelization scheme. A data sparsity measure is used to divide the data before distributing the columns in the network over many computing nodes based on the MapReduce framework. Experimental results on multiple datasets showed an average speedup of 8.1× compared to serial implementations.

[1]  Rong Gu,et al.  A parallel computing platform for training large scale neural networks , 2013, 2013 IEEE International Conference on Big Data.

[2]  Rainer Goebel,et al.  "Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech , 2008, Science.

[3]  D. N. Ranasinghe,et al.  On the Performance of Parallel Neural Network Implementations on Distributed Memory Architectures , 2008, 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid (CCGRID).

[4]  G. Edelman,et al.  The Mindful Brain: Cortical Organization and the Group-Selective Theory of Higher Brain Function , 1978 .

[5]  Mikko H. Lipasti,et al.  Profiling Heterogeneous Multi-GPU Systems to Accelerate Cortically Inspired Learning Algorithms , 2011, 2011 IEEE International Parallel & Distributed Processing Symposium.

[6]  Vipin Kumar,et al.  Introduction to Data Mining , 2022, Data Mining and Machine Learning Applications.

[7]  M E J Newman,et al.  Finding and evaluating community structure in networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[8]  Nadine Hajj,et al.  Weighted entropy cortical algorithms for isolated Arabic speech recognition , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[9]  Honglak Lee,et al.  Unsupervised feature learning for audio classification using convolutional deep belief networks , 2009, NIPS.

[10]  Mikko H. Lipasti,et al.  A case for neuromorphic ISAs , 2011, ASPLOS XVI.

[11]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[12]  P. Werstein,et al.  Parallelization of a Backpropagation Neural Network on a Cluster Computer , 2022 .

[13]  Ashraf A. Kassim,et al.  Gini Index as Sparsity Measure for Signal Reconstruction from Compressive Samples , 2011, IEEE Journal of Selected Topics in Signal Processing.

[14]  Mikko H. Lipasti,et al.  Cortical columns: Building blocks for intelligent systems , 2009, 2009 IEEE Symposium on Computational Intelligence for Multimedia Signal and Vision Processing.

[15]  Mikko H. Lipasti,et al.  Discovering Cortical Algorithms , 2018, IJCCI.