Acceleration of MapReduce Framework on a Multicore Processor

MapReduce framework is widely used in massive data processing, such as financial prediction, online marketing, and so on. Multicore processor is a great platform to implement MapReduce because of its inherent parallelism and flexibility. This book chapter extracts features of MapReduce applications, and proposes a software–hardware co-design framework based on a multi-core processor to improve the performance of MapReduce applications. Experimental results show that the MapReduce framework with hardware accelerators speeds up by 40 times at maximum compared to the pure software solution, and the proposed Topo-MapReduce speeds up further by 29% at maximum compared to the original MapReduce.

[1]  Zhiyi Yu,et al.  An 800MHz 320mW 16-core processor with message-passing and shared-memory inter-core communication mechanisms , 2012, 2012 IEEE International Solid-State Circuits Conference.

[2]  Volker Markl,et al.  Big Data Analytics on Modern Hardware Architectures: A Technology Survey , 2012, eBISS.

[3]  Jussara M. Almeida,et al.  GPU-NB: A Fast CUDA-Based Implementation of Naïve Bayes , 2013, 2013 25th International Symposium on Computer Architecture and High Performance Computing.

[4]  Jie Lin,et al.  Acceleration of Naive-Bayes algorithm on multicore processor for massive text classification , 2014, 2014 International Symposium on Integrated Circuits (ISIC).

[5]  Christoforos E. Kozyrakis,et al.  Evaluating MapReduce for Multi-core and Multiprocessor Systems , 2007, 2007 IEEE 13th International Symposium on High Performance Computer Architecture.

[6]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[7]  Yi Li,et al.  A 65nm 39GOPS/W 24-core processor with 11Tb/s/W packet-controlled circuit-switched double-layer network-on-chip and heterogeneous execution array , 2013, 2013 IEEE International Solid-State Circuits Conference Digest of Technical Papers.