Scalable matrix inversion using MapReduce

Matrix operations are a fundamental building block of many computational tasks in fields as diverse as scientific computing, machine learning, and data mining. Matrix inversion is an important matrix operation, but it is difficult to implement in today's popular parallel dataflow programming systems, such as MapReduce. The reason is that each element in the inverse of a matrix depends on multiple elements in the input matrix, so the computation is not easily partitionable. In this paper, we present a scalable and efficient technique for matrix inversion in MapReduce. Our technique relies on computing the LU decomposition of the input matrix and using that decomposition to compute the required matrix inverse. We present a technique for computing the LU decomposition and the matrix inverse using a pipeline of MapReduce jobs. We also present optimizations of this technique in the context of Hadoop. To the best of our knowledge, our technique is the first matrix inversion technique using MapReduce. We show experimentally that our technique has good scalability, enabling us to invert a 10^5 x 10^5 matrix in 5 hours on Amazon EC2. We also show that our technique outperforms ScaLAPACK, a state-of-the-art linear algebra package that uses MPI.

[1]  Jin-Soo Kim,et al.  HPMR: Prefetching and pre-shuffling in shared MapReduce computation environment , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[2]  Yi Zhang,et al.  Optimizing I/O for Big Array Analytics , 2012, Proc. VLDB Endow..

[3]  Michael D. Ernst,et al.  HaLoop , 2010, Proc. VLDB Endow..

[4]  Jingen Xiang,et al.  Scalable Scientific Computing Algorithms Using MapReduce , 2013 .

[5]  Emmanuel Agullo,et al.  LU factorization for accelerator-based systems , 2011, 2011 9th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA).

[6]  Aart J. C. Bik,et al.  Pregel: a system for large-scale graph processing , 2010, SIGMOD Conference.

[7]  William H. Press,et al.  Numerical Recipes 3rd Edition: The Art of Scientific Computing , 2007 .

[8]  Kun Li,et al.  The MADlib Analytics Library or MAD Skills, the SQL , 2012, Proc. VLDB Endow..

[9]  James T. Dobbins,et al.  Applications of matrix inversion tomosynthesis , 2000, Medical Imaging.

[10]  Peter J. Haas,et al.  Large-scale matrix factorization with distributed stochastic gradient descent , 2011, KDD.

[11]  Erik Elmroth,et al.  Design and Evaluation of Parallel Block Algorithms: LU Factorization on an IBM 3090 VF/600J , 1991, PPSC.

[12]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[13]  Jack J. Dongarra,et al.  The LINPACK Benchmark: past, present and future , 2003, Concurr. Comput. Pract. Exp..

[14]  Yannis Sismanis,et al.  Sparkler: supporting large-scale matrix factorization , 2013, EDBT '13.

[15]  Michael Isard,et al.  DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language , 2008, OSDI.

[16]  Stefan Lüpke,et al.  LU-Decomposition on a Massively Parallel Transputer System , 1993, PARLE.

[17]  Wang Xiang-qian.  Application of HPMR in Parallel Matrix Computation , 2010 .

[18]  Robert A. van de Geijn,et al.  Families of algorithms related to the inversion of a Symmetric Positive Definite matrix , 2008, TOMS.

[19]  Anthony Skjellum,et al.  A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard , 1996, Parallel Comput..

[20]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[21]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[22]  Jack Dongarra,et al.  ScaLAPACK Users' Guide , 1987 .

[23]  Jack J. Dongarra,et al.  High performance matrix inversion based on LU factorization for multicore architectures , 2011, MTAGS '11.

[24]  Thomas A. Hopf,et al.  Protein 3D Structure Computed from Evolutionary Sequence Variation , 2011, PloS one.

[25]  Werner Dubitzky,et al.  A Practical Approach to Microarray Data Analysis , 2003, Springer US.

[26]  Luis Mateus Rocha,et al.  Singular value decomposition and principal component analysis , 2003 .

[27]  Bart De Schutter,et al.  The QR decomposition and the singular value decomposition in the symmetrized max-plus algebra , 1997, 1997 European Control Conference (ECC).

[28]  Zhengping Qian,et al.  MadLINQ: large-scale distributed matrix computation for the cloud , 2012, EuroSys '12.

[29]  Jack Dongarra,et al.  LAPACK: a portable linear algebra library for high-performance computers , 1990, SC.

[30]  Jingen Xiang,et al.  Probing the spatial distribution of the interstellar dust medium by high angular resolution X-ray halos of point sources , 2005 .

[31]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[32]  Robert A. van de Geijn,et al.  A Note On Parallel Matrix Inversion , 2000, SIAM J. Sci. Comput..

[33]  Shirish Tatikonda,et al.  SystemML: Declarative machine learning on MapReduce , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[34]  David Jones.  High performance , 1989, Nature.

[35]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[36]  Randy H. Katz,et al.  Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center , 2011, NSDI.

[37]  R. Venkatesh,et al.  Parallel matrix inversion techniques , 1996, Proceedings of 1996 IEEE Second International Conference on Algorithms and Architectures for Parallel Processing, ICA/sup 3/PP '96.