Vectorized OpenCL implementation of numerical integration for higher order finite elements

In our work we analyze computational aspects of the problem of numerical integration in finite element calculations and consider an OpenCL implementation of related algorithms for processors with wide vector registers. As a platform for testing the implementation we choose the PowerXCell processor, being an example of the Cell Broadband Engine (CellBE) architecture. Although the processor is considered old for today's standards (its design dates back to year 2001), we investigate its performance due to two features that it shares with recent Xeon Phi family of co-processors: wide vector units and relatively slow connection of computing cores with main global memory. The performed analysis of parallelization options can also be used for designing numerical integration algorithms for other processors with vector registers, such as contemporary x86 microprocessors. We consider higher order finite element approximations and implement the standard algorithm of numerical integration for prismatic elements. Original contributions of the paper include the analysis of data movement and vector operations performed during code execution. Several versions of the implementation are developed and tested in practice.

[1]  N.K. Govindaraju,et al.  A Memory Model for Scientific Algorithms on Graphics Processors , 2006, ACM/IEEE SC 2006 Conference (SC'06).

[2]  Krzysztof Banas,et al.  Higher order FEM numerical integration on GPUs with OpenCL , 2010, Proceedings of the International Multiconference on Computer Science and Information Technology.

[3]  Robert Michael Kirby,et al.  From h to p efficiently: Implementing finite and spectral/hp element methods to achieve optimal performance for low- and high-order discretisations , 2010, J. Comput. Phys..

[4]  Lukasz Szustak,et al.  Model-driven adaptation of double-precision matrix multiplication to the Cell processor architecture , 2012, Parallel Comput..

[5]  Gordon Erlebacher,et al.  Porting a high-order finite-element earthquake modeling application to NVIDIA graphics cards using CUDA , 2009, J. Parallel Distributed Comput..

[6]  Victor M. Calo,et al.  Computational complexity and memory usage for multi-frontal direct solvers used in p finite element analysis , 2011, ICCS.

[7]  James Reinders,et al.  Intel Xeon Phi Coprocessor High Performance Programming , 2013 .

[8]  Jens Markus Melenk,et al.  Fully discrete hp-finite elements: fast quadrature , 2001 .

[9]  Krzysztof Banas,et al.  Numerical integration on GPUs for higher order finite elements , 2013, Comput. Math. Appl..

[10]  L. Demkowicz.  One and two dimensional elliptic and Maxwell problems , 2006 .

[11]  Krzysztof Banas,et al.  Finite Element Numerical Integration on GPUs , 2009, PPAM.

[12]  Matthew G. Knepley,et al.  Finite Element Integration on GPUs , 2013, TOMS.

[13]  Eric Darve,et al.  Application of Assembly of Finite Element Methods on Graphics Processors for Real-Time Elastodynamics , 2011 .

[14]  Michal Mrozowski,et al.  Generation of large finite-element matrices on multiple graphics processors , 2013 .

[15]  Anders Logg,et al.  A compiler for variational forms , 2006, TOMS.

[16]  Robert Strzodka,et al.  Exploring weak scalability for FEM calculations on a GPU-enhanced cluster , 2007, Parallel Comput..

[17]  Krzysztof Banaś,et al.  ModFem : a computational framework for parallel adaptive finite element simulations , 2013 .

[18]  Eric Darve,et al.  Assembly of finite element methods on graphics processors , 2011 .

[19]  Michal Mrozowski,et al.  FINITE ELEMENT MATRIX GENERATION ON A GPU , 2012 .

[20]  David A. Ham,et al.  Finite element assembly strategies on multi‐core and many‐core architectures , 2013 .

[21]  Victor Eijkhout,et al.  Towards mechanical derivation of Krylov solver libraries , 2010, ICCS.

[22]  Philippe G. Ciarlet,et al.  The finite element method for elliptic problems , 2002, Classics in applied mathematics.

[23]  David A. Ham,et al.  Towards generating optimised finite element solvers for GPUs from high-level specifications , 2010, ICCS.

[24]  Samuel Williams,et al.  Scientific computing Kernels on the cell processor , 2007 .

[25]  Krzysztof Banaś.  A Model for Parallel Adaptive Finite Element Software , 2005 .

[26]  Maciej Paszyński,et al.  Computing with hp-ADAPTIVE FINITE ELEMENTS: Volume II Frontiers: Three Dimensional Elliptic and Maxwell Problems with Applications , 2007 .

[27]  Krzysztof Banas.  A Modular Design for Parallel Adaptive Finite Element Computational Kernels , 2004, International Conference on Computational Science.

[28]  Krzysztof Banas,et al.  Finite Element Numerical Integration on PowerXCell Processors , 2009, PPAM.

[29]  Krzysztof Banas,et al.  Design and development of an adaptive mesh manipulation module for detailed FEM simulation of flows , 2010, ICCS.

[30]  Robert Strzodka,et al.  Co-processor acceleration of an unmodified parallel solid mechanics code with FEASTGPU , 2009, Int. J. Comput. Sci. Eng..

[31]  Moshe Dubiner.  Spectral methods on triangles and other domains , 1991 .

[32]  Timothy C. Warburton,et al.  Nodal discontinuous Galerkin methods on graphics processors , 2009, J. Comput. Phys..

[33]  Noriyuki Kushida.  Element-wise Implementation of Iterative Solvers for FEM Problems on the Cell Processor , 2011, 2011 19th International Euromicro Conference on Parallel, Distributed and Network-Based Processing.

[34]  Samuel Williams,et al.  Optimization of sparse matrix-vector multiplication on emerging multicore platforms , 2009, Parallel Comput..

[35]  Robert A. van de Geijn,et al.  Designing Linear Algebra Algorithms by Transformation: Mechanizing the Expert Developer , 2012, VECPAR.

[36]  Lukasz Szustak,et al.  Adaptation of Double-Precision Matrix Multiplication to the Cell Broadband Engine Architecture , 2009, PPAM.

[37]  Anders Logg,et al.  Automated Solution of Differential Equations by the Finite Element Method: The FEniCS Book , 2012 .

[38]  Krzysztof Banas,et al.  3D finite element numerical integration on GPUs , 2010, ICCS.

[39]  Krzysztof Banas.  Parallelization of Large Scale Adaptive Finite Element Computations , 2003, PPAM.

[40]  Sean Rul,et al.  An experimental study on performance portability of OpenCL kernels , 2010, HiPC 2010.

[41]  James Demmel,et al.  Benchmarking GPUs to tune dense linear algebra , 2008, HiPC 2008.

[42]  Igor Peterlik,et al.  GPU Acceleration of Equations Assembly in Finite Elements Method -- Preliminary Results , 2009 .

[43]  Anders Logg,et al.  DOLFIN: Automated finite element computing , 2010, TOMS.

[44]  David Pardo,et al.  Out-of-core multi-frontal solver for multi-physics hp adaptive problems , 2011, ICCS.

[45]  George Em Karniadakis,et al.  TetrahedralhpFinite Elements , 1996 .