Parallel finite element computation of incompressible flows

A stabilized finite element formulation for three-dimensional unsteady incompressible flows is implemented on a distributed memory parallel computer. A matrix-free version of the GMRES algorithm is utilized to solve the equation systems in an implicit manner. The scalability of the computations on a 64-processor Linux cluster is evaluated for moderate to large size problems. A method for estimating the speedup for large-scale problems, where computations on a single processor is not possible, is proposed. Superlinear speedup is observed, perhaps for the first time, for a large-scale problem that is associated with more than 44 million nodes and 176 million equations. The performance of the various subactivities of the program is monitored to investigate the cause. It is found that the formation of the RHS vector and the preconditioner achieves a very high level of superlinear speedup as the number of processors increase. As a result, even though the network time for interprocessor communication increases with increase in processors, an overall superlinear speedup is realized for large-scale problems. The superlinear speedup is attributed to cache related effects. A comparison between the performance of matrix and matrix-free versions of the GMRES algorithm is carried out. It is found that for large-scale applications the matrix-free version outperforms its counterpart for reasonable dimensions of the Kyrylov subspace. The effect of mesh partitioning on the scalability is also studied. A significant reduction in communication time is observed with partitioning that leads to an overall improvement of speedup. The parallel implementation is utilized to study the wake instabilities in flow past a stationary circular cylinder at Re=150, 200 and 300. The Re=150 flow is found to be two-dimensional while mode-A and mode-B instabilities are observed at Re=200 and 300, respectively. The Re=300 flow is associated with a low frequency modulation in addition to the vortex shedding frequency.

[1]  C. Williamson.  Three-dimensional wake transition , 1996, Journal of Fluid Mechanics.

[2]  David Levine,et al.  Parallel performance of two applications in the Boeing high performance computing benchmark suite , 2001, Parallel Comput..

[3]  S. Lennart Johnsson,et al.  The Finite Element Method on a Data Parallel Computing System , 1989, Int. J. High Speed Comput..

[4]  Thomas J. R. Hughes,et al.  A data parallel finite element method for computational fluid dynamics on the Connection Machine system , 1992 .

[5]  Marek Behr,et al.  Implementation of implicit finite element methods for incompressible flows on the CM-5 , 1994 .

[6]  M. Braza,et al.  Physical analysis of the transition to turbulence in the wake of a circular cylinder by three-dimensional Navier–Stokes simulation , 1998, Journal of Fluid Mechanics.

[7]  H. Schlichting.  Boundary Layer Theory , 1955 .

[8]  Tayfun E. Tezduyar,et al.  Parallel finite element computations in fluid mechanics , 2006 .

[9]  Yousef Saad,et al.  An Edge Based Stabilized Finite Element Method For Solving Compressible Flows: Formulation and Paral , 2001 .

[10]  S. Mittal,et al.  Prediction of the critical Reynolds number for flow past a circular cylinder , 2006 .

[11]  Marek Behr,et al.  Parallel finite-element computation of 3D flows , 1993, Computer.

[12]  V. Kalro,et al.  Parallel 3D Computation of Unsteady Flows Around Circular Cylinders , 1997, Parallel Comput..

[13]  J. G. Kennedy,et al.  Computation of incompressible flows with implicit finite element implementations on the Connection Machine , 1993 .

[14]  Charbel Farhat,et al.  Transient finite element computations on 65536 processors: The connection machine , 1990 .

[15]  Akira Tezuka,et al.  A platform for parallel CFD FEM computations , 2005 .

[16]  Harry F. Jordan,et al.  Computer systems design and architecture , 1996 .

[17]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[18]  D. Parkinson.  Parallel efficiency can be greater than unity , 1986, Parallel Comput..

[19]  Ryutaro Himeno,et al.  A parallel 3D unsteady incompressible flow solver on VPP700 , 2001, Parallel Comput..

[20]  C. Williamson.  Vortex Dynamics in the Cylinder Wake , 1996 .

[21]  Genki Yagawa,et al.  Large-Scale Finite Element Fluid Analysis by Massively Parallel Processors , 1997, Parallel Comput..

[22]  C. Williamson.  The natural and forced formation of spot-like ‘vortex dislocations’ in the transition of a wake , 1992, Journal of Fluid Mechanics.

[23]  J. Wu,et al.  Parallel three-dimensional direct simulation Monte Carlo method and its applications , 2003 .

[24]  S. Balachandar,et al.  Low-frequency unsteadiness in the wake of a normal flat plate , 1997, Journal of Fluid Mechanics.

[25]  Tayfun E. Tezduyar,et al.  Massively parallel finite element simulation Of compressible and incompressible flows , 1994 .

[26]  B. R. Noack,et al.  On the transition of the cylinder wake , 1995 .

[27]  Walter B. Sturek,et al.  Comments on CFD code performance on scalable architectures , 2000 .

[28]  Ted Belytschko,et al.  Finite element analysis on the connection machine , 1990 .

[29]  S. Mittal,et al.  Massively parallel finite element computation of incompressible flows involving fluid-body interactions , 1994 .

[30]  S. Mittal,et al.  Incompressible flow computations with stabilized bilinear and linear equal-order-interpolation velocity-pressure elements , 1992 .

[31]  Tayfun E. Tezduyar,et al.  Stabilization Parameters in SUPG and PSPG Formulations , 2003 .

[32]  G. P. Wren,et al.  Parallel Implementations of a Finite Element Formulation for Fluid-Structure Interactions in Interior Flows , 1997, Parallel Comput..

[33]  M. Braza,et al.  Successive stages and the role of natural vortex dislocations in three-dimensional wake transition , 2001, Journal of Fluid Mechanics.

[34]  Sanjay Mittal,et al.  On the performance of high aspect ratio elements for incompressible flows , 2000 .