Parallel Computations in 3D Vortex Particle Method

AIAA 2011-3225

20th AIAA Computational Fluid Dynamics Conference 27 - 30 June 2011, Honolulu, Hawaii

Parallel computations in 3D vortex particle method Henryk Kudela∗ and Andrzej Kosior∗ Wroclaw University of Technology, Wroclaw, Poland In paper was presented an attempt to implement Vortex In Cell (VIC) method on Graphics Processing Units (GPUs). The 3D VIC method was shortly described. To show correctness of the code the motion of a vortex ring was simulated. At first the numerical calculation were done on the single processor (x86) architecture. Disadvantage of the single processor computation is time consumption. To manage this problem we switched to the parallel architecture. In our first approach to the multicore architecture we tested algorithms for the solution of the algebraic system of equations that results form discretization of the Poisson equation. For parallel computations CUDA architecture was used. We used the Multigrid and the Full Multigrid methods. It permitted us to speed up the solution of algebraic system using the Full Multigrid method 10-times. Finally, the whole Vortex In Cell was implemented on the GPU. A 15-times speed-up was obtained.

Nomenclature u = (u, v, w) – Velocity vector ρ – Fluid density p – Pressure ν – Kinematic coefficient of viscosity ω – Vorticity vector A – Vector potential Γp = (Γp1 , Γp2 , Γp3 ) – Vorticity particle xp = (xp1 , xp2 , xp3 ) – Position of vorticity particle ϕ(x) – Interpolation kernel Ek – Kinetic energy ψ – Stream function Subscript i Variable number j Variable number l Variable number p Particle number

I.

Introduction

Many problems in fluid dynamics can be analyzed using the vorticity and its evolution in time and space. From this fact results a great importance of the vortex particle methods. In computation the particles that carries information about vorticity are used. Tracing positions of these particles allows one to study evolution of the vorticity field. From distribution of the vorticity one can calculate fluid velocity. We can distinguish two groups of vortex methods: • direct methods where velocity of vortex particles are calculated on the basis of the Biot–Savart law by summing of the contribution to the velocities induced by all particles in the flow, • Eulerian-Lagrangian methods, like Vortex–In–Cell method (VIC), where velocity field is determined on a grid by solving Poisson equation for a vector potential. ∗ Department

of Numerical Modeling of Flow, 50-370 Wroclaw Wybrzeze Wyspianskiego 27.

1 of 9 Copyright © 2011 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

American Institute of Aeronautics and Astronautics

The direct methods are very attractive from a theoretical point of view. They are based on fundamental law of vector analysis, are grid free and they allow exact realization of far–field boundary condition. But direct vortex methods also have a great disadvantage. The amount of a computational time is prohibitively large.2 The Euler-Lagrangian VIC method is about 1000 times faster.3 In 2D simulation the vortex particle method is particularly very efficient. In every time step one must solve only one Poisson equation that combines the component of the vector potential (the stream function) with the third component of vorticity. In 3D computation in each time step we need to solve three Poisson equations, one for each component of the vector potential. In vortex particle method the particles after several steps have a tendency to concentrate in areas where velocity gradient is very high. It may lead to spurious vortex structures. To avoid this situation one should do after arbitrary number of time steps the redistribution of particles to regular positions. From our experience for 2D vortex particle computation5–7 we noted that it is useful to do the remeshing at every time step. At the beginning the vortex particles are put on the grid nodes. After displacement in one discrete time step the intensities of the particles are redistributed again to the initial grid nodes. The fact that we know that particles lay on the regular gird permit on accurate simulation of the fluid viscosity. In vortex particle method the viscous splitting algorithm is used. The solutions is obtained in two steps. At first step the inviscid part of fluid motion equation is solved and in the second step the viscosity of the fluid is taken into account. The position of the particles on regular grid permit on accurate simulation of the fluid viscosity. In present paper we implemented this idea to the case of 3D flow. Depside of the fact that we use VIC method the computation time for 3D case are still very large and to speed-up computations we decided to switch to parallel computations. The VIC method is very good suited for parallelization. The Poisson equation for each component of the vector potential can be solved independently. The remeshing process has a local character and the computations for each particle can be also done independently. The same is related to the displacement of the vortex particles. In the last years it was found that Graphical Processing Units (GPUs) developed for video games are suitable for scientific computations. These GPUs provide cheap and easily accessible hardware for scientific calculations.

II.

Equations of motion

Equations of incompressible viscous fluid motion have the following form: 1 ∂u + (u · ∇)u = − ∇p + ν∆u ∂t ρ

(1)

∇·u=0

(2)

where u = (u, v, w) is velocity vector, ρ – fluid density, p – pressure, ν – kinematic coefficient of viscosity. The equation (1) can be transformed to the Helmholtz equation for the vorticity evolution: ∂ω + (u · ∇)ω = (ω · ∇)u + ν∆ω ∂t

(3)

where ω = ∇ × u. From incompressibility (2) stems the existence of the vector potential A u=∇×A

(4)

Assuming additionally that vector A is incompressible (∇ · A = 0) its components can be obtained by solution of the Poisson equation ∆Ai = −ωi , i = 1, 2, 3 (5) Further we will assume that viscosity of the fluid is equal zero. So that imply that we will solve only inviscid (Euler) equation of fluid motion.

III.

Description of VIC method for three-dimensional case

First we have to discretize our computational domain. To do this we set up a regular 3D grid (j1 ∆x, j2 ∆y, j3 ∆z) (j1 , j2 , j3 = 1, 2, . . . , N ), where ∆x = ∆y = ∆z = h. The same mesh will be used for solving Poisson equation. The continuous field of vorticity is replaced by a discrete distribution of Dirac delta measures2, 8 2 of 9 American Institute of Aeronautics and Astronautics

ω(x) =

N X

Γp (xp )δ(x − xp )

(6)

p=1

where Γp means vorticity particle Γp = (Γp1 , Γp2 , Γp3 ) at position xp = (xp1 , xp2 , xp3 ). The domain of the flow is covered by numerical mesh (Nx × Ny × Nz ) with equidistant spacing h, and the i-th component of the vector particle α is defined by expression: Z Γi = ωi (x1 , x2 , x3 ), dx ≈ h3 ωi (xp ), xp ∈ Vp , |Vp | = h3 (7) Vp

From the Helmholtz theorems13 we know that the vorticity is carried on by the fluid: dxp = u(xp , t), dt

(8)

We must take into account also the fact that due to three dimensionality of the vorticity field the intensity of the particles are changed by stretching effect dΓp = [∇u(xp , t)] · Γp dt

(9)

Velocity at the grid nodes was obtained by the solution of the Poisson equation (5) by the finite difference method and using (4). The system of equations (8), (9) was solved by Runge–Kutta method of 4-th order. A.

Remeshing

In Vortex in Cell method particles have a tendency to gather in regions with high velocity gradients. This can lead to inaccuracies as particles coming too close to one another. To overcome this problem particles have to be remeshed that is distributed back to the nodes of the rectangular mesh. It is done by an interpolation:   ˜p 1 X˜ xj − x Γ pn ϕ ωj = 3 (10) h p h where j is index of grid node and p is index of a particle. The quality of that interpolation strongly depend on the properties of the kernel ϕ. At first the kernel should satisfy X  x − xp  ϕ =1 (11) h p The next the kernel should fulfill the moment conditions: X x − xp (x − xp )α ϕ( ) = 0, for h p

1 ≤ |α| ≤ m − 1

(12)

where m is order of remeshig procedure. The discrepancy between the old and new position of the particle can be measure by the difference2 X X ˜ p δ(x − x Γ ˜p ) − Γpn δ(x − xp ) (13) n p

p

If we multiply (13) by a test function φ and take into account (11) one get:2 " #  X X ˜ x − x j p ˜ p φ(˜ E= Γ xp ) − φ(xj )ϕ n h p p

(14)

˜ p and x where Γ ˜p are values from old distribution. Thus the expression in the brackets characterize the n distortion of the redistribution process. Naming the expression the brackets by f (x), using the Taylor expassion of ϕ and ones again using the property (11) one can get2

3 of 9 American Institute of Aeronautics and Astronautics

f (x) =

XX α

[(xj − x) · ∇φ]α ϕ

j



x − xj h

 (15)

So f (x) = O (hm ) .

(16)

It is regarded that the remeshing procedure is of order m. It means that polynomial functions up to an order m will be exactly represented by this interpolation. In this work we used following interpolation kernel:  2 3  if 0 ≤ |x| ≤ 1  (2 − 5x + 3|x| )/2 ϕ(x) =

2 (2 − |x|) (1 − |x|)/2   0

if 1 ≤ |x| ≤ 2

(17)

if 2 ≤ |x|

Kernel (17) used in this work is of order m = 3.

IV.

Test Problem

To check correctness of our code with remeshing particle position in every time step we carry out test for vortex ring. The calculations were done at first on a single processor architecture. For a thin cored ring with circulation Γ formula for the vortex ring velocity is:4      0 8R0 1 Γ ln − +O (18) U= 4πR0 0 4 R0 where R0 is ring radius and 0 is core radius (0 /R0