DNS of turbulent flows using Artificial Neural Networks

Indian Institute of Technology Kharagpur

Computational Fluid Dynamics : Term Paper

DNS of turbulent flows using Artificial Neural Networks (ANNs)

Name: Kshitij Goel Roll number: 13AE10013

Submitted to: Dr. Somnath Ghosh

November 30, 2015

Abstract This term paper focuses on application of feed-forward artificial neural network in solving partial differential equations and using it’s boundary conditions and/or it’s initial conditions. Also, this is extended to visualize Direct Numerical Simulations (DNS) of turbulent flows by applying this technique on the general non-approximated Navier-Stokes equations. This is, in the end, compared with the traditional computational fluid dynamics techniques. Applications of ANNs in other areas of Aerospace Engineering are also listed.

1

Turbulence and Navier-Stokes Equations

Moin et al. in [7] mention that turbulent flows possess wide range of length and time scales and consequently have a very unusual mix of chaos and order. This is how a typical velocity v/s time plot looks like in such flows :

This chaos is a consequence of the Non-linear partial differential - Navier Stokes - equation [8]: ∂u 1 1 + (u · ∇)u = − ∇p + γ∇2 u + F ∂t ρ ρ where γ = µρ is the kinematic viscosity, F is gravitational potential. We first try to obtain solution a PDE by using feed-forward neural networks and then we try to apply it in flow problems using Navier Stokes equations. The condition for conservation of mass is given by the continuity equation [8]: ∂ρ + ∇ · (ρU) = 0 ∂t An essential feature of turbulent flows is that they are rotational. This phenomena is governed by the vorticity equation [8]: Dω = ν∇2 ω + ω · ∇U Dt These kind of equations turn out to be really computationally expensive for the industry. Therefore, analyzing turbulent flows by DNS has for long been considered just a research tool. There has 1

always been a need of alternate kind of computational method for a more comprehensive turbulent fluid analysis. One such methodology is by modelling such complex non-linear mapping by Artificial Neural Networks - widely considered as a universal way to map any kind of nonlinear mapping quite accurately.

2

Artificial Neural Networks (ANNs)

Artificial neural networks (ANNs) are learning algorithms made from observing biological systems (for example : human brain), and are used to estimate or approximate functions that can depend on a large number of inputs and are generally unknown. Such networks are represented as systems of interconnected ’neurons’ which can compute values from a variety of inputs. In a shorter sentence, these algorithms can learn from data provided to them. In [2], the authors conclude that feed forward networks are capable of arbitrarily well approximating arbitrary functions and it’s derivatives.

2.1

Feed-forward ANNs (Static Neural Nets(SNNs))

As long as the directed edges in the figure below are pointing in one single directions and are not a part of a cycle, the ANN is a feed-forward neural network. Otherwise, the network becomes recurrent, which is much more difficult to analyze. Single hidden layer SNN are usually represented and interpreted like this :

Input layer is where the user input is fed into for the answer at the output end. The hidden layer and the weights of the edges is measured by iterations.

2

3

Steps involved in solving PDEs using ANNs

We are talking about equations of the form: G(x, y(x), ∇y(x), ∇2 y(x)) = 0

(1)

Here is the procedure of applying the methodology: • Selection of an approximate or trial form of a solution is done. Generally for a second order differential equation with specified boundary conditions, it is of the type : y(x, p) = y(x) + F (x)N (x, p)

(2)

where: – p are the number of parameters. – N (x, p) is the feed forward network NN with parameters p. – F (x) is a scalar function in x such that it is zero at all the boundary conditions. • That particular choice is made of F (x) so that y(x) automatically satisfies the function boundary conditions. • Equation (2) is then approximated on a discretized domain D = x(i) ∈ D; i = 1, ..., m. Equation (1) is relaxed to hold only at the points in domain D. • The problem in the above point is further simplified by allowing a trial solution that ”nearly satisfies” the equation (1) at the points in domain D. This is done by defining a error index and then trying to minimize it by changing the parameters p. The error index is : J(p) =

m X

G(x(i) , y(x(i) , p), ∇y(x(i) , p), ∇2 y(x(i) , p))

(3)

i=1

• Now, the parameter p for which J(p) is minimized is obtained by different optimization techniques. These methods may or may not reach global optimum, but a local optimum is assured. Two of these are: – Back-Propagation Algorithm – Quasi-Newton BFGS Algorithm Back Propagation one is the most popular amongst the machine learning community as it’s very intuitive as well as difficult and challenging to implement.

3

4

Extending to DNS and flow prediction

We have seen how to solve a general PDE using artificial neural networks. Now we extend this concept to either one of the following paths : • Solve Navier-Stokes equation (without taking any assumptions) and hence obtain DNS. (Unsupervised Learning) • Perform flow prediction from some information available by some previous DNS. (Supervised Learning) [5] uses intelligent prediction techniques to predict turbulent flow over a backward-facing step using the Direct Numerical Simulation Data. [6] uses ANN for behavior learning from meso-scale simulations. Both of these research show us how the two avenues use the power of ANN for both DNS and flow prediction. In the next two subsections these are described in detail.

4.1

ANN for behavior learning from meso-scale simulations [6]

[6] describes that meso-scale behavior can be ”learned” and transfered to the macro scale. This is an interesting application because they are performing experiments on a very small scale and are able to confirm the behavior of a large scale system by using ANN. More precisely, they are used to learn and predict the transient forces on a particle in a compressible flow field to produce an accurate model for shocked particulate-laden flows. It learns meso-scale information of particle-fluid interactions requiring expensive computations; once the behavior is learnt, the ANN can be interrogated to obtain information by a macro-scale model to accurately produce results without continuing to perform expensive computations in direct numerical simulations. It is here that it saves extra DNS computations and may well give better results than it. ANN is able to evolve and reproduce correlations between the control parameters and particle dynamics. Inside a single neuron

Now, output of a single neuron is given by : On = φ(bk +

m X i=1

where :

4

wki ∗ xi )

On = Output φ = Activation function bk = threshold bias wk = synaptic weights xi = inputs from previous layers of neurons and, i is the number of inputs. Each neuron is capable of editing weights supplied to it based upon the accuracy of the entire network. This enables the neural network to learn the behavior of data provided. General types of activation function φ

tanh is a popular choice as it’s derivative is always positive. A simple application of this network could be the amount or red and blue as inputs of a color mix and the final area would be purple. In [6], back-propagation algorithm is used.When the ANN is in training, it should be learning from every point in a data set otherwise learning will be biased. Every iteration step for an ANN consists of cycling through the total number of data points in a data set. The error produced on every iteration step can be plotted to show a convergence curve on how the ANN is being trained. RMS Error convergence

High-level Algorithm : • The macro-scale calculations access the lifted information by simply querying the ANN (a procedure that rapidly provides information to the macro-scale on fairly complicated behavior of the particles as the meso-scale). 5

• The macro-scale calculations are then advanced further and information is accessed from the ANN at each step of the macro-calculation. • Then the ANN learning process will proceed alongside the querying process and the micro- and macro-calculations will need to be synchronized in some way.

4.2

Intelligent Flow Prediction [5]

In [5], a multivariate Levenberg-Marquardt (LM) neural network is developed to predict the turbulent flow over backward-facing step using information from a DNS. A multi-task learning technique is used to improve generalization performance and the prediction accuracy of flow field properties with low computational cost.Also, the optimum number of hidden neural units are determined in order to prevent ”overfitting”. Most of the physical mechanisms in separated internal flow are included in this case, the so called backward-facing step (BFS) is commonly accepted as a useful benchmark for numerical analyses. Additionally, the BFS is one of the most popular test cases used to evaluate a turbulence models accuracy. An alternative method, other than experimental and numerical, is desirable to aggregate the knowledge from experimental and numerical sources. Artificial Neural Networks are a potential choice when training data are available. Mutli-task Learning enhances the performance of a Back-propagation algorithm. This is how a multi-task learning algo looks like :

Using this in conjunction with Levenberg-Marquardt (LM) neural network, authors were able to predict flow attachment properties for a turbulent flow over a backward-facing step using the data provided by a preliminary not much computationally intensive DNS.

5

Conclusion : Comparison with Conventional Discretized Methods • The benefit of this method is that the trial solution (via the trained NN) represents a smooth approximation that can be evaluated and differentiated continuously on the domain. 6

This is in contrast with the discrete or non-smooth solutions obtained by traditional schemes. • Authors in [1] states that the Algorithms from Numerical Algebra like : Gauss-Seidel, successive over relaxation, Krylov subspace method etc. only give approximate solutions even with parallel programming and supercomputers. • However, the ANN method can also be computationally expensive if the no. of hidden layers or the number of nodes in each hidden layers increase by a considerable amount. • Neural Networks are not a substitute of understanding a problem more deeply. They are sort of a ”black box” that just learns from it’s experience, brings out a pattern between input and output and can give an correct output afterwards. But this approach doesn’t give the right scientific idea of the problem behind the black box.

6

Some other applications of ANN in fluids engineering

Back-propagation neural networks have been used for many applications in fluids engineering. Many attempts have been made to apply ANNs to fluid dynamics problems such as : • Two phase flow • Turbulent flames • Airfoil optimization • Cavitations performance • Turbulent wake • Turbulent flow over cylinder • Scour depth prediction at bridge abutments.

7

Applications in other fields of Aerospace Engineering

Neural Networks are specially studied in Aerospace control systems: • Authors in [4] use it for determination of aerodynamic coefficients. • Fault diagnostic systems • Control system based on neural networks • Parameter estimation from flight data • Trajectory modeling of artillery shell and rockets. But as we have highlighted they have a far more exciting application in Computational Fluid Dynamics too.

7

References [1] Modjtaba Baymani, Artificial Neural Networks Approach for Solving Stokes Problem Applied Mathematics, 2010, 1, 288-292. [2] Lucie P. Aarts, Neural Network Method for Solving Partial Differential Equations Neural Processing Letters 14: 261-271, (2001) [3] M. M. Chiaramonte and M. Kiener, Solving differential equations using neural networks [4] T. Rajkumar and Jorge Bardina, Prediction of Aerodynamic Coefficients using Neural Networks for Sparse Data NASA, (2002) [5] Elham Rajabi and Mohammad R. Kavianpour, Intelligent prediction of turbulent flow over backward-facing step using direct numerical simulation data Engineering Applications of Computational Fluid Mechanics Vol. 6, No. 4, pp. 490503 (2012) [6] Christopher Lu, Artificial neural network for behavior learning from meso-scale simulations, application to multi-scale multimaterial flows Master’s Thesis, The University of Iowa (2010) [7] Parviz Moin and Krishnan Mahesh, DIRECT NUMERICAL SIMULATION : A Tool in Turbulence Research Annu. Rev. Fluid Mech. (1998) [8] Stephen B. Pope, Turbulent Flows Cambridge University Press (2000)

8