Arithmetic Parameterization of General Purpose CFD Code

Arithmetic Parameterization of General Purpose CFD Code Aleksandar Jemcov1 Development Department, ANSYS/Fluent Inc., Lebanon, NH, 03766, USA Email: [email protected]

A BSTRACT In this paper we describe the use of C++ templates to parameterize CFD code to produce specialized versions of the code at the compile time that is capable of defining different arithmetic types. This parameterization is primarily used to compute automatic derivatives in design optimization. However, it is possible to use the same code parameterization to define various arithmetics that can be used to produce additional useful side effects such as automatic stability characterization of the algorithms, uncertainty propagation, higher order derivative computations and interval arithmetics. Here we describe the implementation of the parameterization and the design of these classes and we show results of the use of different arithmetics in CFD and related fields.

1

I NTRODUCTION

Modern software architecture in Computational Fluid Dynamics usually involves use of object oriented software design to promote information hiding, strict modularity and code reuse. Given the fact that CFD codes require high computational performance in both serial and parallel runs, the natural choice for the programming language fall onto C++ language. C++ is a weakly object oriented language that is rich with language constructs and can be used to produce high performance numerical software. However, in addition to support for object oriented programming, C++ offers the possibility of generic programming through the use of templates. The ability to freely mix object oriented and metaprogramming styles of code writing can be exploited successfully to create computational code with many purposes. Templating of the CFD code enables parameterization of the software implementation and depending on how this templating is performed, many parameterizations are possible. Use of automatic differentiation libraries in design

optimization and sensitivity analysis is well established [1, 2, 3, 4]. Differentiation libraries such as ADIFOR [3] and ADOLC [4] are implemented as language parsers that augment original code in oder to compute derivatives. While this approach to computing derivatives is mostly automatic with minimal changes in source code and very efficient, it is not possible to define any other type of arithmetic other than implemented in the library without reimplementing large portions of the parser. Another approach to computing derivatives consists of the use of the operator overloading mechanism offered in several languages such as FORTRAN90 and C++. This mechanism allows redefinition of the basic arithmetic operations such as +, −, ∗, /, etc. Once these operations are overloaded, the new definition of the arithmetic operators are used by the compiler whenever a particular type or a key word is encountered. One such implementation of operator overloading is described in [5, 6, 7]. This approach requires changes in the source code that sometimes are not trivial because they may involve a deliberate design of the code, especially in the case of C++ language. However, once this is achieved, the code becomes very flexible and generic. Operator overloading is a necessary precursor to code parameterization by C++ templates. Introduction of C++ templates into a CFD code allows generic programming to be used in its full potential, even beyond the original intention of automatic differentiation of the code. Moreover, carefully overloaded CFD code together with function and class templating introduces the possibility of the use of advanced concepts such as algorithm stability characterization and interval arithmetic in addition to automatic differentiation. The final result of this effort is a generic numerical library that can be used for advanced computations. Examples of these computations include computation of the first and second order derivatives with respect to flow variables, uncertainty propagation, interval arithmetics and stability characterization of the algorithms.

2

A RITHMETIC T YPE

The basic idea of the exchangeable arithmetics in numerical libraries is based upon the capability of C++ language to declare a new types through the definition of classes [8]. Here we have a specific new type in mind, the one that is capable of replacing the built-in compiler arithmetic with a user defined one. The definition of the class Arithmetic looks like this: class Arithmetic { public: //constructor methods Arithmetic(const double x); Arithmetic(const Arithmetic& x); ... //public methods for operator overloading Arithmetic& operator*=(const Arithmetic& x); Arithmetic& operator+=(const Arithmetic& x); ... //access methods double value() const; double other() const; void setValue(const Arithmetic& x); void setOther(const Arithmetic& x); ... private: //data for floating point arithmetic double _value; //additional data for other arithmetic double _other; ... }; This class definition contains all necessary ingredients required for the definition of the new type in C++ language i.e., it contains the means of creation of the new variable of the Arithmetic type and its destruction (instantiated by the compiler), definition of basic arithmetic operations, access methods and data to store the results of the computations. The intention here is to use this class as a basic building block of the computations that represent the generalization of the built-in types such as float and double. A particular implementation of the concrete Arithmetic type will depend on the type of calculations that are desired. Some examples of different Arithmetic types will be presented in the next several sections. This is achieved by using specific operator overloading that correspond to a particular arithmetic. The ability of the C++ language to define new types that can be used in the same way as a built-in types together with the operator overloading to define operations on the new types is critical in enabling the different arithmetics.

However, Arithmetic types do not replace built-in types in normal operation i.e., when side effects of the new types is not required. In other words, the code must be able to use built-in types whenever it is required without any performance penalties. Therefore, a way to signal to the compiler that a specific type, be it a built-in or newly defined Arithmetic type, must be devised. This is achieved through the use of C++ templates. C++ templates [8] are very useful in situations when there is a generic code that contains a logical structure that is applicable to many different types. For example, in finite volume methods numeric flux could be computed with the use of the following class: template class Flux { public: //constructor Flux(const Args& args); //desctructor ~Flux(); ... //public methods for flux computation Arithmetic inviscidFlux(Arithmetic& waveSpeed, const Arithmetic& metricCoeffs, ...); Arithmetic viscousFlux(const Arithmetic& metricCoeffs, ...); private: ... }; Class Flux together with its methods was parameterized with respect to class Arithmetic and this allows substitution of different arithmetic types: template class Flux; template class Flux; template class Flux; ... Different Arithmetic types are selected at the compile time and the compiler performs the necessary substitutions and instantiations. Close attention must be paid to implementation of the Arithmetic classes in order to avoid function calls in place of overloaded arithmetic operators and this can be achieved by a deliberate use of inline directive. It is important to keep the implementation of the overloaded operators rela-

tively short and avoid using the data that cannot be determined at the compile time. Trait mechanism [8] is used to make sure that the right overloaded functions from the standard library such as sin(x), pow(x,n), min(x,y), etc. are selected at the compile time.

3

AUTOMATIC D IFFERENTIATION

The automatic differentiation class is a particular implementation of the generic Arithmetic that enables the computation of the directional derivatives for the sensitivity analysis and design optimization. Derivative computation is achieved by a specific overloading of all arithmetic operators together with the mathematical library. The basic idea of overloading the arithmetic operators is to insert the line of the code at the compile time that is capable of computing the derivative. If we consider a simple example of the multiplication of two numbers z = xy,

(1)

directional derivative is defined as δz = xδy + yδx

(2)

The form of Eq. (1) and Eq. (2) suggests that whenever the value of the expression z = xy is computed, it is possible to compute the directional derivative of that expression. In other words, operator overloading should produce the code that is capable of computing the value and the derivative at the same time by changing the meaning of the arithmetic operation + to insert the necessary instruction into the code to perform that operation. The resulting class, called Tangent may be implemented as follows: class Tangent { public: //constructor methods Tangent(const double x); Tangent(const Tangent& x); ... //public methods for operator overloading Tangent& operator*=(const Tangent& x){...} Tangent& operator+=(const Tangent& x){...} ... //access methods double value() const; double deriv() const; void setValue(const Tangent& x); void setOther(const Tangent& x);

... private: //floating point data double _v; //derivative data double _dv; }; In this case, operator*= must be implemented as a part of the Tangent class and shown here: Tangent& operator*=(const Tangent& x) { _dv = _dv*x.value() + _v*x.deriv(); _v *= x.value(); return *this; } With the help of the operator*=, it is possible to implement operator*: inline Tangent operator*(const Tangent& x, const Tangent& y) { return Tangent(x)*=y; } Similarly, math library function sin(x) is implemented as follows: inline Tangent sin(const Tangent& x) { return Tangent(std::sin(x.value()), x.deriv()*std::cos(x.value())); } Careful overloading of all necessary operators and math library functions with templating of the code on Arithmetic type results in a parameterized code capable of computing directional derivative in addition to values of the flow field. Sometimes this approach to computing the directional derivatives is called algorithmic differentiation due to the fact that the whole code is considered to be composed of elemental functions: f = fn ◦ fn−1 ◦ · · · ◦ f2 ◦ f1 ◦ f0 (3) Consistent application of the chain rule of differentiation leads to the expression δ f = f $ n,n−1 f $ n−1,n−2 · · · f $ 1,0 δx

(4)

8.31e-01 8.04e-01 7.76e-01 7.49e-01 7.21e-01 6.94e-01 6.67e-01 6.39e-01 6.12e-01 5.84e-01 5.57e-01 5.29e-01 5.02e-01 4.74e-01 4.47e-01 4.19e-01 3.92e-01 3.64e-01 3.37e-01 3.10e-01 2.82e-01 2.55e-01 2.27e-01 2.00e-01 1.72e-01 1.45e-01

Contours of Mach Number

Feb 05, 2008 FLUENT 6.3 (2d, dp, dbns exp)

Figure 1: Mach number contours around NACA0012 at M = 0.7 and α = 0o 5.00e+04 4.59e+04 4.19e+04 3.79e+04 3.39e+04 2.98e+04 2.58e+04 2.18e+04 1.78e+04 1.37e+04 9.73e+03 5.70e+03 1.68e+03 -2.35e+03 -6.37e+03 -1.04e+04 -1.44e+04 -1.84e+04 -2.25e+04 -2.65e+04 -3.05e+04 -3.45e+04 -3.86e+04 -4.26e+04 -4.66e+04 -5.06e+04

Contours of User Memory 0

5.76e+02 5.55e+02 5.35e+02 5.14e+02 4.93e+02 4.72e+02 4.52e+02 4.31e+02 4.10e+02 3.89e+02 3.69e+02 3.48e+02 3.27e+02 3.06e+02 2.86e+02 2.65e+02 2.44e+02 2.23e+02 2.03e+02 1.82e+02 1.61e+02 1.40e+02 1.20e+02 9.88e+01 7.81e+01 5.73e+01

Contours of User Memory 4

Feb 05, 2008 FLUENT 6.3 (2d, dp, dbns exp)

Figure 3: Contours of derivative of u-velocity with respect to Mach number at far field boundary around NACA0012 at M = 0.7 and α = 0o

Feb 05, 2008 FLUENT 6.3 (2d, dp, dbns exp)

Figure 2: Contours of derivative of pressure with respect to Mach number at far field boundary around NACA0012 at M = 0.7 and α = 0o

The Tangent class allows the propagation of the chain rule of the differentiation in accordance to Eq. (4) throughout the computations regardless of the code complexity. An example of the computation of the directional derivative is demonstrated by computing the derivative of pressure and velocities with respect to Mach number at far field boundary. The airfoil under the consideration is symmetric NACA0012 airfoil in inviscid free stream with Mach number M = 0.7 at zero angle of attack. Contours of Mach number around airfoil are given in Fig. (1) whereas derivatives of pressure and velocity fields are given in Fig (2), Fig. (3) and Fig. (4).

1.10e+02 1.01e+02 9.24e+01 8.36e+01 7.48e+01 6.60e+01 5.72e+01 4.84e+01 3.96e+01 3.08e+01 2.20e+01 1.32e+01 4.40e+00 -4.40e+00 -1.32e+01 -2.20e+01 -3.08e+01 -3.96e+01 -4.84e+01 -5.72e+01 -6.60e+01 -7.48e+01 -8.36e+01 -9.24e+01 -1.01e+02 -1.10e+02

Contours of User Memory 5

Feb 05, 2008 FLUENT 6.3 (2d, dp, dbns exp)

Figure 4: Contours of derivative of v-velocity with respect to Mach number at far field boundary around NACA0012 at M = 0.7 and α = 0o

It is possible to further generalize Tangent by introducing the computation of the second order derivatives. The resulting class is called Curvature and it is implemented as follows: class Curvature { public: //constructor methods Curvature(const double x); Curvature(const Curvature& x); ... //public methods for operator overloading Curvature& Curvature*=(const Curvature& x){...} Curvature& operator+=(const Curvature& x){...} ... //access methods double value() const; double deriv() const; void setValue(const Curvature& x); void setOther(const Curvature& x); ... private: //floating point data double _v; //derivative data double _dv; double _sd; };

7.07e+04 6.95e+04 6.82e+04 6.70e+04 6.58e+04 6.46e+04 6.33e+04 6.21e+04 6.09e+04 5.97e+04 5.84e+04 5.72e+04 5.60e+04 5.48e+04 5.36e+04 5.23e+04 5.11e+04 4.99e+04 4.87e+04 4.74e+04 4.62e+04 4.50e+04

Contours of Static Pressure (pascal)

Figure 5: Static pressure contours around NACA0012 at M = 0.7 and α = 1o 1.36e+05 1.22e+05 1.08e+05 9.35e+04 7.94e+04 6.53e+04 5.13e+04 3.72e+04 2.32e+04 9.12e+03 -4.94e+03 -1.90e+04 -3.31e+04 -4.71e+04 -6.12e+04 -7.52e+04 -8.93e+04 -1.03e+05 -1.17e+05 -1.31e+05 -1.45e+05 -1.60e+05

Contours of User Memory 0

Here _v stores values of computations, _dv stores first order derivative, and _sd stores the second order derivative. In the case of the Curvature class, the overloaded operator*= has additional instructions in the body of the function: Curvature& operator*=(const Curvature& x) { _sd = _sd*x.value() + 2.*_dv*x.deriv() + x.sderiv()*_v; _dv = _dv*x.value() + _v*x.deriv(); _v *= x.value(); return *this; } Other operators and math library functions are overloaded in the same manner to produce consistently first and second order derivatives. The use of the Curvature class in computation of the first and second order derivatives is demonstrated for the case of the inviscid flow over the symmetric

Feb 05, 2008 FLUENT 6.3 (2d, dp, dbns exp)

Feb 05, 2008 FLUENT 6.3 (2d, dp, dbns exp)

Figure 6: First order derivative of pressure with respect to angle of attack contours around NACA0012 at M = 0.7 and α = 1o NACA0012 airfoil with Mach number M = 0.7 and at the angle of attack of α = 1.0o. Here the differentiation is performed with the respect to the angle of attack. The pressure field for this case is shown in Fig. (5), the first derivative of the pressure field with respect to angle of attack is shown in Fig. (6) and the second order derivative is shown in Fig. (7).

4

A LGORITHM S TABILITY C OMPUTATION

Another good candidate for defining the new arithmetic is based on the forward error analysis. In forward error analysis the influence of the finite precision arithmetic is examined and the error in the computed

be computed by overloading the appropriate operators similarly to Tangent class. Since the code is considered to be a composition of a large number of elemental functions in accordance to Eq. (3) and the chain rule of differentiation is given by Eq. (4), absolute normwise forward error is given by:

7.04e+05 6.02e+05 5.01e+05 3.99e+05 2.98e+05 1.97e+05 9.51e+04 -6.37e+03 -1.08e+05 -2.09e+05 -3.11e+05 -4.12e+05 -5.14e+05 -6.15e+05 -7.17e+05 -8.18e+05 -9.20e+05 -1.02e+06 -1.12e+06 -1.22e+06 -1.33e+06 -1.43e+06

ϒ ≤ (σn κn + σn−1 σn κn + · · · + σn−1 σn κ0 )ε|| f (x) ˜ (8) where

Contours of User Memory 7

Feb 05, 2008 FLUENT 6.3 (2d, dp, dbns exp)

Figure 7: Second order derivative of pressure with respect to angle of attack contours around NACA0012 at M = 0.7 and α = 1o

f

E

~ f(x)

~ f

~ x

R ~ R

~~ f(x)

result is estimated by computing the relative error [9] (5)

Here ε is machine precision, σ ≤ 1 is a stability indicator that is known for binary operators (+, −, ∗, /) and κr is a relative condition number given by the following expression: κr =

||x|| || f $ (x)|| || f (x)||

(9)

The Stability class is structurally similar to Tangent and Curvature and overloaded operator+= is implemented as follows: Stability& operator+=(const Stability& x) { const double xs = x.stab(); if(xs == 0) _stability += numberTrait::eps; else _stability += (numberTrait::eps + xs); _value += x.value(); return *this; } As before, implementation of operator+ uses operator+=

Figure 8: Forward error illustration

|| f˜(x) ˜ − f (x)|| ˜ ≤ σκr ε. || f (x)|| ˜

ϒ = || f˜(x) ˜ − f (x)||. ˜

(6)

Symbol f˜(x) ˜ represent the perturbed output due to errors in input x˜ and f (x) ˜ represent exact function evaluated at the perturbed input x. ˜ The definition of the forward error is illustrated in a Fig. (8). Norm of the Jacobian matrix || f $ (x)|| is computed in subordinate 1 − norm || f $ (x)x|| || f $ (x)|| = sup . (7) ||x|| x&=0 Since the computation of the normwise relative condition number, Eq. (6) involves computation of the Jacobian of the expression, the condition number can

inline Stability& operator+(const Stability& x, const Stability& y) { return Stability(x) += y; } Other arithmetic operators are overloaded in a similar way and the resulting Stability class is used in analysis of the stability of algorithms. As an example of the use of the Stability class in the algorithm stability analysis, consider the simple problem of error cancellation. The function f (xi ) i = 1, 7 defined as 7

f (xi ) = ∑ xi

(10)

i=1

with the arguments x1 = x2 = 0.5, x3 = x4 = 1.0, x5 = x6 = 2.0, and x7 = −6.999998 is analyzed for the stability. This function is evaluated in single precision and due to subtractive cancellation error, the result of the summation is unreliable in floating point arithmetics. Theoretically, the stability of this algorithm

is proportional to the number of operations which is in this case S = 6 but the application of the Stability class results in a much higher number, S = 6.999997748E06 thus indicating that the result is unreliable. The simple way of correcting this problem is to switch x7 and x1 position in the summation in order to avoid subtractive cancellation error. In this case, the application of Stability class results in stability indicator S = 6. Another example that demonstrates the ability of the Stability class to analyze the stability of the algorithms due to the inherent stiffness in the function is provided by the Rump problem [10]. The Rump problem consists of evaluating the following function: f (x1 , x2 ) = + +

(333.75 − x21)x62 x21 (11x21 x22 − 121x42 − 2) x1 5.5x82 + 2x2

(11)

at the point x1 = 77617 and x2 = 33096. This function can be factored into the following function: f (x1 , x2 ) =

x1 −2 2x2

(12)

From the mathematical point of view, Eq. (11) and Eq. (12) are equivalent and they should produce the same result in exact arithmetic. However, in floating point arithmetic evaluation of Eq. (11) yields the result f (x1 , x2 ) = −0827396, whereas the evaluation of Eq. (12) yields the correct result f (x1 , x2 ) = 1.1726. Application of the Stability class to this problem shows that the stability of the of the algorithm in Eq. (11) is S = 5.81141E19 thus indicating that the algorithm is unstable. The Stability class is used to test the implementation of algorithms and it is very valuable in determining the potential problems with the code implementation. It can be applied equally to function evaluations, iterative algorithms and larger pieces of the code. Stability class is a very valuable tool for both code development and error analysis at the run time.

5

I NTERVAL A RITHMETIC

In interval arithmetic every real number is replaced by an interval so that the value of some variable is contained in the interval defined by the lower and upper interval bounds. The usual notation used in interval arithmetic for an interval is a pair of angled brackets [ ]. Usual arithmetic operations take a very special meaning and addition is defined as follows: [a, b] + [c, d] = [a + c, b + d].

(13)

Similarly, other arithmetic operations are defined by the following expressions: [a, b] − [c, d] = [a − d, b − c],

(14)

[a, b][c, d] = [min(ac, ad, bc, bd), max(ac, ad, bc, bd)],

(15)

[a, b]/[c, d] = [min(a/c, a/d, b/c, b/d), max(a/c, a/d, b/c, b/d)].

(16)

The Interval class is similar in structure to other Arithmetic classes and the implementation of overloaded operators operator+= is shown here: Interval& operator+=(const Interval& x) { unsigned short int mode = fpu_get_mode(); fpu_set_mode(mode_up); _sup += x.sup(); fpu_set_mode(mode_down); _inf += x.inf(); fpu_set_mode(mode); checkInfinity(); return *this; } inline Interval operator+(const Interval& u, const Interval& v) { return Interval(u) += v; } The Interval class is very useful in special computations since interval arithmetic adds some special operations such intersection, subset, etc. that are very useful in construction of robust iterative algorithms. These additional operations belong to the class of set operations and they are not available in real arithmetic. With their help, a robust iterative Newton method can be constructed that is not sensitive to the initial guess. One implementation of interval Newton method is as follows: int main() { Interval x, x_old; x = Interval(2,3); if (criterion(x))

{ do { x_old = x; cout