TIDES, a Taylor Series Integrator for Differential ... - Semantic Scholar

Algorithm 924: TIDES, a Taylor Series Integrator for Differential EquationS ALBERTO ABAD, ROBERTO BARRIO, FERNANDO BLESA, and MARCOS RODRÍGUEZ, IUMA, Universidad de Zaragoza, Spain

This article introduces the software package TIDES and revisits the use of the Taylor series method for the numerical integration of ODEs. The package TIDES provides an easy-to-use interface for standard double precision integrations, but also for quadruple precision and multiple precision integrations. The motivation for the development of this package is that more and more scientific disciplines need very high precision solution of ODEs, and a standard ODE method is not able to reach these precision levels. The TIDES package combines a preprocessor step in M ATHEMATICA that generates Fortran or C programs with a library in C. Another capability of TIDES is the direct solution of sensitivities of the solution of ODE systems, which means that we can compute the solution of variational equations up to any order without formulating them explicitly. Different options of the software are discussed, and finally it is compared with other well-known available methods, as well as with different options of TIDES. From the numerical tests, TIDES is competitive, both in speed and accuracy, with standard methods, but it also provides new capabilities. Categories and Subject Descriptors: G.1.7 [Numerical Analysis]: Ordinary Differential Equations; G.1.0 [Numerical Analysis]: General-Multiple precision arithmetic; G.1.4 [Numerical Analysis]: Quadrature and Numerical Differentiation-Automatic differentiation General Terms: Algorithms Additional Key Words and Phrases: Taylor series method, automatic differentiation, high precision, variational equations, numerical integration of ODEs ACM Reference Format: Abad, A., Barrio, R., Blesa, F., and Rodr´ıguez, M. 2012. Algorithm 924: TIDES, a Taylor series integrator for differential equations. ACM Trans. Math. Softw. 39, 1, Article 5 (November 2012), 28 pages. DOI = 10.1145/2382585.2382590 http://doi.acm.org/10.1145/2382585.2382590

1. INTRODUCTION

The Taylor method has a long history: in the work of Newton [1671, Problema II, Solution Casus II, Ex. I] we already see the solutions of differential equations by means of infinite series, in the works of Euler [1768, Section 663 E342] we can see a nice and complete introduction of the method, and in his proof of the existence of solutions of ODEs, Cauchy studied the convergence of the Taylor series of the solution [Cauchy 1839–42]. Besides, this method has been rediscovered several times (e.g., This work was supported by the Spanish Research projects MTM2009-10767 and AYA2008-05572. Authors’ addresses: A. Abad, Departamento de F´ısica Teórica and IUMA, Universidad de Zaragoza, E-50009 ´ Zaragoza, Spain; R. Barrio (corresponding author), Departamento de Matematica Aplicada and IUMA, Universidad de Zaragoza, E-50009 Zaragoza, Spain; email: [email protected]; F. Blesa, Departamento de F´ısica Aplicada, Universidad de Zaragoza, E-50009 Zaragoza, Spain; M. Rodr´ıguez, Centro Universitario de la Defensa, Universidad de Zaragoza, E-50090 Zaragoza, Spain. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permission may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701, USA, fax +1 (212) 869-0481, or [email protected]. c 2012 ACM 0098-3500/2012/11-ART5 $15.00 DOI 10.1145/2382585.2382590 http://doi.acm.org/10.1145/2382585.2382590

ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5

5:2

A. Abad et al.

in celestial mechanics the Taylor series method is called the recurrent power series method [Barton et al. 1971; Deprit and Zahar 1966; Lara et al. 1999; Roy et al. 1972]). In the field of numerical analysis Corliss and coworkers have studied the case of ODEs [Chang 1974; Chang and Corliss 1994; Corliss and Chang 1982], and it is the basis for the Taylor models of COSY [Makino and Berz 2006]. Moreover, the use of Taylor methods for Differential-Algebraic Equations (DAEs) has been considered in the last few years [Barrio 2005; Nedialkov and Pryce 2005, 2007, 2008; Pryce 1998, 2001; Revol et al. 2005] providing excellent results. Recently, in several studies in dynamical systems, for example, in the determination of periodic orbits [Barrio and Blesa 2009; Barrio et al. 2009b,a], the numerical integration of the differential systems is done using the Taylor method. In this type of problem, it has proven its applicability. Moreover, some extensions of the Taylor series method to solve directly variational equations have been used in Barrio [2006a] and Barrio and Serrano [2007] in the study of several chaos indicators. Another important application of the Taylor method is that it can be executed using interval arithmetic, and thus it allows us to obtain validated numerical methods for differential equations. This subject is currently an area of great interest (see Nedialkov et al. [1999] and references therein). For instance, one important application is in computer-assisted proofs in dynamical systems, like the computer-assisted proof of the existence of the Lorenz attractor [Tucker 2002], the computer-assisted proof of existence of periodic orbits [Galias 2001], and so on. The present article is organized as follows. Section 2 reviews some basic properties of the Taylor method for ODEs and introduces several variable-stepsize and variableorder schemes used in TIDES. Section 3 presents some of the automatic differentiation algorithms and how TIDES computes Taylor series. Section 4 studies the computational complexity of our method. Section 5 explains briefly the use of TIDES. Section 6 presents several numerical tests comparing them with well-established codes. Finally, Section 7 presents conclusions. 2. TAYLOR SERIES METHOD: BASIC PROPERTIES

Let us consider the initial value problem dy(t) = f(t, y(t); p), y(t0 ) = y0 , t ∈ R, y ∈ Rs, p ∈ Rk , (1) dt where p represents a vector of k ≥ 0 parameters. The value of the solution at ti+1 = ti + hi (that is, y(ti+1 )) is approximated from the n-th degree Taylor series of y(t) developed at ti and evaluated at t = ti+1 (the function f has to be smooth enough. In this article we assume f to be analytic). y(t0 )

def

= y0 ,

y(ti+1 ) y(ti) +

dy(ti) 1 dn y(ti) n hi + · · · + hi dt n! dtn

yi + f(ti, yi) hi + · · · +

(2)

f(ti, yi) n def 1 d hi = yi+1 n! dtn−1 n−1

From the formulation of the Taylor Series Method (from now on TSM and TSM(n) for the n-th degree TSM), the problem is reduced to computing the Taylor coefficients {d j y(ti)/dt j}. One procedure for computing them comes from the differentiation of the right-hand side of the differential equation f(t, y(t); p). This approach is useful only in theoretical studies, because when n grows, the complexity of symbolic computation ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

Algorithm 924: TIDES, a Taylor Series Integrator for Differential EquationS

5:3

of the derivatives is too high (exponential). Another approach is to use Automatic Differentiation (AD) techniques [Griewank 2000] as in Chang and Corliss [1994] and Corliss and Chang [1982]. Several preprocessing codes have been designed to obtain the recurrence formulas of Taylor coefficients via AD [Chang and Corliss 1994; Corliss and Chang 1982; Jorba and Zou 2005]. In TIDES (see Section 5) we have used a preprocessor called MathTIDES which is a M ATHEMATICA [Wolfram 1999] package that generates code to obtain the Taylor series solution of an ODE problem for which f is smooth. The output is a Fortran or C program that computes the recurrence formulas used to integrate the differential system by recurrent power series, that is, the coefficients of the Taylor series method. The TSM has several advantages. One of them is that it gives directly a dense output in the form of a power series, and therefore, we can evaluate the solution at any time using the Horner algorithm. Also as TSM of degree n is of order n, the use of TSMs of high degree gives us a numerical method of high order. Therefore, they can be useful for high-precision solution of ODEs [Barrio 2005, 2006b; Barrio et al. 2005]. Now we study briefly several properties of the TSM. One of them is the behavior of the TSM for stiff differential equations. In these situations, it is necessary to use A-stable methods (or methods with good stability properties), but it is clear that TSM is not A-stable (it is only in the infinite case, taking infinite series). As TSMs are not Astable, we are interested in studying the size of the stability domain. Low-order TSMs (order n ≤ 4) have the same stability domain as any explicit Runge-Kutta method of n stages and also order n [Ascher and Petzold 1998], but this is only valid for very low order, and we are interested in high-order TSMs. From Barrio [2005] and Barrio et al. [2005] we have the following. P ROPOSITION 2.1. The stability domain of the TSM(n) tends to a semicircle in the negative complex plane whose radius r behaves asymptotically like O(n), and it can be approximated by r(n) ≈ rapprox (n) = 1.3614 + 0.3725 n. In Figure 1, we represent the stability domain for several TSMs. From these plots we observe that, when the order n grows, the stability domain tends to a semicircle of radius rapprox (n) centered at the origin, showing good approximation of the domain by a semicircle. Other authors [Jeltsch and Nevanlinna 1982] have already studied the stability domain of high-order TSMs but without giving explicit simple formulas for the stability domain. From this analysis, we can establish that high-order TSMs can be used with moderately stiff equations, because the size of the stability domain grows linearly with n. Hence, for large n, the stability domain is reasonably large. Obviously, it cannot be used for highly stiff systems, where it is necessary to use implicit methods. Implicit versions of Taylor methods that are A-stable have been developed in Barton [1980], Corliss et al. [1997], and Griewank [1996]. TIDES does not detect stiffness, as this is out of the scope of the software, but when the stepsize decreases dramatically it can be due to that fact, and it is advisable to use a standard implicit method. In practical implementations of a numerical method for the solution of ODEs, the use of variable stepsize is crucial. Several formulations of Variable-Stepsize (VS) Taylor methods can be found in Barrio [2005], Barrio et al. [2005], Corliss and Lowery [1977a], and Jorba and Zou [2005] and references therein. Note that in literature there are several variable-order codes [Hindmarsh et al. 2005; Shampine and Baca 1986]. ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:4

A. Abad et al.

Fig. 1. Stability domains of several TSM methods of different orders n and approximations using the radius given by rapprox (n).

In TIDES, we use an absolute error tolerance tolabs and a relative tolerance tolrel. With both, we construct the error tolerance TOL at the i + 1 step TOL = tolabs + max( yi, yi−1 ) × tolrel. Another crucial point in the TSM is the selection of the order of the method, n. In TIDES, we adopt a modification of the optimal order used, for example, in Jorba and Zou [2005]. On the one hand, when we use an order that depends only on the requested tolerance TOL, we adopt the simple formula n = − ln(TOL)/2 + nordinc, where nordinc is an increment of the order with respect to the asymptotic formula (this may be adjusted by the user). This is the case in the dp-tides and mp-tides programs (see Section 5), where the complexity of the extended Taylor series algorithm does not justify the use of a more adaptive algorithm. Therefore, in these cases, we use a formula for the order that depends only on the tolerance. In the minf-tides and minctides programs, we use instead a more sophisticated formula tolorder(i + 1) = min tolabs/ min( yi, yi−1 ), tolrel , n = − ln(tolorder(i + 1))/2 + nordinc. In both cases, we use n = max (minord, n) , where we take minord = 6 as default value for the minimum order. We use two strategies for selecting stepsize. The first one is based on estimating the error by taking the last term in the Taylor series (to avoid problems with odd/even functions, we take the last two terms different from zero: n and (n− 1)th, which avoids ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:5

problems with polynomial solutions). Hence, the estimated stepsize at the i + 1 step is given by ⎧ 1/(n−1) 1/n ⎫ ⎨ ⎬ TOL TOL hi = min , , ⎩ y[n−1] ∞ ⎭ (3) yi[n] ∞ i hi), rminstep × hi−1 , hi = fac1 × max min(rmaxstep × hi−1 , with y[n] the normalized derivative y[n] = y(n) /n!, fac1 a safety factor (we use fac1 = 0.9), and rmaxstep = 100 and rminstep = 0.01 stand for the maximum and minimum ratio between the actual stepsize and the previous one. Other authors [Chang and Corliss 1980, 1994; Corliss and Lowery 1977b] use more terms giving more complicated and likely better estimates of the radius of convergence of the Taylor series, but our strategy has given us good results in all the tests and real problems used by our team. In any case, after this selection of the stepsize, we may enter, or not (it can be selected by the user), in a refinement process, which is based on the defect error control [Enright 1989; Enright and Hayes 2007]. In the TSM, there is no rejected step as occurs in any variable-stepsize formulation for Runge-Kutta or multistep methods, because we choose the stepsize once the series are generated to obtain a required precision level. However, to give more guarantee about the stepsize, we may analyze the agreement between the tangent vector to the Taylor polynomial and the vector field at the end of the step. That is, given the Taylor approximation of the solution on the interval [ti, ti+1 ] = [ti, ti + hi] y(t)

n

y[k] (ti) · (t − ti)k ,

y (t)

n

k=0 then, evaluating at the end of the interval yi+1 ≡ rejecting the stepsize is

if

k y[k] (ti) · (t − ti)k−1

k=1

n k=1

k yi[k] · hik−1 , the criterion for

yi+1 − f(ti+1 , yi+1 )∞ > fac2 × TOL then

hi = fac3 · hi,

(4)

where fac2 and fac3 are control factors that reduce the stepsize (we have taken fac2 = 10, fac3 = 0.8). We remark that this strategy uses just the last point, while in a true defect-control error the defect is sampled at several points. Although the stepsize may be rejected, we do not have to recalculate the Taylor coefficients: we only have to consider the new stepsize and check (4) for rejecting the stepsize. Therefore, we cannot say that we reject a complete step, we just reject the estimation of the stepsize, and so the computational cost is small (in fact the cost of and f(ti+1 , yi+1 )). The cost of rejecting the estimation of the stepsize evaluating yi+1 is just O(n) (the cost of the Horner algorithm), while the cost of computing the Taylor coefficients is O(n2 ). This process is done a maximum of nitermax = 5 times. The extra error control (defect error control (4)) avoids the use of too large stepsizes for entire functions, that can lead to large rounding errors in the evaluation process. 3. AUTOMATIC DIFFERENTIATION: PARTIAL DERIVATIVES

In the practical computation of Taylor coefficients of a function, we use the classical rules for Automatic Differentiation (AD) of the elementary operations and standard functions (±, ×, /, log, sin, . . .). For details of this technique see Moore [1966], Rall [1981], Rall and Corliss [1996], and Griewank [2000]. ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:6

A. Abad et al.

The AD technique is based on the decomposition of a function into a sequence of arithmetic operations and calls to standard unary or binary functions. For instance, if we want to evaluate the function f (x) = sin x2 /(x2 + 5), we can decompose it as l1 l2 l3 l4 l5 f (x)

= = = = = =

x l1 l1 5 + l2 sin l2 l4 /l3 l5 .

We call this decomposition a Linked function (LK-function). There is no unanimity in the name of this concept: code list [Rall and Corliss 1996], computational graph [Griewank 2000], computational process [Bauer 1974]. However these names describe different aspects of the concept: the code to evaluate the function, a graph to visualize the iteration, etc. We began to work with this concept from a symbolic point of view, and at that stage we decided to use the name “linked function” referring to the Mathematica expression obtained after simplification of the function and the common subexpressions. The LK-function can be used, together with the chain rule, to evaluate the derivative of the function f . The use of LK-functions gives us a way to construct, term by term, theTaylor series solution of Eq. (1). Assume we have the Taylor series of the solution y = i y[i] ti. Then, by replacing y by its power series expansion in (1), we have

[i]

(i + 1) y[i+1] ti ∼ y[i] ti; p) ∼ f ( y[0] , . . . , y[i] ; p) ti, = = f(t, i

i

i

and finally, equating both sides, we obtain the relation f [i] ( y[0] , . . . , y[i] ; p) , i+1 which gives us the iterative procedure to compute each term of the Taylor expansion beginning from the initial condition y0 and expanding f by applying arithmetic of series, given, for instance, in Moore [1966] and Moore et al. [2009], to the LK-function that represents f. In this article, as we focus our attention on the generation of partial derivatives, we give extended formulas for the Taylor coefficients of a function and/or its partial derivatives of any order based on the formulas in Barrio [2006b]. In the literature, there are several formulas for the AD computation of partial derivatives [Griewank et al. 2000; Makino and Berz 2006; Neidinger 1992, 2005; Tsukanov and Hall 2003]. In the following, we denote by N0 the set of the natural numbers with the zero (N0 := N ∪ {0}), i = (i1 , i2 , . . . , is) ∈ Ns0 a vector with partial derivative orders, i∗ = i − (0, . . . , 0, 1, 0, . . . , 0) = (0, . . . , 0, ik − 1, i k+1 , . . . , is), that is, we subtract 1 from the first nonzero element ik of i if i = 0, |i| = sj=1 i j is the total order of derivation, and i − j = (i1 − j1 , . . . , is − js). The jth normalized Taylor coefficient of the partial derivative of the function f with respect to yi = (yii1 , . . . yiss ) is denoted by y[i+1] =

f [ j] :=

1 dj f , j! dt j

[ j]

fi :=

∂ (i1 +···+is) f [ j] ∂ yi11 ∂ yi22 · · · ∂ yiss

,

and [ j,n] hv,i = h[vj] ,

( j = n or v = i),

h[n,n] = 0. i,i



5:7

In addition, given v = (v1 , . . . , vs) ∈ Ns0 , we define the multicombinatorial number i i1 i2 = v1 · v2 · · · viss , and we consider the classical partial order in Ns0 . v We divide the basic formulas into three groups, as described next. Definition 3.1. We call an LK-independent function any elementary function that does not need auxiliary functions for its LK sequence. In this class of functions we have: x ± y, x × y, x/y, 1/x, xα (α ∈ R), α x (α > 0 ∈ R), exp(x), log x. We call LK-cycle(n) function any function that itself and its n − 1 derivatives form a cycle. In this class of functions, we have the LK-cycle(2) sets: {sin, cos}, {sinh, cosh}. We call LK-order(n) aided function any function that its nth derivative is expressed by elementary LKindependent functions. In this class of functions (LK-order(1)), we have all the inverse trigonometric functions. P ROPOSITION 3.2. Suppose that f (t, y(t)), g(t, y(t)) : (t, y) ∈ Rs+1 → R are functions of class C n. Then, for the basic LK-independent functions, we have [n] [n] (i) If h(t) = f (t) ± g(t) then h[n] i = fi ± gi . [n] (ii) If h(t) = α f (t) with α ∈ R then h[n] i =α ⎫ ⎧ fi . n ⎬

i ⎨ [ j] [n− j] = f · g (iii) If h(t) = f (t) · g(t) then h[n] v i i−v ⎭. v ⎩ v≤ i

(iv) If h(t) = f (t)/g(t) then h[n] i

j=0

⎫⎞ ⎧ ⎛ n 1 ⎝ [n] i ⎨ [n− j,n] [ j] ⎬⎠ hv,i = [0] · gi−v fi − ⎭ v ⎩ g j=0

v≤ i

(v) If h(t) = f (t)α with α ∈ R\{0} and ( f [0] (t))α ∈ R then [0] α h[0] 0 = ( f (t)) ,

h[0] i

h[n] i

1 i∗ [0,0] [0] [0] [0] α hv · fi−v − hi−v, i · fv , = [0] v f v≤ i∗ ⎧ ⎫ n ⎬ ⎨ i 1 [ j,n] [n− j] h n α − j(α + 1) = · f i−v ⎭ , ⎩ v v, i n f [0] j=0

i > 0,

n > 0.

v≤ i

(vi) If h(t) = exp( f (t)) then [0] h[0] 0 = exp( f (t)),

i∗ [0] h[0] v · fi−v , v v≤ i∗ ⎧ ⎫ n−1 ⎨ ⎬

i 1 [n− j] h[vj] · fi−v = (n − j) , ⎩ ⎭ v n

h[0] i =

h[n] i

j=0

i > 0,

n > 0.

v≤ i


5:8

A. Abad et al.

(vii) If h(t) = log( f (t)) then [0] h[0] 0 = log( f (t)),

h[0] i

h[n] i

⎛ ⎞

i∗ 1 ⎝ [0] h[0] · f [0] ⎠ , = [0] fi − v i−v v f ∗ 0 0,

n > 0.

v≤ i

P ROPOSITION 3.3. Suppose that f (t, y(t)), g(t, y(t)) : (t, y) ∈ Rs+1 → R are functions of class C n. Then, for the basic LK-cycle(2) functions, we have (i) If g(t) = cos( f (t)) and h(t) = sin( f (t)) then g0[0] = cos( f [0] (t)), gi[0]

gi[n]

h[n] i

[0] h[0] 0 = sin( f (t)),

i∗

i∗ [0] [0] [0] [0] h · fi−v , g[0] · fi−v = − hi = , i > 0, v v v v ∗ ∗ v≤ i v≤ i ⎧ ⎫ n ⎬ 1 ⎨ i [ j] j] h[n− = − j · f n > 0, i−v ⎭ , ⎩ v v n j=1 v≤ i ⎧ ⎫ n 1 ⎨ i [n− j] [ j] ⎬ g = j · fi−v , n > 0. ⎩ ⎭ v v n j=1

v≤ i

(ii) If g(t) = cosh( f (t)) and h(t) = sinh( f (t)) then g0[0] = cosh f [0] (t) ,

[0] h[0] 0 = sinh f (t) ,

i∗

i∗ [0] [0] [0] h[0] gv[0] · fi−v · f , h = , i > 0, v i−v i v v v≤ i∗ v≤ i∗ ⎧ ⎫ n ⎨ ⎬

i 1 [ j] j] h[n− = j · f n > 0, i−v ⎭ , ⎩ v v n j=1 v≤ i ⎧ ⎫ n ⎨

i [n− j] [ j] ⎬ 1 g = j · fi−v , n > 0. ⎩ ⎭ v v n

gi[0] =

gi[n]

h[n] i

j=1

v≤ i

P ROPOSITION 3.4. Suppose that f (t, y(t)), g(t, y(t)) : (t, y) ∈ Rs+1 → R are functions of class C n. Then, if we consider the function h(t) = ψ(u) ≡ ψ( f (t)) such that ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:9

dψ/du = 1/g(u), we have for the basic LK-order(1) functions given by the inverse trigonometric and hyperbolic functions [0] h[0] 0 = ψ( f (t)),

h[0] i

⎫ ⎧ ⎬

i∗ 1 ⎨ [0] [0] [0] h = − · g , f v i−v v ⎭ g( f [0] ) ⎩ i ∗

i > 0,

0 0,

v≤ i

[ j]

where gv are the Taylor coefficients of g and √ (i) If h(u) = asin(u) then g(u) = √ 1 − u2 . (ii) If h(u) = acos(u) then g(u) = − 1 − u2 . (iii) If h(u) = atan(u) then g(u) = 1√ + u2 . (iv) If h(u) = asinh(u) then g(u) = √1 + u2 . (v) If h(u) = acosh(u) then g(u) = u2 − 1. (vi) If h(u) = atanh(u) then g(u) = 1 − u2 . The proofs of all of these formulas are based on classical arguments of AD for Taylor coefficients of the derivatives, and the extension of most of the cases to partial derivatives can be found in Barrio [2006b]. For i = 0, these formulas reduce to the classical formulas of arithmetic of series. In the case of Proposition 3.4, we also need the coefficients of the auxiliary function g(u), and for the LK-cycle(2) functions, we need to obtain the coefficients of both functions. If a function of an LK-cycle appears, we have to compute simultaneously all the functions of the cycle. For example, taking again the function f (x) = sin x2 /(x2 + 5), the extended LK-function is l1 = x l2 = l1 l1 l3 = 5 + l2 l4 = sin l2 l5 = cos l2 l6 = l4 /l3 f (x) = l6 in which the link l5 is added, because we need it to compute the expansion of l4 . 3.1. Dealing with Partial Derivatives in TIDES

With the previous notation, we identify each element i of the set Ns0 with the corresponding partial derivative of f . Let’s name Dsm = {i | |i| ≤ m} the set of all partial derivatives of f of order less orequal to m. Wedefinea total order relationship in the ik < jk ; or ik = jk and there exists an index p, set Dsm by considering i < j if such as ik = jk for all k < p, and i p < jp ; and i = j if ik = jk for all k. The position of i in Dsm is [Tsukanov and Hall 2003] j−1 s

Kj + j , where K j = Dsm(i) = is−k − 1. j j=1

k=0


5:10

A. Abad et al.

To compute the partial derivative fi we need to know the partial derivatives represented by a subset of Dsm, with m = |i|, given by V(i) = {v | v ≤ i}. The cardinality of Dsm and V(i) are s m+ s m Ds = , V(i) = (i j + 1). s j=1

If we want to compute not only the derivative i, but the set of derivatives D of different orders, we need to compute all the derivatives given by V(D) = i∈D V(i) ⊂ Dsm. We name ord(V(D)) the result of ordering the set V(D) with this order relation. Then we may identify each element of the set V(D) with an integer number between 0 and N−1, with N = D the number of elements of V(D), that represents its position in the ordered set ord(V(D)). In the process of computing partial derivatives in the TSM, only two types of expressions are involved (see Propositions 3.2, 3.3, 3.4)

i Ci,v fv[−] gw[−] , Ci∗ ,v fv[−] gw[−] , w = i − v, Ci,v = , (6) v ∗ v,w∈V(i)

v,w∈V(i )

where i goes over the set V(D), and [−] represents the order. The library LibTIDES contains functions to compute all these sums, but the indexes i and the sets V(i), V(i∗ ) change depending on the function and the desired derivatives. The M ATHEMATICA package MathTIDES computes these sets automatically and uses them to output C code. If we want to compute all the partial derivatives represented by the set D, then MathTIDES performs the following operations. (1) It computes the N elements of the set V(D). (2) It sorts the set V(D) and identifies each element i of the ordered set ord(V(D)) with an index 0 ≤ i < N that represents the position of the element in the ordered set. (3) For every i of ord(V(D)), it constructs the sets V(i), V(i∗ ) and the lists with the coefficients Ci,v , Ci∗ ,v . (4) The sets V(i), V(i∗ ) are represented by two lists of si , s∗i integers in [0, N], that represent the indexes v of the sums. (5) Two more lists, with the same elements as the previous ones, but ordering to represent the elements w = i − v, are computed. With the previous elements, MathTIDES constructs data structures needed to implement the sums (6). The location of the different partial derivatives is assigned also by MathTIDES, and it is stored in C data structures that LibTIDES uses (other approaches in the literature use heavy formulas for pointing to the data [Tsukanov and Hall 2003]). These data structures are based on four lists of integer numbers: PREV ACUM, PREV COEF, PREV VI, PREV IV, associated with the sets V(i), for all i, and four more PREVSTAR ACUM, PREVSTAR COEF, PREVSTAR VI, PREVSTAR IV, associated with the sets V(i∗ ), for all i∗ . In the following, we describe only the first four. (1) PREV ACUM is a list with N + 1 integers. PREV ACUM[i] represents the position in the other three lists, where the information of the set V(i) begins, and i−1 it has the value j=1 sj . The last element of PREV ACUM contains the number N j=1 sj of elements of each of the other lists. (2) PREV COEF contains the list of the coefficients Ci,v . The si coefficients corresponding to the elements of the sum in V(i) are stored from the position PREV ACUM[i] to the position PREV ACUM[i + 1]−1. ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:11

(3) PREV VI contains the list of the elements v ∈ V(i). These elements are stored from the position PREV ACUM[i] to the position PREV ACUM[i + 1]−1. (4) PREV IV contains the list of the elements w ∈ V(i). These elements are stored from the position PREV ACUM[i] to the position PREV ACUM[i + 1]−1. As an example, let us suppose we want to compute the partial derivative ∂3 f , ∂ y1 ∂ y22 that is, the set of derivatives represented by D = {(1, 2)}. After ordering the set V(D) of derivatives necessary in the calculus process, we obtain the following set of 6 elements and its corresponding set of indexes ord(V(D)) = {(0, 0), (1, 0), (0, 1), (1, 1), (0, 2), (1, 2)} = {0, 1, 2, 3, 4, 5}. Taking the fourth element we have V(4) = V((0, 2)) = {(0, 0), (0, 1), (0, 2)} = {0, 2, 4}. The set of elements v are {0, 2, 4}, while the set of elements w are {4, 2, 0}. Joining these lists with the lists corresponding to the rest of the elements of ord(V(D)), we have the following table of elements. PREV ACUM

0

1

3

5

9

12

18

PREV COEF

1

1

1

1

1

1

1

1

1

1

2

1

1

2

1

1

2

1

PREV VI

0

0

1

0

2

0

2

1

3

0

2

4

0

2

4

1

3

5

PREV IV

0

1

0

2

0

3

1

2

0

4

2

0

5

3

1

4

2

0

4. COMPUTATIONAL COMPLEXITY

In the complexity study of the TSM method, the first step is to calculate the computational complexity of the computation of derivatives, that is, the automatic differentiation process using the algorithms of Section 3. From classical references (see Barrio [2006b],Corliss and Chang [1982], and Rall and Corliss [1996], and references therein) we have what follows. P ROPOSITION 4.1. If the evaluation of f (t, y(t)) involves k elementary functions (×, /, log, exp, sin, cos, . . .) then the computational complexity of the evaluation of f [0] , f [1] , . . . , f [n−1] is k · O(n2 ). (In the case of linear functions k · O(n).) This complexity result can be extended to the computation of partial derivatives [Barrio 2006b]. P ROPOSITION 4.2. If the evaluation of f (t, y(t)) involves k elementary functions (×, /, log, exp, sin, cos, . . .) and given i = (i1 , i2 , . . . , is) ∈ Ns0 , then the computational complexity of the evaluation of fi[0] , fi[1] , . . . , fi[n−1] , assuming already known all derivatives of index v < i, is ⎤ ⎡ s ⎣ i j + 1 ⎦ k · O n2 . j=1

Note that, if the function f (t, y(t)) is a linear function (therefore it involves only additions/substractions and multiplication by scalars), then the computational complexity is evidently O(n). ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:12

A. Abad et al.

In the particular case of computing the solution to the first- or second-order variational equations, we may simplify the preceding formulas. C OROLLARY 4.3. The computational complexity of evaluating the Taylor coefficients of a partial derivative of f is twice the complexity of evaluating the Taylor coefficients of f , and the computational complexity of evaluating the Taylor coefficients of a secondorder partial derivative of f is, once the coefficients of the first-order partial derivatives are known, four times the complexity of evaluating the Taylor coefficients of f in the case of ∂ 2 f/∂ yi ∂ y j (i = j) and three times in the case ∂ 2 f/∂ yi2 . Now we can study rigorously the computational complexity of solving an ODE system using the Taylor series method using the results of R.M. Corless [Corless 2002, Corless and Ilie 2008] about the complexity of ODE solvers using the Hölder mean. We assume that the structure of the local truncation error follows the common model LTE = ψ(t)hn+1 , where ψ is the principal error function, n is the order of the method, and h is the stepsize. This model fits well the error estimates of the TSM. Thus, just by adapting Theorem 6.2 of Corless and Ilie [2008] to the solution of ODEs and using the number of digits of the requested accuracy instead of the number of bits, we may obtain the following result. T HEOREM 4.4 [C ORLESS AND I LIE 2008]. The minimal cost of computing the solu˙ = f ( y) on the finite interval [a, b ] with the TSM of order n is tion of the ODE system y polynomial in the number D of requested correct digits (D = − log10 TOL, with TOL the user error tolerance) and is bounded above by C(b − a)(M−1/n( N ))1/n D 4 ,

1 N m 1/m obtained for n = −(1/2) ln TOL , where C ∈ R and Mm( N ) = ( N dei=1 ψi ) notes the Hölder m-mean for the vector of the principal error function at the N steps N = (ψ1 , . . . , ψ N ). The minimum cost is reached on the equidistributing mesh ¯ nM−1/n( N ) = TOL for all 1 ≤ i ≤ N with h ¯ = (b − a)/N). (ψihin = h

We remark that Theorem 4.4 supposes that all the operations are done without roundoff errors. If we add the roundoff errors, we will obtain a similar result that has been recently proved by J.D. Pryce [Pryce et al. 2009] bounding the effect of rounding errors. The value of the optimal order (n = −(1/2) ln TOL ) was also obtained by C. Simó [2001] using assumptions on the radius of convergence and on the form of the Taylor series coefficients. This analysis justifies the estimation of order in the TSM of Section 2. Therefore, the computational complexity of the numerical solution of an ODE system using a TSM with D = − log10 (TOL) number of digits is O(D 4 ) using variable-order (depending on the requested tolerance TOL), variable-stepsize, and variable-precision arithmetic (D + 5 digits); and C · O(D 2 ) using variable-order, variable-stepsize, and fixed-precision arithmetic (505 digits). In the fixed precision case, the O(D 2 ) term is multiplied by a constant term C that includes the computer cost of using always the highest precision, 505 digits, and so it is slower than the use of variable-precision arithmetic. Just for high precision the computer cost of both approaches are close one to each other because this constant term is equivalent to the other D 2 term in the theoretical estimates. In Figure 2, we show CPU time versus number of digits on the classical Lorenz model (using variable-precision arithmetic and fixed-precision arithmetic with 505 digits) up to 500 digits of precision. This figure explains the fact that, ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:13

Fig. 2. (Color online) CPU time versus the error precision in the numerical solution of the Lorenz model using variable-order, variable-stepsize, and variable-precision arithmetic; and using variable-order, variablestepsize, and fixed-precision arithmetic (505 digits); and quartic and quadratic approximations, respectively.

for arbitrary precision calculations, it is much more optimal to use an adaptive precision arithmetic. We observe good fits of the quadratic and quartic approximations to the real CPU time, as predicted by the theory. 5. TIDES

In this section, we show briefly the structure of the software (more details may be found in Abad et al. [2011]), and we give an example of its use. TIDES consists of two main parts: the M ATHEMATICA package MathTIDES and the C library LibTIDES. TIDES Package

Language

MathTIDES

preprocessor

M ATHEMATICA

LibTIDES

library (objects or source code)

C

The preprocessor MathTIDES writes, automatically, all the necessary files containing the code in C or Fortran to integrate an ODE system. With MathTIDES, we may write four different versions of the TSM: two minimal (faster) versions in Fortran (minf-tides) and C (minc-tides) respectively, and two standard (more complete) versions in double (dp-tides) or arbitrary precision (mp-tides) respectively. Version

Contents

MathTIDES generates

linked with

minf-tides

basic TSM

Fortran files

minc-tides

basic TSM

C files

dp-tides

complete TSM + partial derivatives

C files

LibTIDES

mp-tides

complete TSM + partial derivatives + arbitrary precision

C files

LibTIDES MPFR library GMP library


5:14

A. Abad et al.

Files generated using the standard version must be linked with the library LibTIDES (and the high-precision libraries in mp-tides version). The minimal version codes (minf-tides, minc-tides) are used when we ask for the integration of an ODE whose system of equations uses only x ± y, x × y, x/y, 1/x, xα (α ∈ R), α x (α > 0 ∈ R), exp(x), log x, sin(x) and cos(x) functions. When we ask also for the solution of variational equations up to any order (or the functions sinh, cosh, tanh, asin, acos, atan, asinh, acosh, atanh appear in the system of equations), the standard version dp-tides must be used. When we ask for high-precision integrations, mp-tides is required. To illustrate the use of MathTIDES we give, as an example, the Lorenz model given by equations x˙ = −σ x + σ y,

y˙ = −xz + r x − y,

z˙ = xy − b z.

(7)

We need two calls in M ATHEMATICA: one to create the associated differential equation, and another one to generate the files according to the needs of the user. If we want a basic integration and code in Fortran, the commands are as follows. lorenzeq = FirstOrderODE[{s*(y - x), -x*z + r*x - y, x*y - b*z}, t, {x, y, z}, {s, r, b}]; CodeFiles[lorenzeq, "lorenz", MinTIDES->"Fortran", Output->Screen] In the first line, the arguments of the function are: a list with the equations, the independent variable, a list with the dependent variables, and a list with the parameters of the system. The second command, that generates the files, has as arguments: the generated differential equation (same name as previous command), name for the output files, minimal version minf-tides in Fortran, and type of output. The output includes options for dense output, just by choosing the time increment or the number of points. The rest of standard data (as the initial conditions, etc.) can be introduced as options in the driver file generated by MathTIDES. If we want to generate the code to integrate the system in multiple precision, for example, 300 digits, simultaneously with sensitivities up to order 3 with respect to x0 and σ , the second command in M ATHEMATICA should be as follows. CodeFiles[lorenzeq, "lorenzMP", Precision->Multiple[300], Output->Screen, AddPartials->{{x, s}, 3}] Detailed information about installation, use, and more advanced options can be found in the documentation included in the software package. 6. NUMERICAL EXPERIMENTS

In the numerical simulations we have used three different architectures: a Mac OS X 10.6 Core2 Duo 2. GHz, Linux Core2 Duo 2.66 GHz and a Windows XP SP2 Pentium D 3.40 GHz. The tests in double precision have been done using the GNU compilers g77, gfortran (Fortran), and gcc (C), the quadruple precision tests using the Lahey LF95 compiler (on a Windows XP Pentium D 3.40 GHz), and the multiple precision tests using C and the MPFR library [Fousse et al. 2007]. In addition to the MPFR library (the one used by TIDES), there are several excellent multiple precision libraries, including gmp (a GNU multiple precision library in C), mpfun and mpf90 (Fortran 77 and Fortran 90 multiple precision libraries of D. H. Bailey [Bailey 1995]). The TSM is compared with well-established codes dop853 and odex developed by Hairer et al. [1993]. The dop853 code is based on an explicit Runge-Kutta of order 8 given by Dormand and Prince with stepsize control and dense output, and odex is based on an extrapolation method (that is also of variable order). The results have ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:15

Table I. Comparisons on Some DETEST Problems of TIDES in the Fortran Version and DOP853 Prob

RelErr

CPU

DOP853 Steps

RelErr

9 (47)15 8 (48)16 13 15 36 39 7 7 10 12

0.2178E−02 0.2587E−07 0.2159E−08 0.4615E−14 0.1903E−09 0.1399E−14 0.7983E−10 0.1069E−14 0.2305E−11 0.2004E−15 0.2170E−08 0.1590E−13

0.70E−04

27

0.5117E−04

0.25E−03

129

0.8863E−08

0.70E−04 0.23E−03 0.38E−03 0.13E−02 0.50E−04 0.19E−03 0.70E−04 0.23E−03

23 106 78 384 13 67 16 79

0.2816E−11 0.1272E−15 0.3572E−09 0.1016E−13 0.2623E−11 0.1000E−16 0.6621E−09 0.1590E−13

0.90E−04 0.15E−03 0.10E−04 0.20E−04 0.10E−04 0.30E−04 0.80E−04 0.13E−03 0.70E−04 0.11E−03

69 81 14 14 14 16 25 22 42 48

0.4533E−08 0.5747E−14 0.1025E−11 0.4441E−15 0.1049E−10 0.2347E−15 0.2366E−10 0.8080E−15 0.5340E−10 0.4962E−14

0.65E−03 0.23E−02 0.16E−03 0.52E−03 0.15E−03 0.48E−03 0.44E−03 0.18E−02 0.40E−03 0.17E−02

144 723 34 158 29 135 87 454 80 418

0.3274E−08 0.1248E−13 0.1118E−12 0.2220E−15 0.7932E−12 0.4687E−15 0.3540E−08 0.2848E−13 0.2434E−09 0.7443E−14

0.40E−04 0.60E−04 0.11E−03 0.17E−03 0.15E−03 0.25E−03 0.10E−04 0.10E−04 0.10E−04 0.30E−04

17 17 87 100 59 65 4 5 6 7

0.1820E−09 0.2021E−14 0.2319E−09 0.8151E−14 0.4089E−09 0.3971E−14 0.1544E−09 0.3597E−16 0.2842E−09 0.2139E−14

0.32E−03 0.13E−02 0.14E−02 0.45E−02 0.96E−03 0.38E−02 0.80E−04 0.24E−03 0.14E−03 0.36E−03

52 266 184 900 118 610 10 44 13 64

0.7960E−09 0.3096E−13 0.4622E−10 0.1761E−13 0.1653E−08 0.5997E−13 0.4301E−12 0.2093E−15 0.1618E−10 0.1258E−14

Tol

CPU

A1 A1(DE) A1 A1(DE) A2 A2 A3 A3 A4 A4 A5 A5

10−10 10−10 10−15 10−15 10−10 10−15 10−10 10−15 10−10 10−15 10−10 10−15

0.10E−04 0.20E−04 0.10E−04 0.30E−04 0.10E−04 0.30E−04 0.60E−04 0.11E−03 0.10E−04 0.10E−04 0.10E−04 0.20E−04

B1 B1 B2 B2 B3 B3 B4 B4 B5 B5

10−10 10−15 10−10 10−15 10−10 10−15 10−10 10−15 10−10 10−15

E1 E1 E2 E2 E3 E3 E4 E4 E5 E5

10−10 10−15 10−10 10−15 10−10 10−15 10−10 10−15 10−10 10−15

TIDES Steps

All the tests use the standard stepsize estimator (3) except in the case A1(DE), where it is combined with the defect error control strategy (4).

been compared with a reference orbit computed with multiple precision to guarantee sufficient correct digits. The errors in the tables are relative errors using the L ∞ norm at the end of the integration time. 6.1. Preliminary Tests: DETEST

Previous studies of the authors [Barrio et al. 2005] using the TSM presented comparisons on standard test set of problems, like DETEST [Enright and Pryce 1987]. We present a short comparison with DETEST, but having in mind that the objective of TIDES is not to compete on basic ODE problems and requirements. In Table I, we present results on several test problems from DETEST. Each test has been done 104 times, and the CPU time is the average time. From these results, we see that the TIDES software shows a good behavior on all problems, being more efficient at high precision, as expected. Another interesting fact about the TSM is that the variable-stepsize, variable-order formulation (for tolerance 10−10 the order is 16 and for tolerance 10−15 the order is 22) maintains an almost constant the number of steps in the integration process, for different tolerance requirements. We remark that for the only case where the error is large, the case A1, we have also tested TIDES using ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:16

A. Abad et al.

the combination of both the stepsize estimator and the defect error control (4), and this has permitted us to obtain a smaller relative error. This mixed strategy permits us to correct some pathological situations, such as entire function solutions (where the large stepsize can lead to unstable polynomial evaluations). 6.2. Test Problems

Previously we have briefly studied the behavior of TIDES on standard test problems. Now we focus our attention on the extended options of TIDES, which are useful for problems of the type studied here. The problems used in the comparisons are as follows. AR. Arenstorf orbits [Arenstorf 1963] are a particular case of the planar circular restricted three-body problem. One considers two bodies of masses 1 − μ and μ in circular rotation in a plane and a third body of negligible mass moving around in the same plane. The interval of integration is t ∈ [0, 30], and the equations are x+μ x − μ −μ , D1 D2 y y y = y − 2x − μ −μ , D1 D2 ⎧ D 1 = ((x + μ)2 + y2 )3/2 , D 2 = ((x − μ )2 + y2 )3/2 , ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ x(0) = 0.994, y(0) = 0, x = x + 2y − μ

⎪ ⎪ x (0) = 0, y (0) = −2.00158510637908252240537862224, ⎪ ⎪ ⎪ ⎩ μ = 0.012277471, μ = 1 − μ. GA. This is a Galactic dynamics model [Binney and Tremaine 1987]. This problem is a Hamiltonian problem with coordinates q1 , q2 , q3 and moments p1 , p2 , p3 . The interval of integration is t ∈ [0, 1000]. The Hamiltonian function and parameter values (the initial conditions have been fixed to obtain H = 2) are 2 2 2 q q q 1 2 p1 + p22 + p23 + ( p1 q2 − p2 q1 ) + A ln C + 21 + 22 + 23 , H= 2 a b c ⎧ ⎪ a = 1.25, b = 1, c = 0.75 A = 1, C = 1, = 0.25, ⎪ ⎪ ⎪ ⎨ q1 (0) = 2.5, q2 (0) = q3 (0) = 0, ⎪ ⎪ √ ⎪ 1 ⎪ ⎩ p1 (0) = 0, p2 (0) = 25 + 6961 − 3200 ln 5 , p3 (0) = 0.2. 40 HH. This is the well-known Hénon-Heiles problem [Hénon and Heiles 1964] given by the Hamiltonian 1 2 1 1 (X + Y 2 ) + (x2 + y2 ) + ε y(x2 − y2 ), 2 2 3 in the interval of integration t ∈ [0, 70] with the initial values H=

x(0) = 0,

y(0) = 0.52,

X (0) = 0.371956090598519,

Y (0) = 0,


(8)

ε = 1.


5:17

Fig. 3. (Color online) CPU time in seconds vs. relative error in quadruple precision using the minf-tides, dop853, and odex codes.

LO. The classical Lorenz model [Lorenz 1963] is given by Eq. (7). We take the classical Salztman values r = 28, σ = 10 and b = 8/3 of the parameters, and we use the initial conditions of an unstable periodic orbit of period T (shown up to 100 digits thanks to Prof. Divakar Viswanath [Viswanath 2004]). x(0) = −13.76361068213420052501440105436165386410086485409236 84535378642921202827747268115852940239346395038284 y(0) = −19.57875194245179553883804144600955886611424005342764 38649791334295426354746147526415973165506704676171 z(0) = 27 T = 1.55865221071617472756787020921269607052848054899724 39358895215783190198756258880854355851082660142374 KE. The well-known Kepler or two-body problem is given by the Hamiltonian H=

1 2 1 (X + Y 2 ) − , 2 r

(9)

! with r = x2 + y2 , in the interval of integration ! t ∈ [0, 10 · 2π ] with the initial values x(0) = 1 − e, y(0) = 0, X (0) = 0, Y (0) = (1 + e)/(1 − e). ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:18

A. Abad et al.

Fig. 4. (Color online) CPU time in seconds vs. relative error in quadruple precision using the minf-tides, dop853, and odex codes.

6.3. minf-tides vs dop853 and odex in Double and Quadruple Precision

The first test is to compare the minf-tides version with dop853 and odex, all in Fortran. In Figure 3, we show the comparisons in double precision using the g77 compiler, where we plot CPU time (in seconds) versus relative error. In all the programs, we have used the same optimization option -O2 and allowing dense output. For all the problems, we see similar behavior: for low-precision requests, the RK code dop853 is the fastest option. When the precision level is around 10−10 , the TSM becomes the fastest option. The reason is the variable-order formulation of the TSM, obtaining a different slope in the curves (with the dop853 and odex the same happens but we need more precision in order for the extrapolation method to beat the RK code). Finally, in Figure 4, we show comparisons in quadruple precision using the Lahey LF95 compiler. In these plots, we present CPU time in seconds vs. relative error. Now, the TSM is the fastest option. 6.4. minf-tides vs. minc-tides vs. dp-tides

Now we compare the different versions of the TIDES programs. That is, we want to see if there are significant differences between minf-tides, minc-tides and dp-tides. First, we have to say that the extended TSM program dp-tides should be slower than the other options due to the use of more complex data structures and algorithms. The minimal versions minf-tides and minc-tides are designed to be fast, but dp-tides is designed to be easy to use for the solution of a differential system and sensitivities, and the solution of variational equations up to any order. These differences in the complexity of the algorithms appear in the tests shown in Figure 5, where clearly, if we just look at the computer time in the integration of the differential equations, we should use the ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:19

Fig. 5. (Color online) CPU time in seconds vs. relative error in double precision using the minf-tides, minc-tides, and dp-tides codes.

Fig. 6. (Color online) CPU time in seconds vs. relative error in multiple precision using the mp-tides with variable-order, variable-stepsize, and variable-precision arithmetic; and using variable-order, variablestepsize, and fixed-precision arithmetic (505 digits).

minimal versions. Also, we observe that both minimal versions in C or Fortran present the same behavior. 6.5. Arbitrary Precision: mp-tides

There is a growing number of theoretical and practical applications that need a highprecision solution [Bailey et al. 2011]. For instance, D. Viswanath obtained periodic orbits in the classical Lorenz model up to more than 500 digits for several analytic studies [Viswanath 2004]. To reach this goal, he used clever algorithms to avoid numerical ODE integration methods, because a standard method cannot integrate up to such a high-precision level. One of the greatest advantages of the TSM is that it can be used to solve ODEs with very high precision. This is also one of the objectives of TIDES, and in particular of the mp-tides version. ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:20

A. Abad et al.

Fig. 7. (Color online) Top: number of exact digits depending on the number of time periods in the numerical integration of an unstable periodic orbit for the Lorenz model using mp-tides with 200 digits. Bottom: 300 time periods using the mp-tides and 50 time periods using a standard double precision numerical integrator.

From Section 4, the computational complexity of the numerical solution of an ODE system using a TSM with D = − log10 (TOL) number of digits is O(D 4 ) using variableorder (depending on TOL), variable-stepsize, and variable-precision arithmetic (D + 5 digits) and O(D 2 ) using variable-order, variable-stepsize, and fixed-precision arithmetic (but always the highest one in all the simulation process, 505 digits). In Figure 6, we show the CPU time versus the number of digits for four test problems up to 500 digits of precision using variable-order, variable-stepsize, and variable-precision arithmetic; and using variable-order, variable-stepsize, and fixed-precision arithmetic (505 digits). We observe that the behavior is quite similar for all the problems: reliably computing solutions of ODE problems up to high precision. Also, we see that the most adaptive approach gives the lowest computational cost. We cannot compare the mptides code with any standard method (except using a symbolic system such as Maple), because none of them is able to give solutions up to such degree of accuracy. Therefore, for some practical applications, this version of TIDES can be of a great help. As an example, we show in Figure 7 how the use of high precision can give good simulations of unstable periodic orbits foliated to chaotic attractors, as occurring in the Lorenz model for the unstable periodic orbit with the symbolic code LR [Viswanath 2004], given by the initial conditions for the problem LO. Figure 7 (top) shows the number of exact digits that we have depending on the number of time periods in the numerical integration of the periodic orbit using mp-tides with 200 digits. At the bottom, we show 300 time periods using the mp-tides (in blue color) and 50 time periods using a standard double precision numerical integrator (in red color). Obviously, the standard integrator falls into the chaotic attractor, but the accurate numerical integration continues on the periodic orbit. 6.6. Partial Derivatives: dp-tides and mp-tides

TIDES can integrate the solution of a variational equation up to any order automatically, without writing explicitly the equations of the variational equations. Moreover, ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:21

Fig. 8. (Color online) Top: CPU time in seconds vs. relative error in the computation of sensitivities diagrams for the Lorenz model in double precision using the dp-tides code with extended TSM (ETS) or the standard TSM with explicit formulation of the variational equations (VAR). Bottom: time evolution of the different sensitivities.

this is not limited to studying the dependency with respect to the initial conditions, but also the sensitivities with respect to any parameter of the system. Besides, the way of programming the partial derivatives via the extended TSM using the formulas of Section 3 (instead of, for instance, the approach of Berz [1991]) permits computing the minimum number of partial derivatives, if we are just interested in one particular partial derivative and not all up to its differentiation order. In Figure 8, we show some comparisons for the Lorenz model (LO) in double precision of the code dp-tides using the traditional way to compute the solution of the variational equations, that is writing them explicitly, and with the use of the extended TSM and using TIDES. In these plots, we present CPU time in seconds versus relative error. The extended TSM is the fastest option, but the difference is insignificant in CPU time. The most important is the different formulation of the ODE system. It is cumbersome to write variational equations of order one, two, and higher! These plots are produced for two cases: computing the complete order two and only the partial derivative ∂ 2 x/∂ x20 , and computing ∂ x/∂σ or ∂ 2 x/∂σ ∂b . We also compute sensitivities with respect to parameters σ and b , not only with respect to the initial conditions. In Figure 9, we show comparisons for the Kepler problem (KE) in multiple precision of the code mp-tides using the traditional way to compute the solution of the variational equations, and the extended TSM using TIDES. Again, the extended TSM is the fastest option, and again the difference is insignificant. The plots are done for two values of the eccentricity, e = 0.7 and e = 0.99, and computing the complete order two and only the partial derivative ∂ 2 x/∂ x20 . The computations, up to a precision level of 100 digits, ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:22

A. Abad et al.

Fig. 9. (Color online) CPU time in seconds vs. relative error in the computation of sensitivities diagrams for the Kepler problem with two eccentricities in multiple precision with the mp-tides code using the extended TSM (ETS) or just the standard TSM with explicit formulation of the variational equations (VAR). Bottom: pictures time evolution of different sensitivities.



5:23

Fig. 10. (Color online) CPU time in seconds vs. maximum order in the computation of the partial derivatives of the solution of the Kepler problem with the TIDES code using the extended TSM (ETS) version for the partial derivatives.

show that this precision is easily obtained with TIDES. In the bottom plots, we show the time evolution of the solutions. Finally, in Figure 10, we show the evolution of the CPU time in seconds versus maximum order in the partial derivatives for the Kepler problem using the TIDES code with the extended TSM (ETS) version for the partial derivatives. In this plot, we present the CPU time in seconds for computing the derivatives with respect to just one variable (but from order 1 to 9), two, three, or four variables of the problem. Obviously, the time is smaller when computing sensitivities with respect to one variable, and the increment in time follows the complexity results of Section 4. Note that, when we introduce more variables, there are many more intermediate mixed partial derivatives, and so the computer time also grows.

6.7. Preservation of Invariants: Brouwer’s Law

It is quite standard to preserve several geometric properties of a differential system by means of geometric integrators [Hairer et al. 2006]. These kind of methods are especially useful when we want to solve a problem with not very high precision, but with a “constant” value of energy. The problem for very long numerical integrations is that it does not matter how we perform an integration: finally rounding errors affect the integration, giving an increment of the error in the geometric object [Hairer et al. 2008]. The optimal error in these quantities was studied first by Brouwer [1937], who established that the error in energy grows at least as O(t1/2 ). This error is obtained for long integrations of carefully used symplectic integrators [Hairer et al. 2008], or when one is able to suppress the truncation error in any numerical integrator. In other circumstances, we may observe a typical linear growth O(t). In the case of the positions, and for regular orbits, we will have a Root-Mean-Squared (RMS) error O(t3/2 ) in the best case, and a typical error O(t2 ) (as in any nonsymplectic RK code). Here, we show how to reduce the truncation error in the Taylor series method, and therefore, in TIDES. The advantage is that, using the error estimator (3), we may use any tolerance level. Hence, if we fix the tolerance far below the unit roundoff of the machine, the error in the truncation of the Taylor series will be, analytically, smaller than the unit roundoff. This means that the power series expansion will be exact up to machine precision. However, other sources of errors, such as in the generation of the coefficients in automatic differentiation, will affect the integration, but their effect will be small, and the error of both the solution and first integrals will be optimal, according to Brouwer’s law. ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.

5:24

A. Abad et al.

Fig. 11. (Color online) Top: Energy error, using the truncation-free formulation, for the Hénon-Heiles and 200 orbits with random perturbations of the initial conditions, evolution of the standard deviation of 1000 orbits, and the mean of 1000 orbits, as a function of time. Bottom: evolution, in logarithmic scale, of the standard deviation compared with a growth of order O(t1/2 ).

This technique has been used previously by the group of Carles Simó [2001] and others [Grazier et al. 2004/05a, 2004/05b; Sharp 2006]. We have to combine this technique with a “compensated sum” formulation [Higham 2002; Kahan 1965] of the time increment, as we use variable-stepsize strategies (in contrast with symplectic integrators that use fixed-stepsize implementations). Hence, the truncation-free formulation can be described as. (use TOL u, with u = the unit roundoff) + (“compensated sum”). TIDES uses compensated sum in some stages of the method. To preserve some geometric properties of the systems, we fix low sufficiently tolerance level. Obviously this approach is more computationally expensive than other approaches, and it is suitable only if we also look for high-precision numerical results. In Figure 11 we show the error in energy using the truncation-free formulation for the Hénon-Heiles problem for 200 orbits with random perturbations of the initial ACM Transactions on Mathematical Software, Vol. 39, No. 1, Article 5, Publication date: November 2012.


5:25

conditions (all of them with same value of the Hamiltonian) for the interval [0, 104 ], evolution of the standard deviation of 1000 orbits, and the mean as a function of time. The figure shows the random walk nature of the error, and that the mean is zero to within sampling error. The evolution of the standard deviation is of order O(t1/2 ) (shown in the interval [0, 105 ]). Therefore, the test shows that the TSM using the truncated-free formulation achieves the Brouwer’s law, like well-programmed symplectic integrators [Hairer et al. 2008], but it can be used in variable-stepsize formulations. 7. CONCLUSIONS

In this article, we have described our new software package called TIDES. This package is a powerful new kind of numerical ODE code providing easy-to-use interfaces for standard double precision integrations, but also quadruple precision and multiple precision, when needed and when other standard numerical ODE methods and software are not able to provide an accurate solution. The package combines a preprocessor in M ATHEMATICA that generates Fortran or C programs with a library in C. We have presented several numerical tests comparing with other standard software in double and quadruple precision, showing that our software is competitive in double precision, but faster in quadruple precision. In multiple precision, we have shown that TIDES can obtain solutions up to any accuracy demand (we have made several tests up to 500 precision digits) in a reasonable computing time. This multiple precision option is obtained with the combined use of TIDES and the GNU multiple precision MPFR library. Moreover, the numerical tests confirm the polynomial complexity of the TSM (quartic in the number of precision digits). Another new capability of TIDES is the direct computation of sensitivities of the solution of an ODE system. This is an extension of the traditional Taylor series method that permits solving the variational equations up to any order without formulating them, which is useful in practical applications due to the tedious work of obtaining variational equations of order greater than one. We have presented numerical tests for this new option. Finally, we have shown that with the Taylor series method and with TIDES, we can integrate ODEs following Brouwer’s law, which means that the behavior is optimal looking to the asymptotic growing of the error in position and in the integrals of the system, although the method is not symplectic. ACKNOWLEDGMENTS We thank Professor Divakar Viswanath for providing us with initial conditions of a periodic orbit for the Lorenz model up to 500 digits. Also, we thank all the free software and GNU community, and in particular the authors of the MPFR and GMP libraries for providing freely to the scientific community their marvelous software. The authors are grateful to the referees for their numerous suggestions, remarks, and corrections that have contributed to improve the article.

REFERENCES A BAD, A., B ARRIO, R., B LESA , F., AND R ODR Í GUEZ , M. 2011. TIDES web page: http://gme.unizar.es/software/tides. A RENSTORF, R. 1963. Periodic solutions of the restricted three body problems representing analytic continuations of Keplerian elliptic motions. Amer. J. Math. LXXXV, 27–35. A SCHER , U. M. AND P ETZOLD, L. R. 1998. Computer Methods for Ordinary Differential Equations and Differential-Algebraic Equations. (SIAM), Philadelphia, PA. B AILEY, D. 1995. Fortran-90 based multiprecision system. ACM Trans. Math. Softw. 21, 379–387. B AILEY, D., B ARRIO, R., AND B ORWEIN, J. 2011. A high-precision computation: Mathematical physics and dynamics. Appl. Math. Comput. 28, 20, 10106–10121.


5:26

A. Abad et al.

B ARRIO, R. 2005. Performance of the Taylor series method for ODEs/DAEs. Appl. Math. Comput. 163, 2, 525–545. B ARRIO, R. 2006a. Painting chaos: A gallery of sensitivity plots of classical problems. Int. J. Bifur. Chaos Appl. Sci. Engrg. 16, 10, 2777–2798. B ARRIO, R. 2006b. Sensitivity analysis of ODEs/DAEs using the Taylor series method. SIAM J. Sci. Comput. 27, 6, 1929–1947. B ARRIO, R. AND B LESA , F. 2009. Systematic search of symmetric periodic orbits in 2DOF Hamiltonian systems. Chaos Solitons Fractals 41, 2, 560–582. B ARRIO, R., B LESA , F., AND L ARA , M. 2005. VSVO formulation of the Taylor method for the numerical solution of ODEs. Comput. Math. Appl. 50, 1–2, 93–111. B ARRIO, R., B LESA , F., AND S ERRANO, S. 2009a. Bifurcations and safe regions in open Hamiltonians. New J. Phys. 11, 053004. B ARRIO, R., B LESA , F., AND S ERRANO, S. 2009b. Periodic, escape and chaotic orbits in the Copenhagen and the (n + 1)-body ring problems. Comm. Nonlinear Sci. Numer. Simul. 14, 5, 2229–2238. B ARRIO, R. AND S ERRANO, S. 2007. A three-parametric study of the Lorenz model. Phys. D 229, 1, 43–51. B ARTON, D. 1980. On Taylor series and stiff equations. ACM Trans. Math. Softw. 6, 3, 280–294. B AUER , F.L. 1974. Computational graphs and rounding error. SIAM J. Numer. Anal. 11, 1, 87–96. B ARTON, D., W ILLERS, I. M., AND Z AHAR , R. V. M. 1971. The automatic solution of ordinary differential equations by the method of Taylor series. Comput. J. 14, 243–248. B ERZ , M. 1991. Algorithms for higher derivatives in many variables with applications to beam physics. In Automatic Differentiation of Algorithms. SIAM, Philadelphia, PA, 147–156. B INNEY, J. AND T REMAINE , S. 1987. Galactic Dynamics. Princeton University Press. B ROUWER , D. 1937. On the accumulation of errors in numerical integration. Astron. J. 30, 149–153. C AUCHY, A. 1839-42. Several articles in Comptes Rendus de l’Acad. des Sciences de Paris. C HANG, Y. F. 1974. Automatic solution of differential equations. In Constructive and Computational Methods for Differential and Integral Equations, Lecture Notes in Mathematics, vol 430, 61–94. C HANG, Y. F. AND C ORLISS, G. 1994. ATOMFT: Solving ODEs and DAEs using Taylor series. Comput. Math. Appl. 28, 10–12, 209–233. C HANG, Y. F. AND C ORLISS, G. F. 1980. Ratio-Like and recurrence relation tests for convergence of series. J. Inst. Math. Appl. 25, 349–359. C ORLESS, R. M. 2002. A new view of the computational complexity of IVP for ODE. Numer. Algor. 31, 1–4, 115–124. C ORLESS, R. M. AND I LIE , S. 2008. Polynomial cost for solving IVP for high-index DAE. BIT 48, 1, 29–49. C ORLISS, G. AND C HANG, Y. F. 1982. Solving ordinary differential equations using Taylor series. ACM Trans. Math. Softw. 8, 2, 114–144. C ORLISS, G. AND L OWERY, D. 1977a. Choosing a stepsize for Taylor series methods for solving ODE’s. J. Comput. Appl. Math. 3, 4, 251–256. C ORLISS, G. F., G RIEWANK , A., H ENNEBERGER , P., K IRLINGER , G., P OTRA , F. A., AND S TETTER , H. J. 1997. High-Order stiff ODE solvers via automatic differentiation and rational prediction. Lecture Notes in Computer Science, vol. 1196, 114–125. C ORLISS, G. F. AND L OWERY, D. 1977b. Choosing a stepsize for Taylor series methods for solving ODE’s. J. Comp. Appl. Math. 4, 251–256. D EPRIT, A. AND Z AHAR , R. 1966. Numerical integration of an orbit and its concomitant variations. Z. Angew. Math. Phys. 17, 425–430. E NRIGHT, W. AND P RYCE , J. 1987. Two Fortran packages for assesing initial value problems. ACM Trans. Math. Softw. 13, 1–27. E NRIGHT, W. H. 1989. A new error-control for initial value solvers. Appl. Math. Comput. 31, 288–301. E NRIGHT, W. H. AND H AYES, W. B. 2007. Robust and reliable defect control for Runge Kutta methods. ACM Trans. Math. Softw. 33, 1, 1:1–1:19. E ULER , L. 1768. Institutionum Calculi Integralis. Vol. I, St. Petersburg. In Opera Omnia, Ser. I, Vol. XI. ´ ` , V., P ELISSIER , P., AND Z IMMERMANN, P. 2007. MPFR: A multipleF OUSSE , L., H ANROT, G., L EF EVRE precision binary floating-point library with correct rounding. ACM Trans. Math. Softw. 33, 2, 13:1–13:15. G ALIAS, Z. 2001. Interval methods for rigorous investigations of periodic orbits. Int. J. Bifur. Chaos Appl. Sci. Engrg. 11, 9, 2427–2450. G RAZIER , K. R., N EWMAN, W. I., H YMAN, J. M., AND S HARP, P. W. 2004/05a. Long simulations of the solar system: Brouwer’s law and chaos. ANZIAM J. 46, (C), C1086–C1103.



5:27

G RAZIER , K. R., N EWMAN, W. I., H YMAN, J. M., S HARP, P. W., AND G OLDSTEIN, D. J. 2004/05b. Achieving Brouwer’s law with high-order Störmer multistep methods. ANZIAM J. 46, C, C786–C804. G RIEWANK , A. 1996. ODE solving via automatic differentiation and rational prediction. In Numerical Analysis 1995. Pitman Research Notes Mathematics Series, vol. 344, Longman, Harlow, 36–56. G RIEWANK , A. 2000. Evaluating Derivatives. Frontiers in Applied Mathematics, vol. 19, SIAM, Philadelphia, PA. G RIEWANK , A., U TKE , J., AND WALTHER , A. 2000. Evaluating higher derivative tensors by forward propagation of univariate Taylor series. Math. Comput. 69, 231, 1117–1130. H AIRER , E., L UBICH , C., AND WANNER , G. 2006. Geometric Numerical Integration, 2nd Ed. Springer Series in Computational Mathematics, vol. 31, Springer. H AIRER , E., M C L ACHLAN, R. I., AND R AZAKARIVONY, A. 2008. Achieving Brouwer’s law with implicit Runge-Kutta methods. BIT 48, 2, 231–243. H AIRER , E., N ØRSETT, S. P., AND WANNER , G. 1993. Solving Ordinary Differential Equations. I, 2nd Ed. Springer Series in Computational Mathematics, vol. 8, Springer. ´ , M. AND H EILES, C. 1964. The applicability of the third integral of motion: Some numerical experH ENON iments. Astronom. J. 69, 73–79. H IGHAM , N. J. 2002. Accuracy and Stability of Numerical Algorithms, 2nd Ed. SIAM, Philadelphia, PA. H INDMARSH , A. C., B ROWN, P. N., G RANT, K. E., L EE , S. L., S ERBAN, R., S HUMAKER , D. E., AND W OODWARD, C. S. 2005. Sundials: Suite of nonlinear and differential/algebraic equation solvers. ACM Trans. Math. Softw. 31, 3, 363–396. J ELTSCH , R. AND N EVANLINNA , O. 1982. Stability and accuracy of time discretizations for initial value problems. Numer. Math. 40, 245–296. ` AND Z OU, M. 2005. A software package for the numerical integration of ODEs by means of J ORBA , A. high-order Taylor methods. Experiment. Math. 14, 1, 99–117. K AHAN, W. 1965. Further remarks on reducing truncation errors. Comm. ACM 8, 1, 40. L ARA , M., E LIPE , A., AND PALACIOS, M. 1999. Automatic programming of recurrent power series. Math. Comput. Simul. 49, 4–5, 351–362. L ORENZ , E. 1963. Deterministic nonperiodic flow. J. Atmosph. Sci. 20, 130–141. M AKINO, K. AND B ERZ , M. 2006. Cosy infinity version 9. Nucl. Instrum. Meth. Phys. Res. 558, 1, 346–350. M OORE , R. E. 1966. Interval Analysis. Prentice-Hall. M OORE , R. E., K EARFOTT, R. B., AND C LOUD, M. J. 2009. Introduction to Interval Analysis. (SIAM), Philadelphia, PA. N EDIALKOV, N. S., J ACKSON, K. R., AND C ORLISS, G. F. 1999. Validated solutions of initial value problems for ordinary differential equations. Appl. Math. Comput. 105, 1, 21–68. N EDIALKOV, N. S. AND P RYCE , J. D. 2005. Solving differential-algebraic equations by Taylor series. I. Computing Taylor coefficients. BIT 45, 3, 561–591. N EDIALKOV, N. S. AND P RYCE , J. D. 2007. Solving differential-algebraic equations by Taylor series. II. Computing the system Jacobian. BIT 47, 1, 121–135. N EDIALKOV, N. S. AND P RYCE , J. D. 2008. Solving differential algebraic equations by Taylor series. III. The DAETS code. J. Numer. Anal. Ind. Appl. Math. 3, 1–2, 61–80. N EIDINGER , R. D. 1992. An efficient method for the numerical evaluation of partial derivatives of arbitrary order. ACM Trans. Math. Softw. 18, 2, 159–173. N EIDINGER , R. D. 2005. Directions for computing truncated multivariate Taylor series. Math. Comput. 74, 249, 321–340. N EWTON, I. 1671. Methodus Fluxionum et Serierum Infinitarum. edita Londini 1736, Opuscula mathematica, Vol.I, traduit en francais par M. de Buffon, Pasis MDCCXL. P RYCE , J. D. 1998. Solving high-index DAEs by Taylor series. Numer. Algor. 19, 1–4, 195–211. P RYCE , J. D. 2001. A simple structural analysis method for DAEs. BIT 41, 2, 364–394. P RYCE , J., N EDIALKOV, N. AND E NRIGHT, W. 2009. On the Ilie-Corless polynomial complexity proof. Preprint. R ALL , L. B. 1981. Automatic Differentiation-Techniques and Applications. Lecture Notes in Computer Science, vol. 120, Springer. R ALL , L. B. AND C ORLISS, G. F. 1996. An introduction to automatic Differentiation. In Computational Differentiation. SIAM, Philadelphia, PA, 1–18. R EVOL , N., M AKINO, K., AND B ERZ , M. 2005. Taylor models and floating-point arithmetic: Proof that arithmetic operations are validated in COSY. J. Log. Algebr. Program. 64, 1, 135–154.


5:28

A. Abad et al.

R OY, A., M ORAN, P., AND B LACK , W. 1972. Studies in the application of recurrence relations to special perturbation methods I. Celest. Mech. 6, 468–482. S HAMPINE , L. AND B ACA , L. 1986. Fixed versus variable order Runge-Kutta. ACM Trans. Math. Softw. 12, 1–23. S HARP, P. W. 2006. N-body simulations: The performance of some integrators. ACM Trans. Math. Software 32, 3, 375–395. S IM O´ , C. 2001. Global dynamics and fast indicators. In Global Analysis of Dynamical Systems. 373–389. T SUKANOV, I. AND H ALL , M. 2003. Data structure and algorithms for fast automatic differentiation. Int. J. Numer. Meth. Eng. 56, 13, 1949–1972. T UCKER , W. 2002. A rigorous ODE solver and Smale’s 14th problem. Found. Comput. Math. 2, 1, 53–117. V ISWANATH , D. 2004. The fractal property of the Lorenz attractor. Phys. D 190, 1-2, 115–128. W OLFRAM , S. 1999. The Mathematica Book, 4th Ed. Wolfram Media, Inc., Champaign, IL. Received February 2010; revised November 2011; accepted November 2011
