ADIFOR: Generating Derivative Codes from ... - Semantic Scholar

ADIFOR: Generating Derivative Codes from Fortran Programs Christian Bischof Alan Carle George Corliss Andreas Griewank Paul Hovland

CRPC-TR91185-S December 1991

Center for Research on Parallel Computation Rice University 6100 South Main Street CRPC - MS 41 Houston, TX 77005

Revised: August, 1995. Appears in Scienti c Program-

ming, 1(1), 1992.

ADIFOR | Generating Derivative Codes from Fortran Programs Christian Bischofy Alan Carlez George Corlissy Andreas Griewanky Paul Hovlandy Argonne Preprint MCS{P263{0991 CRPC Technical Report CRPC{TR91185 Published in Scienti c Programming, 1(1), pp. 1{29, 1992.

Abstract. The numerical methods employed in the solution of many scienti c computing problems require the computation of derivatives of a function f : Rn ! Rm . Both the accuracy and the computationalrequirements of the derivativecomputation are usually of critical importance for the robustness and speed of the numerical solution. ADIFOR (Automatic Dierentiation In FORtran) is a source transformation tool that accepts Fortran 77 code for the computation of a function and writes portable Fortran 77 code for the computation of the derivatives. In contrast to previous approaches, ADIFOR views automatic dierentiation as a source transformation problem. ADIFOR employs the data analysis capabilities of the ParaScope Parallel ProgrammingEnvironment, which enable us to handle arbitraryFortran 77 codes and to exploit the computationalcontext in the computation of derivatives. Experimental results show that ADIFOR can handle real-life codes and that ADIFOR-generated codes are competitive with divided-dierence approximations of derivatives. In addition, studies suggest that the sourcetransformation approach to automatic dierentation may improve the time to compute derivatives by orders of magnitude. Key words. Large-scale problems, derivative, gradient, Jacobian, automatic dierentiation, optimization, sti ordinary dierential equations, chain rule, parallel, ParaScope Parallel Programming Environment, source transformation and optimization.

1 Introduction The methods employed for the solution of many scienti c computing problems require the evaluation of derivatives of some function. Probably best known are gradient methods for optimization [22], Newton's method for the solution of nonlinear systems [15, 22], and the numerical solution of sti ordinary dierential equations [8, 21]. Other examples can be found in [17]. In the context of optimization, for example, given a function f : R ! R; one can nd a minimizer x of f using variable metric methods that involve the iteration for i = 1, 2, .... do Solve B s = ?rf (x ) x +1 = x +  s n

i

end for i

for suitable step multipliers  > 0. Here i

i

i

i

i

i

0 rf (x) = B @

@ @x1

@ @x

f (x)

.. .

n f (x)

1 CA

(1)

 This work was supported by the Applied Mathematical Sciences subprogram of the Oce of Energy Research, U.S. Department of Energy, under Contract W{31{109{Eng{38, through NSF Cooperative Agreement No. CCR{8809615, and by the W. M. Keck Foundation. y Mathematics and Computer Science Division, Argonne National Laboratory, 9700 S. Cass Avenue, Argonne, IL 60439. z Center for Research on Parallel Computation, Rice University, P. O. Box 1892, Houston, TX 77251.

1

is the gradient of f at a particular point x0 , and B is a positive de nite matrix that may change from iteration to iteration. In the context of nding the root of a nonlinear function i

0f 1 1 B . . f =@ . C A;

f :R !R ; n

n

f

n

Newton's method requires the computation of the Jacobian matrix 0 f1(x)


1 n f1 (x) B . .. 0 .. f (x) = @ . f ( x )


f 1 n (x) @

@

@x

@x

@

@

@x

n

@x

1 CA :

(2)

n

Then, we execute the following iteration: for i = 1, 2, .... do Solve f 0 (x )s = ?f (x ) x +1 = x + s i

end for i

i

i

i

i

Another important application is the numerical solution of initial value problems in sti ordinary dierential equations. Methods such as implicit Runge-Kutta [9] and backward dierentiation formula (BDF) [35] methods require a Jacobian which is either provided by the user or approximated by divided dierences. Consider a system of ODEs y0 = f (t; y); y(t0 ) = y0 : (3) System (3) is called sti if its Jacobian J = (in a neighborhood of the solution) has eigenvalues  with Re( )