An Efficient Solution Process for Implicit Runge-Kutta Methods

An Efficient Solution Process for Implicit Runge-Kutta Methods Author(s): Theodore A. Bickart Source: SIAM Journal on Numerical Analysis, Vol. 14, No. 6 (Dec., 1977), pp. 1022-1027 Published by: Society for Industrial and Applied Mathematics Stable URL: http://www.jstor.org/stable/2156679 Accessed: 03/08/2010 16:03 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=siam. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Society for Industrial and Applied Mathematics is collaborating with JSTOR to digitize, preserve and extend access to SIAM Journal on Numerical Analysis.

http://www.jstor.org

SIAM J. NUMER. ANAL. Vol. 14, No. 6, December 1977

AN EFFICIENT SOLUTION PROCESS FOR IMPLICIT RUNGE-KUTTA METHODS* THEODORE

A. BICKARTt

solutionprocedurefora modifiedNewtonmethod Abstract.Described in thispaper is an efficient orderordinarydifferential solutionofa v-stageimplicitRunge-Kuttamethodappliedto a setofn first foreach time-stepplus order equations.The procedurerequiresonlyordervn3 scalarmultiplications withinthetime-step.Also, itonlyrequiresspace to foreach iteration-step '2n2scalarmultiplications storeorder3n3 variables.

can be achieved Highorder,A -stablenumericalintegration 1. Introduction. with implicitRunge-Kutta methods [7, pp. 243-244]. However, there is an impedimentto usingan implicitRunge-Kuttamethod.It is thata v-stagemethod equationsrequiresat used to solve a systemof n firstorderordinarydifferential each time-stepthesolutionof vn nonlinearalgebraicequations.Ifthesealgebraic equations are solved by the Newtonmethod,or one of the variationson it,it is withineach necessaryto solve vn linearalgebraicequationsat each iteration-step time-step.Unless thesystemof equations is sparse and sparsematrixtechniques We are invoked,each iteration-step requiresorder v 3n3scalar multiplications. shall describe,fora particularmodifiedNewtorimethod,a solutionprocedure foreach time-stepplus order whichrequiresonlyordervn3 scalarmultiplications withinthetime-step.Note: The foreach iteration-step iA2n2scalarmultiplications procedure to be described is similarto that which has been employed with compositemultistepmethods,as alluded to in [1] and describedin [8]. 2. Solution process. Consider the n-vector-valuedfirstorder differential equation

y=f(y,t)

(1) and the initialcondition

y(to)= yo.

(2)

Let Ymdenote the approximationof y(tm),where tm= to+ mh, derived by the followingRunge-Kuttamethod: Ym= Ym-1+ h

(3)

i=l

yiki,

where (4)

ki=f(ym-i+h h

E

j=1

i3ki,tm-i+6ih)

(i=1,.

.

.

* Received by the editorsFebruary26, 1976, and in finalrevisedformNovember19, 1976. t Departmentof Electrical and Computer Engineering,Syracuse University,Syracuse,New York. On leave fromDepartmentof ElectricalEngineeringand ComputerScience, Universityof Californiaat Berkeley,Berkeley,California94720.

1022

IMPLICIT

RUNGE-KUTTA

1023

METHODS

Now,set Pi =Ym-l+ h

(S)

E

j=1

iiki

(i = 1, . *

or,equivalently, Pi Ym-, + h

(6)

Z iifi

j=1

(i = 1,..

where fj= f(pj,tmi + jh).

(7)

Note: pi maybe viewedas an approximationof y(t) fort = tm-1+ 8ih E [tm1, tm]

then(6) is equivalentto Supposethematrix [ilij]is nonsingular;' (8)

hfi=

v E j=1

aipi-

v J=

aqjym-l a., 1y),

(i = 1,

whereaij isthe(i,j)thelement of[,/ij]-l. Thesetofequations(8) canbe expressed inmatrix formas (A i) Un)P-hD-(a

(9)

Un)ym = 0,

where? denotestheKronecker ofordern,A = [ai7], U, isa unitmatrix product,

.=L1P

a=LJ

and D

The (1+1)st NewtoniterateforP, namelyP 1, can be obtainedfrom

- p) of the solution(p"+i _ (10)

{(A 0 Un)-hJ1}(Pi'

-P') =-{(A 0 UO*P -hDf

(a(a Un)ym-},

whereJ' is theblockdiagonalmatrix (1 1)

Jl=diag ff,(pl, tm-, +8lh), **fy(pl, tm-,+,6vh)l.

Note:Underthecorrespondence in(5),theNewtoniteration expressed appliedto (9) forthepi-elementsofP-is equivalent to a Newtoniteration appliedto (4) fortheki. Letthemodified Newtonmethodbe thatobtainedbyreplacing J'bya block JofJacobians offevaluatednotatthevariousintermediate diagonalmatrix points in the time-interval 1 but at one pointtA,where [tm-i, ti] at each iteration-step

mie[0,m-11. Thatis, (12)

J= diag {Ffr,.F*, F

= Uv Fm,

1 It is easilyshownusingButcher'sresults[2] thathisorder2v methods,whichwereshownto be A-stablebyEhle [5], satisfythisassumption.Thus,thesetofmethodsforwhichtheassumptionis valid is not onlynonvacuousbut containsmethodsof some significance.

1024

THEODORE

A. BICKART

whereU, is a unitmatrixof order v and Fm= fy(Ym tm).

(13)

Using (12) the modifiedNewtonmethodthatfollowsfrom(10) is (14) {(A 0 Un)-h (U ,0

Fm)}('+1

-

f")=

-{(A 0 Un)P

-

hD' - (a

Un)Ym-1}.

This is equivalentto the two-sidedmatrixequation (15) (hFA)(P'+' - P')- (Pl+1- P')A' whereP', and E', and D' are the n x v matrices P' = [p,

*,

pt],

El = P'A'

hD

=

El -ym-ia',

and 1

D' = [f,

withA' being,of course,the transposeof A. Let a (A) denote the characteristic polynomialof A'; thatis, let (16)

I akAvk.

a(A)=det{AU,-A'}=

k=O

Then, as Jamesonhas shown[6], the solutionof (15) can be expressedas Pl '-Pl={a(hFh)V'

(17)

{ k=O

akM E-k

where M=

0,

M1

E,

=

Mk = (hFm)Mk- +Mk-1A'-(hFm

)Mk-2A'.

Note thatifm - 1 > m,then{a(hF )}'- was evaluatedat a previoustime-step and need not be evaluated at the currenttime-step.On the other hand, if m - 1 = mii,then {a(hFw)} 1 must be evaluated; this entails 1 Jacobian matrix evaluationand vn3+ n2 scalar multiplications. The numberof scalar multiplications,2vectorfunction evaluations,and Jacobianmatrixevaluationsare displayed in Table 1. The numberofiteration-steps at themthtime-stepis theredenotedIm. TABLE 1

Scalarmultiplications Vector m-1

>m

Old method

function

matrix

(SolutionforP)

(Solutionfor0)

evaluations

evaluations

imv

0

imi

1

pt+v + lm{l,2n2+(v

=m 2

+ t'2)n}

2

A3

vn +n +zn+ +l

Jacobian

New method

{z,2n2+(z(3+z,2)n}

vn+v

+ lm{z2n2+ (,2 + 3 3

)n}

2

vzn +n +zn+z + (v2 +v)}

+ lm{22

Counts of multiplications includethe numberof divisions.

IMPLICIT

RUNGE-KUTTlA

1025

METHODS

Shownalso in Table 1, forpurposesof comparison,are thenumberofmultiplications needed when P' is evaluated by solving(14) using an explicitinverseof {(A 0 Un) - h(U, 0 Fm)}. (The numbersofvectorfunctionevaluationsand Jacobian matrixevaluationsare the same forbothmethods.) For large n and m - 1 = m-the worst case-the dominanttermsin the expressionfor the numberof scalar multiplicationsin a time-stepare vn3+ For purposesof comparison,the imp2n2, as was predicatedin the introduction. dominant terms when solving (14) directlyare p3fn3 + jmz2n2 . Note: If LU decompositionratherthanexplicitinversionis used,thedominanttermsbecome, respectively, (v - 2/3)n3 + imp2 2 and (1/3)v3 n 3 +1 mp2n2. Remark: The value of Im to achieve a solutionhas been observed to be essentiallythe same forboth methods-old and new. Another factor of some significancein evaluating the new method for the pi is the amountof data storagespace required.The numberof determining variablesforwhichspace mustexistwhen usingan explicitinverseis shownin Table 2 for the new and old methods. There, En and Eo denote small amounts of storage space needed duringintermediatecalculations.Observe, for large n, that the dominant terms are 3n2 and (2p2+ 1)n2 for the new and old methods, respectively.(If LU decomposition rather than explicit inversionis used, then these dominanttermsbecome, respectively,2n2 and (z2+ 1)n2.) Thus, the dominantnumberof variables as well as the dominant is smaller for the new method by a multiplicative numberof multiplications, factorof order 1/1,2. Upon convergenceof the iteration-lm iterationsteps-P is assigned the value of P'-. From the columnsof P, whichare thevectorspj(1 = 1,**. , v), the value of Ym can be evaluated using Ym=UrOYm-1+

(18)

Z 07jp1

j=l

an expressionderivedby combining(3) through(8), excluding(6), with v

o,,I

yai,i

i=1

,)

(j=l

and j=l

TABLE 2 Numberofstoredvariables Old method for0) (Solution

Newmethod forP) (Solution

,

(7

3n2

.1X , ,

A

2+,p

,

4A4

(2A2+ 1)n2 , + /2+ ,p

+ 1n

+,4A

1026

THEODORE

A. BICKART

3. Discussion. The improvementbased on operations and storage space countsis renderedsomewhatmorereal bythefollowing, albeitlimited,numerical data. This data is based on realizationof themethodsas APL functionsexecuted on an IBM System370 Model 155 computer.For v = 3 itwas foundthatn could be as largeas 29 forthenew method,versus12 fortheold method,in a fixedsize APL workspace.Note: For storagespace, the dominanttermswould indicatea need forsimilaramountsof space-2523 storedvariablesforthe new method, versus2736 forthe old method.For P = 3, n = 12, im= 1, and m - 1 = miit was foundthattheold methodrequiredabout 5 timesas muchcentralprocessortime as the new method,forotherthanvectorfunctionand Jacobianmatrixevaluations.Note: For operations,the dominanttermswould indicatea need for7.4 timesas manymultiplications by the old method. The new methodengendersa quite significant reductionin the numberof because the matrixto be invertedis of order n X n, not -'nX vn. multiplications Chipman has also proposed a method-alluded to in [3] and described in [4]-which onlyrequiresinversionof an ordern X n matrix;however,notonlyis his method different, it applies to a restrictedclass of implicitRunge-Kutta methods.Added in proof:See also [9], a new result,by Butcher. ImplicitRunge-Kuttamethods,as previouslyfootnoted,can manifestboth high order and A-stability-propertiesassociated with an efficientnumerical solutionof a stiffordinarydifferential equation. Since forsuch an equation the conditionnumberof hFA tendsto be ratherlarge, it can be expected thatthe conditonnumberof a(hFm) increases-significantly forsufficiently large h-as thedegree v of a (A) increases.On occasion a (hFm)maybe too ill-conditioned to invertusingfixedprecisionarithmetic.If thisshould be the case, the inverseof a (hFw) can be soughtin thefollowingmanner.Let a (A) be expressedas a product (of not necessarilyirreducible)real polynomials;thus, (19)

a(A)=

H ci(A),

i=l

where ut_ v. Then, (20)

{a(hF)}'-=

H {ci(hFm)}'.

i=1

The task is to select the factorsci(A) such that each ci(hFm) can be inverted. Ideally, ,t should also be as small as possible, since, evaluated in this way, fora totaloforder {a(hFw)}-1 requires(,u- 1)n3 additionalscalarmultiplications scalar multiplications (v +,t - 1)n3+ lmPf2n2 when m - 1 = m'. However,achievementof a least ,u need not be of paramountconcern,because, as ,u_zv,thetotal cannotexceed (2 v - 1)n3+ lmP,2n2, a value stillless-often significantly so-than the 3f3n3+ jmf2n2 of the old method.It is of some note thatthisvariationon the new method requires but littleadditional space for stored variables-just n2 additionalvariablesmustbe stored. The factoredformof{a (hFm)}- 1, whichcan ofcoursebe used even ifa (hFm) is not ill-conditioned,suggestsyet anothervariationon the new method.This variationwillachievetheconditionimprovement of thefactoredformwith(most

IMPLICIT

RUNGE-KUTTA

1027

METHODS

likely) fewer additional multiplicationsbut at a not necessarilyinsignificant increaseinrequiredstoragespace-(ut - 1)n2 additionalstoredvariables.Now,if, ratherthanfirstevaluating{a(hFf)}'1 as in (20) foruse in (17), the expression as (17) using(20) is rewritten (21)

P'+' -P' ={c(hFF)}_

{c, (hFm)} |

E

akM.-k}

at each iteration-step, and theseveralmatrixproductsare evaluatedright-to-left needed fora totalof are scalar multiplications thenjust lm(tt- 1)vn2additional m - 1 = m. As y-t- v when order Pnf +1m[P2 +(/t - 1)v]n2 scalar multiplications this total cannot exceed vn3+ 1m(2 ,2- v)n2, a numbernot much greaterthan when{a (hFm)}-1is evaluatedbytheexpression(17). IfLU decompositionrather than explicitinversionis used with(21), then the numberof scalar multiplica- l)v]n2. Though this is a tions will be of order [v-(2/3)/u]n 3 +1m[P2+( +1im[2v2- _]n2 (1/3)vn3 small-between rather quantity which could get _ 3 + _Im [(3/2)v2 or (2/3)vn v/2]n2 + ]n2, [(3/2)?2 and [(2/3)v (1/3)]n3 Im of factorization irreducible in an factors of quadratic number dependingupon the use. its mitigate might requirement space a (A)-the largerstorage A commenton the applicabilityof sparse matrixtechniquesin conjunction withthe new methodappears to be an appropriateway to close thisdiscussion. equations, typicallywhen n is verylarge, exhibita Many ordinarydifferential matrix, sparse Jacobian Fw. Because of the powers of that matrix,it can be less sparse. However, if Fm is verysparse-somewhat is that assumed a(hFw) if the degree v of a (A) is not too great,then large-and very is when n likely sparse to take advantage of sparse matrix sufficiently remain may a(hFw) forexample,ifFmwere a band matrixwitha the case, be That would techniques. also that applicabilityof sparse matrix Note bandwidth. small sufficiently be the determiningfactorin storage-might sparse particular, techniques-in than(17). rather decomposition, with LU possibly using(21), REFERENCES methodsfornumerical stablecompositemultistep [1] T. A. BICKART AND Z. PICEL, High orderstiffly equations,BIT, 13 (1973), pp. 272-286. ofstiff integration differential [2] J. C. BUTCHER, ImplicitRunge-Kuttaprocesses,Math. Comp., 18 (1964), pp. 50-64. ofRunge-Kuttaprocesses,BIT, 13 (1973), pp. 391-393. [3] F. H. CHIPMAN, The implementation , Numericalsolutionofinitialvalue problemsusingA -stableRunge-Kuttaprocesses,Univ. [4] of Waterloo Dept. of Applied Analysis and Computer Science Res. Rep. CSRR 2042, Waterloo,Ontario,Canada, June1971. [5] B. L. EHLE, High orderA -stablemethodsforthenumericalsolutionofsystemsofD.E.'s, BIT, 8 (1968), pp. 276-278. [6] A. JAMESON, Solutionof theequationAX + XB = C by inversionof an M x M or N x N matrix, SIAM J. Appl. Math., 16 (1968), pp. 1020-1023. Equations,JohnWiley,New [7] J. D. LAMBERT, ComputationalMethodsin OrdinaryDifferential York, 1973. ordinary of stiff fornumericalintegration [8] Z. PICEL, Block one-stepmethodswithfreeparameters equations,Ph.D. Dissertation,SyracuseUniv., Syracuse,New York, September differential 1975. of implicitRunge-Kutta methods,BIT, 16 (1976), pp. [9] J. C. BUTCHER, On theimplementation 237-240.