Control Strategies for the Iterative Solution of Nonlinear Equations in ...

CONTROL STRATEGIES FOR THE ITERATIVE SOLUTION OF NONLINEAR EQUATIONS IN ODE SOLVERS KJELL GUSTAFSSON AND GUSTAF SO DERLINDy

Abstract. In the numerical solution of ODEs by implicit time-stepping methods, a system of (nonlinear) equations has to be solved each step. It is common practice to use xed-point iterations or, in the sti case, some modi ed Newton iteration. The convergence rate of such methods depends on the stepsize. Similarly, a stepsize change may force a refactorization of the iteration matrix in the Newton solver. This paper develops new strategies for handling the iterative solution of nonlinear equations in ODE solvers. These include  automatic switching between xed-point and Newton iterations  investigating the \optimal" convergence rate with respect to total work per unit step  a strategy for when to reevaluate the Jacobian  a strategy for when to refactorize the iteration matrix  coordination with stepsize control Examples will be given, showing that the new overall strategy works eciently. In particular, the new strategy admits a restrained stepsize variation without refactorizations, thus permitting the use of a smoother stepsize sequence. The strategy is of equal importance for Runge-Kutta and multistep methods. Key Words. Numerical solution of ODEs, iterative solution of nonlinear equations, control strategies, stepsize control

1. Introduction. The numerical integration of an ODE y_ = f (y) by implicit time-stepping methods leads to the problem of solving a nonlinear equation on every step. The generic structure of this equation is y = hf (y) + ; (1) where h is the stepsize, is a constant of moderate size characteristic of the discretization method, and is a known vector. The variable translation y = x + yields the alternative x = hf (x + ), which is sometimes more convenient. The techniques developed below obviously apply to both cases. In nonsti computations, i.e. when hL[f ]  1, where L[f ] is the Lipschitz constant of f , xed-point iterations are used|convergence is rapid and the iterations are inexpensive. In sti computations, on the other hand, hL[f ]  1 for any choice of norm, implying that xed point iterations do not converge. Instead one uses (some variant of) Newton's method. The extra expense incurred by this iteration is motivated by the ability to use much larger time{steps; although there is an increase in work per step , total work per unit step is reduced. An adaptive stepsize selection is central to an ecient numerical integration of ODEs. In principle, the stepsize should be selected with respect to prescribed accuracy requirements. Although a classical problem, a systematic study of advanced stepsize control (or rather error control ) based on control theoretic ideas has appeared only recently, see e.g. [5], [6], [7], [10], [11], and [9]. Most of these analyses deal with explicit time{stepping. For implicit methods, the problem is considerably more dicult, because of the need to solve nonlinear equations. Since the convergence rate of the iterative solver usually depends on the stepsize h, strategies for an ecient solution of the nonlinear equation interact with the stepsize selection strategy. It is necessary to coordinate these strategies so that eciency is maintained. The aim of this paper is to  y

Advanced Studies, Ericsson Mobile Communications AB, S-223 70 LUND, Sweden. Department of Computer Science, Lund University, P.O. Box 118, S-221 00 LUND, Sweden. 1

2

K. GUSTAFSSON AND G. SO DERLIND

address the latter question, i.e. how to control the iterative solver without hampering an advanced stepsize selection scheme. This is of particular importance in connection with Newton iterations. We shall take the conventional approach of iterating \until convergence," i.e. the iterate yk is accepted as an approximate solution to (1) provided that an estimate of the error kek k = kyk yk is suciently small. We note, in passing, that new results on termination criteria have recently appeared, indicating that a minimum number of iterations, related to the order of the method, may be required, cf. [2] or [13]. Such an extra requirement, however, is not in con ict with the schemes proposed below. Applying a modi ed Newton iteration to (1) yields (I hJ ) yk = yk + hf (yk ) + yk+1 = yk + yk ;

(2)

where J is some approximation to the Jacobian @f=@y. Taking the \approximation" J = 0 yields the xed-point iteration yk+1 = hf (yk ) + :

(3)

In the general modi ed Newton iteration (2), the iteration matrix M = I hJ varies with J and h, calling for occasional reevaluations of the Jacobian J and/or refactorizations of M . Forming a new iteration matrix on every iteration is too expensive, and in general M will be kept unchanged for several consecutive integration steps. Many implicit Runge{Kutta methods are also designed to use a single Jacobian and iteration matrix for the computation of all stage values, in spite of the fact that these may not always be close. The convergence rate of the iteration deteriorates if the stepsize and/or true Jacobian deviate(s) from the values used to form M , making the strategy for the administration of the iteration matrix vital for overall eciency. The structure of the paper is as follows. We rst develop models for how stepsize changes and a varying Jacobian aect the convergence rate of xed point and modi ed Newton iterations. The aim is both to devise convergence control algorithms and to investigate from the theoretical point of view how they should be used to enhance eciency and robustness. We then discuss how these convergence control algorithms are coordinated with stepsize control, and how the two dierent types of iterations can be coordinated in problems that change stiness character. Several computations on the van der Pol equation (which is suciently rich in phenomena) support the theory, serve as illustrations of algorithm performance and help clarify how tuning parameters of the control algorithm should be selected. All computations are carried out in a test implementation; the reason why we avoid making comparisons with \established" codes is that it is not possible to draw rm conclusions about strategies and algorithmic features vital to a code's performance unless such code elements are tested under true ceteris paribus conditions. The test environment supports a consistent evaluation as all parameters are under full control. Finally, the central parts of the control algorithms are given in terms of pseudo{code to facilitate their implementation in existing solvers. 2. Modeling the stepsize{convergence relation. For the purpose of controlling the convergence rate of the iterative method we need to establish a model that relates the convergence rate to the parameters that may aect it. The most important such parameter is the stepsize h. Let the error in the solution be denoted ek = yk y.

CONTROL STRATEGIES IN ODE SOLVERS

3

We then rewrite the iteration (2) with the aim of showing how the error is aected in each iteration, and obtain ek+1 = M 1 ( hJ

h J ) ek ;

M = I h J ;

(4)

where h is the stepsize and J the Jacobian used when forming M , h is the current stepsize, and J is a mean value Jacobian [19, Sec. 3.2.6], J=

Z

1 @f




y +  (yk y) d: 0 @y Note that J depends on k, and that Jek = f (yk ) f (y).

Fixed point as well as Modi ed Newton iterations will in general both have a linear asymptotic convergence rate ^ depending on the spectral radius of M 1  (hJ ) = M 1( hJ h J ). To avoid expensive eigenvalue estimates, the estimated convergence rate , denoted by , is obtained by using vector norms; a common practice is to use three consecutive iterates yk 1, yk and yk+1 thus:

ky yk  = max k = max k+1 k : k kyk yk 1k

(5)

A typical iteration possesses three phases: rst a nonlinear transient, second a linear transient and nally the asymptotic phase, in which the error decreases by a factor of ^ in each iteration. This asymptotic rate is often not observed in practice, depending on several factors, e.g. the choice of norm, but also on whether a sucient number of iterations have been taken. In addition, the estimate may deviate considerably from the true asymptotic rate. Thus, estimating  by max k comes close to the asymptotic rate only if the error in the initial phases of the iteration decreases faster than in the asymptotic phase; this depends on the norm but also on the initial approximation y0 . It may occur that  > 1 even if the iteration is convergent. We shall establish models that relate the coarser estimate of the convergence rate   k M 1  (hJ )k to the available control variables: changing h and J respectively. Such models will be constructed for the two cases of main interest, viz. xed-point and modi ed Newton iterations. 3. Fixed point iteration. We let J = 0 in (4) and obtain ek+1 = hJek , from which it follows that

kek+1 k  hkJ k kek k:

(6)

Hence the convergence rate depends on the stepsize and the (unknown) Jacobian J . We model the stepsize{convergence relation by  = #h;

(7)

where # (9) 1  kyk yk 1k  kek k   ke0k; where e0 is the error in the initial guess y0 . From (9) log  log ke0 k ; k>  log  and, consequently, the number of function evaluations per unit step (i.e. per unit time of integration) is proportional to k, where (approximately) k log  log ke0 k : k = = h h log  For an ecient integration we would like to nd the value of  that minimizes k. The convergence  is proportional to h, therefore log  log ke0 k : k   log  The starting value y0 is constructed using an interpolation polynomial, which makes the initial error e0 depend on h (and hence ). The dependence is usually not strong, and hence 1 ; k  (10)  log 

5

CONTROL STRATEGIES IN ODE SOLVERS 4

function calls

2

x 10

1.5 1 0.5 0 0

0.1

0.2

0.3

0.4

0.5

ref

0.6

0.7

0.8

0.9

1

Fig. 2. Total work (number of function calls) as a function of the convergence rate set-point when integrating the van der Pol problem in Example 3.1. The equation solver includes a limit on the number of allowed iterations. The crosses `x' and the rings `o' correspond to setting this limit at 100 and 10, respectively. The actual number of function calls agrees very well with the work predicted by (10) (shown as a full line). The deviation for large ref is a consequence of limiting the number of iterations in the equation solver.

where the negative sign is a consequence of assuming e0 >  . The expression (10) has a minimum at  = e 1  0:36788, which suggests choosing ref  0:4. The function (10) is quite at around the minimum (cf. Fig. 2), and any value 0:2 < ref < 0:4 would probably be acceptable, with robustness and actual experiments favoring the lower value; a stepsize increase should never cause an increased total work per unit step. The recommendation ref ref then hnew := min(hr ; h ) else hnew := hr endif if  jh hLU j=hLU > JAC then

Form new Jacobian and factorize iteration matrix hLU := hnew

elseif jhnew

hLU j=hLU > LU then

Factorize iteration matrix hLU := hnew

else

endif

Calculate h , cf. (19) if diverging then hnew := h elseif fresh-Jacobian then if  > ref then hnew := h else hnew := h=2 endif endif if not-fresh-Jacobian then Form new Jacobian

endif

Factorize iteration matrix

endif

Fig. 4. An outline of the code needed to implement the evaluation and factorization strategy in modi ed Newton iteration. The algorithm has to be augmented with various safety nets, e.g. discard unreasonably large stepsize changes, protect against under ow/over ow, etc. For comments on parameters, see text.

for the iteration [12], is approximately 100, and with  = 0:01 " the contribution from the iteration error is large enough to cause \noisy" variations in r. The rejections can be removed by reducing  , cf. Fig. 7 where  = 0:005 ". A more appropriate value for this method, this termination criterion calls for twice the accuracy in the equation solver, and hence the irregular contribution of the iteration error to r is attenuated. As can be seen from Fig. 5 there are many factorizations when the stepsize changes fast. During the sti part the convergence deteriorates at a faster rate than what could be explained with h=h . When  reaches JAC = 0:4 the Jacobian is evaluated and the convergence rate is immediately improved. The overall behavior agrees well with theoretical considerations. From the behavior in Fig. 5 one may suspect that the iteration matrix is factorized too often. The situation is clari ed by Fig. 6, which depicts a segment of the integration in Fig. 5. Whenever an operation was to be done on the iteration matrix, we have calculated h=h (plotted as `o') and the convergence estimate that one would get if

11

CONTROL STRATEGIES IN ODE SOLVERS 2

10

r=tol

0

10

−2

10

h

−4

10

−6

10

0

50

100

150

200

250

300

350

400

1

0.5

 0

−0.5 0

JAC LU 50

100

150

200 step number

250

300

350

400

Fig. 5. A demonstration of the strategy for administrating the iteration matrix discussed in Example

5.1. The parameters LU and JAC were set to 0:4. The upper plot shows the error and the stepsize, while tic marks in the lower indicate Jacobian evaluations and factorizations of the iteration matrix. The lower plot also depicts the estimate of the convergence rate. Rejected steps during the sti part (steps 35{220) are caused by the iteration error being fed into r.

suppressing the operation (plotted as `x'). It is clear that the factorizations during the nonsti part of the solution (steps 230{330) are unnecessary. The factorizations are triggered by h=h > LU, but   1 in (13) and the stepsize change does not aect the convergence (the crosses lie on top of the  curve). The message is to change factorization strategy when the problem is nonsti, or more naturally, switch to xed point iteration as discussed below. The situation is completely dierent when ramping up the stepsize and entering the sti part of the solution (steps 10{30 and 330{350). As can be seen from the crosses in Fig. 6, the convergence would become poor if omitting any of the factorizations. It is interesting to note how the convergence rate in case of suppressing the factorization (the crosses) approaches the value for h=h (the rings) as the integration enters the sti region. This is a consequence of  ! 1 as kh J k ! 1. In conclusion, we may say that in this case it is not a poor factorization strategy that causes the many factorizations, but rather a (maybe too) strict limit on how much the stepsize may increase in one integration step (the error controller will not allow a stepsize change that would increase the error by more than a factor of 10). The value of  was lowered to reduce the eect on r from the iteration error. As expected the number of rejected steps was reduced. A maximumof 10 iterations in the equation solver were allowed. Normally, 3{5 iterations are done per stage, but when convergence is poor it may sometimes take more than 10 iterations to bring the iteration error below  . To improve the situation the values on LU and JAC were lowered to 0:2, thus forcing a faster convergence. The result is depicted in Fig. 7. The num-

12

K. GUSTAFSSON AND G. SO DERLIND 1

0.5



0

−0.5 250

JAC LU 260

270

280

290

300 310 step number

320

330

340

350

Fig. 6. A segment of the lower plot in Figure 5. At every step where an operation was done on the iteration matrix we have plotted the corresponding value of h=h (`o') and the convergence rate estimate  (`x') one would get if suppressing this operation. It is clear that the factorizations done during the nonsti period (steps 230{330) are unnecessary (  1 in (13)). In contrast, every single factorization is important when ramping up the stepsize (steps 330{350) and entering the sti region.

ber of Jacobian evaluations doubled (41) and the number of factorizations increased by approximately 30 % (129), while the number of function evaluations dropped by some 25 % (5852) due to the improved convergence. A minor reduction of the number of steps is also observed. In this case the exact gures are of no importance; they only serve to demonstrate the very signi cant possibility of trading function evaluations with operations on the iteration matrix and that a balanced overall strategy for combined stepsize and convergence control has a signi cant eect on these classical measures of eciency. Thus, these numbers do not quite measure eciency but rather re ect a particular control design. 6. Switching and coordinating equation solvers. So far we have only considered xed point and Newton iterations separately. Newton iteration is more ecient during the sti phases of the solution and xed point more ecient in nonsi. Therefore, many implicit integration methods support switching between the two techniques. The strategies described in [20, 17] are representative. Fig. 8 shows what happens when such a switching strategy is employed. We will give a short recapitulation of these techniques and comment on a few issues that are sometimes overlooked. 6.1. Switching from xed point to modi ed Newton. The integration is normally started using xed point iteration. The stepsize will be small to resolve initial transients, and xed point iteration is likely to converge. But as stepsize grows, convergence slows down until it becomes necessary to restrain the stepsize to guarantee convergence in the xed point iteration. This is an indication that it might be bene cial to switch to modi ed Newton iteration. Shampine [20] compares the stepsize hr from error control with the stepsize h from convergence control, and if the dierence is too large a switch is made, hoping that the observed behavior is part of a trend of increasing stiness. Suppose that to do the switch we require a possible stepsize increase by a factor  . Consequently, we await a situation where hr =h >  , [20]. To trust that switching will be pro table, we would like  large. It may,however,be dicult to achieve hr =h >  in practice, since the iteration error in the equation solver will prevent r from becoming very small in the case where convergence restricts the stepsize|r will generally always exceed the iteration error. In practice this puts a surprisingly tight upper limit on  .

13

CONTROL STRATEGIES IN ODE SOLVERS 2

10

r=tol

0

10

−2

10

h

−4

10

−6

10

0

50

100

150

200

250

300

350

400

1

0.5

 0

−0.5 0

JAC LU 50

100

150

200 step number

250

300

350

400

Fig. 7. The setup here is similar to the one in Figure 5. The step rejections during the sti part were

removed by lowering the iteration error ( = 0:005 "). This, however, calls for better convergence, and LU and JAC were changed to 0:2. This increases the number of operations on the iteration matrix. Note that the strategy we use has one single convergence failure during steps 35{220 (the sti phase), where the strong nonlinearities force frequent Jacobian evaluations. Conventional strategies typically have one convergence failure at each peak of the  curve.

Let convergence restrict the stepsize, and assume that the convergence rate is kept close to ref , i.e. hr  h . Supppose that hr is calculated using a standard stepsize selection rule [4, pp. 81{84], [8, p. 167] 

hn+1 =

"

1=p

rn+1

(21)

hn ;

where h is the stepsize, r the estimated error, and p related to the order of the integration method. This scheme aims at keeping the error estimate r close to ", where " is chosen less than the user-speci ed tolerance tol, typically " = 0:8tol. Hence when convergence restricts the stepsize, 

hr h



"

1=p

rn+1 h

h



"

1=p

= r : (22) n+1 It follows that the value of rn+1 determines if we will observe hr =h >  or not. The slow convergence in combination with h < hr suggests that the error estimate will be dominated by the contribution from the iteration error ek . This can be seen in Fig. 3, where r   ( = 0:01 ") in the regions where the stepsize was restricted by convergence. Setting rn+1 =  in (22) is an oversimpli cation, but shows that one cannot expect to use  > ("= )1=p , or the switch may never take place. The contribution of the iteration error to the error estimate r is highly method dependent

14

K. GUSTAFSSON AND G. SO DERLIND 2

10

r=tol

0

10



−2

10

h

−4

10

−6

10

0

50

100

150

200

250

300

350

400

1

0.5

Newton JAC LU

0

−0.5 0

50

100

150

200 step number

250

300

350

400

Fig. 8. The setup for this gure is almost identical to the one in Figure 7. The only dierence is

that modi ed Newton iteration was used only during the sti parts of the integration. The switching strategy from Example 6.1 was used.

[12, 16]. It implies a method dependent direct relation between  and  . Note that even for a method like HW-SDIRK(3)4, which has a very small  compared to ", our considerations above imply that  must be less than 3. Consequently, we have chosen  = 2. The strategy proposed in [17] makes a switch as soon as xed point iteration fails to converge. Such a strategy calls for a rather large ref. Otherwise the controller may manage to keep  below 1, and the switch to modi ed Newton iteration never takes place. Another weak point is that convergence failures may be due both to hkJ k > 1 and poor starting values for the iteration. The latter is not a typical indication of stiness and Newton's method may have even greater diculties to overcome that problem. We prefer Shampine's strategy, provided  is chosen carefully in relation to =", ref , and the method coecients. Additional robustness is gained by requiring hr =h >  for a few consecutive steps before doing the switch. Another possibility is to accumulate recent values on hr =h as a kind of ineciency measure, i.e. with convergence restricting stepsize the integration lags more and more behind what might have been possible if that restriction were not present. Switching is then done when this measure indicates a too large ineciency. 6.2. Switching from modi ed Newton to xed point. Most strategies also try to switching back to xed point whenever some indicator suggests that hf is a contraction. The typical indicator is the norm of the Jacobian, cf. (6). Some strategies recommend calculating the norm directly [20], while others estimate it through the ratio between the residual and the correction in the equation solver [17]. Thus, if a residual k = yk hf (yk ) leads to a modi ed Newton correction
yk =

CONTROL STRATEGIES IN ODE SOLVERS

15

yk+1 yk = M 1k , we let k =

kk k ; k
yk k

and note that if h J is a contraction the following lower and upper bounds hold: 1 h kJ k  glbM  k  kM k  1 + h kJ k: Hence k can be used as a stiness indicator, for if h kJ k  1, then 1 h kJ k  k  2: Conversely, if k > 2 then h J is not a contraction. Thus, if we monitor k and observe on a few consecutive steps that max k stays below 2, then it is not unlikely that xed point iterations might converge. Note that monitoring k is usually much cheaper than computing the norm of the Jacobian. We have used this switching approach. In addition, at the switch the stepsize is reduced by the factor ref to secure good convergence. 6.3. Hysteresis. All switching strategies need to include some kind of hysteresis. Otherwise it may happen that the solver toggles between xed point iteration and modi ed Newton iteration. Some common countermeasures are:  Do not allow a switch until the switching condition has been ful lled during a few consecutive steps (say 3).  When switching from xed point iteration to modi ed Newton iteration a switch back is blocked for (say) 5 steps in order to let the controller ramp up the stepsize and enter the region where xed point iteration does not converge [18].  When switching from modi ed Newton to xed point iteration the old Jacobian is stored. Should a switch back occur within not too many steps then a new Jacobian need not be calculated to form the iteration matrix [20]. Example 6.1. Again return to the van der Pol problem and the setup in Example 3.1, where  = 10. Fig. 9 demonstrates some aspects of the switching strategy described above. The number of integration steps is almost halved by switching to modi ed Newton iteration during the mildly sti parts of the solution compared to using xed point iterations throughout, which necessitates stepsizes restricted by convergence. The integration starts using xed point iteration. After the initial transient the stepsize soon gets restricted by convergence (steps 12{22) and the error estimate drops. At t  0:60 (step 23) r has been suciently low for 3 steps and a switch to modi ed Newton is done. The controller ramps up the stepsize and soon r  ". For simplicity a new iteration matrix is formed at every step, and as a consequence the convergence is very good. When the integration approaches the fast state transition the stiness drops. At t  6:7 (step 44) the stiness estimate has been below 2 for 3 steps and a switch back to xed point iteration is done. At rst the stepsize is restricted to assure convergence (steps 45{60), but during the transition at 8:5 < t < 9:4 (steps 61{109) it is accuracy that determines the stepsize. As the integration proceeds out on the at part of the solution, the stepsize is again restricted by convergence and a switch to modi ed Newton iteration is done at t  9:9 (step 120).

16

K. GUSTAFSSON AND G. SO DERLIND

2 1

y1

0 −1 −2 0

20

40

60

80

100

120

20

40

60

80

100

120

20

40

60

80

100

120

1

10

r tol

0

10

−1

10

−2

10

−3

10

0

0

10

−1

h

10

−2

10

−3

10

0

4



3

;



2



1

h < hr

0 0

20

40

60 80 step number

100

120

Fig. 9. The upper plot depicts the solution component y1 from the van der Pol problem in Example

6.1. The erratic look of the curve is due to the time{distortion introduced by plotting y1 as a function of step number. The second plot shows the error estimate. The error r is kept close to the set-point, except at fast transitions and when convergence limits the stepsize. The error and stepsize plots clearly show the regions where the stepsize is restricted by convergence. The lower plot depicts the convergence estimate , and the stiness estimate for modi ed Newton. Vertical bars indicate where the standard stepsize selection rule has been used due to h < hr in the previous step.

CONTROL STRATEGIES IN ODE SOLVERS

17

7. Controller, controller parameters and termination criteria. Turning to the parameters regarding the iteration error and the convergence control it is no longer possible to make the choice independently of integration method and dierential equation. Consider the choice of  . As noted in [9, p. 131] its value aects the eciency of the method. In addition, a large value on  results in a nonsmooth component in r that impairs the performance of the stepsize selection rule for controlling accuracy. The choice  = 0:005 " works well for HW-SDIRK(3)4. It is, however, not possible to directly translate this value to other methods. A dierent method is likely to call for a dierent value of  . For Runge{Kutta methods it is advisable to take =" LU is motivated when the Jacobian is stored on special form, so that a factorization is relatively cheap compared to the Jacobian evaluation. The choice of JAC and LU also aects the average number of iterations needed in the equation solver. A strategy that accepts slow convergence thus has to be accompanied by a fairly large upper limit on the number of iterations. As a starting point for tuning the control strategies we have proposed in this paper, ref = LU = JAC = 0:2 should give a reasonably balanced performance. REFERENCES [1] R. Alexander. \The modi ed Newton method in the solution of sti ordinary dierential equations." Mathematics of Computation, 57, pp. 673{701, 1991. [2] J. L. M. van Dorsselaer M. N. Spijker. \The error committed by stopping Newton iteration in the numerical solution of sti initial value problems." Technical Report TW-92-02, Leiden University, Department of Mathematics and Computer Science, Division of Applied Mathematics, 1992. [3] W. H. Enright. \Improving the eciency of matrix operations in the numerical solution of sti ordinary dierential equations." ACM Transactions on Mathematical Software, 4:2, pp. 127{136, June 1978. [4] C. W. Gear. Numerical Initial Value Problems in Ordinary Dierential Equations. PrenticeHall, Englewood Clis, New Jersey, 1971. [5] K. Gustafsson. \Control theoretic techniques for stepsize selection in explicit Runge-Kutta methods." ACM Transactions on Mathematical Software, 17:4, pp. 533{554, December 1991.

18

K. GUSTAFSSON AND G. SO DERLIND

[6] K. Gustafsson. Control of Error and Convergence in ODE Solvers. PhD thesis, Department of Automatic Control, Lund Institute of Technology, 1992. [7] K. Gustafsson, M. Lundh, G. So derlind. \A PI stepsize control for the numerical solution of ordinary dierential equations." BIT (Nordisk Tidskrift for Informationsbehandling), 28:2, pp. 270{287, 1988. [8] E. Hairer, S. P. Nrsett, G. Wanner. Solving Ordinary Dierential Equations I { Nonsti Problems, volume 8 of Springer Series in Computational Mathematics. Springer-Verlag, 1987. [9] E. Hairer G. Wanner. Solving Ordinary Dierential Equations II { Sti and DierentialAlgebraic Problems, volume 14 of Springer Series in Computational Mathematics. Springer-Verlag, 1991. [10] G. Hall D. J. Higham. \Analysis of stepsize selection schemes for Runge-Kutta codes." IMA Journal of Numerical Analysis, 8, pp. 305{310, 1988. [11] D. J. Higham G. Hall. \Runge-Kutta equilibrium theory for a mixed relative/absolute error measure." Technical Report 218/89, Department of Computer Science, University of Toronto, 1989. [12] N. Houbak, S. P. Nrsett, P. G. Thomsen. \Displacement or residual test in the application of implicit methods for sti problems." IMA Journal of Numerical Analysis, 5, pp. 297{ 305, 1985. [13] K. R. Jackson, A. Kvrn, S. P. Nrsett. \The order of Runge-Kutta formulas when an iterative method is used to compute the internal stage values." Technical Report, Institute for Mathematics, University of Trondheim, Norway, 1992. In preparation. [14] J. von Neumann. \Eine Spektraltheorie fur allgemeine Operatoren eines unitaren Raumes." Mathematische Nachrichten, 4, pp. 258{281, 1951. [15] O. Nevanlinna. \Matrix valued versions of a result of von Neumann with an application to time discretization." Technical Report HTKK-MAT-A224, Institute of Mathematics, Helsinki University of Technology, 1984. [16] S. P. Nrsett P. G. Thomsen. \Local error control in SDIRK-methods." BIT (Nordisk Tidskrift for Informationsbehandling), 26, pp. 100{113, 1986. [17] S. P. Nrsett P. G. Thomsen. \Switching between modi ed Newton and x-point iteration for implicit ODE-solvers." BIT (Nordisk Tidskrift for Informationsbehandling), 26, pp. 339{348, 1986. [18] S. P. Nrsett P. G. Thomsen. User Guide for SIMPLE { a sti system solver, 1987. [19] J. M. Ortega W. C. Rheinboldt. Iterative Solution of Nonlinear Equations in Several Variables. Computer Science and Applied Mathematics. Academic Press, New York, 1970. [20] L. F. Shampine. \Type-insensitive ODE codes based on implicit A-stable formulas." Mathematics of Computation, 36:154, pp. 499{510, 1981. [21] G. So derlind. \On nonlinear dierence and dierential equations." BIT (Nordisk Tidskrift for Informationsbehandling), 24, pp. 667{680, 1984. [22] G. So derlind. \Bounds on nonlinear operators in nite-dimensional Banach spaces." Numerische Mathematik, 50, pp. 27{44, 1986. [23] G. So derlind. \Folklore and fudge factors in the implementation of xed-point and Newton iterations for nonlinear ODEs.". Talk presented at \Workshop on the Numerical Solution of Sti ODEs", Sept 15{17, NTH, Trondheim, Norway, 1986. [24] J. M. Varah. \On the ecient implementation of implicit Runge-Kutta methods." Mathematics of Computation, 33:146, pp. 557{561, April 1979.