a multigrid-lanczos algorithm for the numerical

International Journal of Bifurcation and Chaos, Vol. 13, No. 5 (2003) 1217–1228 c World Scientific Publishing Company

A MULTIGRID-LANCZOS ALGORITHM FOR THE NUMERICAL SOLUTIONS OF NONLINEAR EIGENVALUE PROBLEMS* S.-L. CHANG Center for General Education, Southern Taiwan University of Technology, Tainan, Taiwan 710 C.-S. CHIEN† Department of Applied Mathematics, National Chung-Hsing University, Taichung, Taiwan 402 [email protected] Received October 31, 2001; Revised February 7, 2002 We study numerical methods for solving nonlinear elliptic eigenvalue problems which contain folds and bifurcation points. First we present some convergence theory for the MINRES, a variant of the Lanczos method. A multigrid-Lanczos method is then proposed for tracking solution branches of associated discrete problems and detecting singular points along solution branches. The proposed algorithm has the advantage of being robust and can be easily implemented. It can be regarded as a generalization and an improvement of the continuation-Lanczos algorithm. Our numerical results show the efficiency of this algorithm. Keywords: Multigrid method; Lanczos method; continuation method; nonlinear elliptic eigenvalue problems; bifurcation.

1. Introduction In this paper we are concerned with numerical methods for solving parameter-dependent problems of the following from G(u, λ) = 0 ,

(1)

where u represents the solution (i.e. flow fields, displacements, concentrations of some intermediate chemicals in a reaction–diffusion system, etc.) and λ is a vector of physical parameters (i.e. Reynolds number, load, initial or final products, temperature, etc.) Equation (1) arises, for instance, in homotopy continuation methods and nonlinear eigenvalue problems. In this paper we will concentrate ∗ †

on the latter. We wish to solve Eq. (1) numerically by the continuation method based on parametrizing the solution branch by arc-length, say [u(s), λ(s)]. First it is necessary to discretize Eq. (1), for example, by a finite difference method or a finite element method. In both cases, Eq. (1) is approximated by a finite-dimensional problem H(x, λ) = 0 ,

(2)

where H : RN × Rk → RN is a smooth mapping with x ∈ RN and λ ∈ Rk , k ≥ 1. Viewing some component of λ as the continuation parameter, the continuation algorithm can be implemented to follow solution curves of Eq. (2). Specifically, in the context of predictor–corrector continuation method

Supported by the National Science Council of R.O.C. (Taiwan) through Project NSC 90-2115-M-005-007. Author for correspondence. 1217

1218

S.-L. Chang & C.-S. Chien

[Allgower & Georg, 1997; Keller, 1987] one needs to solve bordered linear systems of the following form By = b ,

(3)

where B ∈ R(N +1)×(N +1) is the augmented matrix of the Jacobian DH = [Dx H, Dλ H] ∈ RN ×(N +1) and b ∈ RN +1 . For simplicity we assume that the matrix Dx H ∈ RN ×N is symmetric. During the past two decades various numerical methods have been proposed for solving Eqs. (1)– (3). Perhaps AUTO97 [Doedel et al., 1997; Doedel et al., 2000] is one of the well-known software packages using continuation methods to treat bifurcation problems in ordinary differential equations and dynamical systems. A comprehensive report of numerical methods for bifurcation problems in dynamical systems was given in [Govaerts, 2000]. Chan and Keller [1982] introduced some arc-length continuation and multigrid algorithms for nonlinear elliptic eigenvalue problems in partial differential equations which contain folds. The algorithm they proposed for elliptic systems can be arbitrarily close to singularity. Glowinski et al. [1985] described continuation methods combined with least squares formulations, preconditioned conjugate gradient algorithms and finite element approximations for computing branches of solutions of nonlinear elliptic eigenvalue problems which contain folds and bifurcation points. Bolstad and Keller [1986] studied continuation methods combined with accommodative full approximation scheme (FAS) multigrid (see e.g. [Briggs et al., 2000]) and the frozen tau technique of Brandt [1982] for computing solutions of nonlinear elliptic eigenvalue problems which contain folds. Allgower and Chien [1986] described numerical methods for the detection of multiple bifurcations on solution paths of certain semilinear elliptic eigenvalue problems. Moreover, Allgower et al. [1989] developed efficient and versatile predictor–corrector continuation methods for large sparse continuation problems. Specifically, they showed how to use a special nonlinear conjugate gradient method to perform the corrector phase. Irani et al. [1991] studied various preconditioned conjugate gradient algorithms for finding zeros or fixed points of nonlinear systems of equations that are globally convergent for almost all starting points, i.e. with probability one. Furthermore, Desa et al. [1992] studied preconditioned iterative methods for solving systems of nonlinear equations. Recently some continuation-Lanczos algorithms [Chien & Chang, to appear; Chien et al.,

1997] were proposed for tracing solution branches of nonlinear elliptic eigenvalue problems, where the Lanczos method [Lanczos, 1950; Golub & Van Loan, 1996, Chap. 9] was used to solve the linear systems and detect singularity of the coefficient matrices as well. Our aim in the current paper is to develop a multigrid-Lanczos algorithm for computing solution branches of nonlinear elliptic eigenvalue problems. Instead of solving the linear systems on a single fine grid as the continuation-Lanczos algorithm did in [Chien & Chang, to appear; Chien et al., 1997], the Lanczos method is used as a relaxation scheme on each grid. Our numerical experiments show that the multigrid-Lanczos algorithm converges for nonlinear eigenvalue problems close enough to singular points, including folds and bifurcation points. In addition, it converges even faster than the continuation-Lanczos algorithm. We also believe that our algorithm is at least as efficient as the multigrid-continuation algorithms of Chan and Keller [1982], and Bolstad and Keller [1986] that were developed in the eighties. However, our method is much easier to implement. The organization of this paper is as follows. In Sec. 2 we give a brief overview of the numerical methods for solving Eq. (2). In Sec. 3 we present some convergence theory for the minimal residual algorithm (MINRES) [Paige & Saunders, 1975], a variant of the Lanczos algorithm. A multigrid-Lanczos algorithm is then described in Sec. 4 for tracing solution branches of nonlinear elliptic eigenvalue problems. Specifically, we discuss how the minimal residual smoothing (MRS) technique [Schönauer, 1987; Weiss, 1990] and the inexact minimal residual smoothing (IMRS) [Zhang, 1997b] can be incorporated in our algorithm. Compared with the continuation-multigrid algorithms of Chan and Keller [1982], and Bolstad and Keller [1986], the multigrid-Lanczos algorithm we proposed has the advantage of being robust and easy to implement. Our numerical results are reported in Sec. 5. Finally, some concluding remarks are given in Sec. 6.

2. An Overview of the Continuation Algorithm We consider finite-dimensional problems of the following form H(x, λ) = 0 ,

(4)

A Multigrid-Lanczos Algorithm for the Numerical Solutions of Nonlinear Eigenvalue Problems

where H : RN × R → RN is a smooth mapping with x ∈ RN and λ ∈ R. We denote the discrete solutions of Eq. (4) by c, where c = {y(s) = (x(s), λ(s))|H(y(s)) = 0, s ∈ I ⊂ R} . Assume that a parametrization via arc length is available on c. Equation (4) arises, for instance, in homotopy continuation methods and discretizations of nonlinear eigenvalue problems. Application of homotopy continuation methods include: finding zeros of nonlinear systems of equations or systems of polynomials and matrix eigenvalue problems. In all cases one is interested only in zeros or eigenpairs of the problems one wishes to solve. Various algorithms have been proposed in the past years for homotopy curve-tracking, e.g. the HOMPACK of Watson et al. [1987], which was updated as HOMPACK 90 in [1997]. Additionally, some homotopy continuation algorithms for symmetric and real nonsymmetric eigenvalue problems were proposed by Li and Rhee [1989] and Li et al. [1992]. We also refer the further references cited therein for details. As we indicated in Sec. 1, the aim of this paper is to develop numerical methods for tracing solution curves of nonlinear eigenvalue problems. In particular, we are interested in detecting singularity along solution curves, and computing singular points such as folds and bifurcation points and so on. Doedel and his coworkers studied numerical methods and developed software packages for treating bifurcation problems in ordinary differential equations and dynamical systems [Doedel & Friedman, 1989; Doedel et al., 1991a, 1991b; Doedel et al., 1997; Doedel et al., 2000]. In this section we will discuss how to use predictor–corrector continuation methods [Allgower & Georg, 1993, 1997; Keller, 1987] to trace solution curves of nonlinear eigenvalue problems in partial differential equations. Specifically we will show how singular points can be detected along solution curves. The linear systems which appear in continuation problems can be written as A p x f = , (5) T λ g q γ where the matrix A = Dx H ∈ RN ×N , p = Dλ H, and [q T , γ] ∈ R(N +1) with γ ∈ R is the constraint vector yet to be determined. See e.g. [Allgower & Georg, 1997; Keller, 1987]. Throughout the remainder of this paper we will assume that the matrix A is symmetric. One may use either direct or iterative

1219

methods to solve Eq. (5). For instance, Allgower and Chien [1986] have proposed a reduction process for the matrix A defined in Eq. (5) via Gaussian elimination, which in turn can be used to detect both simple and multiple bifurcations of Eq. (1) [respectively Eq. (4)]. Alternatively, one can perform a QR factorization with column pivoting on A. See [Golub & Van Loan, 1996, Chap. 5]. More precisely, assume that for some positive integer k the Householder matrices H1 , . . . , Hk−1 and permutation matrices Π1 , . . . , Πk−1 have been computed such that (Hk−1 · · · H1 )A(Π1 · · · Πk−1 ) = R(k−1) " (k−1) R11 = 0

(k−1)

R12

(k−1)

R22

#

∈ RN ×N ,

(6)

(k−1)

where R11 is nonsingular and upper triangular, (k−1) (k−1) (k−1) , . . . , zN ] is a column parti= [zk and R22 tioning. Let l be the smallest positive integer satisfying k ≤ l ≤ N such that (k−1)

kzl

(k−1)

k2 = max{kzk

(k−1)

k2 , . . . , kzN

k2 } .

(k−1)

k2 < ε for some tolIf rank(A) = k − 1, then kzl erance ε > 0 and we are done. Otherwise let Π k be the interchange permutation which is obtained by swapping columns l and k. We determine a Householder matrix Hk such that if R(k) = Hk R(k−1) Πk , then R(k) (k+1 : N, k) = 0. We continue the process (k−1) k2 < ε. In practice, the rank defi(6) until kzl ciency of the matrix A is quite small in the context of continuation problems, say, less than or equal to 4. If a perturbed problem of Eq. (4) is solved near the bifurcation point, then we can determine additional Householder matrices H k , . . . , HN −1 and permutation matrices Πk , . . . , ΠN −1 such that (HN −1 · · · H1 )A(Π1 · · · ΠN −1 ) = R , or QT AΠ = R is upper triangular, where QT = HN −1 · · · H1 = Q and Π = Π1 · · · ΠN −1 are orthogonal matrices. Now the solutions of the linear systems that appeared in block elimination can be obtained by solving the least-squares problems of the following form kAx − bk22 = k(QT AΠ)ΠT x − QT bk22 = kRy − ck22 ,

(7)

where y = Πx and c = QT b. The solution of Eq. (7) can be easily obtained since R is upper triangular.

1220


However, using direct methods to solve Eq. (5) can be prohibitively expensive if the linear systems associated to Eq. (5) are large and sparse. In Sec. 3 we will discuss how to use iterative methods to solve Eq. (6).

3. Some Convergence Theory for the Lanczos Method The Lanczos method [1950] is a very popular technique that can be used to solve certain large, sparse, symmetric eigenproblems Ax = λx. In particular, this method becomes particularly useful in situations where a few of A’s largest or smallest eigenvalues are desired. Besides, the Lanczos method can be used to solve large sparse linear systems. We refer to [Chien & Chang, to appear; Golub & Van Loan, 1996, Chap. 9] and further references cited therein for details. Chien et al. [1997] exploited the properties of the Lanczos method mentioned above to solve large, sparse continuation problems. Specifically, one can monitor singularity along discrete solution curves of Eq. (4) by computing the extremum eigenvalues (respectively, condition number) of the coefficient matrices. In this section we show some convergence theory for the minimal residual algorithm (MINRES) [Paige & Saunders, 1975], a variant of the Lanczos method. The convergence theory will be used in Sec. 4 as a criterion for grid transferring in multigrid method. To start with, we consider the following linear system Ax = b ,

(8)

where A ∈ RN ×N is symmetric and nonsingular, and b ∈ RN . Let x0 ∈ RN be the initial guess to the solution of Eq. (8), and r0 = b − Ax0 be the corresponding residual. Let v1 = r0 /β1 with β1 = kr0 k2 . The Lanczos algorithm generates a sequence of orthonormal vectors v1 , . . . , vj , called Lanczos vectors, for the Krylov subspace K(A, v 1 , j) such that span{v1 , . . . , vj } = span{v1 , Av1 , . . . , Aj−1 v1 }

1. Start: Set r0 := b − Ax0 and v1 := r0 /β1 with β1 := kr0 k2 . 2. Generate the Lanczos vectors: For j = 1, 2, . . . , do rj := Avj − βj vj−1 , αj := (rj , vj ), rj := rj − αj vj , βj+1 := krj k2 .

If βj+1 < τ for some small τ, then set m := j and go to 3 ; else, compute vj+1 := rj /βj+1 . 3. Form the approximate solution: −1 (β e ), xm := x0 + Vm Tm 1 1

where Vm = [v1 , v2 , . . . , vm ] is a Lanczos matrix of order N by m, and Tm is the tridiagonal matrix   α1 β2 0 β  β3  2 α2    .. .. ..   . . . Tm =  (9) .     .. .. . . βm   0 β m αm Under the assumptions given above, we have AVm = Vm Tm + rm eTm ,

The Lanczos algorithm for solving Eq. (8) can be described as follows. Algorithm 3.1.

linear systems

The Lanczos algorithm for solving

(10)

where Tm is defined as in Eq. (9). Note that Eq. (10) can be written as AVm = Vm+1 T m , where T m is an (m+1)×m matrix which is the same as Tm except its last row whose only nonzero entry lies in the position (m + 1, m) with value β m+1 . If we replace step 3 of Algorithm 3.1 by x ˆm := x0 + Vm ym ,

(11)

where ym solves the minimization problem min kβ1 e1 − T m yk2 ,

≡ K(A, v1 , j) ≡ Kj .

(v0 ≡ 0)

y∈Rm

then we obtain the minimal residual algorithm (MINRES) for the Lanczos method. See [Paige & Saunders, 1975] for details. Actually one can easily check that the approximate solution x ˆ m satisfies the following minimization problem min

x∈xo +Km

kb − Axk2 = minm kβ1 e1 − T m yk2 . (12) y∈R


In step 3 of Algorithm 3.1, we let zm = (1) (m) −1 (β e ). Let r [zm , . . . , zm ]T = Tm ˜m and rˆm be the 1 1 residual vectors corresponding to the approximate solutions xm and x ˆm which are obtained from the Lanczos algorithm and the MINRES, respectively. One can readily verify that (m) r˜m = −rm zm

(m) and rˆm = −rm ym

(13)

(m)

hold. Here ym denotes the mth component of the vector ym defined in Eq. (11). We refer to [Chien & Chang, to appear] and further references cited therein for details. Let Pi denote the set of polynomials qi of degree less than or equal to i such that q i (0) = 1, and let σ(A) denote the spectrum of the matrix A. For completeness we show some basic convergence theory for the MINRES. To our knowledge, these results have not appeared in the literature yet. Similar results for nonsymmetric linear systems can be found in [Eisenstat et al., 1983]. Theorem 3.1. Let {ˆ ri } be the sequence of residual

vectors generated by the MINRES, then kˆ ri k2 ≤ Mi · kr0 k2 ,

(14)

where Mi := min

max |qi (λ)| .

qi ∈Pi

λ∈σ(A)

Moreover, if A is symmetric positive definite with eigenvalues λ1 ≥ λ2 ≥ · · · ≥ λN > 0, then " #i λN 2 2 · kr0 k2 . (15) kˆ r i k2 ≤ 1 − λ1 Proof. Since the matrix A is symmetric, there is an orthogonal matrix U such that U T AU = D ≡ diag(λ1 , λ2 , . . . , λN ) where λi ’s are the eigenvalues of A. By Eq. (13) the residual vectors rî generated by the MINRES can be expressed as rî = qi (A)r0 for some qi ∈ Pi . Thus by Eq. (12) we have

kˆ r i k2 =

min

x∈x0 +Ki

kb − Axk2

= min kqi (A)r0 k2 qi ∈Pi

≤ min kqi (A)k2 · kr0 k2 . qi ∈Pi

But we also have min kqi (A)k2 = min kqi (D)k2

qi ∈Pi

qi ∈Pi

= min

max |qi (λ)| .

qi ∈Pi λ∈σ(A)

(16)

1221

From this, the inequality (14) follows. To prove Eq. (15), we let q1 (z) = 1 + αz for some α ∈ R. Then min kqi (A)k2 ≤ kq1 (A)i k2 ≤ kq1 (A)ki2 .

qi ∈Pi

(17)

But ((I + αA)x, (I + αA)x) x6=0 (x, x) (x, Ax) 2 (Ax, Ax) = max 1 + 2α . +α x6=0 (x, x) (x, x)

kq1 (A)k22 = max

Using the positive definiteness of A, we have (Ax, Ax) ≤ λ21 (x, x)

and

(x, Ax) ≥ λN > 0 . (x, x)

Thus if α < 0, then kq1 (A)k22 ≤ 1 + 2λN α + λ21 α2 . The right-hand side of the above equation is minimized by choosing α = −(λN /λ21 ). With this choice of α we have " #1 λN 2 2 . (18) kq1 (A)k2 ≤ 1 − λ1 From Eqs. (16)–(18), the inequality (15) follows. Similar to the relationship between GMRES and GMRES(m) [Saad, 1996, Chap. 6], we can restart the MINRES every m steps for some fixed integer m. We refer to this as the MINRES(m). All we need to do is to add the following step to MINRES: 4. Restart: (m) Compute rˆm = −rm ym ; if satisfied then stop; else, set x0 := x ˆm , compute β1 := kˆ r m k2 , v 1 = rˆm /β1 and go to step 2 of Algorithm 3.1. From Theorem 3.1 we can establish an error bound for the residual vectors generated by the MINRES(m). Corollary 3.2. Let {ˆ ri } be the sequence of residual vectors generated by the MINRES(m). Then we have

kˆ rjm k2 ≤ (Mm )j · kr0 k2 .

(19)

1222


Moreover, if A is symmetric positive definite, then " # 2i λN 2 kˆ r i k2 ≤ 1 − · kr0 k2 , (20) λ1 where i = jm + k with 0 ≤ k < m. The error bound Eq. (19) is a trivial result of Theorem 3.1. From Eq. (15) we have " #k λN 2 2 kˆ ri k2 = kˆ rjm+k k2 ≤ 1 − · kˆ rjm k2 . λ1

Proof.

(21) By Eqs. (16)–(18) we also have kˆ rjm k2 ≤ min kqm (A)k2 · kˆ r(j−1)m k2 qm ∈Pm

≤

min kqm (A)k2

qm ∈Pm

2

· kˆ r(j−2)m k2

≤ ··· j ≤ min kqm (A)k2 · kr0 k2 qm ∈Pm

j ≤ [kq1 (A)km 2 ] · kr0 k2   2 ! m2 j λN  · kr0 k2 ≤ 1− λ1

"

= 1−

λN λ1

2 # mj 2

· kr0 k2 .

Hence, "

λN λ1

2 # k+mj 2

"

λN λ1

2 # 2i

kˆ r i k2 ≤ 1 −

= 1−

· kr0 k2

· kr0 k2 .

4. A Multigrid-Lanczos Algorithm Multigrid methods have been regarded as an efficient solver for elliptic boundary value problems [Brandt, 1977; Hackbusch, 1985; Briggs et al., 2000], where the classical Gauss–Seidel method or the SOR is used as a relaxation scheme. Application of the multigrid methods in the context of continuation methods for tracing solution branches of

nonlinear elliptic eigenvalue problems can be found in [Bolstad & Keller, 1986; Chan & Keller, 1982; Hackbusch, 1985, Chap. 13; Mittelmann & Weber, 1985; Weber, 1985]. However, as was reported by Chan and Keller, straightforward implementations of the multigrid fail near folds. A modified multigrid algorithm was proposed in [Chan & Keller, 1982] which converges for elliptic systems arbitrarily close to folds. Besides, Bolstad and Keller [1986] described a delicate multigrid-continuation method for computing solution branches of nonlinear elliptic eigenvalue problems with folds. Their method combines the frozen tau technique of Brandt [1977] with pseudo-arc length continuation and correction of the parameter on the coarsest grid. Actually the convergence behavior of the Gauss–Seidel or the SOR deteriorates if the condition number of the coefficient matrix becomes large. For symmetric indefinite linear systems our numerical experiments show that the Gauss–Seidel or the SOR method is not as robust as the Lanczos method. The numerical results in [Chien & Chang, to appear; Chien et al., 1997] show that the Lanczos method can be used to solve linear systems as well as to detect singularity in bifurcation problems. The main issue here is to replace the Gauss–Seidel method by the Lanczos method as the relaxation scheme in the context of the multigrid method. That is, instead of solving the linear systems on a single fine grid as the continuation-Lanczos algorithm did in [Chien et al., 1997], the Lanczos method is used as the relaxation scheme on each grid. Now, we adopt the idea of the V-cycle scheme multigrid method for solving the associated symmetric linear systems. We assume that there is a hierarchy of grids Ωh , Ω2h , . . . , ΩM h defined on a domain Ω, where ih denotes the mesh size on Ω. First we relax on the finest grid Ωh using the Lanczos method until convergence deteriorates. Then we compute the finest grid residual and restrict it to the next finer grid Ω2h , which is to be used as the right-hand side for the linear system on Ω 2h . We repeat this process until the coarsest grid Ω M h is reached. Next, we perform the correction scheme by injecting the solution of the linear system obMh tained on ΩM h to Ω 2 , and adding it to the soMh lution obtained in Ω 2 , which is to be used as the initial guess for the Lanczos method. We relax Mh the same linear system on Ω 2 using the Lanczos method again. This process is repeated until the


approximate solution obtained in the finest grid is accurate enough. Our numerical results show that the multigrid-Lanczos algorithm proposed here is more efficient than the continuation-Lanczos algorithm. Furthermore, it can be easily modified to detect singularity of coefficient matrices. All we need to do is to implement the Lanczos method on the finest grid if necessary. We refer to [Chien et al., 1997] for details. One may use either a fixed strategy or an adaptive one to transfer between grids. In a fixed strategy one performs p relaxation sweeps on each grid Ωkh before transferring to a coarser grid Ω 2kh , and perform q relaxation sweeps before interpolating kh back to a finer grid Ω 2 . In an adaptive strategy one transfers to a coarser grid when the ratio of the residual norm of current iterate to the residual norm a sweep earlier is greater than some tolerance η, and transfer to a finer grid when the ratio of the residual norm of current iterate to the residual norm on the next finer grid is less than another tolerance δ. The strategy we propose here is based on the convergence theory of the Lanczos method and is described as follows. Let xkh 0 be an initial guess to the linear system Akh x = f kh

(22)

on the grid Ωkh , and r0kh = f kh − Akh xkh 0 the corresponding residual. In the V-cycle scheme we relax on Eq. (22) a certain number of sweeps by using the Lanczos method, and transfer to a coarser grid Ω2kh if krikh k2 < δk · kr0kh k2

(23)

is satisfied. Here rikh = f kh − Akh xkh i is the residual and δ vector of the ith sweep xkh k ∈ (0, 1). Simii larly, in the correction process we also relax on the associated linear system of Eq. (22) a certain numkh ber of sweeps and transfer to a finer grid Ω 2 if the initial residual vector r0kh and the final residual vector rjkh satisfy krjkh k2 < ηk · kr0kh k2

(24)

for some ηk ∈ (0, 1). Our numerical experiments show that the convergence behavior of the multigrid-Lanczos algorithm depends on the choices of δk and ηk . Our strategy of when to transfer between grids is based on the assumption that the Lanczos method converges smoothly for symmetric linear systems. Since the MINRES is a variant of the

1223

Lanczos method, our assumption is reasonable because of Theorem 3.1. However, for other iterative methods the residual norms of the iterates, say {xk }, may not decrease monotonically to zero. Schönauer [1987] and Weiss [1990] proposed to use a technique which generates a second sequence {y k } via a simple relation y0 = x 0 , yk = (1 − τk )yk−1 + τk xk ,

k = 1, 2, . . . ,

(25)

in which each τk is chosen to minimize kb − A((1 − τ )yk−1 + τ xk )k2 over τ ∈ R, i.e. sTk−1 (rk − sk−1 ) , (26) τk = − krk − sk−1 k22 where rk = b − Axk and sk−1 = b − Ayk−1 . This technique was referred to by Zhou and Walker [1994] as “minimal residual smoothing (MRS)”, and was applied there to investigate certain relationships between the residuals and residual norms of the biconjugate gradient (BCG) and quasi-minimal residual (QMR) methods. Zhang [1997a] employed the MRS technique to accelerate the convergence of a multilevel iterative method by smoothing the residuals of the original iterative sequence on the finest grid. In [Zhang, 1997b] some inexact versions of the minimal residual smoothing (IMRS) technique were proposed to accelerate the convergence of the multigrid method. More precisely, Zhang [1997b] proposed to compute the inner products that appear in Eq. (26) on a subspace of Ωh . This subspace may be chosen as the coarse grid Ω2h or Ω4h . The MRS obtained in this way is referred to as the half minimal residual smoothing (Half-MRS) or Quarter-MRS, respectively. The original MRS is then called FullMRS. Our multigrid-Lanczos algorithm is described as follows. Alogrithm 4.1. A V-cycle scheme multigridLanczos algorithm Input: δk , ηk ∈ (0, 1), k = 1, 2, 3, . . . .

• Relax on Ah uh = f h i1 times by the Lanczos method with initial guess v h so that krih1 k2 < δ1 · kr0h k2 . • Set r h = rih1 . • Compute f 2h = Ih2h r h . • Relax on A2h u2h = f 2h i2 times by the Lanczos method with initial guess v 2h = 0 so that k < δ2 · kr02h k2 . kri2h 2 2 . • Set r 2h = ri2h 2

1224


Algorithm 4.1, then we do not have to do this process. By incorporating Algorithm 4.1 into the continuation method, we obtain a generalization and an improvement of the continuation-Lanczos algorithm [Chien et al., 1997].

4h r 2h . • Compute f 4h = I2h .. .

• Solve AM h uM h = f M h and obtain v M h . .. . 2h v 4h . • Correct v 2h ← v 2h + I4h 2h 2h • Relax on A u = f 2h j2 times by the Lanczos method with initial guess v 2h so that krj2h k < η2 · k r02h k2 . 2 2 h v 2h . • Correct v h ← v h + I2h • Relax on Ah uh = f h j1 times by the Lanczos method with initial guess v h so that krjh1 k2 < η1 · kr0h k2 . jh In Algorithm 4.1, Iih stands for the restriction operator, while the bicubic interpolation opih where j = 2i. That is, erator is denoted by Ijh jh ih : Ωjh → Ωih . We may : Ωih → Ωjh and Ijh Iih incorporate the MRS or the IMRS techniques into Algorithm 4.1. However, if we replace the Lanczos algorithm by the MINRES or MINRES(m) in

T. Time aver. iter.(P)

: :

aver. iter.(C)

:

5. Numerical Results The multigrid-Lanczos algorithm described in Sec. 4 was implemented in the context of the continuation method to trace solution branches of certain nonlinear elliptic eigenvalue problems. These test problems were discretized by the centered difference approximations with h = 1/64 on the finest grid, and a total of five levels of grids, making the coarsest grid with h = 1/4. The accuracy tolerance for solving linear systems was 10−10 , and the parameters in Algorithm 4.1 were chosen to be δk = ηk = 0.5, k = 1 : 4. All of our computations were executed on an IBM PC 586 machine with a High Performance Fortran Compiler and with 64 bit IEEE arithmetic. The following notations are used in Tables 1–2.

the total execution time (in hours) for each method. the average number of iterations or V-cycles required to solve a single linear system using the Lanczos method or Algorithm 4.1 and its variants in the predictor step. the average number of iterations or V-cycles required to solve a single linear system using the Lanczos method or Algorithm 4.1 and its variants in the corrector step.

1. A Fold Problem. The Lanczos method, the multigrid-Lanczos algorithm, and the multigrid-Lanczos algorithm with MRS, Half-MRS, Quarter-MRS techniques, respectively, were implemented to trace the solution curve of the classical Bratu problem [Chan & Keller, 1982] Example

∆u + λ exp(u) = 0 in Ω = [0, 1]2 , u = 0 on ∂Ω ,

(27)

starting with (u, λ) = (0, 0). The fold was detected at λ∗ ≈ 6.8157. The exact location of the fold is λ = 6.808124423 [Doedel & Sharifi, 2000; Fedoseyev et al., 2000]. Actually we could have obtained more accurate solutions near the fold if we chose finer grid, say, h = 1/128, and use relatively small steplength in our predictor–corrector continuation algorithm. After we have passed the fold, we continued to trace the solution curve until λ = 6.7187 was reached. In total we executed 130 continuation steps, and 520 linear systems were

solved. Table 1 shows the performance of these techniques. Example 2. A Bifurcation Problem.

We used the same techniques and parameters as in Example 1 to trace the first solution branch of the following nonlinear eigenvalue problem [Chien & Chang, to appear; Chien et al., 1997] ∆u + λ sinh(u) = 0 u=0

in Ω = [0, 1]2 , on ∂Ω

(28)

bifurcating at (u, λ) = (0, 2π 2 ). We executed 90 continuation steps until λ ≈ 9.8091 was reached, where 183 linear systems were solved in total. The bifurcation point was detected at λ ∗ ≈ 19.7241. Table 2 shows our numerical results. Example 3. Exploiting Symmetry of the Domain and the Solution. It is well-known in bifurcation

1225

A Multigrid-Lanczos Algorithm for the Numerical Solutions of Nonlinear Eigenvalue Problems Table 1.

Implementing Algorithm 4.1 and some of its variants for Example 1.

Methods

Lanczos

Multigrid-Lanczos

Multigrid-Lanczos with MRS

Multigrid-Lanczos with Half-MRS

Multigrid-Lanczos with Quarter-MRS

T. Time aver. iter.(P) aver. iter.(C)

25.97 126.0 86.9

23.77 8.5 6.1

24.38 8.1 5.8

25.34 8.1 5.9

24.95 8.1 5.7

Table 2.

Implementing Algorithm 4.1 and some of its variants for Example 2.

Methods

Lanczos

Multigrid-Lanczos

Multigrid-Lanczos with MRS

Multigrid-Lanczos with Half-MRS

Multigrid-Lanczos with Quarter-MRS

T. Time aver. iter.(P) aver. iter.(C)

15.28 82.1 68.7

14.05 6.3 4.7

14.04 6.4 4.7

14.93 6.3 4.8

14.42 6.1 4.6

theory (e.g. [Bossavit, 1986; Mei, 1991]) that solution branches of certain semilinear elliptic eigenvalue problems defined on a domain can be obtained by solving the same problem defined on subdomains with appropriate boundary conditions. Bifurcation of semilinear elliptic eigenvalue problems of the following form ∆u + λf (u) = 0 in Ω = [0, 1]2 , u = 0 on ∂Ω

(29)

in Ω1 =

∆u + λ sinh(u) = 0 1 , (x, y) 0 ≤ x ≤ y, 0 ≤ y ≤ 2

with boundary conditions [Mei, 1991] 1 ∂u x, = 0, u(0, y) = 0 , ∂x 2 ∂u ∂u (x, x) = (x, x) , ∂x ∂y

0.1

0

−0.1

−0.2 1 0.8

1 0.6

have been investigated numerically and analytically during the past decade. We refer to [Allgower & Chien, 1986; Allgower et al., 1989; Mei, 1991] and further references cited therein for details. For instance, the solution curves of Eq. (28) branching from the double bifurcation point (0, λ 2,4 ) = (0, 20π 2 ) [Isaacson & Keller, 1965, Chap. 9] can be obtained by solving the same problem defined on the subdomain Ω1 defined in Eq. (30) with appropriate boundary conditions (31) and (32) given below, respectively. In this example, we traced the solution curves branching from the second bifurcation point of

0.2

0.8 0.6

0.4

0.4

0.2 0

0.2 0

Fig. 1. Contour of the solution curve of Eq. (30) with boundary conditions (31) bifurcating at (0, λ∗ ) ≈ (0, 191.941).

by the predictor–corrector continuation method together with Algorithm 4.1. The bifurcation point was detected at (u, λ∗ ) ≈ (0, 191.941). Figure 1 shows the contour of the solution curve on the subdomain. Figures 2 and 3 show the three- and twodimensional contours of Eq. (29) on the whole domain [0, 1]2 . Similarly, we traced solution branch of Eq. (29) with boundary conditions [Mei, 1991]

(30) u(0, y) = u(x, x) = 0 ,

(31)

∂u ∂x

1 x, 2

= 0,

(32)

bifurcating at (u, λ∗ ) ≈ (0, 191.941). Figure 4 shows the contour of this solution curve defined on the subdomain Ω1 , while Figs. 5 and 6 show the contour of this solution curve on the whole domain [0, 1]2 .

1226


0.2

0.3 0.2

0.1

0.1 0

0 −0.1

−0.1

−0.2 −0.2 1

−0.3 1 0.8

1 0.6

0

Fig. 5.

0.9

0.9

0.8

0.8

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0

0

0.2

0

0

0

Contour of the solution curve on Ω = [0, 1]2 .

0.7

0

0

0

0.2 0

1

0

0.4

0.2

1

0

0

0

0

0.1

0.1

0

0.6

0.4

Contour of the solution curve on Ω = [0, 1]2 .

0.2

0.8

0.2 0

0.7

1 0.6

0.4

0.2

Fig. 2.

0.8

0.8 0.6

0.4

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 3. Two-dimensional contour of the solution curve on Ω = [0, 1]2 .

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fig. 6. Two-dimensional contour of the solution curve on Ω = [0, 1]2 .

6. Concluding Remarks Our results in Tables 1 and 2 show that the multigrid-Lanczos algorithm is the most efficient one among all of the techniques we have tested. Moreover, it seems that there is no need to incorporate the MRS into the multigrid-Lanczos algorithm.

0 −0.05 −0.1 −0.15

Acknowledgment

−0.2 −0.25 −0.3 1 0.8

1 0.6

We thank Prof. E. Doedel of Concordia University for reading the original manuscript of this paper and giving us valuable suggestions.

0.8 0.6

0.4 0.4

0.2

0.2 0

0

Fig. 4. Contour of the solution curve of Eq. (30) with boundary conditions (32) bifurcating at (0, λ∗ ) ≈ (0, 191.941).

References Allgower, E. L. & Chien, C.-S. [1986] “Continuation and local perturbation for multiple bifurcations,” SIAM J. Sci. Stat. Comput. 7, 1265–1281.


Allgower, E. L., Chien, C.-S. & Georg, K. [1989] “Large sparse continuation problems,” J. Comput. Appl. Math. 26, 3–21. Allgower, E. L. & Georg, K. [1993] “Continuation and path following,” Acta Numerica 2, 1–64. Allgower, E. L. & Georg, K. [1997] “Numerical path following,” in Handbook of Numerical Analysis, eds. Ciarlet, P. G. & Lions, J. L. (North-Holland), Vol. 5, pp. 3–207. Bolstad, J. H. & Keller, H. B. [1986] “A multigrid continuation method for elliptic problems with folds,” SIAM J. Sci. Stat. Comput. 7, 1081–1104. Bossavit, A. [1986] “Symmetry, group and boundary value problems — A progessive introduction to noncommutative harmonic analysis of partial differential equations in domains with geomertical symmetry,” Comput. Meth. Appl. Mech. Engrg. 56, 167–215. Brandt, A. [1977] “Multi-level adaptive solutions to boundary value problems,” Math. Comput. 31, 333–390. Brandt, A. [1982] Guide to Multigrid Development, in Multigrid Methods, Lecture Notes in Mathematics, Vol. 960, eds. Hackbusch, W. & Trottenberg, U. (Springer Verlag, NY), pp. 220–312. Briggs, W. L., Henson, V. E. & McCormic, S. F. [2000] A Multigrid Tutorial, 2nd edition (SIAM Publications, Philadelphia). Chan, T. F. & Keller, H. B. [1982] “Arc-length continuation and multi-grid techniques for nonlinear elliptic eigenvalue problems,” SIAM J. Sci. Stat. Comput. 3, 173–194. Chien, C.-S. & Chang, S.-L. “Application of the Lanczos algorithm for solving the linear systems that occur in continuation problems,” Numer. Lin. Algeb. Appl., to appear. Chien, C.-S., Weng, Z.-L. & Shen, C.-L. [1997] “Lanczos type methods for continuation problems,” Numer. Lin. Algeb. Appl. 4, 23–41. Desa, C., Irani, K. M., Ribbens, C. J., Watson, L. T. & Walker, H. F. [1992] “Preconditioned iterative methods for homotory curve tracking,” SIAM J. Sci. Stat. Comput. 13, 30–46. Doedel, E. J., Keller, H. B. & Kernévez, J. P. [1991a] “Numerical analysis and control of bifurcation problems, Part I: Bifurcation in finite dimensions,” Int. J. Bifurcation and Chaos 1, 493–520. Doedel, E. J., Keller, H. B. & Kernévez, J. P. [1991b] “Numerical analysis and control of bifurcation problems, Part II: Bifurcation in infinite dimensions,” Int. J. Bifurcation and Chaos 1, 745–772. Doedel, E. J., Champneys, A. R., Fairgrieve, T. F., Kuznetsov, Y. A., Sandstede, B. & Wang, X. J. [1997] “AUTO97: Continuation and Bifurcation Software for Ordinary Differential Equations (with HomCont),” User’s Guide (Computer Science Department, Concordia University, Montreal, Canada).

1227

Doedel, E. J. & Friedman, M. J. [1989] “Numerical computation of heteroclinic orbits,” J. Comput. Appl. Math. 26, 159–170. Doedel, E. J., Paffenroth, R. C., Champneys, A. R., Fairgrieve, T. F., Kuznetsov, Y. A., Sandstede, B. & Wang, X. J. [2000] “AUTO2000: Continuation and bifurcation software for ordinary differential equations,” Report, Applied Mathematics, Caltech, Pasadena. Doedel, E. J. & Sharifi, H. [2000] “Collocation methods for continuation problems in nonlinear elliptic PDEs,” in Notes on Numerical Fluid Mechanics, eds. Henry, D. & Bergeon (Vieweg), Vol. 74, pp. 105–118. Eisenstat, S. C., Elman, H. C. & Schultz, M. H. [1983] “Variational iterative methods for nonsymmetric systems of linear equations,” SIAM J. Numer. Anal. 20, 345–357. Glowinski, R., Keller, H. B. & Reinhart, L. [1985] “Continuation-conjugate gradient methods for the least squares solution of nonlinear boundary value problems,” SIAM J. Sci. Stat. Comput. 6, 793–832. Fedoseyev, A. I., Friedman, M. J. & Kansa, E. J. [2000] “Continuation for nonlinear elliptic partial differential equations discretized by the multiquadratic method,” Int. J. Bifurcation and Chaos 10, 481–492. Golub, G. H. & Van Loan, C. F. [1996] Matrix Computations, 3rd edition (The Johns Hopkins University Press, Baltimore, MD). Govaerts, W. J. F. [2000] Numerical Methods for Bifurcations of Dynamical Equilibria (SIAM Publications, Philadelphia). Hackbusch, W. [1985] Multigrid Methods and Applications (Springer-Verlag, Berlin). Irani, K. M., Kamat, M. P., Ribbens, C. J., Walker, H. F. & Watson, L. T. [1991] “Experiments with conjugate gradient algorithms for homotopy curve tracking,” SIAM J. Opt. 1, 222–251. Isaacson, E. & Keller, H. B. [1965] Analysis of Numerical Methods (John Wiley, NY). Keller, H. B. [1987] Lectures on Numerical Methods in Bifurcation Problems (Springer, Berlin). Lanczos, C. [1950] “An iteration method for the solution of the eigenvalue problems of linear differential and integral operators,” J. Res. Nat. Bur. Stand. 45, 255–282. Li, T. Y. & Rhee, N. H. [1989] “Homotopy algorithm for symmetric eigenvalue problems,” Numer. Math. 50, 265–280. Li, T. Y., Zeng, Z. & Cong, L. [1992] “Solving eigenvalue problems of real nonsymmetric matrices with real homotopies,” SIAM J. Numer. Anal. 29, 229–248. Mei, Z. [1991] “Path following around corank-2 bifurcation point of a semilinear elliptic problems with symmetry,” Computing 47, 69–85. Mittelmann, H. D. & Weber, H. [1985] “Multi-grid solution of bifurcation problems,” SIAM J. Sci. Stat. Comput. 6, 49–60.

1228


Paige, C. C. & Saunders, M. A. [1975] “Solution of sparse indefinite systems of linear equations,” SIAM J. Numer. Anal. 12, 617–629. Saad, Y. [1996] Iterative Methods for Large Sparse Linear Systems (PWS Publishing Co., Boston). Sch¨ onauer, W. [1987] Scientific Computing on Vector Computers (North-Holland, Amsterdam, NY, Oxford, Tokyo). Watson, L. T., Billups, S. C. & Morgan, A. P. [1987] “Algorithm 652, HOMPACK: A suite of codes for globally convergent homotopy algorithms,” ACM Trans. Math. Software 13, 281–310 Watson, L. T., Sosonkina, M., Melville, R. C., Morgan, A. P. & Walker, H. F. [1997] “HOMPACK90: A suite of Fortran 90 codes for globally convergent homotopy algorithms,” ACM Trans. Math. Software 23, 514–549.

Weber, H. [1985] “Multigrid bifurcation iteration,” SIAM J. Numer. Anal. 22, 262–279. Weiss, R. [1990] “Convergence behavior of generalized conjugate gradient methods,” Ph.D. thesis, Univ. Karlsruhe, Germany. Zhang, J. [1997a] “Minimal residual smoothing in multilevel iterative method,” Appl. Math. Comput. 84, 1–25. Zhang, J. [1997b] “Multigrid with inexact minimal residual smoothing acceleration,” Appl. Numer. Math. 24, 501–512. Zhou, L. & Walker, H. F. [1994] “Residual smoothing techniques for iterative methods,” SIAM J. Sci. Comput. 15, 297–312.