An Algorithm for Solving Non-Symmetric Saddle-Point Systems

An Algorithm for Solving Non-Symmetric Saddle-Point Systems Jaroslav Haslinger, Charles University, Prague ˇ Tom´asˇ Kozubek, VSB–TU Ostrava ˇ Radek Kuˇcera, VSB–TU Ostrava

HARRACHOV, August 2007

•First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

OUTLINE

Motivation: Fictitious domain method Algorithm PSCM: Schur complement method + Null-space method Inner solver: Projected BiCGSTAB Preconditioning: Hierarchical multigrid Singular matrices: Poisson-like solver based on circulants Numerical experiments


OUTLINE



MODEL PROBLEM 1: Dirichlet problem

− ∆u = f u=g

on ω

(1)

in γ ≡ ∂ω

(2)

Fictitious domain method (FDM): PDE (1) is solved on the fictitious domain Ω, ω ⊂ Ω, with a simple geometry. The corresponding stiffness matrix A is structured. The original boundary conditions (2) on γ are enforced by Lagrange multipliers or control variables. Ω Γ γ

δ ω

Ξ

Smooth FDM with Γ 6= γ

Classical FDM with Γ = γ

A B> γ Bγ 0

u λγ

=

f g

A B> Γ Bγ 0

u λΓ

=

f g


MODEL PROBLEM 2: Signorini problem

− ∆u = f ∂u ∂u u − g ≥ 0, ≥ 0, (u − g) =0 ∂nγ ∂nγ

on ω

(3)

in γ ≡ ∂ω

(4)

FDM formulation uses the non-differentiable max-function to express BC (4):

Au + BΓ λΓ = f Cγ,i u = max {0, Cγ,i u − ρ(Bγ,i u − gi )}, i = 1, . . . , m

(5)

where Bγ,i , BΓ,i and Cγ,i are rows of Dirichlet and Neumann trace matrices, respectively. The equations (5) can be solved by the semi-smooth Newton method, in which

Jacobian =

A

B> Γ

∂G(u) 0

is determined by the generalized derivative ∂G(u). •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

MODEL PROBLEM 2: Newton method = Active set algorithm (0) Set k := 1, ρ > 0, εu > 0, u(0) ∈ Rn , λ(0) ∈ Rm . (1) Define the inactive and active sets by:

I k := {i : Cγ,iuk−1 − ρ(Bγ,iuk−1 − gi) ≤ 0} Ak := {i : Cγ,iuk−1 − ρ(Bγ,iuk−1 − gi) > 0} (2) Solve:



A

B> Γ



 Bγ,Ak 0  Cγ,I k

0

uk λkΓ





f =  gAk  0

(3) If kuk − uk−1 k/kuk k ≤ εu , return u := uk . (4) Set k := k + 1, and go to step (1). Remark: The mixed Dirichlet-Neumann problem is solved in each Newton step, that is described by the non-symmetric saddle-point system. •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

FORMULATION: Non-symmetric sadle-point system

A B> 1 B2 0

u λ

=

f g

General assumptions A . . . non-symmetric (n × n)–matrix . . . singular with p = dim Ker A B1 , B2 . . . full rank (m × n)–matrices . . . B1 6= B2 Special FDM assumptions

• n is large (n = 4198401) • m n (m = 360) • p m (p = 1) • A is structured so that actions of A† or (A−1) are ”cheap” • B1, B2 are highly sparse so that their actions are ”cheap” •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

OUTLINE



ALGORITHMS based on the Schur complement reduction Case 1: A non-singular, symmetric Case 2: A non-singular, non-symmetric Case 3: A singular, symmetric Case 4: A singular, non-symmetric


Case 1: A non-singular, symmetric

A B> B

0

u λ

=

f g

=⇒

u = A−1 (f − B> λ)

=⇒

BA−1 B> λ = BA−1 f − g

| {z }

negative Schur complement S Algorithm

1◦ Assemble d := BA−1f − g. 2◦ Solve iteratively Sλ = d with S := BA−1B>. 3◦ Assemble u := A−1(f − B>λ). If A is positive defined, then CGM can be used. Matrix-vector products Sµ are performed by: Sµ := B A−1 B> µ


Case 2: A non-singular, non-symmetric

A B> 1 B2

u λ

0

=

f

=⇒ =⇒

g

u = A−1 (f − B> 1 λ) −1 B2 A−1 B> 1 λ = B2 A f − g

|

{z

}

negative Schur complement S Algorithm is analogous.

• an iterative method for non-symmetric matrices is required (GMRES, BiCG, BiCGSTAB, ...)

A :=

A B> 1 B2

0

=

I

0

B2 A−1 I

A B> 1

0 −S

Theorem 1 Let A be non-singular. Then A is invertible iff S is invertible. •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Case 3: A singular, symmetric

• a generalized inverse A† satisfying A = AA†A • an (n × p)–matrix N whose columns span Ker A Au + B> λ = f

f − B> λ ∈ Im A ⊥ Ker A

⇐⇒

m

m

u = A† (f − B> λ) + Nα

N> (f − B> λ) = 0

&

The reduced system:

Bu = g

BA† B> −BN −N> B>

0

λ α

† BA f − g = > −N f

⇓ BA† B> λ − BNα = BA† f − g If A is positive semidefinite, then it corresponds to the algebra in FETI DDM. •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Case 4: A singular, non-symmetric

• a generalized inverse A† • columns of (n × p)–matrices N, M span Ker A, Ker A>, respectively Au + B> 1λ = f

> f − B> 1 λ ∈ Im A ⊥ Ker A

⇐⇒

m

m

u = A † ( f − B> 1 λ) + Nα

M > ( f − B> 1 λ) = 0

&

The reduced system:

B2 u = g

B2 A† B> −B2 N 1 −M> B> 1

0

λ α

B 2 A† f − g = > −M f

⇓ † B2 A † B> 1 λ − B2 Nα = B2 A f − g


Theorem 2 The saddle-point matrix A :=

A B> 1 B2

0

   

B1 has full row-rank Ker A ∩ Ker B2 = {0} A Ker B2 ∩ Im B> 1 = {0}

is invertible iff

(NSC)

  

Remark: The third equality is equivalent to the MinMax condition that is wellknown in the continuous setting:

∃C > 0 :

min

max

u∈Ker B2 ,u6=0 v∈Ker B1 ,v6=0

v> Au ≥C kvkkuk


The generalized Schur complement: the matrix of the reduced system

S :=

−B2A†B>1 B2N M > B> 1

0

Theorem 3 The following three statements are equivalent:

• The necessary and sufficient condition (NSC) holds. • A is invertible. • S is invertible.

Remark: The generalized Schur complement S is not defined uniquely.


First step of the algorithm = Schur complement reduction:

A B2

B> 1 0

u λ

=

f g

⇐⇒

 >  λ d F G1   = G2

α

0

e

   u = A†(f − B>λ) + Nα 1

How to solve the reduced system again with the saddle-point structure?

• matrix-vector products via Fµ := B2 A

†

B> 1µ

• G1, G2, d, e may be assembled   1) Again the Schur complement reduction (the second elimination)    > −1 > −1    Eα = r with E := G2F G1 , then λ := F (d − G1 α) and u (R.K., Appl. Math. 50(2005))

   2) Null-space method    

(Farhat, Mandel, Roux: FETI DDM, 1994) •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Second step of the algorithm = Null-space method:

F G2

G> 1

0

λ α

=

d e

Two orthogonal projectors P1 and P2 onto Ker G1 and Ker G2 : Pk : Rm 7→ Ker Gk ,

> −1 Pk := I − G> k (Gk Gk ) Gk ,

Ker Pk = Im G>k

Property:

k = 1, 2

⇐⇒ Pk G>k = 0

• P1 splits the saddle-point structure:

P1 Fλ + P1 G> 1 α = P1 d

−1 P1 Fλ = P1 d, G2 λ = e, α := (G1 G> 1 ) (G1 d − G1 Fλ)

• P2 decomposes λ = λIm + λKer , λIm ∈ Im G>2 , λKer ∈ Ker G2 =⇒ λIm := G>2 (G2G>2 )−1e

At first:

G2 λ = G2 λIm = e

At second:

P1 FλKer = P1 (d − FλIm ) on Ker G2 •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Theorem 4 Let A be invertibel. The linear operator P1 F : Ker G2 7→ Ker G1 is invertible. Proof. As both null-spaces Ker G1 and Ker G2 have the same dimension m − p, it is enough to prove that P1 F is injective. Let µ ∈ Ker G2 be such that P1 Fµ = 0. Then Fµ ∈ Ker P1 = Im G> 1 and, p therefore, there is β ∈ R so that Fµ = G> 1 β and G2 µ = 0. We obtain

F G> 1 G2 0

µ −β

=

0 0

,

where the matrix is the (negative) Schur complement −S that is invertible iff A is invertibel. Therefore µ = 0.


Algorithm PSCM > > Step 1.a: Assemble G1 := −N> B> 2 , G2 := −M B1 .

Step 1.b: Assemble d := B2 A† f − g, e := −M> f. > −1 −1 Step 1.c: Assemble H1 := (G1 G> 1 ) , H2 := (G2 G2 ) .

˜ Step 1.d: Assemble λIm := G> 2 H2 e, d := P1 (d − FλIm ). ˜ on Ker G2. Step 1.e: Solve P1 FλKer = d Step 1.f: Assemble λ := λIm + λKer . Step 2:

Assemble α := H1 G1 (d − Fλ).

Step 3:

Assemble u := A† (f − B> 1 λ) + Nα.

• an iterative projected Krylov subspace method for non-symmetric operators can be used in Step 1.e


OUTLINE



Find

λ ∈ Rm so that Fλ = d, where d ∈ Rm .

Algorithm BiCGSTAB [, λ0 , F, d ] → λ Initialize: r0 := d − Fλ0 , p0 := r0 , ˜r0 arbitrary, k := 0 While krk k >

1◦ 2◦ 3◦ 4◦ 5◦ 6◦ 7◦ 8◦ 9◦ 10◦

˜k := Fpk p αk := (rk )>˜r0/(˜pk )>˜r0 ˜k sk := rk − αk p ˜sk := Fsk ωk := (˜sk )>sk /(˜sk )>˜sk λk+1 := λk + αk pk + ωk sk rk+1 := sk − ωk˜sk βk+1 := (αk /ωk )(rk+1)>˜r0/(rk )>˜r0 ˜k ) pk+1 := rk+1 + βk+1 (pk − ωk p k := k + 1

end (Van der Vorst, 1992) •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Find

˜, where d˜ ∈ Ker G1. λ ∈ Ker G2 so that P1 Fλ = d

˜] → λ Algorithm ProjBiCGSTAB [, λ0 , F, P1 , P2 , d ˜ − P1Fλ0, p0 := r0, ˜r0 arbitrary, k := 0 Initialize: λ0 ∈ Ker G2 , r0 := d While krk k >

1◦ 2◦ 3◦ 4◦ 5◦ 6◦ 7◦ 8◦ 9◦ 10◦

˜k := P1Fpk p αk := (rk )>˜r0/(˜pk )>˜r0 ˜k sk := rk − αk p ˜sk := P1Fsk ωk := (˜sk )>sk /(˜sk )>˜sk λk+1 := λk + αk P2 pk + ωk P2 sk rk+1 := sk − ωk˜sk βk+1 := (αk /ωk )(rk+1)>˜r0/(rk )>˜r0 ˜k ) pk+1 := rk+1 + βk+1 (pk − ωk p k := k + 1

end •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

˜, with λ0 ∈ Ker G2. Formally solve P2 P1 Fλ = P2 d ˜] → λ Algorithm ProjBiCGSTAB [, λ0 , F, P1 , P2 , d ˜ − P2P1Fλ0, p0 := r0, ˜r0, k := 0 Initialize: λ0 ∈ Ker G2 , r0 := P2 d While krk k >

1◦ 2◦ 3◦ 4◦ 5◦ 6◦ 7◦ 8◦ 9◦ 10◦

˜k := P2P1Fpk p αk := (rk )>˜r0/(˜pk )>˜r0 ˜k sk := rk − αk p ˜sk := P2P1Fsk ωk := (˜sk )>sk /(˜sk )>˜sk λk+1 := λk + αk pk + ωk sk rk+1 := sk − ωk˜sk βk+1 := (αk /ωk )(rk+1)>˜r0/(rk )>˜r0 ˜k ) pk+1 := rk+1 + βk+1 (pk − ωk p k := k + 1

end •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

OUTLINE



Consider a family of nested partitions of the fictitious domain Ω with stepsizes:

hj , 0 ≤ j ≤ J • the first iterate is determined by the result from the nearest lower level • the terminating tolerance on each level is := νhpj Algorithm: Hierarchical Multigrid Scheme 0,(0)

(0)

Initialize: Let λKer ∈ Ker G2 be given. p

0,(0)

(0)

(0)

(0)

(0)

˜ ] → λKer . ProjBiCGSTAB[νh0 , λKer , F(0) , P1 , P2 , d For j = 1, . . . , J,

1◦

(j−1) ˜ 0,(j) prolongate λKer → λ Ker

2◦

˜ Ker → λKer := P2 λ ˜ Ker project λ

3◦

˜ ] → λKer ProjBiCGSTAB[νhj , λKer , F(j) , P1 , P2 , d

0,(j)

0,(j) p

(j)

0,(j)

0,(j) (j)

(j)

(j)

(j)

end (J)

Return: λKer := λKer . •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Motivation u∗ . . . exact solution of PDE problem u . . . FEM approximation with respect to h with the convergence rate p

ku∗ − uk ≤ Chp,

Au = f

uk . . . the k -th iteration uk −→ u,

Auk = f + rk

When should be iterations terminated? krk k ≤ ,

=???

ku∗ − uk k ≤ ku∗ − uk + ku − uk k ≤ Chp + kA−1rk k ≤ Chp + kA−1k · ≤ (C + kA−1kν)hp

if := νhp

Control parameter ν may by choosen experimentally;

ν ≈ KC/kA−1k.


OUTLINE



Circulant matrices and Fourier transform

a1  a2  A =  a3  ... 

an a1 a2

 . . . a2 . . . a3  2 n−1 . . . a4   = a, Ta, T a, · · · , T a . . . ... 

.. .

an an−1 . . . a1 Z d T k f (ω) =

f (x − k)e−ixω dx = e−ikω fb(ω)

R

XA = (Dx0 , Dx1 , Dx2 , · · · , Dxn−1 ) = DX Lamma: Let A be circulant. Then A = X−1 DX, where X is the DFT matrix and D = diag(b a), b a = Xa, a = A(:, 1). •First •Prev •Next •Last •Go Back •Full Screen •Close •Quit

Multiplying procedure:

0◦ ◦

1 2◦ 3◦ Multiplying procedures:

A† v := X−1 D† (Xv)

. . . Moore-Penrose

d := fft(a) v := fft(v) v := v. ∗ d−1

  

O(2n log2 n)

  A† v := ifft(v) Nα,

N> v

(and

Mα, M> v)

As AN = 0, the matrix N may be formed by eigenvectors corresponding to zero eigenvalues. I − DD† = diag(1, 1, 1, 0, . . . , 0) =⇒ X−1 = (N, Y), X−1 =

Therefore we can define the operation:

1◦ 2◦

vα := ind(α) Nα := ifft(vα )

1◦ 2◦

ind(α) =

α 0

>

N Y

v := ifft(v) N> v := ind−1 (v)

∈ Rn O(n log2 n)


Kronecker product of matrices: Ax ∈ Rnx ×nx , Ay ∈ Rny ×ny

 Ax ⊗ Ay = 

Lemma 1:

ay11Ax . . . ay1ny Ax



ayny 1Ax . . . ayny ny Ax



.. .

...

.. .

(Ax ⊗ Ay )(Bx ⊗ By ) = AxBx ⊗ Ay By (Ax ⊗ Ay )† = A†x ⊗ A†y N = Nx ⊗ Ny

Lemma 2:

(Ax ⊗ Ay )v = vec(AxVA>y ), where V = vec−1(v).  

V = (v1 , . . . , vny ) ∈ Rnx ×ny

⇐⇒

v1 vec(V) =  ...  ∈ Rnxny vn y


Kronecker product and circulant matrices:

Let Ax , Ay be circulant then:

A = Ax ⊗ Iy + Ix ⊗ Ay −1 −1 −1 = X−1 x Dx Xx ⊗ Xy Xy + Xx Xx ⊗ Xy Dy Xy −1 = (X−1 x ⊗ Xy )(Dx ⊗ Iy + Ix ⊗ Dy )(Xx ⊗ Xy ) = X−1DX

with

X = Xx ⊗ Xy D = Dx ⊗ Iy + Ix ⊗ Dy

(DFT matrix in 2D) (diagonal matrix)

where Xx , Xy are the DFT matrices, Dx = diag(Xx ax ), Dy = diag(Xy ay ) and ax = Ax (:, 1), ay = Ay (:, 1), respectively.


Multiplying procedure:

0◦

A† v := X−1 D† (Xv)

dx := fft(ax ), dy := fft(ay ) V := vec−1 (v)

1◦ 2◦ 3◦ 4◦ 5◦

V := fft(V) V := fft(V> )> V := vec−1 (D† vec(V)) V := ifft(V) V := ifft(V> )> A† v := vec(V)

Number of arithmetic operations :

O(2n(log2 nx + log2 ny ) + n) ≈ O(n log2 n), Multiplying procedures:

Nα, N> v, Mα, M> v

n = nx ny

. . . analogous


OUTLINE



CONCLUSIONS

• The method for solving non-symmetric sadddle-point systems with singular diagonal blocks was presented. It combines the Schur complement reduction with the null-space method.

• It can be understood as a generalization of the algebraic description of FETI DDM for non-symmetric and possibly indefinite cases.

• In connection with FDM, it presents the highly efficient solver for solving separable PDE problems. The fast implementation based on the Poissonlike solver is ”matrix free” as the stiffness matrix is not needed to be formed explicitly.


REFERENCES J. Haslinger, T. Kozubek, R.K., G. Peichel (2007): Projected Schur complement method for solving non-symmetric systems arising from a smooth fictitious domain approach, acceptted in Num. Lin. Algebra Appl.. Farhat, C., Mandel, J., Roux, F., X. (1994): Optimal convergence properties of the FETI domain decomposition method, Comput. Meth. Appl. Mech. Engrg., 115, 365–385. R. Glowinski, T. Pan, J. Periaux (1994): A fictitious domain method for Dirichlet problem and applications. Comput. Meth. Appl. Mech. Engrg. 111, 283–303. R. K. (2005): Complexity of an algorithm for solving saddle-point systems with singular blocks arising in wavelet-Galerkin discretizations, Appl. Math. 50, 291–308. H. A. Van der Vorst (1992): BiCGSTAB: a fast and smoothly converging variant of BiCG for solution of nonsymmetric linear systems, SIAM J. Sci. Statist. Comput., 13, 631–644. G. Meurant(1999): Computer solution of large linear systems, North Holland. M. Benzi, G. H. Golub, J. Liesen(2005): Numerical solution of saddle point systems, Acta Numerica, pp. 1–137.


An Algorithm for Solving Non-Symmetric Saddle-Point Systems

An Algorithm for Solving Non-Symmetric Saddle-Point Systems

Suggest Documents

An Iterative Method for Nonsymmetric Systems

An Optimization Algorithm for Solving Systems of Singular Boundary ...

An Algorithm for Solving Quadratic Programming

An Algorithm for Solving the Shortest Path

An Infeasible Interior-Point Algorithm for Solving

An Approximate Algorithm for Solving the

AN INDEPENDENT LOOPS SEARCH ALGORITHM FOR SOLVING ...

An efficient meta-heuristic algorithm for solving

On solving strong multistage nonsymmetric ... - Semantic Scholar

A self-concordant interior point algorithm for nonsymmetric circular ...

A Branch-and-Prune Algorithm for Solving Systems of Distance ...

The Finite Termination Property of an Algorithm for Solving the ...

GENACE: An Efficient Cultural Algorithm for Solving the Flexible Job ...

An Efficient Two-Objective Hybrid Local Search Algorithm for Solving ...

An Adaptive Local Search Based Genetic Algorithm for Solving Multi ...

An algorithm for solving nonsmooth variational inequalities arising in ...

An Iterative Algorithm for Solving Constrained Decentralized Markov ...

An algorithm for symbolic solving of differential ... - Google Sites

An Efficient Algorithm for Solving the Dyck-CFL

An Algorithm for Solving Initial Value Problems Using ... - CiteSeerX

An elitist teaching-learning-based optimization algorithm for solving ...

An Algorithm for Solving the System-Level Problem in Multilevel ...

An Algorithm Development Environment for Problem-Solving-Software ...

A Graphical Algorithm for Solving an Investment Optimization Problem