Partitioning Sparse Matrices with Eigenvectors of ... - SNAP: Stanford

Partitioning Alex

Sparse Pothen

Matrices

1, Horst Report

Ames

RNR-89-009,

Systems

Research Moffett

first

and

third

author

July

98128

Paul

Liu

1989

Center,

Mail

Stop

T-045-1

CA 94035

25, 1989

are with

the

Computer

Science

Pennsylvania State University, University Park PA 16802 2The second author is an employee of Boeing Computer WA

of Graphs

Division

Field, July

1The

Eigenvectors

D. Simon 2, and Kang-Pu

NAS NASA

with

Department,

Services,

The

Bellevue,

PARTITIONING

SPARSE

MATRICES

WITH

EIGENVECTORS

OF

GRAPHS ALEX

POTHEN.,

HOILST

SIMONt

AND

KANG-PU

PAUL

LIOU$

Abstract. The problem of computing a small vertex separator in a graph arises in the context of computing a good ordering for the parallel factorization of sparse, symmetric matrices. An algebraic approach to computing vertex separators is considered in this paper. It is shown that lower bounds on separator sizes can be obtained in terms of the eigenvalues of the Laplacian matrix associated with a graph. The Laplacian eigenvectors of grid graphs can be computed from Kronecker products involving the eigenvectors of path graphs, and these eigenvectors can be used to compute good separators in grid graphs. A heuristic algorithm is designed to compute a vertex separator in a general graph by first computing an edge separator in the graph from an eigenvector of the Laplaeian matrix, and then using a maximum matching in a subgraph to compute the vertex separator. Results on the quality of the separators computed by the spectral algorithm are presented, and these are compared with separators obtained from Automatic Nested Dissection and the Kernighan-Lin algorithm. Finally, we report the time required to compute the Laplacian eigenvector, and consider the accuracy with which the eigenvector must be computed to obtain good separators. The spectral algorithm has'the advantage that it can be implemented on a medium size multiprocessor in a straight forward manner. AMS(MOS)

subject

Keywords. separator,

parallel

1.

in

matrix

to employ

whose in the

matrices

needs

to find

a vertex A,

B,

with

of large,

65F15,

Laplacian

68R10.

matrix,

sparse

the

is employed

strategy sparse

processor,

matrix,

into

number

in the

for and

of vertices two

nearly

the

vertices

well-known

systems that

to compute

a set

graph

recursively

such

required

strategy

Find

definite

matrix

storage

of a single One

paradigm:

and

positive

of the

the

capacities

disconnects last,

sparse,

an ordering problems,

conquer

vertex

Nested

on

it can

the

be

parallel factored

structure

this

of

parallel

a good

adjacency

equal

parts.

in the

ordering

parallel

in the

Dissection

order graph

Number

two

the

parts

is of the

by

algorithm

the for

factorization.

an ordering to

65F05,

spectra,

in parallel.

removal

for

computing

problem

large

storage

and

This

solution

computed

separator

strategy.

ordering

parts

divide

graph

to compute

the

to be

the

matrix,

vertices

In

the

For

exceed

need

65F50,

algorithms.

In

parallel.

may will

same

reordering

it is necessary

efficiently

itself

partitioning,

Introduction.

computers,

the

graph

classifications:

by the

be solved.

Given

separator

S

nearly

equal

such

above

approach,

an adjacency that

numbers

S

has

at

graph few

vertices

each

step,

the

G of a sparse and

following matrix,

S disconnects

Partitioning this

problem

G \ S into

is two

of vertices.

* Computer Science Department, The Pennsylvania State University, Whitmore Lab, University Park, PA 16802. Electronic address: [email protected], [email protected]. The research of this author was supported by National Science Foundation Grant CCR-8701723 and U.S. Air Force Office of Scientific Research Grant AFOSR-88-0161. t Numerical Center, Moffett

Aerodynamic Simulation Field, CA 94035. The

address: simon_orville.nas.nasa.gov Computer Science Department, 16802.

(NAS) author

Systems Division, Mail-Stop 258-5, is an employee of Boeing Computer

The Pennsylvania

State

University,

Whitmore

NASA Ames Research Services. Electronic

Lab,

University

Park,

PA

In this paper, associate the

we consider

with the given

Laplacian

joining from

A _ and

that

to initially

separator

spectral

are worthy First,

of a vertex,

neighbors

about

one part,

and

discrete

choice

be used

to move

(zero

in the

with

the

to.

other

or one)

part.

to belong from

of the separator

the dominant

algorithms

in a fairly

operations,

straight

This

The

about

employs

global

components.

in the

graph

that

are

approaches. about

weights

a vertex

which

below

the

in the graph

part in the initial

median

method,

weights

each

weight

vertex

in the

spectral

if a slightly

different

forward

form

makes

method

a

can

partition

is an eigenvector

the

new

is

of the computation

Becasue

of its algebraic

most

is well suited

nature

standard

the algorithm

multiprocessors

computations

for vector

from

is based on standard

on medium-grain

of the

computation

algorithm

Most manner

since

method

distinguishes

numbers.

Furthermore, this algorithm

with

or the

eigenvector

in which

+1 and -1,

in the spectral

algorithm. point

computing.

as an approach

Dissection

algorithm.

computation

or similar

by previous

to one set.

on floating

used in

are also vector

supercomputers

is

floating

used for large

scale

computing.

This

paper

properties

is organized

of Laplacian

earlier

as follows.

matrices

work on computing

In § 3, we obtain of the eigenvalues

lower

bounds

the second

employs

show

the spectra

that spectra

of path

in § 4. We proceed computed

from

to compute Results

to show

then

uses

about

the

the proof

and square

how good

a maximum quality

graph

and vertex

graphs. matching

of the

separators

Two

The

spectral

in § 2.

We also

vertex

separators

for

minimax

criterion,

and

theorem.

in a subgraph computed

and

in the grid

graphs

spectral

can

be

algorithm

computes

to compute

the

algorithm

from

products

initially by the

_Ve then

explicitly

Kronecker

our heuristic

algorithm

separators

techniques

can be computed products

ma-

different

of the Wielandt-Hoffman graphs

on the

of the adjacency

size of the best

matrix.

In § 5, we describe

in general

partitioning

Courant-Fischer

grid

suitable

edge

information.

separators

on the

uses the

by employing

the spectral

vertex and

graphs

One

material

the eigenvectors

of the Laplacian

from

of rectangular

to graph

from

lower bounds

are illustrated:

an inequality

background

relevance

edge separators

of a graph

in terms

We include

and their

trix in this section.

rator.

of edges

information

method

from

one part to the other,

operations

separator,

previous

viz.

separators

In the Kernighan-Lin

vector

the

set

and

S is computed

as Nested

spectral

a separator

All vertices

computationally.

proving

called

matrix,

separator

it from

such

The

between

algorithms

review

a matrix

A _, B _. The

in the graph,

of finding

method

a weight

theoretical

scientific

distinguish

obtained

graph

point

sets

A vertex

separators,

potential

the spectral

a few vertices

course

a Lanczos

scientific

G.

that

it computes

the

to belong

the rest,

parallelizable

into two

separators.

the separators

choice,

it is going

since has

from

we can view

Third,

Laplacian

use of local information

graph,

different

partition

by

graph),

of the

We

technique. features

to compute

method

a continuous

desired

(and its adjacency

vertices

for computing

make

the

spectral

Second, makes

problem.

eigenvector

in the graph

has three

algorithms

the

qualitatively

the

the partitioning

of comment.

previous

the

matrix

partition

by a matching

algorithm

Thus

for solving

a particular

separator

algorithm

Kernighan-Lin information

algorithm

symmetric

We compute

B _ is an edge

the edge This

sparse,

matrix.

use its components

a spectral

an edge

vertex

sepa-

is presented

in § 6. by

In this

section,

Automatic

obtained

Nested

recently

the Laplacian vector and

2.

Dissection

by Liu good

directions

Background.

Let matrix

otherwise.

By convention,

associated

with

Let

d(v) Let

a graph

denote

el,,,,, = d(v).

the

The

the

incidence

with

edges

of the

are

separators

Kernighan-Lin and

with

separators

algorithm,

Lewis

algorithm

addressed

[29].

The

and the

in § 7. The

as well

computed

as with

results

time

required

to compute

accuracy

needed

in the eigen-

final

§ 8 contains

our

conclusions

work.

G :

(V,E)

be an undirected

has

a_.,_ is zero, are indexed

element

graph

and

= D -

G be directed graph.

arbitrarily,

The

IV I x

and columns

of the

matrices

their

order

being

n x n diagonal

and let

if v is the head

The

and

matriz

IEI matrix

vertices. E E,

D to be the

A is the Laplacian

-

if (v,w)

of the graph,

defne

IvI =

on

to one

for all v E V. The rows

of a matrix,

directed

graph

a_.,_ equal

by the vertices

Q = Q(G)

of the

matrix

the

spectral

Leiserson

A = A(G)

degree

matrix

the

the Lanczos

separators for future

n x n adjacency

and

[31] and

eigenvectors

to obtain some

we also compare

zero

arbitrary.

matrix

with

of G.

C denote

C has

the

vertex-edge

elements

of e

if v is the tail of e +1 _-i otherwise.

c.,,,e = It is easy edges

to verify

[7] that

in C. The spectral

Q(G)

properties

= CC t, and

of Q have

been

z_tOg

that

studied

Q is independent by several

of the

authors.

direction

of the

Since

x_CC%_

=

= =

Z

-

(v,,.)eE it is easily

seen

that

Q is positive

semidefiuite.

Let the eigenvalues

of Q be ordered

$1 = 0 __ O, define Similarly,

for

investigated

of his results

Let G be a connected

A2. For

of the second

V_(r)

graph,

and

and the

eigenvalue relates

of interest

let y be an

a corresponding vertex

of G generated

in this

paper

eigenvector Then

is the

and by

edge the

following.

corresponding

the subgraph

r -r}.

a real number

A2 and

to

induced

= {v E V : y_,
O},

N

=

{vsV:w 0}, with n/2

the

The

proof

sign of the

of the

upper

eigenvector

bound

chosen

such

makes that

use

W has

of no

vertices.

relationship by several

of these

of the authors;

Laplacian two

spectrum

recent

to several

survey

articles

by

graph

properties

Mohar

[34] and

have Bien

been

con-

[6] describe

results.

Spectral

methods

for

computing

edge

[12] obtain

lower

separators

have

been

considered

by

several

re-

searchers. Donath

and

Hoffman

of the eigenvalues

of the

matrix

property

that

matrix

with

with weights

the

on the edges,

of the eigenvectors Here,

A is the

of A + D sum

and

A + U, where trace(U) obtain

corresponding

weighted to zero.

adjacency They

bounds

A is the

= -2[E[. lower

to the matrix, formulate

on the

bounds

algebraically and the 4

size of an edge

adjacency

Barnes

matrix,

and

on the weights largest

separator and

Hoffman of edge

separators of a matrix

D is an n x n matrix

such

partitioning

as

problem

U is a diagonal

[4] consider

eigenvalues

in terms

that

graphs in terms A + D.

the

elements

a quadratic

integer

programming problems,

problem, whose

Barnes number

solutions

[3] shows

of edges

problem,

a transportation

the

3.

class

Lower

of the in the

which

Bounds.

shortest

path

less than

solution

he proves

used

A) the

from

the

George

matrix

edges

with

similarly.

and

in A.

Let

The vertex

from

the

Gilbert set

endpoint

disjoint

sets

other. p from

the

by solving

k algebraically

2.

A,

of

for almost

all graphs

separators

in terms

of vertex The

apply

and

with the

degree

of vertices

is a lower

bound

A and B which

Let S denote

lower

bounds

to the smallest

i.e.,

the

fewest

the set of vertices

hold for separator

between

A from

have

number which

of edges

in a

are at a distance

been

B.

used

A and

B,

p(A,B)

If p = 2, we get the in sparse

matrix

= p. commonly

algorithms

by

endpoints

other

in S.

The

to work

in A, sets

and EAs

with

EB,

Es,

denote

and Ess

the fractional denoted

sizes

by d(v),

the are

set

vector

of

defined

a = IAI/n, and

A win

any

pair

in G.

are

size of a wide

at a distance subsets

p from

of vertices

of vertices

not

separator each

separating

of G which

belonging

>0,

of all ones

are at a distance

to A which

p > 2

are at a distance

where/3=(A/A2)+p2a-1.

and

all zeros,

of

other.

Then

Let _e, 0 be the

If

[22].

both

on the

the set

of its vertices.

A) < p}.

separates

and Zmijewski

Let A, B be disjoint A.

and eigenvectors

and let A be a subset

distance

it wiU be convenient

s 2+_s-p2a(1-a)

Proof:

0, then

of edges

in A,

sizes

= ISl/n. The degree of a vertex v win be

maximum result

less than

#

v from

e v \ A:

that

first

each

The part,

to A. Hence

separators

TrlEOREM

to the

eigenvalues

bounds

n vertices,

S denote

not belonging

if B

In the following, the

P. same

problem

size bisection

on the

these

IVl =

Wide

b - IBI/n, and denote

matrix to the

the

G to be connected. on

separator

denote

the

in this section.

in particular,

of separators.

one

approximation

by a partition

using

Q(G)

set S is a wide

EA

to a matrix

corresponding

bounds

of a vertex

v to a vertex

and Ng [21], and

Let

such that

i and j belong

will find a minimum

graph

distance A,

method

lower

a graph

B = V \ (A U S); notion

programming

graphs.

denote

p > 2 from

p > 2, the

linear

to this approximation

eigenvectors

a spectral

s = Define

by

into k > 2 parts

is equivalent

is approximated

the

Laplacian

We assume

by p(v,

a graph

to one if vertices

a heuristic

in the graph;

G = (V, E)

Denote

matrix

We obtain

of the

separator

graph.

Let

problem

bounds.

is minimum

involving

of random

eigenvalues

any vertex

latter

of A.

matrix

in a certain

the

of partitioning

Pld equal

[9] has described

the adjacency

lower

adjacency

He finds

eigenvalues

the

by the partition

problem

Boppana

approximate

the problem

P has element

and zero otherwise.

largest

then

yield

that

cut

in which

n × n matrix

and

respectively.

The

Courant-Fischer

minimax

principle

states

that =

_2

rain

z_JQz_

e_'z_=o

=

(I)

rain E(+,_)_s(z+- zj) _ _• _0_ E;'=_ _+ e_t z._=O

Using

the Lagrange

identity

2 --

n

X i

Xi

(z+ -- =j)2,

-:

i=1

Fiedler

i,j= l i-4 Z (_,- _)_. (i,j)EE

We prove

the result

Choose

the

v-th

by making

i,jEV i_ A2 ((I -- 8)8 q- p2a(1

this yields that

the desired

[33] proved

graphs

except

that

the

-

adjacency

A/A2,

hence

and

for all graphs

complete

for all the

graph,

graphs

3.

larger

1)) rain

{d(v)

except

the

ratio

A/)_2

the

of sparse

j3, is much

COROLLARY

result.

matrices

than

in Theorem

sl < 82. Then

__ 2p(a(1

-

It remains value

a)) 1/2, then to verify

1/2 when The

second p and

exhibits

the

size.

corollary

the fractional

The

common

inequality

some

+ p2a-

equation

number.

partitions,

Indeed, the

ratio

1"

corresponding

to the inequality

- a)) 1/2)

the rhs in power of the series

expansion

dependence

the ratio also shows

is valid

of vertex

A/A2,

series

yields

the result.

Since (a(1 - a))in

coronary.

separator

and the smaller

the dependence

when sizes

its maximum

73 __ p. • on A2: the smaller

the lower

on the lower

has

bound bound

the

on the vertex on the distance

size of the set A. situation

of a separator

corresponds

to p = 2. In this case,

the

quadratic

becomes s 2+fls-4a(1-a)>_O,

After

computed

for all

p2_(1-_)

+ (732 -b 4p_a(1

the condition

the larger

The

A > A2. Thus

s _> 82, and

expanding

eigenvalue,

separator

Kn,

one.

of the quadratic

0 __ a g 1, the power

corollary

we have

(A/_2)

1 (-/7

82_

If/3

graph

If fl > p, then

sl, 82 be the roots

2, with

: v • V}.

>_ 1, and fl is a positive

that

/7

Let

•

complete

8 >__ p:a(1- _) _

Proof:

s)).

A2 satisfies )_2 _ (n/(n

Mohar

-a-

simplification,

and Milman [11 is equivalent the lower bound

it can

with_=(A/A2)+4a-1. be seen

to the

above

s >

that

the

inequality

inequality.

4a(1

- a)

- (A/4) + 4a 7

In this

-- i"

in Theorem case,

when/3

2.1 of Alon,

Galil,

>_ 2, we obtain

Mohax the

(Lemma

ratio _,,/_2. We can also

equation

(3),

2.4, obtain

and

lower

replacing

disjoint

Alon

Milman

and

bounds

the

IEI for two vertex

obtained a lower bound

[32]) has

sum

[EA[ -

sets

on

]Esl

> ),,n

techniques

of vertices

obtained

can also be used

from each

A second influencing ttoffman

the

bound.

size of vertex

to bound

the number

in

IEBI, we have

,

p2 )_2nab

=

(a + b)"

of edges

denote

number

of edges

a lower

The

previously

the

d(v)

the

obtain

separators.

IAI > IBI > ISI. Let

separating

two disjoint

sets

technique

used

the

Hoffman

G into

v, and

on v with

exhibits

is derived

and

graph

of a vertex

incident

that

used

by Donath

separates degree

bound

the

two

let i(v)

other

from

another the

factor

Wielandt-

[12] to obtain sets denote

endpoint

A and the in the

lower

B,

with

'internal' same

set

_.

Recall

that

the eigenvalues

of the Laplacian

matrix

_1 = 0 < _2 _< _3... Let the n × n matrix Jc are

similarly

J = diag(Ja,

defined.

The

Jb, Jc), where

eigenvalues

Q are ordered

THEOREM with

4.

Let S be a vertex

IAI > IB[ _> IS[.

as

(1 - a))_2 - 2A -- ($z --)_:)"

Proof:

and

of J are

_tl = na > tz2 = nb > tz3 = ns > l_4 ....

(5)

step

p apart.

((i/a) + (lib))

that

aS

last

of

the inequality

We now

and has been

of v, i.e.,

the

IEAI-

(a + b) 2 + p2ab)

p2 )_2n

bounds on edge separators. Let S be a vertex separator degree

+ b) -

by IEI-

in terms

other.

lower

theorem,

((a

separators

By omitting

+ IEs[

axe a distance

IEI-IEAI-IEBI> These

separators.

lEAs[ + IEBsl

A, B which

[2] have

edge

on vertex

J)

_ _ i=1

8

A,#_.

[26] (see

also

[121),

A,

B,

We now compute The

both

right-hand

sides

side

of the

above

inequality.

is n

=

na.O+nb.)_2+ns.)t3

=

nbX2 +nsX3

=

n(1

i=I

(6) To evaluate

the left-hand

side,

- a-

we partition

the

symmetric

o Q_ Qb. Q.. OQ..)

Q

Qi,

trace(Q

s)X2 + nsXs.

Q_.

matrix

Q to conform

to J:

•

Q-

= trace( QooJo) + trace(Q_Jb) + trace( Q.J. )

J)

(_

+ _-_ + _)

yEA

d(v)

yES

yES

- i(v)

- 2(IEI- IEAI- IEBI- IEsl) _< 2(IEI- IEAI- IEBI) (7)

= Substituting

the

2nsA.

inequalities

(6) and (7) in (5), 2nsA

After

some This

rearrangement

last

lower

A2 influences The

the

bound

lower

bounds

are large

Donath

and

partition matrix only

when

this

bound; when

by the

a word

error

[44] (cited

analysis

for certain

accompanying 4.

same

Partitions

of grid

matrix

obtained

nested

dissection

(ND)

that

lower

the

'gap'

bounds into

as before

that

between

one

treats

[23]). will result graphs.

about

lower

They

the

Xz and

In this

to find good

magnitude

A2 has

of

an effect.

lower

bounds

of the

section

bounds.

cut

by the

eigenvalues

matrix

error

obtained

are

however, best

we show and

These

on the

sizes for the

edge

in terms

of edges

of a

constrained

sum to -2[El. bound

to the

number

U is a diagonal

do illustrate,

in large

will be identical

and

these

an upper

The

on the k sets,

matrix elements

is in order

scheme.

•

size shows

is partitioned

can be used

separators

+ nsXs.

gap is large.

of graphs.

A/A2,

vertex

separator

its diagonal

in

classes

small

the Laplacian

way

s)X2 result.

A is the adjacency that

the

except

the

of caution

be considered

final

it also shows

set of vertices

requirement

the

[12] obtained

M = A + U, where

Finally,

yields

on a vertex

Hoffman the

> n(1 -- a--

we obtain

vertex

separators

in an a priori not

that

the

separators at the

should roundoff

likely to be tight, a large

separators

that

bounds

),2, with

an

in a graph.

second

eigenvector

in grid graphs.

of The

first level in a theoretical

Fic_ile_ vec_r of the pal:h 0.3

0.2

0.I

8

_

o

•_

.0.1

-0.2

.0.3 10

0

1 verlex

20

25

FIG. 1. The second Laplacian eigenveetor

To compute We will show of the

separators

that

Laplacian

the grid

graphs

in terms

As

5.

The

its i-th

( cos((/-

from

shown

zt denote

the

ponents with

less than

the

other

If z, denotes is a measure

in Fig.

into

on n vertices. For concreteness, We number the vertices of the

we path

graph.

Cn --- _r/n.

fork=

(1/2)(k-

We denote

1,...,n,

1)¢.