Multilevel Algorithms for Wavefront Reduction

RAL-TR-2000-031

Multilevel Algorithms for Wavefront Reduction by Y.F. Hua and J.A. Scottb

Abstract Multilevel algorithms are proposed for reordering sparse symmetric matrices to reduce the wavefront and pro le. A graph representation of the matrix is used and two graph coarsening methods are investigated. A multilevel algorithm that uses a maximal independent vertex set for coarsening and the Sloan algorithm on the coarsest graph is shown to produce orderings that are of a similar quality to those obtained using the best existing combinatorial algorithm (the hybrid Sloan algorithm). Advantages of the proposed algorithm over the the hybrid Sloan algorithm are that it does not require any spectral information and is signi cantly faster, requiring on average half the CPU time.

Keywords: Wavefront and pro le reduction, multilevel algorithm, row ordering, frontal method.

a Computational Science and

Engineering Department, Daresbury Laboratory, Warrington WA4 4AD, England.

b Computational Science and Engineering Department, Atlas Centre, Rutherford

Appleton Laboratory, Oxon OX11 0QX, England. August 31, 2000.

CONTENTS

1

Contents

1 Introduction 2 Background

1 2

3 The Multilevel Ordering Algorithm

5

2.1 De nitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 The Sloan algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.3 The Hybrid algorithm . . . . . . . . . . . . . . . . . . . . . . . . 4 3.1 The Multilevel Approach . . . . . . . . . . . . . 3.1.1 The coarsening phase . . . . . . . . . . 3.1.2 The coarsest graph ordering . . . . . . . 3.1.3 The prolongation and re nement phase 3.1.4 The multilevel algorithm . . . . . . . . .

4 Numerical results

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

4.1 Multilevel algorithm with coarsening by edge collapsing . . . . . 4.2 Multilevel algorithm with coarsening based on the maximal independent vertex set . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Sloan versus Hybrid on coarsest graph . . . . . . . . . . . . . . . 4.4 Sensitivity of the multilevel algorithm . . . . . . . . . . . . . . .

5 Conclusions and Future Work 6 Acknowledgements A The test problems

5 6 8 8 9

14 15

17 20 24

27 28 31

1 INTRODUCTION

1

1 Introduction We consider multilevel algorithms for ordering sparse symmetric matrices for small wavefront and pro le. The resulting ordering may be used to construct a row order for use with the row-by-row frontal method applied to a matrix with a symmetric sparsity pattern (see [22]). Since we are primarily concerned with matrices that are positive de nite, we work only with the pattern of the matrix and do not take into account permutations needed for stability. In cases where the matrix is non-de nite, or is symmetric only in its sparsity pattern, the actual factorization may be more expensive and require more storage. Minimizing the pro le of a matrix is known to be an NP-complete problem [18]. A number of heuristic algorithms have been proposed, including the Cuthill-McKee [4], Reverse Cuthill-McKee [6, 19], Gibbs-King [8], Gibbs-PooleStockmeyer [7], and Sloan [26] algorithms. More recently, spectral orderings based on the Fiedler vector of the Laplacian matrix associated with A have been developed [1, 20, 21]. Kumfert and Pothen [17] propose combining the Sloan algorithm with the spectral ordering. The resulting hybrid Sloan algorithm (hereafter referred to as the Hybrid algorithm) has been shown to give much better orderings for large problems than either the spectral method or the Sloan method alone. This has been con rmed by Reid and Scott [22], who implemented the Sloan and Hybrid algorithms within the Harwell Subroutine Library [11] code MC60. One reason for the success of the Hybrid algorithm is that the spectral algorithm takes a global view of the graph of the matrix. This global view is fed into the Sloan algorithm as a priority vector, and the Sloan algorithm then performs local optimizations. The spectral algorithm has also been used in the area of graph partitioning [9, 25]. More recently, researchers have found that, for large graphs, a multilevel approach [2, 10, 13, 27] can provide an equally good global view, while being much faster because the calculation of the spectral vector of a large matrix is avoided. A number of ecient and high-quality graph partitioning codes based on the multilevel approach have been developed [10, 13, 27]. The success of the multilevel approach in graph partitioning motivated the work reported in this paper. We propose a multilevel algorithm for the ordering of sparse symmetric matrices. Numerical tests on a wide range of problems from dierent application areas con rm that our multilevel algorithm yields orderings of comparable quality to the Hybrid algorithm. Moreover, our algorithm does not require any spectral information and is signi cantly faster than the Hybrid algorithm. So far as the authors know, the only other attempt at using a multilevel approach in this context has been by Bolman and Hendrickson [3]. Their algorithm combined a multilevel approach with a 1-sum local re nement procedure. However, they were not able to consistently improve on the quality of the original Sloan algorithm. This paper is organised as follows. In Section 2, de nitions and terminology are introduced, and the Sloan and Hybrid algorithms are recalled. In Section 3, our multilevel approach is presented. Numerical results comparing our method

2 BACKGROUND

2

with the Sloan and the Hybrid algorithms are given in Section 4. Section 5 summarizes our ndings and considers future directions for research.

2 Background 2.1 De nitions

We rst need to introduce some nomenclature and notation. Let A = faij g be an n  n symmetric matrix. At the i-th step of the factorization of A, row k is said to be active if k  i and there exists a column index l  i such that akl 6= 0. The i-th wavefront fi of A is de ned to be the number of rows that are active during the i-th step of the factorization. The maximum and root-mean-squared (RMS) wavefronts are, respectively,

F (A) = 1max ff g in i and

Pn f ! 12 rmsf (A) = i n i : =1

2

The pro le of A is the total number of entries in the lower triangle when any zero ahead of the rst entry in its row is excluded, that is,

P (A) =

n X

max fi + 1 ? j g:

i=1 aij 6=0

It is straightforward to show that

P (A) =

n X i=1

(1)

fi:

For a frontal solver these statistics are important because  the memory needed to store the frontal matrix is F 2  P is the total storage needed for the factorised matrix  the number of oating-point operations when eliminating a variable is proportional to the square of the current wavefront size. Our goal therefore is to construct an ecient ordering algorithm that reduces the above quantities. It is often convenient when developing ordering algorithms to treat the matrix A in terms of its adjacency graph. An undirected graph G is de ned to be a pair (V; E ), where V is a nite set of vertices (or nodes) and E is a nite set of edges de ned as unordered pairs of distinct vertices. In a weighted graph, each vertex and edge has a weight associated with it. The adjacency graph G (A) of the square symmetric matrix A comprises the vertices V (A) = f1; 2; : : : ; ng and the edges E (A) = f(i; j ) j aij 6= 0; i > j g :

2 BACKGROUND

3

Two vertices i and j are said to be neighbours (or to be adjacent) if they are connected by an edge. The notation i $ j will be used to show that i and j are neighbours. The adjacency set for i is the set of its neighbours, that is, adj (i) = fj j j $ i; i; j 2 V g: The degree of i 2 V is deg(i) = jadj (i)j, the number of neighbours. If X is a subset of V , its adjacency set is de ned to be adj (X ) = [j 2X adj (j ) n X: Observe that, given the graph representation of a symmetric matrix, the i-th wavefront can be de ned as the vertex i plus the set of vertices adjacent to the vertex set f1; 2; : : : ; ig, that is, fi = adj (f1; 2; : : : ; ig) [ fig : A path of length k in G is an ordered set of distinct vertices fv1 ; v2 ; : : : ; vk ; vk+1 g, with vi $ vi+1 (1  i  k). Two vertices are connected if there exists a path between them. A graph G is connected if each pair of distinct vertices is connected. The distance, dist(u; v), between two vertices u and v in G is the length of the shortest path connecting them. The eccentricity of a vertex u is de ned to be e(u) = maxfdist(u; v) j v 2 Gg: The diameter of G is then (G ) = maxfe(u) j u 2 Gg: A vertex u is a peripheral vertex if its eccentricity is equal to the diameter of the graph, that is, e(u) = (G ). A pseudoperipheral vertex u is de ned by the condition that, if v is any vertex for which dist(u; v) = e(u), then e(v) = e(u). The pair u; v of pseudoperipheral vertices de ne a pseudodiameter. Throughout our discussion, it is assumed that the matrix A of interest is irreducible so that its adjacency graph G (A) is connected. Disconnected graphs can be treated by considering each component separately.

2.2 The Sloan algorithm

The Sloan algorithm [26], as presented in [17, 22], orders the vertices of a weighted adjacency graph. The weighted graph is derived from the unweighted graph by \condensing" vertices to form supervertices. Vertices i and j are condensed into a supervariable if i [ adj (i) = j [ adj (j ): (2) The weight of a supervertex is the number of unweighted vertices it represents. The use of condensing can sometimes reduce the size of the graph considerably, thus reducing the time required for reordering [5, 22]. This is particularly true for matrices arising from nite-element calculations where it is common for there to be several degrees of freedom at each node in the nite-element grid. The Sloan algorithm for reordering a graph has two distinctive phases:

2 BACKGROUND

4

1. selection of a start vertex s and an end vertex e 2. vertex reordering. Step 1 looks for a pseudodiameter of the weighted graph and chooses s and e to be the endpoints of the pseudodiameter. A pseudodiameter may be computed using a modi cation of the Gibbs-Poole-Stockmeyer algorithm (see [22]). In Step 2, the pseudodiameter is used to guide the reordering. Each vertex of the weighted graph is given a priority P (i) where

P (i) = ?W1 inc(i) + W2 dist(i; e)

(3)

and (W1; W2) are positive weights. The rst term, inc(i), is the amount by which the wavefront will increase if vertex i is ordered next. The second term, dist(i; e), is the distance between i and the end vertex e. The start vertex s is ordered rst then, at each stage, the next vertex is chosen among eligible vertices with the highest priority. Thus, a balance is maintained between the aim of keeping the wavefront small and bringing in vertices that have been left behind (far away from e). The list of eligible vertices comprise those that are in the front (neighbours of one or more renumbered vertices) or neighbours of one or more vertices in the front. The best choice for the weights (W1; W2) is problem dependent. Based on experimentation, Reid and Scott [22] suggest that the two pairs of weights (2; 1) and (16; 1) should be tried and the better ordering chosen. By default, when implementing the Sloan algorithm, the code MC60 tries both these pairs of weights but the user may also choose to select other weights. Once an ordering for the vertices of the weighted graph has been obtained, an ordering for A can be constructed.

2.3 The Hybrid algorithm

The rst term in (3) aects the priority function in a local way, by giving higher priority to vertices that will result in a small (or negative) increase to the current wavefront. This is done in a greedy fashion, without consideration of the longterm eect. The second term acts in a more global manner, ensuring vertices lying far away from the end vertex are not left behind. Step 2 of the Sloan algorithm can therefore be viewed as an algorithm that re nes the ordering implied by the distance function dist(i; e). The distance function in (3) can be replaced by other orderings that provide a global view. In particular, the spectral ordering may be used. The spectral algorithm associates a Laplacian matrix L = flij g with the symmetric matrix A as follows: 8 > if i 6= j and i $ j; < ?1; if i = j; (4) lij = > deg(i); : 0; otherwise: An eigenvector corresponding to the smallest positive eigenvalue of the Laplacian matrix is called a Fiedler vector. The spectral algorithm orders the

3 THE MULTILEVEL ORDERING ALGORITHM

5

vertices of G (A) by sorting the components of the Fiedler vector into monotonic order. This approach has been found to produce small pro les and wavefronts [1]. The spectral algorithm has been used in the context of graph partitioning [9], where it has been found that results can be improved by incorporating a local re nement step. This re nement performs local optimizations and smoothes out local oscillations that may be present. In the context of wavefront reduction, the Hybrid algorithm of Kumfert and Pothen [17] combines the spectral algorithm with Step 2 of the Sloan algorithm. In our view, it is this combination of global and local ordering algorithms that accounts for the good performance of the Hybrid algorithm, particularly for very large problems. The Hybrid algorithm, as presented in [22], chooses as the start vertex s the rst vertex in the spectral ordering and replaces (3) with the priority function P (i) = ?W1 inc(i) ? W2  p(i): (5) Here  is a normalising factor and p(i) is the position of vertex i in the spectral ordering, also referred to as its global priority value.  is chosen so that the factor for W2 varies up to dist(s; e), as in (3) (see [22]). On the basis of their numerical experimentation, Reid and Scott [22] propose the pairs of weights (1; 2) and (16; 1). Note that (1; 2) is recommended instead of the pair (2; 1) used by the Sloan algorithm. Reid and Scott argue that the global priority based on the spectral ordering has been found to be better than that obtained from a pseudodiameter, justifying a larger value for W2 in this case. Numerical experiments have shown that, for large problems, the Hybrid method can signi cantly outperform the original Sloan algorithm, although it requires signi cantly more CPU time. This is illustrated in Section 4.

3 The Multilevel Ordering Algorithm The matrix ordering algorithm proposed in this paper is based on a multilevel approach. Given the adjacency graph G (A), a series of graphs are generated, each coarser than the preceding one. The coarsest graph is then ordered. This ordering is recursively prolonged to the next ner graph, local re nement is performed at each level, and the nal ordering on the nest graph gives an ordering for A.

3.1 The Multilevel Approach

In the context of graph partitioning, the multilevel approach generates a series of coarser and coarser graphs [2, 10, 14, 27]. The aim is for each successive graph to encapsulate the information needed to partition its \parent", while containing fewer vertices and edges. The coarsening continues until a graph with only a small number of vertices is reached. This can be partitioned cheaply. The partitions on the coarse graphs are recursively prolonged (usually by injection) to the ner graphs, with further re nement at each level. One of the rst uses of a multilevel approach for the partitioning of undirected graphs was reported by Barnard, Pothen and Simon [2]. Motivated

3 THE MULTILEVEL ORDERING ALGORITHM

6

by the need to reduce the time for computing the Fiedler vector, Barnard et al. combined a multilevel approach with a spectral bisection algorithm. It was soon realized [10, 15, 27] that the multilevel approach provides a global view of the problem, and therefore can be used to advantage with a good local optimizer. In graph partitioning, the Kernighan-Lin algorithm [16] is used and, combined with the multilevel approach, has proved very successful at rapidly computing high quality partitions. Given the success of the multilevel approach for graph partitioning, it is perhaps surprising that, as far as the authors are aware, there has only been one reported attempt at applying it to the problem of pro le and wavefront reduction. Bolman and Hendrickson [3] introduce a weighted 1-sum metric

1(A) =

n X X

i=1 j $i; j