An Empirical Study of Dynamic Graph Algorithms

Chapter 1

An Empirical Study of Dynamic Graph Algorithms  (Extended Abstract) David Albertsy

Giuseppe Cattaneoz

Abstract

Giuseppe F. Italianox

1 Introduction

A dynamic graph algorithm maintains a certain property { like connectivity, bipartiteness, or a minimum spanning tree { of a dynamically changing graph. Usually, it is realized by a data structure allowing three kinds of operations: inserting an edge, deleting an edge, and asking a query. For the dynamic connectivity problem, for example, a query takes two nodes u and v as its arguments and returns \True", if there is a path connecting u and v in the current graph. The eld of dynamic graph algorithms has been a blossoming eld of research in the last years [2, 5, 7, 9, 10, 11, 12, 14, 18, 19], motivated by theoretical and practical questions (see for instance [17]). However, despite this blend of theoretical and practical interests, we are aware of no implementations and experimental studies in this eld. In this paper, we aim at bridging this gap by studying the practical properties of theoretically interesting dynamic graph algorithms that were developed recently. The contributions of this paper are both of theoretical and of experimental nature. From the theoretical point of view, we analyze the average case running time of sparsi cation [7] in a fairly general random graph model, and prove that in this case the worst-case logarithmic overhead vanishes. More precisely, we prove that the average number of sparsi cation tree nodes affected by an update is a small constant, independent of the size of the graph: this is precisely con rmed by our experiments on random inputs. Our second contribution is more of an experimental nature: we implemented sparsi cation and the recent randomized polylogarithmic dynamic connectivity algorithm of Henzinger and King [14]. Moreover, we propose a simpli ed variant of the algorithm of [14]. The worst case running time of this variant is O(log n) for insertions and queries, and O(m log n) for deletions, where n is the current number of vertices and m the current number of edges in the graph. For random inputs however, we conjecture an expected running time of O(log2 n) per deletion. This variant was indeed faster than the original algorithm in our experiments with random inputs. We tested our implementation of sparsi -

We conduct an empirical study on some dynamic graph algorithms which where developed recently. The following implementations were tested and compared with simple algorithms: dynamic connectivity, and dynamic minimum spanning tree based on sparsi cation by Eppstein et al.; dynamic connectivity based on a very recent paper by Henzinger and King. In our experiments, we considered both random and non-random inputs. Moreover, we present a simpli ed variant of the algorithm by Henzinger and King, which for random inputs was always faster than the original implementation. Indeed, this variant was among the fastest implementations for random inputs. For non-random inputs, sparsi cation was the fastest algorithm for small sequences of updates; for medium and large sequences of updates, the original algorithm by Henzinger and King was faster. Perhaps one of the main practical results of this paper is that our implementations of the sophisticated dynamic graph algorithms were faster than simpler algorithms for most practical values of the graph parameters, and competitive with simpler algorithms even in case of very small graphs (say graphs with less than a dozen vertices and edges). From the theoretical point of view, we analyze the average case running time of sparsi cation and prove that the logarithmic overhead for simple sparsi cation vanishes for dynamic random graphs.

 Work partly supported by the Commission of the European Communities under the ESPRIT-LTR Project no. 20244 (ALCOM-IT) and by the Italian MURST Project \Ecienza di Algoritmi e Progetto di Strutture Informative". y Institut f ur Informatik, Freie Universitat Berlin, Germany. E-mail: [email protected]. Most of this work was done while the author was a member of the Graduiertenkolleg \Algorithmische Diskrete Mathematik", supported by the Deutsche Forschungsgemeinschaft, grant We 1265/2-1. z Dipartimento di Informatica ed Applicazioni, Universit a di Salerno, Italy. E-mail: [email protected]. x Dipartimento di Matematica Applicata ed Informatica, Universita \Ca' Foscari" di Venezia, Italy. Most of this work was done while at University of Salerno and while visiting ICSI, Berkeley, CA 94704. E-mail: [email protected].

1

2 cation on dynamic connectivity, and dynamic minimum spanning tree, and our implementation of [14] on dynamic connectivity. Finally, we compared the performance of our implementations to other static and dynamic algorithms. We performed extensive tests under several variations of graph and update parameters. Our experiments were run both on randomly generated graphs, and on more structured graphs that could be taken as benchmarks for future experimental work on dynamic graph algorithms. Our variant of the Henzinger and King's algorithm was among the fastest implementations for random inputs. For non-random inputs, sparsi cation yielded better times for small sequences of updates; for medium and large sequences of updates, the original algorithm of Henzinger and King took over. More speci cally, given the size of the input graph, there is a threshold 0 (depending on the graph size) such that the algorithm of Henzinger and King is faster for sequences of more than 0 updates, and sparsi cation is better for sequences of up to 0 updates. This is not surprising, as the analysis of Henzinger and King [14] holds only for sequences of (m0 ) updates (where m0 is the number of edges in the starting graph) and it is amortized, while sparsi cation yields worst-case bounds. What was somehow unexpected was the fact that the original algorithm of Henzinger and King started being competitive with sparsi cation well before m0 updates. We remark that our implementation of sparsi cation used p a simple O(n+m(m; n)) algorithm rather than the O( m ) algorithm of Frederickson [10] (as no implementation of this algorithm is currently available). This produced a boundpof O(n(n; n) log(m=n)) 1 per update rather than O( n ). We are currently implementing the algorithm by Frederickson to check whether sparsi cation on top of it will produce faster algorithms for structured graphs. In our experiments, both sparsi cation and the algorithm by Henzinger and King proved to be really practical: indeed, both these algorithms compared favorably to simpler methods for most practical values of the graph parameters, and even in case of very small graphs (i.e., graphs with less than a dozen vertices and edges!). This came as a surprise to us, as we are used to the fact that theoretically interesting algorithms need much larger inputs to become competitive with simple methods. All our implementations are written in C++ and are based on LEDA, the library of ecient data types and algorithms developed by Mehlhorn and Naher [15]. Our source codes are available via anony-

Alberts et al.

mous ftp: from

in the directory in .

ftp.inf.fu-berlin.de /pub/misc/dyn con/ ftp.dia.unisa.it pub/italiano/sparsification

the directory

, and from

2 Sparsi cation

Sparsi cation [7, 8] is a general technique that applies

to a wide variety of dynamic graph problems, including minimum spanning forests, edge- and vertex- connectivity, bipartiteness, dynamic matroid intersection problems, and to static problems as well, such as sampling the space of spanning trees, and computing the k best spanning trees. Sparsi cation is used on top of a given algorithm, in order to speed it up considerably; it has the desirable property that it can be used as a black box , i.e., it does not require the algorithm designer to know the internal details of the algorithm that he/she wants to speed up. Let A be an algorithm that solves a certain problem in time f(n; m) on a graph with n vertices and m edges. There are two versions of sparsi cation: simple sparsi cation [7] and improved sparsi cation [8]. When simple sparsi cation is applied to A, it produces a better bound of O(f(n; O(n)) log(m=n)) for the same problem. This is achieved by calling algorithm A as many as O(log(m=n)) times on graphs with O(n) edges (rather than once on a graph with m edges). Improved sparsi cation uses more sophisticated data structures to eliminate the logarithmic factor, thus yielding a O(f(n; O(n))) time bound. Sparsi cation is based on the notion of certi cate: Definition 2.1. Let P be any given graph property, G be a graph, and c > 0 be a given constant. A c-sparse certi cate for P in G is a graph G0 on the same vertex set as G such that the following is true: (i) for any H , G [ H has property P if and only if G0 [ H has the property; and (ii) G0 has no more than cn edges.

Let G be a graph with m edges and n vertices. Sparsi cation works as follows: we partition the edges of G into a collection of O(m=n) subgraphs with n vertices and O(n) edges. The information relevant for each subgraph is summarized in a c-sparse certi cate. We next merge certi cates in pairs, producing larger subgraphs which are made smaller by again computing their certi cate. This is applied recursively, yielding a balanced binary tree in which each node is represented by a csparse certi cate. We call this tree the sparsi cation tree. The sparsi cation tree has O(m=n) leaves, and hence its height is O(log(m=n)). Each update involves examining the certi cates contained in at most two tree paths from a leaf up to the tree root: this results in considering at most 2dlog(m=n)e small graphs with O(n) edges each, instead of one large graph with m edges, 1 Throughout the paper we let log x stand for maxf1; log xg, and explains how a f(n; m) time bound can be sped up 2 so log(m=n) is never smaller than 1 even when m < 2n. to O(f(n; O(n)) log(m=n))). We say that a tree node

Empirical Study of Dynamic Graph Algorithms

3

is aected by an update if the certi cate at the node must be changed as a result of the update. If ` tree nodes are aected by an update, the update will cost O(f(n; O(n)) `). We refer the reader to [7] for the full details of the method. One contribution of this paper is of theoretical nature and bounds the average number of sparsi cation tree nodes aected by a random update. More speci cally, we prove that this number is at most a small constant, as opposed to the worst-case bound of 2dlog(m=n)e. We need the following de nition rst: Definition 2.2. Let a graph G be given, let G+ be the graph obtained from G after inserting an edge e+ in G, and let G? be the graph obtained from G after deleting an edge e? from G. Given a property P to be maintained for G, and a certi cate C for P in G, C is said to be a good certi cate if the following two conditions are met: (1) If C is no longer a certi cate for G+ , then the edge e+ must be contained in all the certi cates for G+ ; and (2) If e? is not in C , then C is still a certi cate for G? . We remark that De nition 2.2 is rather general as most of the certi cates that we know of for unweighted graphs are good certi cates (e.g., the certi cates for connectivity, minimum spanning trees and k-edge connectivity). We prove that for good certi cates, simple sparsi cation calls the input algorithm A only a very small number of times in the average during each update. The analysis is carried out in a general model for updates in random graphs, namely Mulmuley's expected case model of dynamic computation [16], which was already adapted to fully dynamic graph problems by Eppstein [6] and by Alberts and Henzinger [2]. Let G = (V; E) be a graph with n vertices and m edges, which is dynamically changing? because of edge inser
?
tions and edge deletions. Let V2 be the set of all n2 edges de ned on on the vertex ?
set V . A pattern for G is e EE e  V , consisting of m given by a set E, e edges, 2 together with a string S consisting of `+' and `?'. The set Ee is called the universe and determines the set of edges to be inserted and deleted in the random graph, while the string S determines the types of update operations applied to the random graph: each `+' represents an edge insertion, and each `?' an edge deletion. Each pattern determines a space of update sequences as follows. The string is examined from left to right: if the current symbol is a `+', then an edge is chosen uniformly at random among the edges in the universe Ee not yet in E, and it is added to the graph G. If the current symbol is a `?', then an edge is chosen uniformly at random among all the edges in E, and the graph G is updated by removing this edge. Given an initial graph G, and a pattern for G, the expected running time of an algo-

rithm in this model is given by the time averaged over all possible update sequences consistent with that pattern. We remark that in this model the pattern is not necessarily known to the algorithm but may be revealed only as the actual sequence of updates proceeds. This is an important point in the average-case analysis of simple sparsi cation. Our theoretical result is summarized in the following theorem, whose proof relies on combinatorial and structural properties of sparsi cation trees and on sophisticated average-case analysis arguments; the proof is omitted here for lack of space.

Theorem 2.1. Let P be a property to be maintained on an unweighted graph G for which there are c-sparse good certi cates, and let e be an edge to be either deleted uniformly at random from G or inserted uniformly at random in G. Then, the expected number of sparsi cation tree nodes aected by this update is at most 209 c if c  1, and at most ( 209 + dlog2 ce) if c > 1.

Theorem 2.1 implies that simple sparsi cation has an average running time of O(f(n; O(n))) per update, thus matching the time of improved sparsi cation. However, it is simpler than improved sparsi cation and therefore it is likely to be faster in practice for reasonable inputs. This explains why we decided to implement simple sparsi cation rather than improved sparsi cation. Our implementation follows closely the algorithm of Eppstein et al. [7], with very few exceptions. First, each time an edge is deleted from a given group, we insert a pointer to the corresponding tree leaf in a queue. This queue will contain pointers to empty slots in edge groups: during an edge insertion, we pop the rst item from the queue and insert the new edge in the corresponding leaf. If the queue is empty, there are no available slots from previous deletions, and the new edge is inserted as in the algorithm of Eppstein et al. either in the small group or in a new leaf, if the small group is full. Because of this implementation of edge deletions, edge groups may be impoverished throughout the sequence of operations. Namely, update sequences having more deletions than insertions may result in edge groups with less than n edges. We rebuild the sparsi cation tree from scratch each time the dierence between the actual height and the supposed height (i.e., dlog(m=n)e) is larger than one. Table 1 shows the results of some of our experiments, where we measured the average number of sparsi cation tree nodes aected by a random update. For sake of brevity, we represented only a small but signi cant set of data in Table 1: for n = 10, 25, 50, 250 and 500 we choose the number of edges in the initial graph to be m = n, n log n, n3=2 and n2 =4, and the total number of updates to be 500. We measured other graph and update sizes and reported similar data. As shown

4

Alberts et al. n0 10 10 10 10 25 25 25 25 50 50 50 50 100 100 100 250 250 250 250 500 500 500 500

m0 10 23 25 31 25 80 124 156 50 195 353 625 100 1000 2500 250 1380 3952 15625 500 3107 11180 62500

Up ins 1.004 0.946 1.036 0.909 0.971 0.768 0.709 0.672 0.904 0.669 0.565 0.666 0.936 0.545 0.584 0.734 0.571 0.466 0.538 0.814 0.615 0.544 0.534

Up del 1.214 1.693 1.592 1.695 1.252 1.824 1.849 1.844 1.032 1.605 1.753 1.848 1.086 1.951 1.867 0.889 1.906 1.798 1.875 0.832 1.836 1.881 1.825

Table 1: Experiments with simple sparsi cation and csparse certi cates, for c = 1. Up ins [respectively Up del] denotes the average number of sparsi cation tree nodes aected by an edge insertion [respectively edge deletion]. Each data set is the average of ten samples. in Table 1, the experimental data about certi cates for unweighted graphs con rm the theoretical predictions of Theorem 2.1, thus showing that the computed bounds are extremely tight. This gives practical evidence that the model we adopt is not only theoretically convenient, but more importantly it suites well the evolution of dynamic unweighted random graphs as it yields signi cant estimates in practice. Theorem 2.1 still holds if the graph is weighted (e.g., also for the dynamic minimum spanning tree problem) in case of deletions. However, it does not extend to weighted graphs in the case of insertions, as for the same analysis to hold, we need the cost of every edge to be xed, an assumption which we believe unrealistic. The experimental data we collected on dynamic minimum spanning trees con rm this criticism. Indeed, the average number of sparsi cation tree nodes aected by a deletion is still no more than 209 , while the number of tree nodes aected by an insertion does not comply at all with the bounds computed in Theorem 2.1.

3 The Randomized Dynamic Connectivity Algorithm by Henzinger and King In this section we present the main ideas of the randomized algorithm of Henzinger and King, described in [14] and implemented in [1]. It achieves an expected amortized update time of O(log3 n) for a sequence of at least m0 updates, where m0 is the number of edges in the initial graph, and a query time of O(logn). It needs (m + n logn) space. For more details see the references. Like all previous dynamic connectivity algorithms, the algorithm by Henzinger and King also maintains a spanning forest. All trees in the spanning forest

are maintained in a data structure, which allows one to obtain logarithmic updates and queries within the forest. All we have to do is to keep it spanning, so the crucial case is the deletion of a forest edge. One main idea is to use random sampling among the edges incident to the tree T containing the forest edge e to be deleted, in order to nd a replacement edge quickly. A single random edge adjacent to T can be selected and tested whether it reconnects T in logarithmic time. The goal is an update time of O(log3 n), so the number of sampled edges is O(log2 n). However, the set of possible edges reconnecting the two parts of T, which is called the candidate set of e in the following, might only be a small fraction of all non-tree edges which are adjacent to T. In this case it is unlikely to nd a replacement edge for e among the sampled edges. If there is no candidate among the sampled edges, the algorithm checks all adjacent edges of T. Otherwise it would not be guaranteed to provide correct answers to the queries. Since there might be a lot of edges which are adjacent to T, this could be an expensive operation, so it should be a low probability event. This is not yet true, since deleting all edges in a relatively small candidate set, reinserting them, deleting them again, and so on will almost surely produce many of those events. The second main idea prevents this undesirable behavior. The algorithm maintains an edge decomposition of the current graph G into O(logn) edge disjoint subgraphs Gi = (V; Ei). These subgraphs are hierarchically ordered. Each i corresponds to a level. For each level i, there is a forest Fi of Gi such that the union [ikFi is a spanning forest of [ik Gi; in particular the union F of all Fi is a spanning forest of G. A spanning tree at level i is a tree in [j iFj . The weight w(T) of a spanning tree T at level i is the number of pairs (e0 ; v) such that e0 is a non-tree edge in Gi adjacent to the node v in T. If T1 and T2 are the two trees resulting from the deletion of e, we sample edges adjacent to the tree with the smaller weight. If sampling is unsuccessful due to a candidate set which is non-empty but relatively small, then the two pieces of the tree which was split are reconnected on the next higher level using one candidate, and all other candidate edges are copied to that level. The idea is to have sparse cuts on high levels and dense cuts on low levels. Non-tree edges always belong to the lowest level where their endpoints are connected or a higher level, and we always start sampling at the level of the deleted tree edge. After moving the candidates one level up, they are normally no longer a small fraction of all adjacent non-tree edges at the new level. If the candidate set on one level is empty, we try to sample on the next higher level. There is one more case to mention: if sampling was unsuccessful despite the fact

Empirical Study of Dynamic Graph Algorithms

5

that the candidate set was big enough, which means that we had bad luck, we do not move the candidates to the next level, since this event has a small probability and does not happen very frequently. We present the pseudocode for replace(u,v,i), which is called after the deletion of the forest edge e = (u; v) on level i : 1. Let Tu and Tv be the spanning trees at level i containing u and v, respectively. Let T be the tree with smaller weight among Tu and Tv . Ties are broken arbitrarily. 2. If w(T) > log2 n then (a) Repeat sample and test(T) for at most 16 log2 n times. Stop if a replacement edge e is found. (b) If a replacement edge e is found then do delete non tree(e), insert tree(e; i), and return. 3. (a) Let S be the set of edges with exactly one endpoint in T. (b) If jS j  w(T)=(8 logn) then Select one e 2 S, delete non tree(e), and insert tree(e; i). (c) Else if 0 < jS j < w(T)=(8 logn) then Delete one edge e from S, delete non tree(e), and insert tree(e; i + 1). Forall e0 2 S do delete non tree(e0 ) and insert non tree(e0 ; i + 1). Update added edges[i + 1] and rebuild level i + 1 if necessary. (d) Else if i < l then replace(u; v; i + 1). The \constants" depending only on n (i.e., log2 n; 16 log2 n; 8 logn) can be changed in the implementation. If there are only deletions of edges, a bound of d2 log ne for the number of levels is guaranteed. If there are also insertions, there have to be periodical rebuilds of parts of the data structure to achieve the same bound. There is a function called move edges taking the number, i say, of a level. It moves all edges at levels i and above to level i ? 1. This is a rebuild of level i. An edge is added to level i, if it is either newly inserted into level i or moved up from level i ? 1 during a replace. At each level there is a bound which is some constant for the highest level and twice the bound of level i + 1 for level i. Level i is rebuilt when the number of edges added to levels i and above reaches the bound for level i. The cost for moving edges up or down is also O(log3 n) per update in the amortized sense, if there are at least

(m0 ) updates. In the implementation the number of levels as well as the bound for rebuilds on the highest level can be set to other values than those proposed by [14] (possibly invalidating the analysis). By using dierent parameters, we realized a simpli ed variant of the algorithm which performs well for random inputs.

For convenience, we used an initialization procedure with a bound of O((m + n logn) log n). We simply insert the initial edges into the empty graph. Since there are no deletions, we get the above bound. Using a more sophisticated procedure would give a bound of O(m + n logn) [13].

3.1 A Simpli ed Version of Henzinger-King

Preliminary experiments with random inputs showed that moving edges to a higher level almost never occurred. Moreover, the average number of edges sampled during a successful sampling was less than 2. This lead to a simpli ed variant of the Henzinger and King algorithm: there is just one level, so we do not move edges up or down, and we sample only at most c log n edges instead of 16 log2 n. On random inputs this was always faster than the original algorithm and was more robust to input variations. In this variant, insertions, deletions of non-tree edges and queries take logarithmic time, but currently there is no better bound for the running time of a tree edge deletion than O(m log n) worst-case time. We conjecture that this operation takes O(log2 n) expected time for random inputs. This is motivated by the following observation. Consider a random subset S consisting of at most half of the vertices of a random graph. The ratio between the expected number of edges with exactly one endpoint in S and the expected number of edges with both endpoints in S is greater than 1, and it grows as S becomes smaller. This variant takes O(m + n) space and O(m log n + n) preprocessing time using the simple initialization scheme.

4 Simple Algorithms

Beside the sophisticated algorithms, we also considered some simple algorithms, which were very easy to implement in only a few lines of code. Usually, this means that their implementation constants are extremely good, and thus these algorithms are likely to be very fast in practice for reasonable inputs. For the dynamic connectivity problem these simple algorithms were a fast{update algorithm and a fast{query dynamic algorithm. They are de ned as follows. In case of an update the fast{update algorithm does nothing, except for updating two adjacency lists. It then answers a query(u; v) by doing a breadth rst search starting from u. The fast{query dynamic algorithm maintains a spanning forest and component labels at the vertices. These can be computed by a breadth rst search for each connected component. In case of a deletion, the algorithm checks whether the edge to be deleted is in the current spanning forest (those edges are marked). In case of an insertion, the algorithm checks whether the inserted edge connects two distinct connected components of the

6 former graph by using the component labels. In either of these cases, the spanning forest may be forced to change: if this is the case, the algorithm recomputes from scratch the trees and component labels for the affected components. If an update does not aect the spanning tree, it is handled as in the fast{update algorithm. A query is answered using the component labels of the input nodes. Clearly, the fast{update algorithm takes constant time for updates and O(n + m) for queries in the worst case, while the fast{query algorithm takes constant time for queries and non-tree updates and O(n + m) for tree updates. Both algorithms take O(m + n) space and preprocessing time. From a worst-case point of view, the fast{query algorithm is not better asymptotically than the fast{update algorithm provided that the number of updates is approximately the same as the number of queries. For random inputs however, the fast{query algorithm is much faster, as it takes only O(n) expected time per update independently of the number of edges (cf. [2]). These two algorithms are extreme cases, as one is optimal for updates only and one is optimal for queries only. When we consider a particular application, we have to take this into account. If there are no queries at all, this is no longer a connectivity problem. If there are only queries, the problem becomes static, and the fast{ query algorithm turns into an optimal static algorithm, which we certainly cannot beat with any dynamic algorithm. Moreover, it is a common phenomenon that a theoretically very fast algorithm is outperformed on all small input sizes by a simpler method, because the constants hidden in the O{notation can be really large. Given a graph of a certain size, one can get experimental results on the approximate ratio of updates and queries needed for the simple algorithms to become competitive. For instance, for random graphs of 300 vertices and 40000 edges, our experiments indicate that one needs sequences of about 97% queries for fast{query to become competitive with the simpli ed variant of Henzinger and King, and sequences of more than 99% updates for fast{update to become competitive. For random graphs of 100 vertices and 4000 edges, one needs about 95% queries for fast{query and about 99% updates for fast{update. The above gures are computed for sequences of 5000 operations.

5 Experimental Results 5.1 Test Environment Most of our experiments

were run on a DEC APX 4000/720 machine, with a 190 MHz CPU alpha, and 128 Mbytes of RAM. This machine has a hierarchical memory architecture with a 4 Mbytes cache external to the processor. We measured both the preprocessing time and the time that the

Alberts et al.

algorithms spent in recomputing a new solution after each update (i.e., we did not measure the times required for loading the initial graphs). All our times were CPU times measured in seconds with the Unix getrusage() command.

5.2 Sparsi cation We rst present our experiments

with sparsi cation. We ran sparsi cation on top of a partially dynamic algorithm based on the LEDA Spanning Tree function. This partially dynamic algorithm supports each insertion in O((m; n)) time, and each deletion in O(n + m(m; n)) time. In our implementation this underlying function can be easily changed, and we plan to make further experiments with algorithms that are more ecient than Spanning Tree, such as fast{query for random inputs (this choice being motivated by practical experiments) and Frederickson's algorithm [10] for non-random inputs (this choice being motivated by theoretical arguments). We compared the average running times of sparsi cation and the underlying algorithm. We performed these tests on random sequences of updates starting from random graphs of dierent densities. Since the height of the sparsi cation tree of a graph with m edges and n vertices is dlog(m=n)e, dierent graph parameters, and more precisely dierent m=n ratios give rise to sparsi cation trees of dierent heights. By Theorem 2.1, the average number of sparsi cation tree nodes examined during an update is a small constant, independent of the tree height: indeed we have already seen that each update involves less than 2 tree nodes in the average for c = 1 (see Table 1). An important consequence of this property is that the running times of sparsi cation are fairly independent of the number of edges: if the number of vertices n is xed, the running times of sparsi cation show very little variations as m increases. This was clearly con rmed by our experiments. As predicted from the theory, the denser the graph (i.e., the larger the value m=n), the better was the performance of sparsi cation over the underlying algorithm. For n = 500, for instance, the speed-up given by sparsi cation ranges from around 10% for very small values of m=n (say m=n  3) up to incredible speed-ups for larger values of m=n: when m=n = 249:5 (the maximum value of this ratio for n = 500), sparsi cation produces algorithms that are around 60 times faster. It was surprising for us to notice that sparsi cation outperforms the underlying algorithm even for very sparse graphs: indeed in our experiments, sparsi cation is no worse than the underlying algorithm even in the range 2 < m=n  3, a case where theoretical arguments would suggest the opposite to be true. This has an explanation for random inputs, however. Indeed, in this case the sparsi cation tree has

Empirical Study of Dynamic Graph Algorithms Algorithm fast-query fast-update sparsif. HK HK var.

Preprocessing O(n + m) O(n + m) O(n + m(m; n)) O((m + n log n) log n) O(m log n + n)

7 is impossible for us to illustrate in tables or pictures for lack of space. In what follows we will try to summarize the main points of our experiments.

Table 2: Theoretical bounds for preprocessing in the 5.4 Very Small Graphs One of the main results of our implementations is that sparsi cation and the implementations considered in our experiments. algorithm of Henzinger and King were really practical even for very small graphs. For instance, to process a Algorithm Query Insert Delete sequence of 500 uniformly mixed updates starting from fast-query O(1) O(n + m) O(n + m) fast-update O(n + m) O(1) O(1) a graph with 10 vertices and 5 edges, sparsi cation sparsif. O((n; n)) O((n; n) log(m=n)) O(n(n; n) log(m=n)) 3 HK O(log n) O(log n) O(log3 n) took 0:09 seconds, and Henzinger-King 0:12 seconds. HK var. O(log n) O(log n) O(m log n) This was comparable to the running times of simple Table 3: Theoretical worst-case or amortized bounds for algorithms, such as fast{query, which took 0:08 seconds, the operations in the implementations considered in our and LEDA Spanning Tree, which took 0:15 seconds. It is truly remarkable that sophisticated algorithms are experiments. competitive with simple algorithms already on such small graphs. height equal to three, and the two non-root nodes in 5.5 Distribution of Running Times for Dierthe rightmost path have exactly m ? 2n  n edges. An ent Densities In our experiments, the original algoupdate involving this rightmost path is thus likely to by Henzinger and King rst increases for a xed be faster than the update performed by the underlying rithm n as m grows up to a certain value, and then slightly algorithm which operates on the whole graph. decreases. This apparently strange behavior can be exThis implementation of sparsi cation needs O(m + plained with the help of the theory of random graphs n) space and O(n+m(m; n)) preprocessing time. The (cf. [3, 4]). Indeed, get the following distribution of theoretical bounds of the implementations considered in running times for a we vertex set of xed size n. If there our experiments are summarized in Tables 2 and 3. are strictly less than n edges, then the size of each concomponent in a random graph is almost always 5.3 Random Inputs We performed our tests on ran- nected O(log n). Almost every update involves a tree edge, but dom inputs as follows. We ran four types of experi- they are cheap for the algorithm, since only small comments, according to dierent lengths of the update se- ponents are involved. If there are around n edges, there quence: small sequences of updates (i.e., 50 updates), is almost always a component on (n2=3) vertices, for medium sequences of updates (i.e., 500 updates), large slightly more than n edges even on (n) vertices, the sosequences of updates (i.e., 5000 updates), and very large called giant component. There is still at least a constant sequences of updates (i.e., 50000 updates). We tried fraction of updates involving tree edges and a constant each such experiment on dierent sizes of the input fraction of those deals with the giant component, we graph. For a xed value n of the number of vertices, we get a relatively slow running time. These are in asoway found out in our experiments that the interesting graphs the hardest random instances, and also the most interwere sparse graphs (with around n=2 or n edges), mod- esting ones, since the whole graph is still disconnected erately dense graphs (with around n log n edges), dense and components merge or split quite often. If there are graphs (with around n3=2 edges), and very dense graphs more than n logn edges the whole graph is almost al(with around n2 =4 edges). For each choice of parame- ways connected. The fraction of updates involving tree ters, we generated ten samples. We ran our implementadrops to 1= logn, and the non-tree updates betions on each sample, and retained the average for each edges come dominant: overall running time drops because of them. We did not consider queries, as once a span- these operations the are faster. ning forest is maintained all the algorithms can perform The fast{query algorithm shows a somewhat similar queries roughly at the same cost: thus, the main dier- behavior, but there is a much bigger between ences in the running times of the algorithms depend on non-tree updates (which take O(1)imbalance time) and tree their behavior on updates. The updates were uniformly updates (which can take as much as O(m + n) time). mixed, namely 50% were edge insertions and 50% were In practice, this bigger imbalance lead to a high input edge deletions. For each experiment, we measured the sensitivity and big uctuations in the running times for preprocessing time and the time that each algorithm took to carry out the sequence of updates. These ex- dense graphs and small sequences of updates. periments produced an enormous amount of data, which

8

Alberts et al. n0 10 10 10 10 25 25 25 25 25 50 50 50 50 50 100 100 100 100 100 250 250 250 250 250 500 500 500 500 500

m0 5 10 23 31 12 25 80 124 156 25 50 195 353 625 50 100 460 1000 2500 125 250 1380 3952 15625 250 500 3107 11180 62500

Sparsif. 0.00 0.09 0.00 0.10 0.00 0.10 0.00 0.10 0.00 0.10 0.00 0.11 0.00 0.13 0.01 0.13 0.01 0.13 0.00 0.12 0.00 0.13 0.00 0.18 0.01 0.18 0.02 0.18 0.01 0.14 0.01 0.18 0.02 0.28 0.03 0.28 0.08 0.27 0.02 0.23 0.02 0.33 0.05 0.57 0.14 0.59 0.57 0.66 0.04 0.41 0.05 0.61 0.14 1.09 0.44 1.23 2.43 1.36

HK variant 0.00 0.10 0.00 0.10 0.00 0.10 0.00 0.10 0.00 0.10 0.00 0.10 0.01 0.10 0.01 0.10 0.01 0.10 0.00 0.11 0.01 0.11 0.01 0.11 0.02 0.11 0.02 0.11 0.01 0.11 0.01 0.12 0.02 0.12 0.05 0.10 0.12 0.11 0.02 0.11 0.02 0.13 0.08 0.11 0.22 0.11 0.91 0.12 0.05 0.11 0.07 0.13 0.22 0.12 0.71 0.12 4.44 0.13

0.00 0.00 0.00 0.01 0.01 0.01 0.02 0.02 0.02 0.02 0.03 0.03 0.04 0.06 0.04 0.06 0.10 0.13 0.22 0.11 0.21 0.32 0.49 1.32 0.27 0.51 0.82 1.42 5.88

HK

0.12 0.12 0.16 0.16 0.12 0.14 0.19 0.17 0.14 0.13 0.18 0.21 0.17 0.14 0.15 0.28 0.23 0.17 0.13 0.21 0.48 0.28 0.19 0.15 0.28 0.67 0.29 0.19 0.17

fast query 0.00 0.08 0.00 0.08 0.00 0.08 0.00 0.08 0.00 0.08 0.00 0.08 0.00 0.08 0.00 0.07 0.00 0.08 0.00 0.08 0.00 0.10 0.00 0.09 0.00 0.09 0.00 0.09 0.00 0.09 0.00 0.13 0.01 0.10 0.01 0.10 0.01 0.10 0.01 0.10 0.01 0.23 0.02 0.17 0.02 0.16 0.05 0.14 0.03 0.10 0.03 0.38 0.04 0.28 0.07 0.25 0.31 0.39

Table 4: Experiments with sparsification, our variant of Henzinger-King, Henzinger-King and fast-query on dierent graphs and medium sequences of updates (i.e., 500 updates). For each algorithm the left colum is the preprocessing and the right colum is the processing time in seconds. Each data set is the average of ten dierent samples.

5.6 Small Update Sequences For small sequences

of updates, fast{query was generally slightly faster than the other algorithms. Our variant of Henzinger-King was faster than fast{query in processing the updates but much slower in preprocessing, while sparsi cation was faster than the variant of Henzinger-King in preprocessing but slower in processing.

5.7 Medium Update Sequences For medium se-

quences of updates, we had a similar behavior for graphs up to 100 vertices. From this point on, our variant of Henzinger and King took over in the intermediate range. For larger graphs (say graphs with more than 50000 edges), the variant of Henzinger-King lost ground and fast{query became competitive again. This can be easily explained as the analysis of the algorithm of Henzinger and King holds only for sequences of at least (m0 ) updates, where m0 is the initial number of edges. The results of this experiment are summarized in Table 4, where for each algorithm the rst column denotes the preprocessing and the second column denotes the processing time.

5.8 Large and Very Large Update Sequences For large sequences of updates, fast{query was slightly better than the other algorithms for graphs up to 100 vertices. After this point, our variant of HenzingerKing was the fastest algorithm, except for very sparse

n0 10 10 10 10 25 25 25 25 25 50 50 50 50 50 100 100 100 100 100 250 250 250 250 250 500 500 500 500 500

m0 5 10 23 31 12 25 80 124 156 25 50 195 353 625 50 100 460 1000 2500 125 250 1380 3952 15625 250 500 3107 11180 62500

Sparsif. 0.00 0.99 0.01 1.00 0.01 1.00 0.00 1.01 0.00 1.12 0.00 1.12 0.01 1.28 0.01 1.31 0.01 1.32 0.00 1.27 0.00 1.36 0.01 1.77 0.01 1.80 0.02 1.85 0.01 1.44 0.01 1.81 0.02 2.93 0.03 2.91 0.08 2.83 0.02 2.33 0.01 3.65 0.05 6.09 0.14 6.01 0.58 6.75 0.04 4.17 0.04 5.97 0.14 11.68 0.44 12.74 2.43 13.65

HK variant 0.00 1.01 0.00 1.02 0.00 1.01 0.00 1.00 0.00 1.08 0.00 1.09 0.01 1.07 0.01 1.05 0.01 1.05 0.00 1.09 0.01 1.16 0.01 1.10 0.02 1.08 0.03 1.08 0.01 1.10 0.01 1.20 0.02 1.12 0.05 1.10 0.12 1.13 0.02 1.12 0.03 1.29 0.08 1.16 0.22 1.15 0.94 1.22 0.06 1.17 0.07 1.36 0.22 1.19 0.76 1.22 4.65 1.35

0.00 0.01 0.01 0.00 0.01 0.02 0.02 0.02 0.02 0.02 0.02 0.04 0.05 0.07 0.04 0.07 0.10 0.12 0.22 0.11 0.21 0.33 0.49 1.32 0.28 0.51 0.82 1.42 5.85

HK

1.13 1.12 1.20 1.16 1.20 1.22 1.56 1.61 1.51 1.25 1.37 2.14 1.71 1.44 1.32 1.55 2.31 1.71 1.42 1.44 2.10 2.61 1.77 1.48 1.62 2.94 2.89 1.83 1.58

fast query 0.00 0.80 0.00 0.80 0.00 0.80 0.00 0.79 0.00 0.86 0.00 0.87 0.00 0.84 0.00 0.84 0.00 0.82 0.00 0.91 0.00 0.94 0.00 0.92 0.00 0.89 0.00 0.88 0.00 0.99 0.01 1.27 0.00 1.08 0.01 1.03 0.01 1.03 0.02 1.04 0.00 2.23 0.01 1.60 0.02 1.64 0.07 1.60 0.03 1.02 0.03 3.87 0.04 2.84 0.07 2.68 0.41 3.62

Table 5: Same experiment as Table 4, with large sequences of updates (i.e., 5000 updates). graphs (i.e., graphs with n vertices and n=2 edges), where fast{query was again slightly better. This can be explained with the fact that the implementation constants of fast{query are certainly smaller than the implementation constants of Henzinger-King; thus for small graphs fast{query is likely to be faster than our variant of Henzinger-King. If also the preprocessing time was taken into account, then fast{query became again competitive with our variant of Henzinger-King for graphs with more than 10000 edges. This is due to the fact that, for the dense graphs, a sequence of 5000 update operations is not large enough to amortize the bigger initialization cost. This phenomenon did not happen for very large sequences of updates, thus con rming our explanation. Indeed, for sequences of 50000 updates, fast{query was the fastest algorithm for small graphs (i.e., graphs with less than 100 vertices) and then our variant of Henzinger-King took over; for graphs with more than 10000 edges, our variant of Henzinger-King was still the fastest algorithm, even when the preprocessing was taken into account.

5.9 A Detailed Test Figure 1 depicts the through-

put (i.e., the number of operations per second) of our variant of Henzinger-King and fast{query on graphs with 100 vertices and dierent densities. The experiment for a sequence of 50 updates was run on a Sun Sparc 2, which in our experience is about four times slower than a DEC APX. This was done since the DEC APX running times for very small graphs and small sequences were too small to yield signi cant estimates for the throughput. The throughput of each algorithm was computed by taking into account both the preprocessing

Empirical Study of Dynamic Graph Algorithms HK variant Fast Query

# op / sec 5000 4500 4000 3500 3000 2500 2000 1500 1000 500 0 50000 5000 10

20

50 100 200 edges

500 5001000 2000

5000

# updates

50

Figure 1: The throughput (number of operations per second) of our variant of Henzinger-King and fast-query on graphs with 100 vertices and dierent densities, and for dierent lengths of the update sequence. Each data set is the average of ten dierent samples. and the processing times on small, medium, large and very large sequences of updates. It is clear from this gure that for a small number of updates fast{query is faster, and that as the number of updates gets larger, the throughput of our variant of Henzinger-King improves signi cantly so as to outperform fast{query for very large sequences of updates. Once again, this is in accordance with the fact that Henzinger-King has a bigger preprocessing time and that its amortized analysis holds only for long enough sequences of updates. This behavior gives a clear cut-o between the performance of our variant of Henzinger-King and of fast{query. Indeed, for graphs with 100 vertices our experiments indicate that one would rather choose fast{query for small sequences of updates, while our variant of HenzingerKing seems preferrable for very large sequences of updates. For medium and large sequences of udpates, our variant of Henzinger-King is faster than fast{query in an intermediate density range of the graph, and this range grows with the number of operations. More in detail, for sequences of 500 updates, our variant of HenzingerKing is faster only for graphs with between 50 and 100 edges, while for sequences of 5000 updates our variant of Henzinger-King is faster for graphs with between 80 and 2500 edges. Outside these ranges, fast{query achieves a better throughput.

5.10 Non-Random Inputs We wanted to test our

9 vertices, and an integer s, s > 0. We de ne a family of graphs G (n; s) for each value of n and s. Each graph G(n; s) in G (n; s) is as follows. G(n; s) has exactly n vertices, grouped in s dierent groups V1 , V2 , : : :, Vs : Vj , 1  j  s ? 1, has exactly dn=se vertices, while Vs has the remaining n ? (s ? 1)dn=se vertices. For 1  j  s, let Gj be the subgraph of G(n; s) induced by Vj : Gj is the complete graph. Furthermore, there are (s ? 1) edges in G(n; s): edge ei , 1  i  s ? 1, is a bridge connecting one vertex of Vi and one vertex of Vi+1 . Note that when s is much smaller than n, G(n; s) has strong structural properties, while when s is of the same order as n, G(n; s) looses completely its structural properties and degenerates into a sparse graph. The dynamic operations we perform on G(n; s) are deletions and insertions of bridges, which will force the algorithms to handle dense subgraphs, while the connectivity of the whole graph keeps changing. With high probability this situation does not happen for random inputs. We ran many experiments on these non-random graphs, measuring the time that each algorithm took in performing a sequence of updates, including the initial preprocessing. In our experiments, sparsi cation and the original Henzinger-King algorithm were faster than the other algorithms by more than one order of magnitude for small values of s (i.e., highly structured graphs). As s gets larger, the graph G(n; s) tends to loose its structured properties, and the gap between all the algorithms became smaller in our experiments. When s becomes of the same order as n, G(n; s) degenerates into a sparse graph: in this case our variant of Henzinger-King became again the fastest algorithm. For more structured graphs (i.e., small values of s), sparsi cation was slightly faster than HenzingerKing up to a certain number of updates, and then for larger sequences of updates (that leave room for amortization) the original algorithm of Henzinger-King took over. For instance, we reported the following behavior for s = 2. On graphs with 200 vertices, we needed around 400 updates for Henzinger-King to outperform sparsi cation; on graphs with 400 vertices, this threshold moved to around 700 updates; on graphs with 600 vertices, around 1000 updates were needed; for 800 vertices, around 1750 updates; nally for graphs with 1000 vertices, around 2500 updates were needed. For larger values of s, all these thresholds separating sparsi cation and Henzinger-King became smaller. The experiments for s = 2 and n = 400 and 1000 are illustrated in Figures 2 and 3.

implementations also on structured, non-random in- References puts. We now describe the graphs and sequences of updates we used for these experiments. These graphs [1] D. Alberts. Implementation of the dynamic connectivare characterized by two parameters: n, the number of ity algorithm by Henzinger and King. TR B 95{10,

10

Alberts et al.

n = 400 Sparsification Henzinger-King

# op / sec

1000

100

10

1

0.1 10

100 # updates

1000

10000

Figure 2: The throughput (number of operations per second) of Henzinger-King and sparsification on non-random graphs with 400 vertices for dierent lengths of the sequence of updates.

n = 1000 Sparsification Henzinger King

# op / sec

1000

100

10

1

0.1 10

100 # updates

1000

10000

Figure 3: The throughput (number of operations per second) of Henzinger-King and sparsification on non-random graphs with 1000 vertices for dierent lengths of the sequence of updates.

Freie Universitat Berlin, Inst. f. Informatik, 1995. [2] D. Alberts, M. R. Henzinger, \Average Case Analysis of Dynamic Graph Algorithms", Proc. 6th Symp. on Discrete Algorithms (1995), 312{321. [3] N. Alon and J. H. Spencer. The Probabilistic Method. J. Wiley & Sons, New York, 1991. [4] B. Bollobas. Random Graphs. Academic Press, London, 1985. [5] E. A. Dinitz, \Maintaining the 4-edge-connected components of a graph on-line", Proc. 2nd Israel Symp. on Theory of Computing and Systems (1993), 88{99. [6] D. Eppstein, \Average Case Analysis of Dynamic Geometric Optimization", Proc. 5th Symp. on Discrete Algorithms (1994), 77{86. [7] D. Eppstein, Z. Galil, G.F. Italiano, and A. Nissenzweig, \Sparsi cation|A technique for speeding up dynamic graph algorithms", Proc. 33rd IEEE Symp. on Foundations of Computer Science (1992), 60{69. [8] D. Eppstein, Z. Galil, and G.F. Italiano, \Improved Sparsi cation", Tech. Report 93-20, Dept. of Inform. and Comp. Sci., University of California, Irvine, 1993. [9] D. Eppstein, Z. Galil, G. F. Italiano, T. H. Spencer, \Separator based sparsi cation for dynamic planar graph algorithms", Proc. 25th Annual ACM Symposium on Theory of Computing (1993), 208{217. [10] G.N. Frederickson, \Data structures for on-line updating of minimum spanning trees, with applications", SIAM J. Comput. 14 (1985), 781{798. [11] G.N. Frederickson, \Ambivalent data structures for dynamic 2-edge-connectivity and k smallest spanning trees", Proc. 32nd IEEE Symp. Foundations of Computer Science (1991), 632{641. [12] M. R. Henzinger. Fully dynamic cycle equivalence in graphs. Proc. 35th IEEE Symp. Foundations of Computer Science (1994), 744{755. [13] M. R. Henzinger. Personal communication. 1995. [14] M. R. Henzinger and V. King. Randomized dynamic graph algorithms with polylogarithmic time per operation. Proc. 27th Symp. on Theory of Computing, 1995, 519{527. [15] K. Mehlhorn, S. Naher, \LEDA, A platform for combinatorial and geometric computing", Comm. ACM (1995). [16] K. Mulmuley, Computational Geometry, an Introduction through Randomized Algorithms, Prentice Hall, 1993. [17] G. Ramalingam, \Bounded incremental computation", Ph.D. Thesis, Department of Computer Science, University of Wisconsin at Madison, August 1993. [18] M. Rauch, \Improved data structures for fully dynamic biconnectivity", Proc. 26th Symp. on Theory of Computing (1994), 686{695. [19] J. Westbrook and R. E. Tarjan, \Maintaining bridgeconnected and biconnected components on-line", Algorithmica 7 (1992), 433{464.