we omit the second subscript. Sets will be represented as relations with a distinguished one-point target 1. Every relation R 2 X $1] is a vector. Thus Rx is true ...
Chapter 4
Algorithms from Relational Speci cation Rudolf Berghammer, Burghard von Karger
4.1 Introduction The purpose of a speci cation is to state a problem as clearly as possible. In many cases, the most direct and intuitive way to specify a problem is by writing down a logical predicate like in the phonebook example in Chapt. ?? that describes its possible solutions. We employ the calculus of binary relations for developing ecient algorithms from logical problem speci cations, using the procedure sketched below. The rst task is to obtain a relational speci cation from the original problem description. There are a number of correspondences between logical and relation-algebraic operations, which in many practical examples yield the desired relational speci cation easily. In more complex cases one can use relation algebra extended with direct products and their associated projections (see Sect. 2.5 ). Relational speci cations are very compact, and if the carrier sets are not too large, they can be executed directly in the RELVIEW system [4]. In this way, the desired algorithm can be prototyped rapidly. In some cases the resulting algorithm is good enough, and then the development is complete. More often the relational speci cation is not acceptable as an implementation, and we apply the welldeveloped apparatus of relational algebra to derive a more ecient algorithm. Relational calculations are completely formal manipulations of abstract symbols; they can be checked and pretty-printed by the proof assistant RALF [6]. We illustrate our approach with numerous graph- and order-theoretic examples, including bounds and extremal elements, tests for cycle-freeness and reachability, and algorithms to compute transitive closures and kernels for speci c classes of directed graphs. [chapt:background]
[sect:hetrel prod]
4.2 Preliminaries
Apart from the material in Sect. ?? and Sect. ?? this chapter is mainly self-contained. Therefore we collect some preliminaries here. Matrix notation. A relation R stored on a computer is frequently represented as a Boolean matrix. The yth element of the xth row of a matrix R is traditionally denoted Rxy . Since the entries in a Boolean matrix range over 0 (false) and 1 (true) the proposition Rxy simply states that R relates x to y, or (x; y) 2 R. In the special case where R is a vector, R = R;U, we omit the second subscript. Sets will be represented as relations with a distinguished one-point target 1. Every relation R 2 [X $ 1] is a vector. Thus Rx is true just when x is [sect:background setcalc]
51
[sect:background binrel]
BIBLIOGRAPHY
52
a member of the set represented by R. Residuals. The residual operations were introduced in Sect. ?? . In Boolean matrix notation their de nitions read (S=R)yx () 8 z (Rxz ! Syz ) (RnS )xy () 8 z (Rzx ! Szy ): (4.1) These equivalences are invaluable for removing universal quanti cations. There are two special cases that deserve explicit mention, namely (S=U)y () 8 z Syz (Rn;)x () 8 z Rzx : (4.2) We recall the universal property of left residuals Q;S T () Q S=T; (4.3) and its corollary, the cancellation rule (Q=S ) ;S Q: (4.4) Fixed points. Let (V; v) and (W; v) be complete lattices. Recall from Chapt. ?? that every order-preserving function f on a complete lattice has least and greatest xed points f and f (Tarski's xed point theorem). When f is suciently well-behaved (for example, when V is nite) we can compute f and f iteratively using the formulas in the following theorem. Theorem 4.2.1 Assume f is an order-preserving function on V . a) If f preserves limits of ascending chains (is continuous ) then f = Fi0 f i (? ). b) If f preserves limits of descending chains (is co-continuous ) then f = i0 f i(> ). [sect:background binrel]
[chapt:background]
Another useful way to compute a xed point is to express it in terms of another { hopefully more tractable { xed point. The -fusion rule is most helpful for this. Lemma 4.2.2 (-Fusion Rule) If f : V ! V and g : W ! W are order-preserving, and h : V ! W is strict (h(? ) = ? ) and continuous then h f = g h implies h(f ) = g. There are a number of laws concerning least xed points of composite functions, of which the most useful and well-known is the rolling rule. Lemma 4.2.3 (Rolling Rule) If f : V ! W and g : W ! V are order-preserving then (f g) = f ((g f )). Graph theory. A directed graph is a pair g = (X; R) where X is a nonempty set and R 2 [X $ X ]. The elements of X and R are the vertices and arcs of the graph. A ( nite) path of length n from vertex x to vertex y in g is a sequence p = (p0; p1; : : :; pn ) such that x = p0 , y = pn , and Rp ;p +1 for all i with 0 i n ? 1. In this case, y is called reachable from x via p. If n 6= 0, then p is said to be non-empty , and if p0 = pn , then p is called closed . A cycle is a non-empty and closed path. An in nite path in g = (X; R) is an in nite sequence p = (p0; p1; : : :) such that Rp ;p +1 for all i 0. The following lemma explains the connection between reachability and transitive closure. Lemma 4.2.4 a) y is reachable from x if and only if Rxy . b) y is reachable from x via a nonempty path if and only if R+xy . i
i
i
i
4.3. RELATIONAL PROBLEM SPECIFICATION
53
4.3 Relational Problem Speci cation In the rst stage of every development we rewrite a logical formula into a relational speci cation and the purpose of this section is to illustrate the underlying technique through a collection of examples involving graphs and partial orders. The translation eliminates quanti cation over elements of the carrier sets and often produces very concise \code" that lends itself to further formal manipulation. We roughly devide our speci cations into three classes: Some are ecient algorithms as they stand, others can be executed as prototypes but are too slow for daily use, and some can only be enacted after further re nements.
4.3.1 Eciently Executable Speci cations
In this subsection we present some of the happy cases where the transition from a predicate logic problem description to a relational expression immediately yields an ecient algorithm. Example 4.3.1 (Cycle-freeness) A directed graph g = (X; R) is cycle-free if every path p = (p0 ; : : :; pn ) with n > 0 satis es p0 6= pn . In terms of vertices, this predicate can be expressed as follows: Forall vertices x and y, if y is reachable from x via a non-empty path then x 6= y. Using Lemma 4.2.4 we can rewrite this predicate to 8 x; y (R+xy ! Ixy ): Removing the universal quanti cation, we arrive at a purely relational speci cation of cyclefreeness: cycle-free : [X $ X ] ! IB cycle-free (R) () R+ I: Obviously, the costs of evaluating cycle-free (R) are of the same order as for computing the transitive closure of R, namely cubic time if S. Warshall's well-known algorithm is used (see Sec. 4.4.1). Example 4.3.2 (Difunctionality) Difunctional relations were introduced in [11] and have found many applications in computer science, see for instance [12]. A relation R is difunctional if there are partial functions (univalent relations) f and g such that R = f ;g[ or, equivalently, if the Boolean matrix associated with R can be transformed into block diagonal form by rearranging rows and columns. The latter property can be restated as follows: Whenever two rows x and y have a \1" in the same place, they must be equal. This is expressed by 8 x; y (9 m (Rym ^ Rxm) ! 8 n (Ryn $ Rxn)): By symmetry, the equivalence sign in the above formula may be replaced with an implication. Using elementary predicate calculus the resulting formula can be rewritten to 8 x; n (9 m; y (Rxm ^ R[my ^ Ryn ) ! Rxn): According to the de nition of relational composition, the antecedent becomes (R;R[;R)xn . We have now arrived at the following relational speci cation: difunctional : [X $ X ] ! IB difunctional (R) () R;R[;R R: This is an ecient difunctionality test, which has the same complexity as an algorithm for computing the composition of two relations.
BIBLIOGRAPHY
54
Example 4.3.3 (Bounds and extremal elements) J. Riguet discovered that many concepts involving preorders can be expressed relationally [10]. His de nitions are widely used in computer science [14, 5, 2] and we present them now as relational speci cations. Assume we are given a xed preorder E 2 [X $ X ]. Then y 2 X is a lower bound of s X if
8 z (z 2 s ! Eyz ): Let us represent subsets of X as vectors so that we can write sz instead of z 2 s. Then (4.1) may be used to eliminate the quanti cation in 8 z (sz ! Eyz ) and we see that the set of all lower bounds of s is represented by the vector E=s[. The formal relational speci cation is lwbd : [X $ X ] [X $ 1] ! [X $ 1]
lwbd (E; s) = E=s[:
This is an ecient algorithm and its computational complexity depends on the cost of computing the residual. Now that we have an algorithm for computing the lower bounds of a set, we can exploit it to generate various related sets. For example, the set of all least elements of s (there may be more than one because E is only a preorder) is obtained by computing the intersection of s with its set of lower bounds: least : [X $ X ] [X $ 1] ! [X $ 1]
least (E; s) = s \ lwbd (E; s):
By transposing E we obtain algorithms for computing upper bounds and greatest elements, for example greatest (E; s) = least (E[; s). Hence, a function inf computing the vector of all in ma , or greatest lower bounds, of s can be written as inf : [X $ X ] [X $ 1] ! [X $ 1]
inf (E; s) = greatest (E; lwbd (E; s)):
Now assume that E is antisymmetric (a partial order). Then z 2 X is a minimal element of s X if z 2 s ^ 8 y (Eyz ^ y 6= z ! y 62 s): When we write sy for s 62 y and Iyz for y 6= z, the second conjunct becomes 8 y ((E \I)yz ! sy ) and (4.1) transforms it into ((E \ I)ns)z . Hence, we arrive at the relational speci cation min : [X $ X ] [X $ 1] ! [X $ 1]
min (E; s) = s \ (E \ I)ns:
If s is not a vector but an arbitrary relation R 2 [X $ Y ], then the above derived functions compute bounds and extremal elements column-wise. For instance, if E and R are Boolean matrices, then every column of least (E; R) 2 [X $ Y ] is a point or the empty vector from [X $ 1]. In the rst case the point represents the least element of the corresponding column of R; in the latter case such a least element does not exist. For example, we can take R to be the membership relation in inf ; we then obtain an algorithm for checking if a given nite partial order (X; E ) is a lattice.
4.3.2 Prototyping Relational Speci cations
Using ideas from [2] we will now present relational speci cations of some higher-order objects (sets of sets). Unlike the previous examples, these are not ecient enough to be used in a production environment. They are rapid prototypes, and as such they serve a double purpose. Firstly, they may be executed directly in the RELVIEW workbench for concrete relations,
4.3. RELATIONAL PROBLEM SPECIFICATION
55
allowing tests and experiments (with small inputs). Secondly, they provide a formal starting point for the derivation of more ecient algorithms. Examples of this will be presented in Sections 4.4 and 4.5. In order to write higher-order speci cations, we incorporate the set-theoretic is-element and is-subset-of relations into the calculus. To avoid confusion, we introduce " and on the object-level as synonyms for the meta-level symbols 2 and . Hence, " and may be used in relation-algebraic expressions and formulae like = "n" or 2 . Furthermore, the powerset of a set X is denoted by 2X instead of P (X ).
Example 4.3.4 (Kernels of directed graphs) Let g = (X; R) be a directed graph. A vertex set a X is absorbant if every vertex outside a has at least one successor inside a: 8 x (x 2= a ! 9 y (y 2 a ^ Rxy )): A vertex set s X is stable if no two elements of s are neighbours: 8 x (x 2 s ! 8 y (y 2 s ! Rxy )): If k is stable and absorbant, then k is said to be a kernel . Kernels correspond to winning strategies in certain combinatorial games (\move into the kernel each time it is your turn"); for an overview see [12]. Now, let's construct an algorithm that takes a directed graph g = (X; R) as its input and returns the set of its kernels (represented as a vector over 2X ). First we observe that 8 y (y 2 s ! Rxy ) is equivalent to (R;")xs. Thus, s is stable if and only if 8 x (" \ R;")xs. This, in turn, is equivalent to ((" \ R;")n;)s due to (4.2). Thus, a function stable that returns the vector that represents the set of all stable vertex sets of its argument is de ned by stable : [X $ X ] ! [2X $ 1]
stable (R) = (" \ R;")n;:
An analogous argument shows how to calculate the set of all absorbant vertex sets: absorb : [X $ X ] ! [2X $ 1]
absorb (R) = (Un(" [ R;"))[:
These relational speci cations are algorithmic (though not very ecient), and they can be combined into an algorithm that computes the set of all kernels of a given graph. We obtain kernel : [X $ X ] ! [2X $ 1]
kernel (R) = absorb (R) \ stable (R):
For speci c classes of graphs, more ecient algorithms to compute a single kernel of a directed graph will be derived in Section 4.5.
Example 4.3.5 (Vertex bases) A vertex base of a directed graph g = (X; R) is a minimal set b of vertices from which every vertex in X can be reached via a path. We note that every vertex can be reached from b if and only if
8 x 9 y (y 2 b ^ Ryx ): Since b 2 y can be written as "by , this is equivalent to 8 x ("[;R)bx. Using the residual properties (4.2) we can further re ne this expression to (("[;R)=U[)b, where U is the top element of [X $ 1]. Among all b with this property we require one that is minimal wrt. the
56
BIBLIOGRAPHY
set inclusion relation . Fortunately we already have a relational speci cation for computing minimal elements of a set (see Sec. 4.3.3). So we obtain the relational speci cation base : [X $ X ] ! [2X $ 1] base (R) = min (; ("[;R )=U[) of a function that computes the vector of vertex bases. An even simpler speci cation can be built with the aid of the kernel function of Example 4.3.4. Observe that b 2 2X is a vertex base of g if and only if every vertex in b can be reached from a vertex in b and no vertex in b can be reached from some other vertex in b. In other words: A set b is a vertex base of g = (X; R) just when b is a kernel of g0 = (X; (R[)+ \ I). Therefore we have the equation base (R) = kernel ((R[)+ \ I).
4.3.3 Non-Algorithmic Relational Speci cations
So far, we have considered only relational speci cations without quanti cation, because these can be regarded as algorithmic (provided the underlying sets are nite). To increase expressive power we shall now drop this restriction, and we shall also consider implicit speci cations, where the desired relation is described as the solution of some equation or formula using also quanti cation over relations. Even though they cannot be executed directly, these speci cations are valuable for calculational reasoning about graphs and order relations. But their real interest is that they can serve as starting points for the formal development of ecient algorithms. We shall illustrate this by two examples. In Example 4.3.6 we give a relational de nition of wellfoundedness, and in Section 4.4.3 we will see how it can be re ned into an algorithm for testing cycle-freeness of a nite graph. Furthermore, in Example 4.3.7 we encounter an implicit speci cation of the kernel function, and in Section 4.5 we shall derive from it ecient algorithms for computing kernels of particular types of graphs. Example 4.3.6 (Well-foundedness) A relation R 2 [X $ X ] is called well-founded if there is no in nite path p = (p0; p1; p2; : : :) through the directed graph g = (X; R). In the literature, well-founded relations are also known as progressively nite, noetherian, or de nite. They were introduced into the relational calculus by D. Park as a tool for proving program termination [7]. Since then, well-founded relations have been used in relation-algebraic works on semantics and rewriting systems [12], and for \making induction calculational" [1]. A little re ection shows that R is well-founded if and only if :9 s (s 6= ; ^ 8 x (x 2 s ! 9 y (y 2 s ^ Rxy ))): Note that we have used quanti cation over subsets s of X . In a rst step, we rewrite the above predicate to 8 s (8 x (x 2 s ! 9 y (y 2 s ^ Rxy )) ! s = ;): If we represent sets by vectors, then x 2 s and y 2 s become sx and sy . In combination with the component-wise de nition of composition and inclusion of relations we obtain the test well-founded : [X $ X ] ! IB well-founded (R) () 8 s (s R;s ! s = ;) for well-foundedness. This speci cation involves quanti cation over relations, and therefore cannot be executed directly. Strictly speaking, the quanti cation should range over vectors only, but it is easy to show that this is irrelevant.
4.3. RELATIONAL PROBLEM SPECIFICATION
57
Example 4.3.7 (Kernels as xed points) In Example 4.3.4 we speci ed the vector of all
kernels of a given directed graph g = (X; R). Now, suppose we wish to compute just one kernel (if one exists). For this task it is appropriate to represent vertex sets as vectors v 2 [X $ 1] (rather than points p 2 [2X $ 1]). Then we may replace the predicate x 2 v by vx, so a vector a is absorbant if and only if 8 x (ax ! 9 y (ay ^ Rxy )) which is equivalent to a R;a. Similarly, s is stable if and only if 8 x (sx ! :9 y (Rxy ^ sy )) or s R;s. As a consequence, a vector k 2 [X $ 1] is a kernel of g if and only if k = (k), where the kernel function is de ned by : [X $ 1] ! [X $ 1] (v) = R;v: We would like to de ne a function that computes a single kernel of a relation implicitly by kernel : [X $ X ] ! [X $ 1] kernel (R) = (kernel (R)): Unfortunately, the kernel function is order-reversing in the sense that v w implies (w) (v), so the above task is not solved by Tarski's xed point theorem. Indeed not every directed graph has a kernel.
Example 4.3.8 (Discrete relations) Let p = (p0; : : : ; pn ) and q = (q0; : : :; qm) be paths through the directed graph g = (X; R). Then q re nes p if p0 = q0, pn = qm and p can be obtained from q by removing one ore more vertices. A relation R 2 [X $ X ] is de ned to be discrete if no path through g = (X; R) can be re ned in nitely often. In the special case where R is a strict order relation, [3] gives a component-free relational speci cation of discreteness. This will now be generalized to arbitrary relations. Assume rst that R is discrete. Then every Q 2 [X $ X ] satis es Q R \ (R;Q [ Q;R)Q = ;; (4.5) because otherwise we can construct inductively an in nite sequence of re nements. Conversely, assume that (4.5) holds for every Q 2 [X $ X ]. We claim that R is discrete. To prove this, we de ne a relation S 2 [X $ X ] by Sxy () Rxy and (x; y) can be re ned in nitely often for all x; y 2 X . Clearly R is discrete if and only if S = ;. In order to deduce S = ; from (4.5), we have to show S R \ (R;S [ S ;R). By de nition, we have S R. To prove that S R;S [ S ;R let x; y 2 X with Sxy . As (x; y) can be re ned in nitely, there exists a two-edge path (x; z; y) through g such that either (x; z) or (z; y) is in nitely re nable. So we have Rxz and Rzy , and either Sxz or Szy . In the rst case, we obtain (S ;R)xy , and in the second case we obtain (R;S )xy . Thus, we have proved that R is discrete if and only if (4.5) is true for all Q, so we can specify discreteness by the relational predicate discrete : [X $ X ] ! IB discrete (R) () 8 Q (Q R \ (R;Q [ Q;R) ! Q = ;): This speci cation is not algorithmic because it uses quanti cation over relations.
BIBLIOGRAPHY
58
4.4 Ecient Algorithms for Path Problems Many practical problems that can be formulated in terms of graph theory are concerned with paths. For example, we might wish to determine if there is a path between two given vertices, or to compute the set of all vertices reachable from a given one, or to check if there are any cyclic paths. We shall now present formal derivations of algorithms solving these path problems.
4.4.1 Algorithms for Transitive Closures
Assume g = (X; R) is a directed graph and we are asked to produce a program for testing whether there is a path between two given vertices. If many such questions will be asked about the same graph, and response time is critical, then it is a good idea to compute the transitive closure R+ of R once and for all, so that subsequent queries can be answered by simple look-up (see Lemma 4.2.4). S + i Recall that R = i1 R : A simple induction in combination with the xed point theorem 4.2.1 shows that [ [ R+ = Ri = %i1(;) = %1; i1
i1
where %1; is the least xed point of the continuous function
%1 : [X $ X ] ! [X $ X ]
%1(Q) = R [ R;Q:
Assume that X has n < 1 elements. Then the iteration ; %1(;) %21(;) : : : computes %1 in O(n) steps. We use Pascal-like notation to present the algorithm we have developed in its nal form: function closure (R); Q := ;; S := R; while S 6= Q do Q := S ; S := R [ R;Q end; return S end: To run this program, we can represent the relation-valued variables Q and S by Boolean matrices and use standard functions on matrices for equality tests, multiplication and union. The run time complexity is then O(n4). Let's look for more ecient algorithms. As noted in Sect. ?? , the transitive closure of R is the smallest transitive relation above R. Therefore [sect:background binrel]
\
\
R+ = fQ : R Q ^ Q is transitiveg = fQ : R [ Q;Q Qg: By Tarski's xed point theorem, the last term is precisely the least xed point of the function %2 de ned by %2 : [X $ X ] ! [X $ X ] %2(Q) = R [ Q;Q:
Theorem 4.4.1 %1 = %2.
4.4. EFFICIENT ALGORITHMS FOR PATH PROBLEMS
59
We have outlined above a free-style proof of Theorem 4.4.1, and it is a nice exercise to redo the proof within the relational calculus. The relational style is much more economic, as it uses no general closure arguments, continuity, in nite intersections and suchlike. The only rules concerning xed points are f (f ) = f (computation) and f x provided f (x) x (induction). Therefore relational proofs are easy to check by machine, and we try to stay within the calculus whereever we can. The relational proof of Theorem 4.4.1 requires a lemma which is interesting in itself. Lemma 4.4.2 %1 is transitive. Proof. By the de nition of %1 and the cancellation rule (4:4) we have %1(%1=%1 );%1 = R [ R;(%1 =%1);%1 R [ R;%1 = %1(%1 ) = %1; whence %1(%1=%1 ) %1=%1 by the universal property (4:3). Now the xed point induction rule yields %1 %1=%1. Applying (4:3) again yields %1;%1 %1 , as desired. Proof (of Theorem 4.4.1). From R R [ %2 ;%2 = %2 (%2 ) = %2 we get %1(%2) = R [ R;%2 R [ %2 ;%2 = %2(%2) = %2 which in turn implies %1 %2, by the induction rule. For the converse, we use transitivity of %1 and the inclusion R %1 (which follows from the computation rule) and obtain %2(%1) = R [ %1;%1 R [ %1 = %1: Now the induction rule yields %2 %1. Hence, the transitive closure is computed as the limit of the iteration ; %2(;) %22(;) : : : for the least xed point %2 , which is nite provided X is nite, and we have the second algorithm function closure (R); Q := ;; S := R; while S 6= Q do Q := S ; S := R [ Q;Q end; return S end: The complexity of this algorithm is only O(n3 log n). An even more ecient algorithm for computing transitive closures was proposed by S. Warshall. It employs a clever problem generalization similar forms of which occur in many transformational developments. With relational algebra we can capture this idea in a concise calculation. Denote the vertices of g = (X; R) by x1; : : : ; xn. For 0 i n let Qi be the element of [X $ X ] which relates x to y if and only if there exists a path from x to y all \inner" vertices of which are members of fx1; : : :; xig. If we interpret the vertices as points xi 2 [X $ 1] in the relational sense, then a purely relation-algebraic de nition of Qi is Qi = (R;Ii);R; where Ii = Sij=1 xj ;x[j is the partial identity associated with fx1; : : : ; xig. The following theorem shows how the Qi can be used for iteratively computing the transitive closure of R.
60
BIBLIOGRAPHY
Theorem 4.4.3 The relations Q0; : : :; Qn de ned above satisfy (i) Q0 = R (ii) Qi = Qi?1 [ Qi?1;xi;x[i ;Qi?1 (iii) Qn = R+ : Proof. Equation (i) follows from I0 = ; and ; = I. In the proof of (ii) we will use the star
decomposition rule,
(S [ T ) = (S ;T );S ; (4.6) which appears as theorem (18) in [9]. We also need that, for any vector v, (S ;v;v[) = I [ S ;v;v[: (4.7) To prove (4.7) note that (S ;v;v[) = I [ (S ;v;v[)+ and (S ;v;v[)2 S ;v;v[. Now we can calculate the recursive equation (ii) by Qi = (R;Ii);R = (R;Ii?1 [ R;xi;x[i );R de nition of Ii [ ; ; ; ; ; ; ; = ((R Ii?1) R xi xi ) (R Ii?1) R star decomposition (4:6) by (4:7) = (I [ (R;Ii?1);R;xi;x[i );(R;Ii?1);R = (R;Ii?1);R [ (R;Ii?1);R;xi;x[i ;(R;Ii?1);R = Qi?1 [ Qi?1;xi;x[i ;Qi?1: Finally, equation (iii) follows from In = I and R+ = R;R. By this theorem, the iteration R = Q0 Q1 : : : Qn = R+ yields the transitive closure of R. Using an initialization and a for-loop, it can be implemented as Q := R; for i := 1 to n do Q := Q [ Q;xi;x[i ;Q end: The relation Q [ Q;xi;x[i ;Q contains additionally to the pairs of Q all pairs (y; z) such that y is a predecessor and z is a successor of xi. Given Q as matrix and the points xi as the columns of the identity matrix of the same dimension, the \updated" matrix Q [ Q;xi;x[i ;Q can be computed in quadratic time. Now, we suppose that the for-loop of the above algorithm is performed on a Boolean array representation for relations such that single entries Qjk { representing whether Q holds for the pair (xj ; xk ) or not { can be overwritten. Then we are able to implement the assignment Q := Q [ Q;xi;x[i ;Q joining every predecessor to each successor of xi by two nested loops: function closure (R); Q := R; for i := 1 to n do for j := 1 to n do for k := 1 to n do Qjk := Qjk _ (Qji ^ Qik ) end end end; return Q end: This is precisely Warshall's algorithm for computing the transitive closure of a relation. In the literature, one usually nds a version in which { by a if-then-command { the inner loop is only performed if Qji holds and the assignment is replaced by Qjk := Qjk _ Qik .
4.4. EFFICIENT ALGORITHMS FOR PATH PROBLEMS
61
4.4.2 Reachability Algorithms
Let g = (X; R) be a directed graph and s X . Then y 2 X is reachable from s if Rxy for some x 2 s. If we represent sets by vectors then x 2 s becomes sx and the existential quanti cation can be expressed in terms of relational multiplication, viz. by ((R[);s)y . Removing the index y we arrive at the relational speci cation reach : [X $ X ] [X $ 1] ! [X $ 1]
reach (R; s) = (R[);s
of a function reach that computes the set of vertices reachable from s. This speci cation is executable but su ers from two ineciencies. First, for computing a vector the big intermediate object \matrix" is used. Secondly, its asymptotic time complexity is dominated by the costs for computing the re exive-transitive closure R. We can save memory space by avoiding the computation of R. There may be a huge speed up as well, particularly when R is sparsely populated. De ne
1 : [X $ X ] ! [X $ X ]
1(Q) = I [ R[;Q
Clearly,
reach (R; s) = (R[);s = 1 ;s: We can eliminate the intermediate data structure by merging the xed point computation on relations and the vector multiplication with s into a single iteration. This technique is known as deforestation though in our case it's not trees but matrices that we wish to dispose of. To this end we require the relational version of a well-known lemma from transformational programming. It is a special case of the -fusion rule 4.2.2.
Theorem 4.4.4 If and are order-preserving function on relations with (Q);S = (Q;S )
for every Q then ;S = .
Now let us calculate an appropriate function . For every Q we must have
(Q;s) = 1(Q);s = (I [ R[;Q);s = s [ R[;(Q;s); so we de ne This gives us
: [X $ 1] ! [X $ 1]
(v) = s [ R[;v:
reach (R; s) = 1 ;s = : When X is nite, can be computed as the limit of the iteration ; (;) 2(;) : : : . Thus we have derived the following algorithm for computing the set of all vertices of the graph g = (X; R) that can be reached from s:
(R; s); v := ;; w := s; while v 6= w do v := w; w := s [ R[;v end; return w end:
function reach
BIBLIOGRAPHY
62
Using Boolean matrices and vectors for implementation, this reachability algorithm has the same worst case run time than the original algorithmic speci cation, but we have saved the memory space for storing R. What can we do about time-wise eciency? The techniques we used in the previous section for speeding up the computation of transitive closures don't seem to work here, so we shall try a di erent tack. A lot of e ort can be saved by keeping track of the vertices already visited. Consider the function which maps a pair of vertex sets u and v to the set of all vertices contained in u or reachable from v via a path without vertices from u: f (u; v) = u [ reach (R \ U;u[; v): Here R \ U;u[ contains only those edges of g = (X; R) that end outside u. Since reach (R; s) = f (;; s); any implementation of the function f will solve the reachability problem. We already saw that reach (R; s) = for any R and s. We can use this fact to compute the second disjunct of f as reach (R \ U;u[; v) = (u; v) where the function (u; v) is de ned by (u; v) : [X $ 1] ! [X $ 1] (u; v)(w) = v [ (R \ U;u[)[;w: This de nition generalizes our earlier de nition of . Since u is a vector, we can simplify as follows: (u; v)(w) = v [ (R \ U;u[)[;w = v [ (u;U \ R[);w = v [ (u \ R[;w): (4.8) To implement f (u; v) = u [ (u; v) (4.9) more eciently, we aim at fusing the outer operation (joining u) into the xed point computation. This will yield a recursion which computes f (u; v) directly. We terminate the recursion when v = ;. Since (u; ;)(;) = u \ R[;; = ; implies (u; ;) = ; we have f (u; ;) = u [ (u; ;) = u: As long as v 6= ;, we recur, using the following identity:
f (u; v) = u [ (u; v) = u [ (u; v)((u; v)) = u [ v [ (u \ R[;(u; v)) = u [ v [ (u \ v \ R[;(u; v)) = u [ v [ (u [ v; u [ v \ R[;v) = f (u [ v; u [ v \ R[;v)
by (4.9) (u; v) is a xed point de nition of (4.8) v [ X = v [ (v \ X ) by -fusion 4.2.2, see below by (4.9):
It remains to verify the -fusion step. We used the \transfer" function h de ned by h(Q) = u \ v \ R[;Q: Clearly h is strict and continuous. So the fusion step in the preceding calculation is justi ed by the -fusion rule 4.2.2, provided we prove that h (u; v) = (u [ v; u [ v \ R[;v) h:
4.4. EFFICIENT ALGORITHMS FOR PATH PROBLEMS
63
This is achieved by the following calculation:
h((u; v)(w)) = u \ v \ R[;(v [ (u \ R[;w)) = u [ v \ R[;(v [ (u \ v \ R[;w)) = u [ v \ R[;(v [ h(w)) = (u [ v \ R[;v) [ (u [ v \ R[;h(w)) = (u [ v; u [ v \ R[;v)(h(w)): This calculation completes the proof that for all vectors u and v, f (u; v) = f (u [ v; u [ v \ R[;v): If the vertex set X is nite, then we de nitely reach the termination case eventually, because the rst argument of f strictly increases with every call. Since the recursion is tail-recursive, we can easily rewrite it as a while loop. Here is the reachability algorithm in its nal form: (R; s); u := ;; v := s; while v 6= ; do u := u [ v; v := u \ R[;v end; return u end:
function reach
In each iteration the set u of vertices visited so far is enlarged by its proper successors v, so the number of iterations it takes to discover some vertex x is just the distance between x and s. This is typical of a breadth- rst search. If we implement R by successor lists, the time consumed by this algorithm is quadratic in the number of vertices.
4.4.3 Well-Foundedness and Testing Cycle-Freeness
In Example 4.3.6 we characterized well-founded relations by the property s R;s =) s = ; for every vector s 2 [X $ 1]: In the following, we aim at an algorithmic test. The antecedent s R;s and Tarski's xed point theorem draw our attention to the function 1 de ned by 1 : [X $ 1] ! [X $ 1] 1(v) = R;v: Since 1 = 1(1) = R;1 well-foundedness of R implies 1 = ;. The reverse implication also holds, because 1 is the greatest vector v with v R;v. Thus, we have proved well-founded (R) () 1 = ;: Based on this equivalence, an algorithm for testing well-foundedness has to compute the chain U 1(U) 12(U) : : : until it becomes stationary and afterwards to test whether ; is the last chain element, i.e., the greatest xed point 1. Using graph-theoretic notation, the kth chain element represents the set of vertices from which a path of length k emerges. If no vertex of the directed graph g = (X; R) lies on an in nite path, then g is cycle-free. We can state and prove this claim formally within the calculus.
BIBLIOGRAPHY
64
Theorem 4.4.5 Let g = (X; R) be a nite directed graph. If R is well-founded then g is
cycle-free. If X is nite, the converse is also true. Proof. Assume R is well-founded. We claim that R+ is also well-founded. For assume x = R+ ;x. Then x = R;(I [ R+ );x = R;x, whence x = ;. Thus x = ; is the only solution of x = R+ ;x and R+ is indeed well-founded. Since we have (R+ \ I);U = (R+ \ I);(R+ \ I);U R+ ;(R+ \ I);U; it follows that (R+ \ I);U = ;. Now Tarski's rule yields R+ I, so g = (X; R) is cyclefree. Conversely, if X is nite and R+ I then some power of R vanishes, hence so does T i 1 = i0 R ;U. Together with the greatest xed point characterization of well-foundedness, this theorem yields an algorithm for testing cycle-freeness of a nite directed graph g = (X; R): function cycle-free
(R);
v := U; w := R;U; while v 6= w do v := w; w := R;v end; return w = ; end:
If n = jX j, then this algorithm runs in time O(n3) which is the same time complexity as the algorithmic speci cation R+ I given in Example 4.3.1. However, we have saved the memory space for storing the transitive closure. Instead with the entire vertex set represented by U, the exhaustion process can also be started with the empty set. This approach is taken is [12] and is based on a least xed point characterization of well-foundedness. A relation R is well-founded if and only if 2 = U, where the function 2 on vectors is de ned by 2(v) = R;v: Compared with 1, the disadvantage of the function 2 is that it is non-continuous, making sometimes proofs and calculations more dicult.
4.5 Ecient Algorithms for Kernels In Example 4.3.7 we characterized kernels as xed points of the function (v) = R;v. For speci c classes of graphs, the rolling rule 4.2.3 allows us to transform this speci cation into an ecient algorithm. We present two cases in which this approach works.
4.5.1 Well-Founded Graphs
The kernel function (v) = R;v of the directed graph g = (X; R) is order-reversing. Not every order-reversing function on a complete lattice has a xed point, but the following theorem helps to nd one when one exists. We use the following notation: The square f 2 of a function is de ned by f 2(x) = f (f (x)).
4.5. EFFICIENT ALGORITHMS FOR KERNELS
65
Theorem 4.5.1 Let V and W be two complete lattices and assume f : V ! W and g : W ! V are order-reversing. Then f g and g f are order-preserving, and we have (i) f ( (g f )) = (f g) (ii) f ((g f )) = (f g):
In the special case where V = W and f = g we obtain (iii) f (f 2 ) = f 2 (iv) f (f 2 ) = f 2 : In particular, if f 2 has a unique xed point, then this is also the only xed point of f . Proof. (Due to R. Backhouse.) Let W op denote the lattice obtained from W by reversing the order. Then f : V ! W op and g : W op ! V are order-preserving and (i) is precisely the rolling rule 4.2.3 (note that least xed points in W correspond to greatest xed points in W op). The second claim can be proved by a dual argument and the rest are obvious corollaries. The following theorem serves as a tool for establishing below a criterion for kernels to exist. Theorem 4.5.2 If R is well-founded then the function (v) = R;v has a unique xed point. Proof. Starting with the computation rule 2 2( 2 ) the de nition of and contraposition yield R;R; 2 2 which in turn is equivalent to R[; 2 R; 2 due to the Schroder equivalence. Now, we use the Dedekind rule and this inclusion and get R; 2 \ 2 R;( 2 \ R[; 2) R;( 2 \ R; 2): Since R is well-founded, this shows R; 2 \ 2 = ; and the de nition of and Theorem 4.5.1 (iv) imply 2 ( 2) = 2. Hence, 2 has a unique xed point and this is also the only xed point of because of Theorem 4.5.1. The following theorem follows immediately from Theorem 4.5.2 and the fact that every order-preserving function between nite sets is continuous. Theorem 4.5.3 Assume that R is well-founded. Then the directed graph g = (X; R) has a unique kernel. Moreover, if X is nite then the kernel of g is the limit of the sequence ; 2(;) 4(;) : : :, where (v) = R;v. Assume that the vertex set X is nite. We have noted before that in this case R is wellfounded just when g = (X; R) is cycle-free. Thus, a nite acyclic directed graph has a unique kernel, which is computed by the algorithm function kernel (R); k := ;; v := R;U; while k 6= v do k := v; v := R;R;k end; return k end: To estimate the complexity of this program note that k increases in each turnaround of the while loop, so that there can be at most jX j iterations. However, when R is transitive, there will be at most one iteration! This follows from the Lemma 4.5.5 below.
BIBLIOGRAPHY
66
Lemma 4.5.4 If R is well-founded then U = R;R;U (from each vertex there is a path to a terminal vertex). Proof. See Prop. 6.3.3 in [12].
Lemma 4.5.5 If R is transitive and well-founded then R;R;P = R;U for every P . Proof. From the previous lemma and transitivity we obtain
R;U = R;R ;R;U = R+ ;R;U = R;R;U R;R;P : The converse inequation is trivial, so the claim follows.
4.5.2 Bipartite Graphs
A bipartite directed graph is a quadruple g = (X; Y; R; S ) where X and Y are disjoint vertex sets, R 2 [X $ Y ] is the set of arcs from X to Y and S 2 [Y $ X ] is the set of arcs from Y to X . Let us represent the given bipartite directed graph g = (X; Y; R; S ) as an \ordinary" directed graph g = (V; B ); where V = X + Y , the disjoint union of X and Y , and the matrix B 2 [V $ V ] has the special form !
R : B = OS O In the previous section we found a kernel of g by looking at the square of the kernel function (v) = Bv. In the present case, we have
2
!
vX = O R S O vY
!
;
O R S O
!
;
!
vX = R;S ;vX vY S ;R;vY
!
which suggests considering the following pair of order-reversing functions on vectors:
: [Y $ 1] ! [X $ 1]
: [X $ 1] ! [Y $ 1]
(v) = R;v
(w) = S ;w:
From this we immediately obtain two kernels ( xed points of ), namely,
k1 = (( ))
!
!
k2 = (( )) :
To prove that, say, k1 is indeed a xed point of , we appeal to Theorem 4.5.1, and calculate
; R S ;
!
;
!
!
!
!
( ) = R;( ) = (( )) = ( ) : ( ) S ; ( ) ( ( )) ( )
The above argument relies on a set-theoretical interpretation of the direct sum V = X + Y and uses component-wise reasoning. We will now show that this kernel construction can be achieved within pure relational algebra. To achieve this, we need the relational analogue of
4.5. EFFICIENT ALGORITHMS FOR KERNELS
67
disjoint union. A direct sum X + Y of X and Y is given by two injections 2 [X $ (X + Y )] and 2 [Y $ (X + Y )] satisfying
;[ = I
;[ = I
[; [ [; = I
;[ = ;
;[ = ;:
These ve equations characterize the direct sum up to isomorphism [14]. Note that the fourth and fth equations are equivalent. With the aid of the injections and , the relations B , k1, and k2 (which we previously de ned in terms of components) can be expressed in abstract relational algebra as
B = [;R; [ [;S ;
k1 = [; ( ) [ [;( )
k2 = [;( ) [ [; ( ):
We have to show that B ;k1 = k1, and in doing so we shall only use relational calculus, the characteristic properties of and , and the preceding de ning equations. By the Dedekind rule we have [;U \ [;U [;(U \ ;[;U) = ;: On the other hand,
[;U [ [;U = ([;U [ [;U);U ([; [ [;);U = U; which shows that [;U and [;U are each other's complements. It follows that
[;U = [;U [; ( )
[;U = [;U [;( ):
(4.10)
From ;[ = I and ;[ = I we get that and are univalent relations. It is easy to show that every univalent relation Q satis es the equation Q;R = Q;R \ Q;U, whence
[; ( ) = [; ( ) \ [;U
[;( ) = [;( ) \ [;U:
(4.11)
Combining this with the preceding formulas we obtain
[; ( ) [ [;( ) = ([; ( ) \ [;U) [ ([;( ) \ [;U) = ([; ( ) [ [;( )) \ ([; ( ) [ [;U) \ ([;U [ [;( )) \ ([;U [ [;U) = ([; ( ) [ [;( )) \ [; ( ) \ [;( ) = [; ( ) \ [;( );
by (4:11) by (4:10)
and this enables us to prove the original claim, since
B ;k1 = ([;R; [ [;S ;);([; ( ) [ [;( )) = [;R;( ) [ [;S ; ( ) = [; (( )) [ [; ( ( )) = [; ( ) [ [;( ) = [; ( ) \ [;( ) = [; ( ) [ [;( ) = k1:
axioms of the direct sum Theorem 4.5.1 see above
Thus, we have proved a generalization of Richardson's well-known theorem, which states that every nite bipartite directed graph has at least one kernel. Unlike Richardson, we did not need to assume niteness. The full statement of our theorem reads as follows:
BIBLIOGRAPHY
68
Theorem 4.5.6 Every bipartite directed graph g = (X; Y; R; S ) has two distinguished (but
not necessarily distinct) kernels k1 and k2, given by
k1 = [; ( ) [ [;( ) k2 = [;( ) [ [; ( ); where and denote the canonical injections from X and Y into X + Y and and are given by (v) = R;v and (w) = S ;w. An early version of this theorem goes back to [13]. If the vertex set X + Y is nite, and are continuous, and their least and upper xed points can be obtained by iteration. More precisely, the sequence U ( )(U) ( )2(U) : : : converges in nitely many steps to ( ), and similarly ; ( )(;) ( )2(;) : : : stabilizes at ( ). Here is the nal algorithm, which is of polynomial complexity: function kernel
(R; S );
k := U; v := R;S ;U; while k 6= v do k := v; v := R;S ;k end; k := ;; w := S ;U; while k 6= w do k := w; w := S ;R;k end; return [;v [ [;w end:
Our representation of a bipartite directed graph uses two disjoint sets X and Y and two relations R 2 [X $ Y ]; S 2 [Y $ X ]. In contrast, [12] take an \ordinary" directed graph g = (V; B ) and use two complementary vectors v; w to de ne bipartiteness, and proceed to prove an accordingly modi ed version of Theorem 4.5.6. Interestingly, their representation yields a di erent algorithm, which rst computes the bounds 2 and 2 of the set of all kernels using the original kernel function and then obtains the speci c kernels k1 and k2 as k2 = (v \ 2 ) [ (w \ 2): k1 = (v \ 2) [ (w \ 2 ) This procedure is less ecient than ours, because during the iterations it works with the entire relation B and a vector representing the entire vertex set, whereas our algorithm uses only the smaller parts R and S and two \shorter" vectors for the disjoint parts of the vertex set.
4.6 Conclusion We have proposed relational algebra as a practical means for formal problem speci cation, prototyping, and algorithm development. Naturally, relations are best suited for reasoning about discrete programming problems, particularly concerning graphs and order relations. We have shown that many common programming tasks can be speci ed very tersely in terms of relations, and that relation algebra is a smooth and formal environment for transforming speci cations into executable programs. The success of relational reasoning is due to the
4.6. CONCLUSION
69
simple and \linear" nature of relational expressions, which allows formal and often concise manipulations. Quanti ers are packed into relational operations and a law involving these often compresses a series of logical inferences into a single step. Since the entire development process works only with formal objects, one of the basic requirements for machine support is ful lled. A typical mechanical task for a computer is validating a speci cation by building a rapid prototype to see whether the intention is met in special cases. The RELVIEW system is such a tool. Another important task is interactive proof support, which is greatly facilitated by the small and simple set of relational axioms. The RALF system allows the user to construct a proof interactively, at each step clicking the mouse on the next rule from a selection o ered by the system. The resulting proof is correct by construction and can be pretty-printed in LaTEX. The relational approach is wonderful when it works, but it is limited by its lack of expressiveness. Its expressive power can be increased enormously by including direct products and projections, but experience has shown that these tend to produce complicated encodings and lose much of the elegant simplicity of the pure calculus. Other authors are experimenting with n-ary relations and even formal languages. Our own work concentrates on the sequential calculus [8] which has slightly weaker axioms than the relational calculus, but a much wider range of computationally relevant models.
Acknowledgement: We thank R. Backhouse for reading a draft version of this chapter and for his stimulating comments.
70
BIBLIOGRAPHY
Bibliography [1] Backhouse R.C., Doornbos H.: Mathematical induction made calculational. Report Nr. 94/16, Department of Mathematics and Computing Science, Eindhoven University of Technology (1994) To appear in Theoret. Comput. Sci. [2] Berghammer R., Gritzner T., Schmidt G.: Prototyping relational speci cations using higher-order objects. In: Heering, J., Meinke, K., Moller, B., Nipkow, T. (eds.): Proc. Int. Workshop on Higher Order Algebra, Logic and Term Rewriting (HOA 93), Amsterdam, The Netherlands, Sept. 1993, LCS 816, Springer, 56-75 (1994) [3] Berghammer R., Schmidt G.: Discrete ordering relations. Discrete Mathematics 43, 1-7 (1983) [4] Berghammer R., Schmidt G.: The RELVIEW-system. In: Cho rut C., Jantzen M. (eds.): Proc. 8th Annual Symposium on Theoretical Aspects of Computer Science (STACS '91), Hamburg, Febr. 1991, LCS 480, Springer, 535-536 (1991) [5] Bird R., de Moor O.: From dynamic programming to greedy algorithms. In: Moller B., Partsch H., Schuman S. (eds.): Formal program development, Proc. of an IFIP TC2/WG2.1 State of the Art Seminar, Rio de Janeiro, Jan. 1992, LCS 755, Springer, 43-61 (1993) [6] Hattensperger C., Berghammer R., Schmidt G.: RALF { A relation-algebraic formula manipulation system and proof checker. In: Nivat M., Rattray C., Rus T., Scollo G. (eds.): Proc. 3rd Conf. on Algebraic Methodology and Software Technology (AMAST '93), University of Twente, The Netherlands, June 1993, Workshops in Computing, Springer, 407-408 (1993) [7] Hitchcock P., Park D.: Induction rules and termination proofs. In: Nivat M. (ed.): Proc. Automata, Languages and Programming (ICALP '72), Rocquencourt, France, July 1972, orth Holland, 225-251 (1973) [8] von Karger, B., Hoare, C.A.R.: Sequential Calculus. Information Processing Letters 53, 123{130 (1995) [9] Mathematics of Program Construction Group Eindhoven: Fixed-point Calculus. Information Processing Letters 53, 131{136 (1995) [10] Riguet J.: Relations binaires, fermetures, correspondances de Galois. Bull. Soc. Math. France 76, 114-155 (1948) [11] Riguet J.: Quelques proprietes des relations difonctionelles. C. R. Acad. Sci. Paris 230, 1999-2000 (1950) [12] Schmidt, G., Strohlein, T.: Relations and graphs. Discrete Mathematics for Computer Scientists, EATCS Monographs on Theoret. Comput. Sci., Springer (1993) [13] Strohlein T.: Untersuchungen uber kombinatorische Spiele. Doctoral Thesis, Technische Universitat Munchen (1970) 71
