Certifying Algorithms (A Paper under Construction)

Certifying Algorithms (A Paper under Construction) Kurt Mehlhorn Arno Eigenwillig Kanela Kanegossi Dieter Kratsch Ross McConnel Uli Meyer Jeremy Spinrad 1998 to 2005 Abstract

TODO Contents 1 Introduction

2

2 Some Warning Examples

6

3 Relation to Extent Work

6

4 Examples of Certifying Algorithms 4.1 Maximum Cardinality Matchings in Graphs . . . . . . . . . . . . . . . 4.2 Planarity Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Five Coloring a Planar Graph . . . . . . . . . . . . . . . . . . . . . . . 4.4 Verification of Higher-dimensional Convex Polyhedra and Convex Hulls 4.5 Solving Linear Systems of Equations . . . . . . . . . . . . . . . . . . . 4.6 Approximation Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Traveling Salesman Tours . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.9 Composition of Functions . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Further Examples and Guide to Literature . . . . . . . . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

. . . . . . . . . .

7 7 8 9 10 14 15 16 16 17 17

5 Correct Checkers 19 5.1 The Pragmatic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 5.2 Formal Verification of Checkers . . . . . . . . . . . . . . . . . . . . . . . . . . 19 6 Programs with Preconditions

20 1

7 Verification and Certification

20

8 An Attempt of a Theory of Certifying Algorithms 8.1 Towards a Definition . . . . . . . . . . . . . . . . . . . . 8.2 Monte Carlo Algorithms resist Certification . . . . . . . . 8.3 Every Deterministic Program has a Certifying Counterpart 8.4 And with Non-Trivial Preconditions . . . . . . . . . . . .

21 21 23 23 24

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

9 Reactive Programs and Data Structures 25 9.1 The Dictionary Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 9.2 Priority Queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 10 Conclusions and Open Problems

30

1 Introduction We call a program certifying if it does not only give the output, it is supposed to give, but also a certificate or proof that this output is the correct one for the given input. We advocate certifying algorithms and programs as a pragmatic approach to program correctness. A conventional Y receives an input x
X and produces an algorithm or program computing a function1 f : X output y
Y . The intention is that y is equal to f x  . However, given the pair x  y  , the caller of  the program has, in general, no way to know whether y is indeed equal to f x  . He has to believe the program. A certifying program, see Figure 1, for f produces not only y but an additional output w 2 (called the witness  or certificate) which certifies (gives convincing  evidence for, proves ) the First, given a triple x  y w  it is easy to check equality “y  f x  ”. This means two things.    whether w is a valid certificate for “y f x ”, and, second, if w is a valid certificate, then y is  indeed equal to f x  . Observe that these are exactly the requirements on a formal proof w for a  statement ϕ (here “y  f x  ”): it is easy to check whether w is a valid proof for ϕ and if w is a valid proof then ϕ is actually true. We will attempt a formal definition of certifying programs in Section 8. What  does it mean that is is easy to check whether w is a valid certificate for the equality “y  f x  ”? We postulate that we have a  correct program C which accepts a triple x  y w  if and only if w is a valid certificate for “y  f x  ”. We call C the checker and discuss the assumption of having a correct checker in Section 5. It is time for a simple example to illustrate the notion.

 Testing Bipartiteness of a Graph: A graph G  V  E  is called bipartite if its vertices can be colored with two colors, say red and green, such that every edge of the graph connects a red  and a green vertex. Consider the function is bipartite : set of all finite graphs 0  1  which 1 We

2 We

will later also consider algorithms specified by pre- and postconditions. prefer the word “certify” over “prove” because is does not have a mathematical meaning.

2

for a graph G has value 1 if the graph is bipartite and has value 0 if the graph is non-bipartite. A conventional algorithm for this function returns a boolean value and so does a conventional program. Does the bit returned by the conventional program really give additional information about the graph? We believe that it does not. What are witnesses for being bipartite and for being non-bipartite? What could a certifying algorithm for testing bipartiteness return? A witness for being bipartite is a two-coloring of the vertices of the graph, indeed this is the definition of being bipartite. A witness for  being non bipartite is an odd cycle, i.e., an odd-length sequence of edges v 0  v1  , v1  v2  , . . . , v2k  v2k  1  forming a closed cycle, i.e., v0  v2k  1 . Indeed, for all i, vi and vi  1 must have distinct colors, and hence all even numbered nodes have one color and all odd numbered nodes have the other color. But v0  v2k  1 and hence v0 must have both colors, a contradiction. Three simple observations can be made at this point: First, a two-coloring certifies bipartiteness (in fact, it is the very definition of bipartiteness) and it is very easy to check if an assignment of colors red and blue to the vertices of a graph is a two-coloring. Second, an odd cycle certifies non-bipartiteness, as we argued above. Also, it is very easy to check that a given set of edges is indeed an odd cycle in a graph. Observe that there is no need to know that a non-bipartite graph always contain an odd cycle. One only needs to understand, that an odd cycle proves non-bipartiteness. It is quite reasonable to assume that a program C for testing the validity of a two-coloring and for verifying that a given set of edges is an odd cycle can be implemented correctly. It is even within the reach of formal program verification to prove the correctness of such a program. Let us finally argue that there is certifying algorithm for bipartiteness. In fact, a small variation of the standard greedy algorithms will do: (1) choose an arbitrary node r and color it red; also declare r unfinished. (2) as long as there are unfinished nodes, choose one of them, say v, and declare it finished. Go through all uncolored neighbors, color them with the other color (= the color different from v’s color), and add them to the set of unfinished nodes. Also, record that they got their color from v. When step (2) terminates, all nodes are colored. (3) Iterate over all edges and check that their endpoints have distinct colors. If so, output the two-coloring. If not, let e  u  v  be an edge whose endpoints have the same color. Follow the paths p u and pv of color-giving edges from u to r and from v to r; the paths together with e form an odd cycle. Why? Since u and v have the same color, the paths pu and pv either have both even length (if the color of u and v is the same as the color of r) or have both odd length (if the color of u and v  differs from r’s color). Thus the combined length of pu and pv is even and hence the edge u  v  closes an odd cycle. We summarize: There is a certifying algorithm for testing bipartiteness of graphs and it is reasonable to assume the availability of a correct checker for the witnesses produced by this algorithm. We next turn to a first discussion of the advantages of certifying algorithms. Assume that we have a certifying program P for a function f and a correct checker C. What do we gain?

3

x

x

program for f

certifying prog for f

y

y

accept

checker C w

reject

Figure 1: The top figure shows the I/O behavior of a conventional program for computing a function f . The user feeds an input x to the program and the program returns an output y. The  user of the program has no way to know whether y is equal to f x  .  A certifying program computes y and a witness w. The checker C accepts the triple x  y w  if and only if w is a valid certificate for the equality y  f x  .

  Instance Correctness: If the checker accepts x  y w  , w is a valid proof of “y  f x  ” and  hence y is equal to f x  . We are certain that our (certifying) program worked correctly on the instance x. We emphasize that we do not know that the  program will work correctly for all inputs, we only know it for instance x. If the checker rejects x  y w  , w is not a valid certificate for the equality “y  f x  ” and we know that our program erred either in the computation of y or the computation of the certificate w. Testing on all Inputs: We can now test our program on every input x and not just on inputs for which we already know the right answer by other means. We can even test our program whenever it is executed. Testing is the most popular method to ensure correctness of implementations. The implementation is run on inputs for which the correct result is already known and only released if the correct output is produced for all test inputs. The standard objection to testing is, of course, that it can only prove the presence of errors and not the absence of errors. Although true, there is also a second more pragmatic objection: one can only test on inputs for which the answer is already known. What if your algorithm is the first for a certain task and you know only a very small set of inputs with known answers or if your algorithm is much more efficient than all previous algorithms and hence you can run the other algorithms only for small inputs. Confinement of Error: Whenever a certifying program fails on an input, the checker catches the error. In a system consisting of certifying components, errors are caught by the failing module and does not spread over other modules. This greatly simplifies the task of finding the source of the error.

4

More Effective Program Development: Program modules are usually incorrect or incomplete during much of their development phase. Confinement of error is particularly important during this phase. Trust with Minimal Intellectual Investment: Which intellectual investment does a user have to make in order to trust a certifying program. Not  much. First, he has to understand, what constitutes a valid certificate for the equality “y  f x  ”, and second he has to verify that the checker is correct. Verified Checkers: As we will see in the examples in later sections, checkers are frequently so simple, that their formal verification is feasible. Also, they are frequently so efficient compared to the program itself, that we may be willing to write them in a less efficient programming language which eases verification. Black-Box Programs: In order to develop trust in a certifying program there is no need to have access to the source text of the program. It suffices to have access to the source of the checker since only its correctness needs to be verified. In other words, the intellectual property can be kept secret and yet trust can be developed. Certifying Algorithms, a Challenge for Algorithmics: Most existing algorithms are noncertifying. It is a challenge for algorithmics to find certifying algorithms which are as efficient as the existing non-certifying ones. Safer Programming: Turning a correct algorithm into a correct program is an error-prone task. An algorithm is the description of a problem solving method intended for human consumption. It is usually described in natural language, pseudo-code, mathematical notation or a mixture thereof. A program is the description of a problem solving method intended for execution by a machine. It is written in a programming language and usually much more detailed than the algorithm. Certifying algorithms are easier to implement correctly than non-certifying algorithms because they can be tested on all inputs. Practical Experience: We developed the concept of certifying algorithms during the design of the LEDA [MN99, LED] library of efficient data types and algorithms, the name “certifying algorithm” was coined in [KMMS03]. We started to build LEDA in 1989 [MN89, NM90]. We implemented conventional algorithms and some checks for assertions and post-conditions and we tested extensively. Nevertheless at least two major programs were incorrect when we first releases them: the planarity test, see Sections 2 and 4.2 and the convex hull algorithms in arbitrary dimensional spaces, see Section 4.4. In the attempt to correct the errors in these programs we developed the concept of certifying program and reported about it in [MNU97, MN98]. For the LEDA book [MN99], many algorithms were reimplemented and a large number of the algorithms in LEDA were made certifying, in particular, the graph and geometry algorithms. However, there 5

are still large parts which are non-certifying, e.g., all algorithms working on the number types of LEDA. Hidden Assumptions: A checker can only be written if the problem at hand is rigorously defined. We noticed that some of our specifications in LEDA contained hidden assumptions which were revealed during the design of the checker. For example, an early version of our biconnected components algorithm assumed that the graph contains no isolated nodes.

2 Some Warning Examples We give some examples of failures in algorithmic software. Planarity Test in LEDA 2.0: The planarity test in LEDA 2.0 was incorrect. It declared some planar graphs non-planar. We corrected the error in LEDA 2.1 and made the planarity test certi- check the fying. This was our first use of a certifying algorithm. Our first solution was far from satisfying. history planarity threshold. Intersection of Solids: Rhino3d (a CAD systems) fails to compute correct intersection of two cylinders and two spheres Linear Programming Solvers: CPLEX (a linear programming solver) fails on benchmark problem etamacro. Review [DFK  03]. Mathematica 4.2 (a mathematics systems) fails to solve a small integer linear program   In[1] := ConstrainedMin[ x , x==1,x==2  , x  ]   Out[1] = 2, x->2     In[1] := ConstrainedMax[ x , x==1,x==2  , x  ] ConstrainedMax::"lpsub": "The problem is unbounded."   Out[2] = Infinity, x -> Indeterminate   Pentium Bug: ware [BW96].

A version of the Pentium chip contained an error in the division hard-

3 Relation to Extent Work The theoretically most satisfying method is to prove the correctness of the program and not just the algorithm. Unfortunately, this is still out of scope for complex algorithms, but see [ver]. There is another argument which limits the usefulness of verification. If you want to convince the user of a program of its correctness, you must publish the program (in source-code or binary 6

form) and its correctness proof. Most likely, the proof will reveal the logic and the algorithm behind the program, which you may want to keep secret to protect your intellectual property. Linear programming; good characterization in Shrijver’s book. connection to Manuel Blum and the other work on program checking. Programs that check their work. connection to proof-carrying code connection to good characterizations in OR At the time of this writing LEDA contains checkers for most network algorithms (mostly based on linear programming duality), for planarity testing, for priority queues, and for the basic geometric algorithms (convex hulls, Delaunay diagrams, and Voronoi diagrams). Program checking has greatly increased our confidence in the correctness of our implementations. For further reading on program checking we refer the reader to [SM90, BS94, SM91, BSM97, BS95, BSM95, SWM95, BK89, BLR90, BW96, WB97, MN99, AL94, MNS  99, DLPT97, MNS  96].

4 Examples of Certifying Algorithms We discuss further illustrative examples.

4.1 Maximum Cardinality Matchings in Graphs A matching in a graph G is a subset M of the edges of G such that no two share an endpoint. A matching has maximum cardinality if its cardinality is at least as large as that of any other matching. Figure ?? shows a graph and a maximum cardinality matching. Observe that the matching leaves two nodes unmatched and hence there is the possibility of a a matching of larger cardinality. What is a witness for a matching being maximum cardinality. Edmonds [Edm65b, Edm65a] gave the first efficient algorithm for maximum matchings. The algorithm is certifying. An odd-set cover OSC of G is a labeling of the nodes of G with non-negative integers such that every edge of G is either incident to a node labeled 1 or connects two nodes labeled with the same i and i 2. Let ni be the number of nodes labeled i and consider any matching N. A simple   argument shows  N n1  ∑ ni  2 i 2 Indeed, for i, i 2, let Ni be the edges in N that two nodes labeled i and let N1 be the  connect   remaining edges in N. Then Ni ni  2 and N1 n1 and the bound follows. It can be shown (but this is non-trivial) that for any maximum cardinality matching M there   is an odd-set cover OSC with  M  n1  ∑ ni  2  (1) i 2 thus proving the optimality of M. In such a cover all ni with i 2 are odd, hence the name. A certifying algorithms for maximum cardinality matching returns a matching M and an odd-set cover OSC such that (1) holds. By the argument above, the odd-set cover proves the optimality of the matching. Observe, that is it not necessary to understand why odd-set covers 7

proving optimality exist. It is only required to understand the three three line argument given above that equation (1) proves optimality. Also, a correct program which checks whether a node labelling is an odd-set cover and satisfies (1) is easy to write.

4.2 Planarity Testing A graph is called planar if it can be drawn in the plane without edge crossings, see Figure ?? for some examples. A planarity tester takes a graph G and returns true if G is planar and returns false if G is non-planar. The planarity test played a crucial role for our thinking about certifying algorithms and the development of LEDA. There are several linear time algorithms for planarity testing [HT74, LEC67, BL76]. An implementation of the Hopcroft and Tarjan algorithm was added to LEDA in 91. The implementation had been tested on a small number of graphs. In 93 we were sent a graph together with a planar drawing of it. However, our program declared the graph non-planar. It took us some days to discover the bug. More importantly, we realized that a complex question of the form ”is this graph planar” deserves more than a yes-no answer. We adopted the thesis that a program should justify (prove) its answers in a way that is easily checked by the user of the program. What does this mean for the planarity test? If a graph is declared planar, a proof should be given in the form of a planar drawing or a combinatorial embedding. If a graph is declared non-planar, a proof should be given in the form of a Kuratowski subgraph. Kuratowski [Kur30] has shown that every non-planar graph contains a subdivision of either K5 , the complete graph on five nodes, or K3  3 , the complete bipartite graph with three nodes on either side. Figure ?? shows the Kuratowski graphs and exhibits a subdivision of a K3  3 in a non-planar graph. A subdivision of a graph is obtained by subdividing    edges, i.e., by replacing an edge u  v  by edges u  z  and z  v  where z is a new vertex. Linear time algorithms [Wil84, Kar90] for the computation of Kuratowski subgraphs were known in 93; unfortunately, we found the algorithms too complicated to implement. There is however, a trivial quadratic time algorithm which is very simple to implement. for all edges e of G do if G  e is non-planar then remove e from G; end if end for The output of this algorithm is a subgraph of the original graph. The subgraph is non-planar, since an edge is only removed if non-planarity is preserved, and removal of any edge of it, makes it planar. Thus the subgraph is a minimal (with respect to edge inclusion) non-planar subgraph and hence a subdivision of a Kuratowski graph. So in 94, we had a quadratic time certifying algorithm for planarity and a linear time noncertifying one. We later developed our own linear time algorithm for finding Kuratowski sub8

graphs [MN99, Section 8.7]. The algorithms and its implementation are discussed in the LEDA book. Let us turn to witnesses of planarity. The obvious witness is a planar drawing. There are two drawbacks of using a planar drawing as the witness. First, there is no linear time algorithm to produce planar drawings of planar graphs and second it is non-trivial to check whether a drawing is actually planar. A less obvious form of witness is a combinatorial planar embedding. A combinatorial embedding of a graph is a cyclic ordering of the edges incident to any vertex, see Figure ?? for an example. A combinatorial embedding of a graph G is called planar if there is a planar drawing of G in which the clockwise ordering of the edges incident to any vertex agrees with the cyclic ordering specified in the combinatorial embedding. Already, Euler observed that there is a simple algorithm for checking whether a combinatorial embedding is planar. We view every edge as a pair of oriented edges  and define a successor function on oriented edges. The cyclic successor of the oriented edge u  v  (= the edge e   uv in the orientation from u to v) is the oriented edge v z  where v z  is the edge following v u  in the cyclic order  of edges around v. We write cyclic succ u  v  v z  . Observe that the function is bijective. It therefore decomposes the set of oriented edges of G into disjoint cycles: two oriented edges belong to the same cycle if one can be obtained from the other by zero or more applications of cyclic succ. Each cycle is called a face cycle and f denotes the number of face cycles. Lemma 1 (Euler) A combinatorial embedding of a graph G is planar iff n  m  f  2, where n and m are the number of vertices of G, respectively, and f is the number of face cycles of the embedding. We refer the reader to [MN99, section 8.6] for a proof. The Lemma is the key for checking whether a combinatorial embedding is planar. We simply compute the number of face cycles declare all edges unconsidered and set f to zero; while there is an unconsidered edge e0 do set e to e0 and declare e considered repeat  e  cyclic succ e  ; declare e considered; until e is equal to e0 increment f ; end while and then check the equality n  m  f  2. Linear time algorithms for computing planar embeddings are described in [CNAO85, NC88, MM95, MN99]. The complete linear time certifying algorithm for planarity is now as follows. If the input graph G is planar, the algorithm returns a planar combinatorial embedding of G. The embedding is verified using Euler’s relation. If G is non-planar the algorithm returns a Kuratowski subdivision contained in G.

4.3 Five Coloring a Planar Graph

 Given a planar graph G  V  E  , the goal is to compute a five-coloring of the vertices of G,  i.e., an assignment of the colors 1  2  3  4  5  to the vertices of G such that the endpoints of any 9

a K5 and a K33 Figure 2: The Kuratowski graphs K5 and K3  3 are known to be non-planar. Every non-planar graph contains edge have distinct colors. Every planar graph has a five-coloring, in fact even a four-coloring. A conventional algorithm for five-coloring either produces a five-coloring or fails. A certifying algorithm for our task either produces a five-coloring or a proof that the input graph is non-planar. How can a proof of non-planarity look like? There are two well-known non-planar graphs, the Kuratowski graphs K5 and the K3  3 , see Figure 2. Also, if G can be transformed to a Kuratowski graph by edge deletions and contractions, G is non-planar3 . An edge deletion is the removal of an edge and an edge contraction identifies the endpoints of the edge, see Figure ?? for illustrations. It is worth observing that the implication “G can be transformed into a Kuratowski graph by edge deletions and contractions implies G is non-planar” is trivial, since a planar drawing of G would directly yield a planar drawing of the Kuratowski graph. There is another useful witness of non-planarity. If a simple graph has more than 3n  6 edges, it is non-planar. Alternatively, if a simple graph has no vertex of degree five or less, it is non-planar. We next describe a certifying algorithm for five-coloring a planar graph. We assume the input graph to be simple. The algorithm either outputs a five-coloring of the input graph G (this may happen even if G is non-planar) or a proof that G is non-planar. If G has no node of degree five or less, declare G non-planar. If G has a node v of degree at most four, delete it and color G  v. Then insert v back in and use a color which is not used on one of its neighbors. Finally, consider a node v of degree five. There must be neighbors u and w of v which are not connected by an edge. Otherwise, the neighbors form a K5 and the graph is not planar. Consider the graph G  obtained by deleting v and by collapsing u and w. Observe that G is a contraction of G: first delete all edges incident to v except those connecting it to u and w and then contract the edges connecting it to u and w. If G is planar, so is G  , since edge deletion and contraction preserve planarity. Simplify G  and then five-color G , uncollapse u and w (they have now the same color) and put v back in. There are only four colors used on its neighbors and hence we have a color to use on v.

4.4 Verification of Higher-dimensional Convex Polyhedra and Convex Hulls This section is [MNS  99, Section 2.3]. We first deal with the question whether a simplicial piecewise linear hyper-surface  without boundary in d-dimensional space is the boundary of a convex polytope. We assume that the hyper-surface is given by its facet graph. The facet graph is a d-regular graph whose nodes are labeled by d-tuples of affinely independent points, i.e., each node corresponds to an oriented  d  1  simplex (= a facet of the surface). The hyperplane supporting a facet divides d-space 3 Actually, the converse is also true: if a graph is non-planar it can be transformed to a Kuratowski graph by edge deletions and contractions. The converse is non-trivial to prove but not needed for our discussion.

10

into a positive and a negative halfspace. Neighboring nodes differ in their labeling by exactly one point and for everyone of the d  vertices of a facet there must be such a neighboring facet. In other words, edges correspond to d  2  -simplices shared by two facets. Neighboring nodes must be labeled consistently, i.e., the negative halfspace corresponding to adjacent facets must agree locally. Task 1 Given the facet graph of a simplicial piecewise linear hyper-surface  space decide whether  is the surface of a convex polytope.

in d-dimensional

We take a theorem of differential geometry as the starting point of our solution. Theorem 1 (Hadamard) Let  be a smooth compact surface in  d without boundary and let d  2. If  is locally convex at everyone of its points then  is the surface of a convex body. This theorem suggests that it suffices to check local convexity at every ridge of a simplicial surface. Although this is clearly a necessary condition for global convexity it is not sufficient as Figure 3 shows, i.e., the non-smoothness of simplicial surfaces complicates matters. We will prove the following theorem for the polyhedral case:

o

Figure 3: Local convexity at ridges does not suffice: The figure shows a locally convex yet self-intersecting polygon in the x  y-plane. A smooth version of this curve demonstrates that the condition d  2 is necessary in Theorem 1. Extending the polygon to a bipyramid by adding two points, one each on the negative and positive z-axis, and constructing two triangles for each polygon edge yields a simplicial surface that is locally convex at every ridge but has self-intersections. The point o shown is on the negative side of all facets of the surface.

 Theorem 2 Let  be a simplicial d  1  -dimensional surface without boundary in  d that is consistently oriented, let o be center of gravity of all the vertices of surface  and let p be the center of gravity of some facet of  . Then  is the surface of a convex body iff  

is locally convex at all its ridges, 11

 o is on the negative side of all its facets, and  the ray emanating from o and passing through p intersects only the facet containing p. Before we prove this result we illustrate it on Figure 3. Let o be any point in the x  y-plane that is on the negative side of every facet of the surface shown. All but two rays emanating from o intersect the surface twice and hence witness the non-convexity of the surface. The two exceptional rays go through the two tips of the bipyramid, i.e., pass through a lower dimensional feature of the surface. The key insight underlying our criterion is that this observation is generally true. The conditions listed in Theorem 2 are clearly necessary. We will prove their necessity as follows. We first show that the last two conditions imply that  is the boundary of a star-shaped d-polytope. Next we show that a star-shaped polytope that is convex at all its ridges is convex. Lemma 2 Let  be a simplicial piecewise linear hyper-surface without boundary in  d that is consistently oriented, and let o be a point that is on the negative side of each facet of  . Every ray emanating from o that intersects facets of  only in their relative interiors intersects the same number of facets of  . Proof:  For each facet f of  let c f be the cone with apex o spanned by f . Let A o  F  be the  arrangement formed by all those cones. Each d-dimensional cell c in A o  F  is a cone with apex o. By construction every ray emanating from o that lies in c intersects the same number of facets of  , say nc (although the order in which a ray intersects  these facets is not constant over c). We now claim that nc  nc for all cells c  c in A o  F  . It suffices to show this for cones c  c that share a d  1  -face, say g. In the generic case g must be part of a cone with apex o that is spanned by some ridge of  . This ridge is contained in exactly two facets f  f  of  , one of which, say f , intersects c, and the other intersects c  . (This is a consequence of  being consistently oriented and of f and f  both having o on the same side.) Since except for f and f  the remaining facets of  that intersect c are the same as the ones that intersect c  we can conclude that in the generic case we indeed have nc  nc . In the non-generic case g can be part of several cones spanned by ridges of the surface  . But as before for each of those ridges one of the two containing facets intersects c and the other intersects c  with the remaining facets intersecting c being the same as the ones for c  . Corollary 3 If the last two conditions of Theorem 2 hold then  polytope whose kernel contains o.

is the boundary of a star-shaped

In view of the above it suffices to prove: Let  be a simplicial d  1  -dimensional surface without boundary in  d that is consistently oriented, and let o be a point that is on the negative side of each facet of  . Assume further that  is locally convex at all its ridges and that every ray emanating from o that intersects facets of  only in their interiors intersects exactly one facet of  . Then  is the surface of a convex body. 12

Lemma 3 For every facet f of  let h f be the supporting hyperplane, let A be the arrangement defined by these hyperplanes, and let K be the cell of A containing o. Then every facet of K is part of a facet of  . Proof: Assume otherwise. Let g be a facet of K that is not contained in any facet of  and let C be the other cell of A incident to g. There must be a facet f of  such that g is contained in h f and a ridge r of f that separates f from g in h f . Let f  be the other facet of  incident to r and view the situation in the 2-plane orthogonal to r. In this plane r is a point and f and f  are edges incident to r, K is in the negative halfspace of f and f  , g is a line segment on the boundary of K, on the same line as f and separated from f by r. Thus  is not locally convex at r. We next strengthen the preceding Lemma. Lemma 4 The boundary of K is contained in 

.

Proof: Let x be any point in the boundary of K. Then x is contained in the closure of some facet of K and hence in the closure of some facet of  . We complete the argument by showing Lemma 5 

is the boundary of K.

Proof: The boundary of K is contained in  simplicial surfaces. Hence they are equal.

. Also 

and the boundary of K are closed

The verification procedure described above is easy to program.

 Check local convexity at every ridge. If local convexity does not hold at some ridge declare  non-convex.  Set o to the center of gravity of the vertices of  side of all facets. If not, declare  non-convex.

and check whether o is on the negative

 Choose any facet and let p be the center of gravity of its vertices. Let r be the ray emanating from o and passing through p. If r intersects the closure of any other facet of  declare  non-convex.  If 

passes both tests declare it the surface of a convex polytope.

We next turn to the convex hull problem. Task 2 Given a set S of points and a hyper-surface  convex hull of S. 13

verify that 

is the boundary of the

We know already how to check the convexity of  . So let us assume that  has passed the convexity test and let P be the convex polytope whose boundary is  . It suffices to verify that  every vertex of P is a point in S and that

 every point of S is contained in P. The first item is fairly easy to check. If the vertices of P are specified as pointers to elements in S the check is trivial. If the vertices in S are specified by their coordinate tuples the check involves a dictionary lookup. This is a first example where the representation of the output influences the complexity of testing. The second condition is much harder to check. In fact, we found no easy and efficient method to verify it. A simple method would be to check every point of S against every facet of  . However, the complexity of this method is an order of magnitude larger that the complexity of the most efficient convex hull programs4 . An alternative method is to use linear programming to check that all non-vertices are non-extreme5 . For fixed dimension the alternative method is quadratic in the number of vertices. For variable dimension one might hope that a simplexbased verification procedure has good expected running time. Nevertheless, both approaches essentially resolve the original problem. We conclude that convex hull programs that output the hull as a convex polytope are hard to check. Gift-wrapping falls in this category. Can a convex hull program help the checker? The algorithms based on randomized incremental construction compute a simplicial complex6 comprising the hull, i.e., a set of simplices whose union is P. They also compute for each point of S a simplex of the complex containing the point. With this representation of the hull the second item is easy to check. One only has to check whether every point of S is indeed contained in the simplex it is supposed to be in. This amounts to the solution of a linear system.

4.5 Solving Linear Systems of Equations We consider a system Ax  b of m linear equations in n unknowns x; here A is a m by n matrix of real numbers and b is an m-vector. We want to know whether there is solution to this system. Again, a conventional algorithm would just return a single bit, telling whether the system is solvable or not. A certifying algorithm would do a lot more. If it declares the system solvable, it would return a solution, i.e., an n-vector x0 such that Ax0  b. Observe that it is very easy to verify whether a given x0 is a solution. We simply plug x0 into the equation. If it declares the system non-solvable, it could return an m-vector c such that c T A  0 and cT b  0. Observe that such a vector witnesses non-solvability. Indeed, assume the existence of a 4 Algorithms

based on randomized incremental construction [CS89, CMS93, BMS94] run in time related to the size of the output and the size of intermediate hulls and the algorithm of [Sei86] is guaranteed to construct the hull in logarithmic time per face. 5 The linear program has d variables corresponding to the coefficients of a linear function. For each vertex of ! there is a constraint stating that the function value at the vertex is negative. For each non-vertex consider the linear program that maximizes the function value at this point. 6 A simplicial complex is a set of simplices the intersection of any two is a face of both.

14

  solution x0 . Then cT Ax0  cT A  x0  0T x0  0 and cT Ax0  cT Ax0 " cT b  0, a contradiction. Thus there is no solution7 A standard way to test whether a system is solvable is Gaussian elimination. We all know that it returns a solution when given a solvable system. It is less well known that it also returns a witness for non-solvability when given an unsolvable system. We describe Gaussian elimination as a recursive procedure. If all entries of A and b are zero, the zero-vector 0 n is a solution. If all entries of A are zero and b is non-zero, say bi  0, the m-vector ei having a one in position i and zero everywhere else witnesses non-solvability. So assume that A has a non-zero entry, say Ai j  0. We subtract a suitable multiple of the i-th from all other equations (Al j  Ai j times the i-th equation for the l-th equation equation for 1 l m and l  i) so as to eliminate the j-th variable from the other equations. The transformation  yields a system with m   1 equations in n  1 unknowns, say A  x  b . Here Alk   Alk  Al j  Ai j  Aik and bl  bl  Al j  Ai j  bi for l  i and all k. Also, row index i and column index j are deleted from the index set of the reduced system. Assume first that the reduced system is solvable and x0 is a solution. We plug x0 into i-th original equation, solve for x j and obtain a solution to the original system. Assume next that the reduced system is unsolvable and that c witnesses it, i.e., c T A  0 and c T b  0. We define the m-vector c by cl  cl for l  i  and ci # ∑l % $ i Al j  Ai j  cl . Then for any k, 1 k n,     Al j  Ai j  Aik  ci Aik  ∑ cl Alk  ci Aik  ∑ cl Alk (2) l%$ i l %$ i     ci  ∑ cl Al j  Ai j  Aik  ∑ cl Alk (3) l%$ i l %$ i   ∑ cl Alk (4) l %$ i  0 (5) An analogous computation shows that cT b  c T b and hence cT b  0. We have now shown that Gaussian elimination is a certifying solver for linear systems of equations.

4.6 Approximation Algorithms Approximation algorithms compute nearly-optimal solutions of optimization problems. They come with a guarantee for the quality of approximation. We may assume w.l.o.g. that we deal with a minimization problem. In the proof of correctness the quality of the solution is usually measured against an easy to compute lower bound for the problem. A certifying approximation algorithm should output this bound (in a certifying way) in addition to the solution. We give an example. The traveling salesman problem asks for a shortest cycle visiting all nodes of a weighted   undirected graph G  V  E  c  . We assume that c satisfies the triangle inequality, i.e, c uv  7 Why does such a c exist? If Ax & b has no solution, b does not belong to the space spanned by the columns of A. Thus we can write b as b & b ')( b' ' , where b' is in the span of the columns of A and b ' ' is orthogonal to all columns of A. We can take c & b ' ' . Then cT A & 0 and cT b & b' ' T b' ' .

15

  c uw   c wv  for any triple of vertices u, w and z. There are approximation algorithms which produce a solution whose cost is at most 1.5 times the optimum [LKS85]. A good lower bound is provided by the so-called 1-trees. A one-tree anchored at a vertex v is defined a spanning tree of G  v plus two edges incident on v. Let π  be any real-valued   function  on the nodes of G and consider the modified cost function cπ uv * c uv   π u   π v  . We call π a potential function.  Lemma 6 For any potential function π and any 1-tree T , cπ T + 2 ∑v , for the cost of any traveling salesman tour.

 π v  is a lower bound V

Proof: A C be any traveling salesman tour. Since c satisfies the triangle inequality, there is a tour D visiting every node exactly once  and having  cost no more than C. D is also aπ 1-tree  π π c D  . Finally, observe, that c D - c D . (with respect to any anchor) and hence c T   2 ∑v , V π v  . Combining the equalities and inequalities, we obtain       c C / c D 0 cπ D " 2 ∑ π v - cπ T " 2 ∑ π v 1 v, V v, V

A certifying approximation algorithm for TSP outputs a tour C, a potential function π and a 1-tree T .

4.7 Traveling Salesman Tours Review principle of a branch-and-cut algorithm for TSP.

4.8 Linear Programming Linear programming duality is a general technique for certifying algorithms. A linear program is defined by an m 2 n real matrix A and real vectors c and b of dimension n and m, respectively. The goal is to solve: max cT x subject to Ax b and x 0 The so-called dual linear program is defined as T

T

min y b subject to y A



cT and y 0

Lemma 7 (Linear Programming Duality) If x0 and y0 are solutions to a linear program and its dual then cT x0 yT0 b. Moreover, if both programs are feasible, then there are solutions x 0 and y0 with cT x0  yT0 b.

16

Proof: We only prove the first part of the Lemma (weak duality). The second part runs under the name strong duality and is much harder to prove. We refer the reader to any textbook on linear programming, e.g., [Sch86]. Weak duality is easy to prove. We have T T c x0 y0 Ax0 yT0 b  The first inequality follows from cT yT0 A and x0 0 and the second inequality follows from Ax0 b and y0 0. So a certifying algorithm for solving linear programs outputs a primal solution x 0 and a dual solution y0 with cT x0  yT0 b. Observe, that it is trivial to check whether x0 is a solution for the primal, that y0 is a feasible solution for the dual, and that cT x0  yT0 b. Also the proof that this certifies optimality is trivial as we have seen above. It is only the existence of x 0 and y0 which is hard to prove, but this is not needed to be convinced.

4.9 Composition of Functions Suppose that we have certifying algorithms for functions g and f . How can we obtain a certifying algorithm for f 3 g. That’s easy.    Assume that our input is x. Let y  g x  , y  g x  , and z  f y  . Let Qg and Q f be certifying algs for f and g, respectively, and let Cg and C f be the corresponding checkers. A certifying algorithm for f 3 g works as follows: Run Qg on x. This produces y and a witness wg . Run Q f on y. This produces z and a witness w f .  Output z and the witness w  wg  y w f  .    We accept w  y w4 5 as a witness if Cg accepts x  y w  and if C f accepts y z  w4  .

4.10 Further Examples and Guide to Literature also discuss LEDAbook, [KMMS03], [DLPT98], [MNS  99], [Met97] does a systematic study for simple graph problems. He describes checkers for

 single source shortest paths with non-negative edge weights (Dijkstra)  single source shortest paths with arbitrary edge weights (Bellman-Ford)  the first two checkers can be found in refined form in the LEDAbook  maximum flow based on min-cut-max-flow  minimum cost flow; a flow has minimum cost iff there is no negative cycle in the residual graph  maximum and maximum weight matchings via reduction to flow problems 17

 faster checkers are discussed in the LEDAbook. The best algorithms for the matchings are primal-dual algorithms and compute not only a primal solution but also a dual solution. Then checking reduces to verifying. complementary slackness. Metzeler discusses checkers which are given only the optimal matching and are not given any side information. He checks the existence of an augmenting path or the existence of a negative alternating cycle.

 input F 6 E and a labeling num : V 98 7 : is num a labeling according to connected components and is F a spanning forest? b Check: 1. any edge must connect nodes with the same number. Then nodes with distinct numbers must belong to distinct components of G. 2. check the subgraph defined by F. As long as there is a node with degree one in this subgraph, remove it. If this removes all edges, F is cycle-free, otherwise it is not.

 input F 6 E. Is F a spanning forest of G. Again, we check first that F is cycle-free. If we know that F is cycle-free, we also determine the components by a simple graph search (since there are no cycles there is no need to keep track of nodes visited earlier). Then we check that no edge in E connects nodes in distinct components.  input F 6 E in an edge weighted graph. Is F a minimum spanning tree. He discusses  O m log n  algorithms for checking the cycle rule. There are better ways of doing it.  BFS: given a tree T and a distance labeling d, is T a breadth-first search tree?  given a tree T and a numbering dfsnum of the vertices, is T a depth-first search tree in the given undirected graph.  as above, but completion numbers are also given.  a digraph G   V  E  and a node labeling strongcomp. Is strongcomp, the labeling according to strongly connected components? b Check: derive several graphs: the shrunken graph (nodes are the numbers 1 to maximal strongcomp and we have edges .... ): verify that this graph is acyclic with strongcomp is topological sort for each integer l verify that the edges connected nodes with label l form a single strongly connected component. He does so by essentially redoing the strongly connected components algorithm. A better way to do it is to require that the SCC-algorithm returns a certificate (suggestion by Ross Mc Connell in a phone conversation of March 1st, 04. For example, for each SSC, a cycle passing through all nodes of the component. Or two trees rooted at the same node, say v. In the first tree all edges are directed away from v; it demonstrates that all nodes can be reached from v. In the second tree all edges are directed towards v; it demonstrates that v can be reached from all nodes. 18

I don’t like this certificate because it requires two DFS’s to compute it. An alternative is as follows: Consider the SCC-algorithm by Cheriyan and myself [CM96]. It is also discussed in the LEDAbook [MN99, Section 7.4]. It identifies a subgraph consisting of at most 2n  2 edges and having the same strongly connected components. The subgraph consists of the DFS-tree plus some cross- and back-edges. I think this algorithms can be extended so that it also computes the in-tree as a witness.

5 Correct Checkers The concept of certifying algorithms relies on our ability to write correct checkers. What approaches do we have?

5.1 The Pragmatic Approach The pragmatic answer is that the task of checking a witness should be so simple that the question of having a correct checking program is not really an issue. The checking program is short and has a simple logical structure and hence every programmer should be able to get it right. In LEDA we followed the pragmatic approach. What kind of primitives can one use in a checker? We have to assume that there is a basic layer of correct primitives, e.g., simple data structures such as linear lists, graphs, union-find and simple algorithms such as connectivity of graphs and minimum spanning trees. We can also use non-trivial primitives as long as they are certifying. Assume that you want a function f in a checker C and that you have a certifying algorithm Q f and a correct checker C f  for it. When you need to compute f x  in C, you call Q f and obtain y and a witness w. Then you   use C f to check the triple x  y w  . If C f accepts the triple, you know that y is equal to f x  and are allowed to proceed. If C f rejects the triple, C gives up and rejects.

5.2 Formal Verification of Checkers We can actually do more. Since checkers are so simple, we might be able to prove them correct. There is an obvious objection. Only programs written in a programming language with a formally defined semantics can be proven correct, but most algorithms are written in languages, e.g., C or C++, whose semantics is only informally defined. However, there is no need to write the checker in the same language as the program computing the solution and the witness. We may write the checker in a language that supports verification. Since checking programs are simple, this should not be a big obstacle. Also since the time complexity of the checking task is frequently much smaller than the complexity of computing the solution, the efficiency of the language is also not an obstacle.

19

6 Programs with Preconditions So far, we dealt with programs computing functions and having trivial preconditions. A notable example was five-coloring of planar graphs. In general, the intended behavior is specified by preand postconditions.  φ x  the program is supposed to return an On an input x satisfying a certain precondition  output y satisfying a certain postcondition ψ x  y  . If the input does not satisfy the precondition, the program may do anything. The requirements for a certifying algorithm are more stringent. On input x, it is supposed to either return a y and a witness which proves ψ x  y  or a witness which proves : φ x  . It is also  supposed to tell us which alternative holds. There is no need that the algorithm decides φ x  . Even if : φ x  , it may return a y and a witness proving ψ x  y  . As an example, reconsider the problem of five coloring a planar graph. The input is a graph G. The output is either a five-coloring of G (which witnesses itself) or a proof that G is nonplanar. The algorithm does not decide planarity. For example, it will five-color the Kuratowski graphs K5 and K3  3 .

7 Verification and Certification We give examples where part of the post-condition is easily verified and part of it is easily certified. Sorting: The example is due to Gerhard Goos (personal communication). The input to a sorting algorithm is a set of elements from a linearly ordered set. The output is (1) the same set but (2) in sorted order. It is trivial to check that a sequence of elements is sorted, say in ascending order. We simply step through the sequence and check that we see successively larger elements. It is difficult to check that two sets agree. In fact, the most efficient way to check equality of sets is to sort them and check equality of the sorted sets. General sorting algorithms frequently use a subroutine swap which exchanges the position of two elements: swap is an extremely short program (less than three lines in most programming languages) which we can prove correct. Any program which only uses (a correct) swap to move elements around, outputs a permutation of its input set. Sorting is an example where part of the post condition is easily proved and part is easily verified.

 Minimum Cut: A cut is an undirected graph G  V  E  is any subset S of the vertices which is neither empty nor the full set of vertices, i.e., 0/  S  V . Let c be a cost function on the edges of G. The weight   of a cut  S is the total cost of the edges having exactly one endpoint in S, i.e., c S  ∑ e; e ; S  1c e  . A minimum cut is a cut of minimum weight. We do not know of a certificate for minimum cuts. However, consider the following generic approach to computing a minimum cut. 20

The algorithm works recursively. It uses a subroutine, which does the following: it determines two vertices s and t (we do not prescribe these vertices, but the subroutine determines them) and a cut S which is minimal among all cuts containing s but not t. Assume that we call the subroutine for G and it returns s, t and a cut S. There are two cases: either S is a minimum cut in G or the minimum cut in G does not separate s and t. In the former case we are done (however, we do not know this) and in the latter case, we may collapse s and t into a single vertex, i.e., re  move s and t from the graph, add a new vertex z and for any v V s t assign c s v
      c t  v  8 as cost of the edge z  v  , and determine a minimum cut in the reduced graph. The minimum cut  in the reduced graph will also be a minimum cut in the original graph (after replacing z by s  t  , of course). An iterative version of the algorithm is as follows: while G has at least two vertices do determine two vertices s and t in G and a cut S which is minimal among all cuts containing s but not t; collapse s and t. end while output the smallest cut found in any iteration; Given the correctness of the subroutine, the algorithm is correct. Stoer and Wagner [SW97], simplifying an earlier algorithm of Nagamochi and Ibaraki [NI92], gave an efficient algorithm for the subroutine. Arikati and Mehlhorn [AM99] made it certifying. The certifying version computes s, t, the cut S and a flow f from s to t of value c S  . The flow proves the minimality of S by the min-cut-max-flow theorem.

8 An Attempt of a Theory of Certifying Algorithms We attempt a definition of certifying algorithm. We will not give a single definition but instead try to delineate the possible formalizations. We will then derive some consequences which hold for all proposed formalizations.

8.1 Towards a Definition We define the notion of a certifying algorithm for a function f : X Y . We start with the notion of a witness predicate W for f . We call a 3-ary predicate W x  y w  witness predicate for f if the following conditions hold:    1. w deserve the name witness, i.e., if W x  y w  holds then y  f x  and if y  f x  then there is a w with W x  y w  . >=    < x y w W x  y w 5 iff y  f x ? (6)

 2. Given x, y, and w, it is trivial to decide whether W x  y w  holds. We list a number of conceivable formalizations of “being trivial to decide”. 8 Of

course, if v is not connected to either s or t, there is no need for introducing an edge connecting v and z.

21

 There is a decision algorithm for W with linear running time.

 W has a simple logical structure. For example, require that x, y, and w are  might       we strings, that W is a conjunction of O x  y  w  elementary predicates and that each predicate tests a simple property of relatively short substrings.  There is a simple logical system, in which we can decide whether W  x  y w  holds.  The correctness of the program deciding W  x  y w  is obvious or can be established by an elementary proof. 3. The witness property is easily verified, i.e., the implications    < x  y w W x  y w  y  f x    =  y  f x 5 w W x  y w @

(7) (8)

have elementary proofs. Observe that we make no assumption about the difficulty of establishing the implication   =  y  f x  w W x  y w A A user of a certifying algorithm does not need to understand why a witness exists in all cases. He only needs to convince himself that it is easy to recognize witnesses and that a witness, indeed,  certifies the equality y  f x  . The “definition above” rules out some obviously undesirable situations: 1. Assume we have a complicated program P for  evaluating f . We could take w as the record of the computation of P on x and define W x  y w  as “w is the computation of P on input x and y is the output of this computation”. This predicate certainly satisfies the second property. If satisfies the first if P is correct. However, it does not satisfy the third, as the proof of the first implication is tantamount to proving the correctness of P.   2. Another extreme is to define W x  y w  as “y  f x  ”. Then the first and the third property holds. However, the second property does not hold. We next define the notion of a certifying algorithm for a function f . We call Q a certifying program  for f if there is a witness predicate W for f such that  for all inputs x, Q computes a triple x  y w  with W x  y w  . Observe that this implies y  f x  . We can now ask: Question 1 Does every computable function have a certifying algorithm. A more stringent version of this question asks, in addition, about the resource requirements of the certifying program. Question 2 Does every program P have a certifying counterpart with essentially the same running time? 22

More precisely, let f be the function computed by P. We want a program Q and a predicate W such that 1. W is a witness predicate for f .

  2. On input x, Q computes a triple x  y w  with W x  y w  and hence y 

 f x .

3. On input x, the resource consumption (time, space) of Q on x is at most a constant factor larger than the resource consumption of P.

8.2 Monte Carlo Algorithms resist Certification Consider a Monte Carlo algorithm for a function f , i.e., a randomized algorithm which on input  at least 3  4 and may output anything otherwise. The running x outputs f x  with probability    time is bounded by T x  . Assume now that we would have a certifying algorithm Q with the same complexity: on   probability at least 3  4 and input x, Q outputs a triple x  y w  passing the witness predicate   with may output anything else otherwise. It has running time O T  x  . We can turn Q into a Las Vegas algorithm for f as follows: We first run Q. If the triple x  y w  returned by Q passes W , we return y. Otherwise, we rerun Q.  The resulting randomized algorithm always return the correct result f x  and        has   expected running time O T x  . Observe that each round has expected running time O O T x    since   the length  of w is at most O T x  by the assumption Q  and  since the verification of         about the triple x  y w  has linear running time O x  y  w @ O T x  by property 2) of a witness predicate. Also observe that the expected number of rounds is constant since the success probability is at least 1  4. Does this observation imply that Monte Carlo algorithms are inherently unsafe? No, it only says that the concept of certification does not apply to them. Program verification does apply and Monte Carlo algorithms are frequently quite simple so that verification may be(come) feasible.

8.3 Every Deterministic Program has a Certifying Counterpart We show that every deterministic program can be made certifying with only a constant overhead in running time. This sounds like a very strong result. It is not really; the argument formalizes the intuition that a formal correctness proof is a witness of correctness. Let f be a function and let P be a program (in a programming language L with well-defined semantics) computing f . We assume that we have a proof (in some formal system S) that P computes f , i.e., a proof of the statement9    < < x y P halts on input x and y  P x  y  f x 5@ We use w2 to denote the proof. 9 We

use P B x C to denote the output of P on input x.

23

  We extend P to a program Q which on input x outputs f x  and a witness w  w 1  w2  w3  of the following form: w1 is the program text P, w2 is as above, and w3 is the computation of P on input x.   The witness predicate W x  y w  holds if w has the required format, i.e., w  w 1  w2  w3  , where w1 is the program text of some program P, w2 is a proof in S that P computes f , w3 is the computation of P on input x, and y is the output of w3 . We have  1. the check whether a triple x  y w  satisfies the witness predicate is trivial.     y  f x 5 is elementary. Assume W x  y w  . Then w  2. the proof of W x  y w   w1  w2  w3  , where w1 is the program text of some program P, w2 is a proof (in system S) that P computes f , w3 is the computation of P on input x, and y is the output of w3 . Thus y  f x .   3. W is clearly a witness predicate, i.e.,   if y  f x  then there is a w with W x  y w  , and if y  f x  then there is no w with W x  y w  . 4. The witness property is trivial to decide. It is essentially a proof checker for the formal system S. 5. the running time of Q is asymptotically no larger than the running time of P. The same holds true for the space complexity. Observe that Q produces a large output; however, the workspace requirement is the same as for P. We admit that the construction above leaves much to be desired. It is not a practical way of constructing certifying algorithms. After all, certifying programs are an approach to correctness when a formal correctness proof is out of reach. However, the construction is also quite assuring and gives strong moral support. Every deterministic program can be made certifying with only a constant loss in time and space. So, we only have to try hard to enough; we are guaranteed to succeed. The construction captures the intuition that certification is no harder than a formal correctness proof of a program.

8.4 And with Non-Trivial Preconditions The goal of this section is to show that the positive result of the previous section does not apply to programs with non-trivial preconditions. Recall the  requirements put forward in Section 6. φ x  the program is supposed to return an On an input x satisfying a certain precondition  output y satisfying a certain postcondition ψ x  y  . If the input does not satisfy the precondition, the program may do anything. The requirements for a certifying algorithm are more stringent. On input x, it is supposed to either return a y and a witness which proves ψ x  y  or a witness which proves : φ x  . It is also  supposed to tell us which alternative holds. There is no need that the algorithm decides φ x  . Even if : φ x  , it may return a y and a witness proving ψ x  y  . 24

A Program without a Certifying Counterpart: Consider the following task. The input is a string s. The precondition states that the string is the description of a Turing machine which halts on the empty string. The postcondition states that the output is the result of running s on the empty string. This behavior is easy to implement, take any universal Turing machine. It is even conceivable to prove the correctness of the implementation. After all, universal Turing machines are quite simple programs. However, there is no certifying algorithm implementing this behavior. What would a certifying algorithm have to do on input s? It has to prove either that s is not the description of a Turing machine halting on empty input or that the output of s on empty string is a certain value. Thus it would have to solve the halting problem.

   A Paradox? Let us write φ x  P  ψ x  y  for a program P with precondition φ and postcondition ψ . Aren’t the two statements       φ x  P  ψ x  y  and true P D: φ x FE ψ x  y  equivalent and didn’t we say that every deterministic program can be made certifying. So why doesn’t this imply that programs with preconditions can also be made certifying? The following example explains the situation. Consider     φ x   do nothing  φ x  versus true  do nothing D: φ x GE φ x @ The latter program can be made certifying. In fact, there is nothing  to do. For the former program, this is not clear. A certifying program would have to decide φ x  because it either has to give  a witness which proves φ x  or it would have to give a witness which proves : φ x  . So, if φ is undecidable, the latter program can be made certifying, in fact, is certifying, but the former program cannot. Decidable Preconditions: In the following situation, the construction of Section 8.3 can be repeated. The precondition is decidable and we have a formal correctness proof for the decision procedure. Also, we have a formal correctness proof for the program itself.

9 Reactive Programs and Data Structures Reactive programs run forever. They keep state information and receive inputs. They react and return outputs. In the algorithms community, reactive programs are called data structures and this is the terminology which we are going to use. We distiguish between abstract data types (= a definition of the intended behavior of the data structure) and implementations (= program exhibiting a certain behavior). The question is how to certify that a certain implementation implements a certain data type. Akin to certifying programs, we consider certification of data structures. 25

A C

D

Figure 4: D is a data structure implementation and C monitors its behavior. Any input is passed to C which then forwards it, maybe in modified form to D. D reacts to it, C inspects the reaction of D and returns an answer to the environment. If D is correct, the combination of C and D realizes the abstract data type A. If D is incorrect, C catches the error. We explore the following szenario, see Figure 4. We have an abstract data type A, a data type implementation D, and a monitor or checker C. The pair C  D is supposed to implement the behavior A as follows: The checker C interacts with the environment and offers the interface of the abstract data type A. It also interacts with D through the interface of D. If for a input stream s, the pair C  D does not implemenent the behavior A, the checker is supposed to raise an exception. If it implements the correct behavior, the checker is supposed to keep quiet. D might be implementation of A, but in the interest of certify-ability it might also be the implementation of a stronger abstract data type. This is akin to the situation for algorithms. In order to achieve certify-ability, we solve a more general problem.  We denote operations performed on D by D op argumentlist   and operations performed on  the abstract data type by op argumentlist . The checker monitors the execution of the data structure D. We may also call it a watchdop.

9.1 The Dictionary Problem The dictionary problem for a universe U and a set I of informations asks to maintain a  set S   ? 
2  I with pairwise-distinct keys (= first elements) under operations insert x  i  , of pairs x i U  delete h  , and find x  . Here, h is a handle to a pair in the dictionary. Insert x  i  adds the pair x  i  to S and returns a handle h to the pair in the data structure.  h If there is already a pair in S with key x, the information of the pair is replaced by i. Delete   deletes the pair with handle h and invalidates h. Key h  and inf h  returns the key and information component of the pair handled by h, set inf h  i  sets the information of the pair with handle h to i. The last four operations have the precondition that h is a valid handle of a pair in S. If they are supplied with an invalid  handle, the outcome of the operation and all subsequent operations is unspecified. Finally, find x  returns a handle to the pair with key x (if any) and returns a special 26

element nil otherwise. In the ordered dictionary problem, we assume that U is linearly ordered and require an ad ditional operation locate x  . It returns a handle to the pair y/A
S with minimal key y x, if there is such a pair. Otherwise, the special value nil is returned. Checking Ordered Dictionaries with Constant Overhead: We show that ordered dictionaries can be checked with constant overhead per operation.  We first define the state of the data structure and the state of the checker. For each pair x  i   in S, we have an entry x  h  in the internal dictionary and an entry x  i  r  in the checker. Here h is a handle to the triple x  i  r  and r is a handle to the pair x  h  . The checker stores its entry in a linear list L sorted by key. We next discuss the implementations of the operations:    Insert x  i  : The checker calls D locate x  and D returns a handle r (or nil). In the former   case r points to a pair y h  . Let z  j  r H be the entry of L handled by h. If r 0 r or z  y, the checker declares  a failure. Otherwise, it checks that x y and that x is larger than the key in the triple preceding y j  r  . If not, the checker declares failure. So assume otherwise. If y  x, it  replaces j by i, and if x I y, it inserts a triple x i into L just before the triple with key y. Let s  J    be a handle to this triple. Next it inserts x  s  into D and stores the returned handle in the triple with handle s. It returns s. If D returns nil, we check that x is larger  that the key in the last item of L. If not, the checker declares failure. If so, we insert a triple x  i K into L after the last item  of L. Let s be a handle to this triple. Next, we insert x  s  into D and store the returned handle in the triple with handle s. We return s.   Delete s  : s is a handle to an item x  i  r  in the list L. We remove the pair with handle r from  D and we remove the item x  i  r  from L. We leave the implementation of the other operations to the reader. Checking Dictionaries in the Comparison Model in the comparison model. Can we prove this?

We show that there is no efficient checker

9.2 Priority Queue

   A priority queue offers the operations insert x  i  , delete h  , and findmin  . Findmin returns a handle to a pair with minimum key and an insert returns a handle to the pair inserted. We will show that there is a simple checker which reports errors with linear delay. The checker has constant overhead. We also show that any checker which catches errors with no delay, must incur logarithmic amortized cost per operation. A Simple Checker with Linear Delay: Review [FM99].

27

A Lower Bound for On-Line Checkers: We use and extend the lower bound argument from Brodal et al. [BCR96]. We make the following assumptions about the checker: it cannot invent elements, i.e., only arguments of insert operations can be passed on to the data structure for insertion. However, not all arguments need to be passed on. Similarly, not all deletions need to be passed on. In this way the set of elements stored in D is a subset of the elements inserted by the environment. We use SD to denote the elements stored in D and SA to denote the elements stored in A. The adversary fixes 1. the outcome of the comparisons made by C. We use I C for the partial order determined by C.  2. the outcome of D findmin  operations. We assume that this outcome is consistent with  the partial order already determined by C and the returns of all preceding D findmin  operations. Note that this restriction is reasonable because the checker can detect a violation without making any further comparisons. The checker interacts with the environment and with D. It has the following obligations.  1. With respect to the environment, it must implement a priority queue, i.e., every findmin  operation must return a handle to the minimum element in SA . 2. With respect to the data structure, it must catch errors immediately, i.e.,  after every D findmin   , C performs some comparisons and then commits to either accepting the answer of the operation or to rejecting it.  Let m be the result of the findmin-operation. C can commit if and only if all linear  orders extending I C and the answers to previous D findmin  -operations have the same minimum and this minimum is equal to m or have a minimum different from m; the “or” in the predecing sentence is exclusive. In the former case it accepts, in the latter case is rejects. Is this the right definition of catching errors immediately? YES. First, C’s only source of knowledge are the comparisons made by itself (collected in I C ) and the answers to D findmin  ops posed to the data structure. The answer of the current findmin is under scrutiny and hence only the answers of previous findmins are relevant. This justifies the basis on which C has to decide. We next explain, that C cannot decide as long as there are two linear extensions I 1 and I 2 , one having m as the minimum and one not having m as the minimum. If C commits and accepts and the real linear order is I 2 , then C missed an error of D and if C commits and rejects and the real order is I 1 , then C rejected a correct priority queue. In either case we have a contradiction.

 Lemma 8 When the checker commits to the answer of a D findmin  , the answer must be the unique minimum of SD with respect to I C . 28

 Proof: Consider the first operation D findmin  , where the claim is false, i.e, I C does not determine a unique minimal element of SD and for all preceding operations D findmin  it did at the time of commit.  By the definition of “immediate error catching”, the result of the current D findmin  must be determined by I C and the answers to all preceding D findmin  operations. The result of all preceding findmin-operations was determined by I C at the time of their commit and hence is determined by the current I C . Thus the result of the current findmin on D is determined by the current I C . We conclude from the above, that the interactions with the environment are completely determined by the sequence of insert,  deletes, and the comparisons made by C. In particular, whenever an environment-findmin  is answered, I C must determine a unique minimal element  in SA . However, which comparisons are made by C depends on the outcome of the D findmin  operations. We next define the strategy of the adversary. We essentially want to use the strategy defined in [BCR96]. In BCR, the adversary only needs to fix the outcome of comparisons. It now also has to fix the outcome of D findmin  operations. If the adversary wants to stay consistent with itself, this will fix outcomes of later comparisons. I next recall the strategy used in Brodal et al. The elements sit in the nodes of a binary tree, we use v x  to denote the node containing x. An insert puts the new element into the root. When elements x and y are compared and

 x and y sit in the same node, we move one to the left and one to the right  x sits in a proper ancestor of y, x is moved to the child which is not an ancestor of y,  neither of the above holds (the two cases above, make sure that the third case applies), the   comparison in the obvious way, i.e., x I y iff v x  precedes v y  in a pre-order traversal of the tree. (in-order gives the same answer).  Assume now that the adversary has to fix the outcome of a D findmin  operation. By Lemma 8, the only logical constraint is that the outcome must be consistent with I C . Consider the elements in SD and how they are distributed over the occupancy tree. Consider the path p defined as follows. We call it the D-min-path. It starts in the root. Assume that we have extended p up to a node v. If no elements in SD are stored in proper descendants of v, the path ends. If some are stored in the left subtree, we proceed to the left child, otherwise, we proceed to the right child. The elements of SD lying on p are exactly the elements in SD which are minimal elements with respect to I C . Observe that the D-min-path  changes over time. Lemma 8 tells us that whenever C commits to the outcome of a D findmin  -operation, the D-min-path contains a single element in SD .  The adversary fixes the outcome of D findmin  as follows: it returns an element m in S D in the lowest non-empty node of p for which the left child is not on p. This node exists since the last node of p has this property. Moreover, it implicitly moves m to the left child of v and all elements of SD different from m sitting in ancestors of v and including v to their right child. Let 29

k, k 1, be the number of elements moved. I say implicitly, because these moves are hidden from C. The adversary commits to these moves but it does not make them yet. Let R be the set of elements moved in this way. We call this move a hidden move. When an element in R is involved in the next comparison, the adversary actually performs the hidden move and then follows the Brodal-et-al strategy. We need to show that any element on the path (except maybe one) must be involved in a further comparison and that this comparison takes place before the next hidden move of this element. In this way we can charge the hidden move to the comparison. Consider the comparisons made by C. By Lemma 8, I C determines the minimum of SD when C commits, i.e., at the time of commitment, the D-min-path contains a single element. Consider the elements in SD different from m and stored in v or an ancestor of v and assume that one of them is not involved in a comparison before the time of commitment. Then the D-minpath contains at least two elements at the time of commitment: either m was not involved in a comparison and then it still sits on the min-path or m was involved and hence moved to the left child and then maybe further and again we have at least two elements on the D-min-path. Thus we have a contradiction.  We conclude that if we perform k hidden moves, we can charge them to at least L k  1   2M comparisons. This is fine except if k  1, i.e., if no other elements in S D is stored in an ancestor of the node containing m. If m is the only element on the D-min-path, there is no need for a hidden move. If not, we simply charge the move of m to the next comparison made by C. In this way, we may charge at most 5 moves to a single comparison (the two moves in the Brodal-et-al strategy, the hidden moves of the two arguments of the comparison, and one hidden move left  over from the preceding D findmin  . It is now easy to complete the lower bound. We consider the sequence     insert a1  ; 5 ; insert an  ; findmin; delete b1  ;  ; findmin; delete bn @ where each bi is the result of the preceding findmin operation. This sequence sorts. When a findmin is answered, the element returned must be the unique minimum on the A-minimum path (this is the path containing the potential minima under I C in SA ). Thus there is no element stored in an ancestor of the element returned. By the definition of the adversary strategy, this is also true at the end of operation sequence. Let li be the depth  of element bi , when it is returned by the i-th l any comparison made findmin-operation. Then ∑i 2 N i 1 and hence 1  n  ∑ li log n. Since  by C increases the sum by at most five, we conclude that C performs 1  5  n logn comparisons. Theorem 4 In the comparison model, any checker for priority queues which reports errors im mediately, must perform 1  5  n logn comparisons.

10 Conclusions and Open Problems TODO

30

References [AL94]

N.M. Amato and M.C. Loui. Checking linked data structures. In Proceedings of the 24th Annual International Symposium on Fault-Tolerant Computing (FTCS’94), pages 164–173, 1994.

[AM99]

S. Arikati and K. Mehlhorn. A correctness certificate for the Stoer-Wagner mincut algorithm. Information Processing Letters, 70:251–254, 1999. www.mpi-sb.mpg. de/˜mehlhorn/ftp/mincut.ps.

[BCR96]

G. Brodal, S. Chaudhuri, and J. Radhakrishnan. The randomized compleixity of maintaining the minimum. Nordic Journal of Computing, 3(4):337–351, 1996.

[BK89]

M. Blum and S. Kannan. Designing programs that check their work. In Proceedings of the 21th Annual ACM Symposium on Theory of Computing (STOC’89), pages 86–97, 1989.

[BL76]

K.S. Booth and G.S. Lueker. Testing for the consecutive ones property, interval graphs, and graph planarity using PQ-tree algorithms. Journal of Computer and System Sciences, 13:335–379, 1976.

[BLR90]

M. Blum, M. Luby, and R. Rubinfeld. Self-testing/correcting with applications to numerical problems. In Proceedings of the 22nd Annual ACM Symposium on Theory of Computing (STOC’90), pages 73–83, 1990.

[BMS94]

C. Burnikel, K. Mehlhorn, and S. Schirra. On degeneracy in geometric computations. In Proceedings of the 5th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA’94), pages 16–23, 1994.

[BS94]

J.D. Bright and G.F. Sullivan. Checking mergeable priority queues. In Proceedings of the 24th Annual International Symposium on Fault-Tolerant Computing (FTCS’94), pages 144–153, Los Alamitos, CA, USA, June 1994. IEEE Computer Society Press.

[BS95]

J.D. Bright and G.F. Sullivan. On-line error monitoring for several data structures. In Proceedings of the 25th Annual International Symposium on Fault-Tolerant Computing (FTCS’95), pages 392–401, Pasadena, California, 1995.

[BSM95]

J.D. Bright, G.F. Sullivan, and G.M. Masson. Checking the integrity of trees. In Proceedings of the 25th Annual International Symposium on Fault-Tolerant Computing (FTCS’95), pages 402–413, Pasadena, California, 1995.

[BSM97]

J.D. Bright, G.F. Sullivan, and G.M. Masson. A formally verified sorting certifier. IEEE Transactions on Computers, 46(12):1304–1312, 1997.

[BW96]

M. Blum and H. Wasserman. Reflections on the pentium division bug. IEEE Transaction on Computing, 45(4):385–393, 1996. 31

[CM96]

J. Cheriyan and K. Mehlhorn. Algorithms for dense graphs and networks on the random access computer. Algorithmica, 15(6):521–549, 1996. www.mpi-sb.mpg. de/˜mehlhorn/ftp/CM-dense-graphs.ps.

[CMS93]

K. Clarkson, K. Mehlhorn, and R. Seidel. Four results on randomized incremental constructions. Computational Geometry: Theory and Applications, 3:185–212, 1993. www.mpi-sb.mpg.de/˜mehlhorn/ftp/CMS-FourResults.ps.

[CNAO85] N. Chiba, T. Nishizeki, S. Abe, and T. Ozawa. A linear algorithm for embedding planar graphs using PQ-trees. Journal of Computer and System Sciences, 30(1):54– 76, 1985. [CS89]

K.L. Clarkson and P.W. Shor. Applications of random sampling in computational geometry, II. Journal of Discrete and Computational Geometry, 4:387–421, 1989.

[DFK  03] M. Dhiflaoui, S. Funke, C. Kwappik, K. Mehlhorn, M. Seel, E. Sch¨omer, R. Schulte, and D. Weber. Certifying and repairing solutions to large LPs, How good are LPsolvers? In SODA, pages 255–256, 2003. www.mpi-sb.mpg.de/˜mehlhorn/ftp/ LPExactShort.ps. [DLPT97]

O. Devillers, G. Liotta, F.P. Preparata, and R. Tamassia. Checking the convexity of polytopes and the planarity of subdivisions. Technical report, Center for Geometric Computing, Department of Computer Science, Brown Universi ty, 1997.

[DLPT98]

Devillers, Liotta, Preparata, and Tamassia. Checking the convexity of polytopes and the planarity of subdivisions. CGTA: Computational Geometry: Theory and Applications, 11, 1998.

[Edm65a]

J. Edmonds. Maximum matching and a polyhedron with 0,1 - vertices. Journal of Research of the National Bureau of Standards, 69B:125–130, 1965.

[Edm65b]

J. Edmonds. Paths, trees, and flowers. Canadian Journal on Mathematics, pages 449–467, 1965.

[FM99]

U. Finkler and K. Mehlhorn. Checking priority queues. In Proceedings of the 10th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA’99), pages 901– 902, 1999.

[HT74]

J.E. Hopcroft and R.E. Tarjan. Efficient planarity testing. Journal of the ACM, 21:549–568, 1974.

[Kar90]

A. Karabeg. Classification and detection of obstructions to planarity. Linear and Multilinear Algebra, 26:15–38, 1990.

[KMMS03] D. Kratsch, R. McConnell, K. Mehlhorn, and J.P. Spinrad. Certifying algorithms for recognizing interval graphs and permutation graphs. In SODA, pages 158–167, 2003. full version, www.mpi-sb.mpg.de/˜mehlhorn/ftp/KMMS.ps. 32

[Kur30]

C. Kuratowski. Sur le probl`eme the courbes guaches en topologie. Fundamenta Mathematicae, 15:271–283, 1930.

[LEC67]

A. Lempel, S. Even, and I. Cederbaum. An algorithm for planarity testing of graphs. In P. Rosenstiehl, editor, Theory of Graphs, International Symposium, Rome, pages 215–232, 1967.

[LED]

LEDA (Library of Efficient Data Types and Algorithms). www.mpi-sb.mpg.de/ LEDA/leda.html.

[LKS85]

E. L. Lawler, J. K. Lenstra A. H. G. Rinooy Kan, and D. B. Shmoys. The Traveling Salesman Problem. John Wiley & Sons, New York, 1985.

[Met97]

M. Metzler. Ergebnis¨uberpr¨ufung bei Graphenalgorithmen. Master’s thesis, Fachbereich Informatik, Universit¨at des Saarlandes, Saarbr¨ucken, 1997.

[MM95]

K. Mehlhorn and P. Mutzel. On the embedding phase of the Hopcroft and Tarjan planarity testing algorithm. Algorithmica, 16(2):233–242, 1995.

[MN89]

K. Mehlhorn and S. N¨aher. LEDA: A library of efficient data types and algorithms. In Antoni Kreczmar and Grazyna Mirkowska, editors, Proceedings of the 14th International Symposium on Mathematical Foundations of Computer Science (MFCS’89), volume 379 of Lecture Notes in Computer Science, pages 88–106. Springer, 1989.

[MN98]

K. Mehlhorn and S. N¨aher. From algorithms to working programs: On the use of program checking in LEDA. In Proceedings of the 23rd International Symposium on Mathematical Foundations of Computer Science (MFCS’98), volume 1450 of Lecture Notes in Computer Science, pages 84–93, 1998. www.mpi-sb.mpg.de/ ˜mehlhorn/ftp/MFCS.ps.

[MN99]

K. Mehlhorn and S. N¨aher. The LEDA Platform for Combinatorial and Geometric Computing. Cambridge University Press, 1999. 1018 pages.

[MNS  96] K. Mehlhorn, S. N¨aher, T. Schilz, S. Schirra, M. Seel, R. Seidel, and C. Uhrig. Checking geometric programs or verification of geometric structures. In Proceedings of the 12th Annual Symposium on Computational Geometry (SCG’96), pages 159–165, 1996. [MNS  99] Kurt Mehlhorn, Stefan N¨aher, Michael Seel, Raimund Seidel, Thomas Schilz, Stefan Schirra, and Christian Uhrig. Checking geometric programs or verification of geometric structures. Computational Geometry, 12(1-2):85–103, 1999. www.mpi-sb.mpg.de/˜mehlhorn/ftp/programc.ps.

33

[MNU97]

K. Mehlhorn, S. N¨aher, and C. Uhrig. The LEDA platform for combinatorial and geometric computing. In Proceedings of the 24th International Colloquium on Automata, Languages and Programming (ICALP’97), volume 1256 of Lecture Notes in Computer Science, pages 7–16, 1997.

[NC88]

T. Nishizeki and N. Chiba. Planar Graphs: Theory and Algorithms. Annals of Discrete Mathematics (32). North-Holland Mathematics Studies, 1988.

[NI92]

H. Nagamochi and T. Ibaraki. A linear-time algorithm for finding a sparse kconnected spanning subgraph of a k-connected graph. Algorithmica, 7:583–596, 1992.

[NM90]

S. N¨aher and K. Mehlhorn. LEDA: A library of efficient data types and algorithms. In Michael S. Paterson, editor, Proceedings of the 17th International Colloquium on Automata, Languages, and Programming (ICALP’90), volume 443 of Lecture Notes in Computer Science, pages 1–5. Springer, 1990.

[Sch86]

A. Schrijver. Theory of Linear and Integer Programming. Wiley, 1986.

[Sei86]

R. Seidel. Constructing higher-dimensional convex hulls at logarithmic cost per face. In Proceedings of the 18th Annual ACM Symposium on Theory Computing (STOC’86), pages 404–413, 1986.

[SM90]

G.F. Sullivan and G.M. Masson. Using certification trails to achieve software fault tolerance. In Brian Randell, editor, Proceedings of the 20th Annual International Symposium on Fault-Tolerant Computing (FTCS ’90), pages 423–433. IEEE, 1990.

[SM91]

G.F. Sullivan and G.M. Masson. Certification trails for data structures. In Proceedings of the 21st Annual International Symposium on Fault-Tolerant Computing(FTCS’91), pages 240–247, 1991.

[SW97]

M. Stoer and F. Wagner. A simple min-cut algorithm. Journal of the ACM, 44(4):585–591, July 1997.

[SWM95]

G.F. Sullivan, D.S. Wilson, and G.M. Masson. Certification of computational results. IEEE Transactions on Computers, 44(7):833–847, 1995.

[ver]

verisoft. http://www.verisoft.de/index_en.html.

[WB97]

H. Wasserman and M. Blum. Software reliability via run-time result-checking. Journal of the ACM, 44(6):826–849, 1997.

[Wil84]

S.G. Williamson. Depth-First Search and Kuratowski Subgraphs. Journal of the ACM, 31(4):681–693, 1984.

34