FUN with Implementing Algorithms Rudolf Fleischer? Max-Planck-Institut fur Informatik, Im Stadtwald, 66123 Saarbrucken, Germany. E-mail:
[email protected].
Abstract. In this paper, I would like to argue { based on my personal pro-
gramming experience { that implementing algorithms is FUN. Moreover, it can lead to a better understanding of the problem to be solved, it can help to avoid embarrassing mistakes, and the implemented algorithms can be used as a tool for teaching.
1 Introduction Computer science is essentially the science of algorithms. The only purpose of developing an algorithm is to have it run on a computer and solve the problem it was designed for. So most algorithms sooner or later end up being implemented. Many years ago, this implementation would usually be done by other people than the inventors of the algorithm who called themselves theoretical computer scientists, meaning that they never bothered with implementation details { or diculties { of their algorithms. The result were nice theoretical papers describing clever algorithms which from time to time were found to be wrong by other people (often while trying to implement the stu ) doing a more thorough analysis, for example trying to ll in the \obvious" details ([1, 4, 7, 8, 11, 12, 18, 29] is a short list of such corrections; for sake of completeness, I would like to note that not only theoretical descriptions of algorithms are error prone, but proofs of \pure" mathematical theorems can have hidden holes as well, see for example [5, 21, 24]). Note that I included one of my own theoretical algorithm papers in the list above. This seems to have changed in the last decade, to some degree due to the pressure of more practically oriented funding policies. Some great theoreticians now spend at least some of their precious time thinking about the implementation aspects of algorithms (take [20] as an example). The result is a much higher awareness of possible pitfalls in the design of algorithms. And it can be reassuring for a reader who has just failed to understand the abstract description of a complicated algorithm if the author claims to have implemented and extensively tested his ideas; it is even better, if the author also provides access to his program to let other people play with it. Of course, implementing and testing an algorithm does not guarantee correctness, as it sometimes even seems to be dicult to correctly implement a correct algorithm. ?
The author was partially supported by the EU ESPRIT LTR Project No. 20244 (ALCOMIT). He was also supported by a Habilitation Scholarship of the German Research Foundation (DFG).
1
I remember my rst implementation of a red-black tree which was included in some central component of the HILL system for VLSI design [25] where it worked perfectly for half a year before mysterious crashes started to annoy innocent users. A buggy implementation of the graph planarity test in LEDA [19] survived even longer. As a positive result, this experience lead to a fruitful new area of research, namely programs which also provide kind of proof for the correctness of their output (try the graph algorithms in [17]). When an algorithm is nally correctly implemented, it can be used to show other people how it functions, for example in a classroom; experience shows that students appreciate such demonstrations (of course, a thorough theoretical analysis is still needed). The LEDA demo programs [17] can easily be used for this purpose. A recent textbook by Ottmann [23] pushes the multimedia approach to its limits.
2 Just (for) FUN Let me start with a simple programming exercise. Given two ordered lists of length m and n, respectively, what is the best way to merge the lists into one sorted list ? We measure the quality of an algorithm by the number of comparisons needed in the worst case, denoted by M (m; n). In [15, p. 202], Knuth de ned four functions :M:, =M:, =M n, and =M= in two variables and also gave a set of 24 recursively de ned lower bounds (see Appendix A). Since by de nition :M:(m; n) M (m; n), he could after some maximizing and minimizing of the lower bounds conclude that M (m; m + d) = 2m + d ? 1 for m 6 and d 4. Hence the trivial Scan-Merge algorithm is optimal in these cases. Later, Stockmeyer and Yao [28] extended this result to the case of 2d m + 2. When I rst encountered this fascinating problem some eight years ago, I tried the obvious, namely to compute tables of lower bounds for the M -functions as Knuth had done in his book for N = 10. Here, N is a parameter de ning the size of the tables, i.e., we want to compute lower bounds for the M -functions for all N 2 values of the parameters m; n 2 f1; : : :; N g. Naively believing Knuth's claim that \a computer can grind out tables of the M -functions rather quickly", I started with a straightforward implementation of the recursively de ned functions, just copying them from the book into my program1. After xing the inevitable minor bugs (this part of programming is less FUN), I could indeed quickly reproduce the (small) tables given by Knuth. Unfortunately, I could not produce much larger tables | and large tables are needed to nd interesting patterns of `abnormal' behaviour, because the lower bounds are surprisingly sharp | because my programs became too slow. This seemed to contradict Knuth's claim, but a quick analysis of the formulas convinced me that I could not expect a better performance from a program with running time
(N 6). 1
The programs mentioned in Sections 2 and 3 can be obtained from my web page at http://www.mpi-sb.mpg.de/~rudolf/fun.html.
2
After a more thorough analysis of the web of entangled recursive function calls I realized that many partial results were computed again and again in di erent recursive calls. Storing these partial results for later use shaved o a factor of (n2 ) from the running time. This speedup was sucient to compute larger tables up to size N = 150 (I do no remember the running times of eight years ago, but computing tables of this size takes eight hours on my current workstation, whereas the rst straightforward implementation would run for two months), so I did not try further optimizations. I could not extract any useful new theorem from these tables, but I got a much deeper insight into the problem by trying to speed up the program. Finding some hidden structure in an abstract problem description is an intellectual challenge which is FUN per se | at least for those people who have chosen by their own free will to be researchers in the eld of computer science. And if this leads to a piece of code which spills out results which otherwise seemed to be uncomputable in a reasonable amount of time, then it can produce a feeling of great pleasure and satisfaction. So I would like to conclude from this rst example :
1. Implementing algorithms is FUN. 2. Implementing algorithms gives deeper insight.
3 More (than) FUN In 1985, in my third year as a student of computer science at the University of Saarbrucken, I attended a course on parallel algorithms, given by our great teacher. Since I prefered to think sequentially, I thought it would be nice to have a tool for visualising and animating the parallel algorithms presented in the lectures. The SBPRAM [26] was not even thought of at that time, neither was a parallel language like Fork95 [14] available. So I spent a few afternoons in the computer room designing a preprocessor which could translate the parallel pseudocode used by our great teacher into Pascal2 programs simulating the parallel steps. For this preprocessor, I chose a minimalistic approach. Since the only parallel construct used in class was a loop of the form \for some subset of processors do in parallel : : : od"3, I just had to translate each assignment within such a loop into 2 3
We didn't use C in Saarbrucken at that time. This was later called \dataparallel loop" in Fork95.
3
a procedure which rst gets the values of all the righthand sides of the assignment (for all processors involved) and then assigns these values to the variables on the lefthand sides. This can elegantly be done by a recursive function. Appendix B gives an example of a parallel program for list ranking using pointer jumping [10, p. 92] and its translation into sequential Pascal. As a matter of fact, the great teacher was very impressed by my program. However, he also had reason to feel embarrassed when I demonstrated that one of his carefully elaborated algorithms (computing connected components of an undirected graph, a variant of the algorithm by Shiloach and Vishkin [27]) was less correct than proved (it never terminated), but that added only to my FUN. So I would like to conclude from this second example my two claims from the end of the previous section, and I would like to add my third claim :
3. Implementing algorithms helps to avoid mistakes.
4 FUN with Online Algorithms Competitive analysis of online algorithms has drawn much attention in recent years. An online algorithm must produce its output incrementally, i.e., it must immediately react to an event with some piece of output without knowledge of future events. Therefore, an online algorithm usually cannot compute an optimal solution to a problem, where the optimal solution would be the output of an optimal oine algorithm with complete knowledge of all events. If the online algorithm is a c-approximation to the optimal oine solution then it is said to be c-competitive. I assume that the reader is familiar with the topic; if not, the book chapters [9] or [22] are a good introduction. After I got interested in this eld a few years ago, I started to think about the implementation of online algorithms. Together with a few students I designed OnVis, a system for visualising and animating online algorithms. The philosophy behind OnVis is similar to the Balsa system [3]. It is a distributed system of many clients whose interactions are coordinated by a central server. In our case, clients are request generators, algorithms (online and oine), and views (visualising the actions of the algorithms). All clients run synchronously, so it is possible to compare the behaviour of di erent algorithms on the same request sequence simply by viewing at their corresponding views. Of course, it is possible to have several views connected to one algorithm or vice versa. A quick and dirty prototype of the system was implemented by the students [16], using LEDA. They also implemented online algorithms for the unidirectional money conversion problem [6], the lost-cow problem (see [13] and references therein), and 4
the problem of navigating in unknown scenes with rectangular obstacles [2]. The lostcow demos were ok, but the implementations of the other two algorithms immediately draw my attention because they often produced solutions which were so obviously non-optimal that it would be unlikely that a potential customer would pay money for such \optimal" algorithms. The problem with the navigation algorithm were unnecessary detours which could easily be detected even by an online algorithm. But since these detours do not in uence the asymptotical behaviour of the algorithm, the authors never mentioned these possible shortcuts, and the students had just copied the code from the paper. The problem with the money conversion algorithm is explained in Appendix C. Again, it is not an error in the paper, just a missing helpful remark. Currently, one group of students is working on implementing more online algorithms, while another student (Pavel Hyska) has completely redesigned the whole system. I expect to have a rst version available in the near future. Long term plans are to design a library of high-level superclasses for all kinds of online problems to make the implementation of his own algorithms for a user as simple as possible. Then, the OnVis system could be a great help for researchers playing around and testing new ideas as well as for teachers showing the animated algorithms to their students. So I would like to conclude from this third example my three claims made before, and I would like to add my last claim :
4. Implemented algorithms can be used for teaching. Acknowledgements. I would like to thank the organizers of this conference for
providing the opportunity of writing such a paper where the emphasis is not entirely on the scienti c content.
References 1. S.G. Akl. Two remarks on a convex hull algorithm. Information Processing Letters, 8(2):108{109, 1979. 2. A. Blum, P. Raghavan, and B. Schieber. Navigating in unfamiliar geometric terrain. SIAM Journal on Computing, 26(1):110{137, 1997. 3. M.H. Brown and R. Sedgewick. A system for algorithm animation. Computer Graphics, 18(3):177{186, 1984. 4. A. Bykat. Convex hull of a nite set of points in two dimensions. Information Processing Letters, 7(6):296{298, 1978. 5. H. Edelsbrunner, R. Seidel, and M. Sharir. On the zone theorem for hyperplane arrangements. SIAM Journal on Computing, 22(2):418{429, 1993.
5
6. R. El-Yaniv, A. Fiat, R. Karp, and G. Turpin. Competitive analysis of nancial games. In Proceedings of the 33rd Symposium on the Foundations of Computer Science (FOCS'92), pages 327{333, 1992. 7. R. Fleischer. A simple balanced search tree with O(1) worst-case update time. International Journal of Foundations of Computer Science, 7(2):137{149, 1996. A preliminary version was published in Proceedings of the 4th International Symposium on Algorithms and Computation (ISAAC'93). Springer Lecture Notes in Computer Science 762, pages 138{146, 1993. 8. M. Hammar and B.J. Nilsson. Concerning the time bounds of existing shortest watchman route algorithms. In Proceedings of the 11th International Conference on Fundamentals of Computation Theory (FCT'97). Springer Lecture Notes in Computer Science 1279, pages 210{221, 1997. 9. S. Irani and A.R. Karlin. Online computation. In D.S. Hochbaum, editor, Approximation Algorithms for NP-hard problems, chapter 13. PWS Publishing Company, Boston, MA, 1996. 10. J. JaJa. An Introduction to Parallel Algorithms. Addison-Wesley, Reading, MA, 1992. 11. B. Joe and R.B. Simpson. Corrections to Lee's visibility polygon algorithm. BIT, 27:458{473, 1987. 12. D.B. Johnson. A note on Dijkstra's shortest path algorithm. Journal of the ACM, 20(3):385{388, 1973. 13. M.-Y. Kao, J.H. Reif, and S.R. Tate. Searching in an unknown environment : An optimal randomized algorithm for the Cow-Path Problem. Information and Computation, 131:63{79, 1996. 14. C.W. Kessler. Practical PRAM programming in Fork95 { A tutorial. Technical Report Mathematik/Informatik, Forschungsbericht Nr. 97, ISSN 0944 0488, Universitat Trier, May 1997. See also http://www.informatik.uni-trier.de/~kessler/fork95.html. 15. D.E. Knuth. The Art of Computer Programming, Volume 3 : Sorting and Searching. Addison-Wesley, Reading, MA, 2. edition, 1973. 16. K. Kursawe and M. Reinstadtler. OnVis 1.1 : Ein System zur Visualisierung von OnlineAlgorithmen. 66123 Saarbrucken, Germany, December 1995. Manual of the prototype OnVis 1.1. 17. LEDA demo programs. See http://www.mpi-sb.mpg.de/~calvin/. 18. K. Mehlhorn and P. Mutzel. On the embedding phase of the Hopcroft and Tarjan planarity testing algorithm. Algorithmica, 16:233{242, 1996. 19. K. Mehlhorn and S. Naher. LEDA : A platform for combinatorial and geometric computing. Journal of the ACM, 38(1):96{102, 1995. 20. K. Mehlhorn and S. Naher. LEDA { A Platform of Combinatorial and Geometric Computing. Cambridge University Press, Cambridge, England, 1998. To appear. A preview is possible at http://www.mpi-sb.mpg.de/~mehlhorn/LEDAbook.html. 21. J.L. Monta~na, L.M. Pardo, and T. Recio. A note on Rabin's width of a complete proof. Computational Complexity, 4:12{36, 1994.
6
22. R. Motwani and P. Raghavan. Randomized algorithms, chapter 13. Cambridge University Press, Cambridge, England, 1995. 23. T. Ottmann. Prinzipien des Algorithmenentwurfs. Spektrum Lehrbuch. Spektrum Akademischer Verlag, Heidelberg, 1998. Includes 2 CD-ROMS. 24. N. Reingold and J. Westbrook. O -line algorithms for the list update problem. Information Processing Letters, 60:75{80, 1996. 25. W. Rulling. Einfuhrung in die Chip-Entwurfssprache HILL. Technical Report SFB 124, Nr. 04/87, University of Saarbrucken, 1987. 26. SB-PRAM. See http://www-wjp.cs.uni-sb.de/sbpram/sbpram.html. 27. Y. Shiloach and U. Vishkin. An O(log n) parallel connectivity algorithm. Journal of Algorithms, 3:57{67, 1982. 28. P.K. Stockmeyer and F.F. Yao. On the optimality of linear merge. SIAM Journal on Computing, 9(1):85{90, 1980. 29. R.E. Tarjan and C.J. van Wyk. Correction to \A linear-time algorithm for triangulating a simple polygon". In Proceedings of the 28th Symposium on the Foundations of Computer Science (FOCS'87), page 486, 1987.
Appendix A Lower Bounds for Merging In [15, x5.3.2], Knuth describes several adversaries whose strategies lead to the following lower bounds. For ; 2 f:; =; ng, we have for any i 2 f1; : : :; mg and j 2 f1; : : :; ng : { M(m; n) 1 + M:(k; l ? 1) + :M(m ? k; n + 1 ? l) for i k m and 1 l j. { M(m; n) 1 + M n(k; l) + =M(m ? k; n + 1 ? l) for i k m and 1 l < j. { M(m; n) 1+ M=(k; l ? 1)+ nM(m +1 ? k; n +1 ? l) for i < k m and 1 l j. { M(m; n) 1 + M=(k ? 1; l) + :M(m + 1 ? k; n ? l) for 1 k i and j l n. { M(m; n) 1+ M n(k ? 1; l)+ =M(m +1 ? k; n +1 ? l) for 1 k i and j < l n. { M(m; n) 1 + M=(k; l) + nM(m + 1 ? k; n ? l) for 1 k < i and j l n. We obtain a lower bound for M(m; n) by taking the maximum over all k; l of the righthand sides of the inequalities for xed i; j , and then taking the minimum over all i; j . This clearly costs (m2 n2 ). Hence, the total complexity for computing all lower bounds of M(m; n) for m; n N is (N 6). Let A[i; j ] denote the maximum over all k; l of the righthand side of the rst inequality. Then it can be seen that this value can be iteratively computed by A[i; j ] = max(A[i; j ? 1]; A[i + 1; j ]; 1+ M:(i; j ? 1) + :M(m ? i; n + i ? j )), with i running from m ? 1 to 1 and j running from 2 to n. This reduces the time to compute A[i; j ] for all values of m; n N to O(N 4). The other ve inequalities can be handled similarly. 7
Appendix B Parallel List Ranking The parallel Pascal code : program rank(input,output); (* list ranking *) const num = 10;
(* just 10 processors *)
var d : array[0..num] of integer; r : array[0..num] of integer; k : integer;
(* array of successors *) (* array of ranks *) (* loop variable *)
beginmain r[0] := 0; d[0] := 0;
(* here starts the main program *) (* initialize the head of the list *)
(* we need at most ln(num)/ln(2) iterations *) (* invariant : r[i] is the number of elements between i and d[i] *) for k := 1 to 1+trunc(ln(num)/ln(2)) do begin pardo(i,1,num,1); (* all num processors in parallel *) r[i] := r[i] + r[d[i]]; (* pointer jumping *) d[i] := d[d[i]]; odpar; end; (*for*) end.
The sequential Pascal code after preprocessing : program rank(input,output); (* list ranking *) const num = 10;
(* just 0 processors *)
var d : array[0..num] of integer; r : array[0..num] of integer; k : integer;
(* array of successors *) (* array of ranks *) (* loop variable *)
procedure xxx1(i, xxxend, xxxstep : integer); var xxxhelp : integer; xxxflag : boolean; xxxindex : integer; begin xxxindex := i; xxxhelp := r[i] + r[d[i]]; if i + xxxstep
