A Finite-Di erence Algorithm for the Traveling Salesman Problem Eric Baxand Joel Frankliny December 11, 1997 Abstract
We develop an algorithm for the traveling salesman problem by applying nite di erences to a generating function. This algorithm requires polynomial space. In comparison, a dynamic programming algorithm requires exponential space. Also, the nite-di erence algorithm requires less space than a similar inclusion and exclusion algorithm.
Key words algorithms, combinatorial optimization, traveling salesman probAMS subject classi cations 05,68
lem, NP-hard .
Computer Science Department, California Institute of Technology 256-80, Pasadena, California, 91125 (
[email protected]). y Department of Applied Mathematics, California Institute of Technology 217-50, Pasadena, California, 91125 (
[email protected])
1
2
1 Introduction An instance of the traveling salesman problem (TSP) consists of a directed graph with vertices representing a starting city s, a terminal city t, and n interior cities 1; : : : ; n. Each directed edge has a nonnegative distance dij . A tour is an s-t Hamiltonian path. The length of a tour is the sum of distances over edges in the tour. The solution is the minimum tour length. The TSP is NP-hard [5]. There are many variations of the problem, and several restricted versions are also NP-hard [5]. Bellman [4] developed a dynamic programming algorithm for the problem. The algorithm requires exponential space. Also, there are many cross-references among terms, so we do not know how to achieve good speedup by implementing the algorithm on a message-passing parallel computer [1]. In contrast, the nitedi erence algorithm requires only polynomial space, and there are no crossreferences among its terms. However, the nite-di erence algorithm requires integer edge distances, while the dynamic programming algorithm does not. The nite-di erence algorithm is similar to an inclusion and exclusion algorithm by Karp [6]. Both algorithms sieve tours from a larger set of walks. In addition, the nite-di erence algorithm sieves tours by length. This reduces the space requirement by eliminating the need for vectors associated with the interior cities that are used to count walks by length in the inclusion and exclusion algorithm. We develop the algorithm by applying nite di erences to a generating function. First, we show how to nd coecients of generating functions by applying nite-di erence operators. Then we develop a generating function for the TSP. The algorithm uses very large integers { too large to store directly as integers on many machines. We outline a method to circumvent this problem. The method also applies to nite-di erence algorithms for other problems and to inclusion and exclusion algorithms.
3
2 Finite-Di erence Operators Let P () be a polynomial in x1 ; : : : ; xn with every term of degree n or less. De ne the nite-di erence operator with respect to xj as follows. P ()jx =u ? P ()jx =v for uj 6= vj (1) Dj P () = u ?v j
j
j
j
j
j
The nite-di erence parameters uj and vj are constants that we are free to choose. Note that Dj is a linear operator. For c constant with respect to xj ,
Dj cxpj =
0 if p = 0 c if p = 1
(2)
Also, Di Dj = Dj Di . Observe that the nite-di erence with respect to xj is similar to the partial derivative with respect to xj . Recall that
@ @ (3) @x1 @xn P () is the number of multilinear terms x1 xn in P (). A similar result holds for
nite di erences. Consider D1 Dn P (). Since the operators are linear, we can apply them to each term separately. Consider a multilinear term.
D1 Dn x1 xn = D1 Dn?1 x1 xn?1 = : : : = 1 (4) Now consider a non-multilinear term. Since each term has degree n or less, the
non-multilinear term must lack some variable. The nite di erence with respect to the missing variable zeroes the term. Hence, D1 Dn P () counts multilinear terms in P (). Finite di erences can also count non-multilinear terms. De ne the kth nitedi erence operator with respect to xj as follows. k k X Dj(k) P () = k1! (u ?1 v )k (?1)k?m P ()jx =v +m(u ?v ) j j m=0 m j
j
j
j
(5)
In [1], it is shown that for c constant with respect to xj ,
Dj(k) cxpj =
0 if p < k c if p = k
(6)
It is also shown that if P () is a polynomial in x1 ; : : : ; xn with every term of degree k1 + : : : + kn or less, then
D1(k1 ) Dn(k ) P () n
(7)
4 counts terms of type xk11 xkn . We solve the TSP by developing a generating function P () with terms corresponding to walks. Each term has indicator variables for cities visited and walk length. We apply nite-di erence operators to count terms corresponding to tours of various lengths. n
5
3 A Generating Function for the TSP Each term in the generating function corresponds to a length n + 1 s-t walk. The generating function has n + 1 variables, x1 ; : : : ; xn+1 . The rst n variables indicate visits to cities. They are used to sieve out tours. The last variable indicates the sum of edge distances. It is used to nd the minimum tour length. Let M be the maximum edge distance. Let A be the adjacency matrix of the directed graph induced by vertices representing interior cities. De ne an associated variable matrix as follows. ?d [A(x)]ij = [A]ij xj xM n+1
ij
(8)
where dij is the distance of the edge from city i to city j . De ne a 1 n \starting" vector as follows. [s(x)]j =
?d xj xM n+1
sj
0
if there is an edge from s to j otherwise
(9)
De ne an n 1 \termination" vector as follows. [t(x)]i = Let
M ?d xn+1 0
it
if there is an edge from i to t otherwise
(10)
P () = s(x)A(x)n?1 t(x)
(11) Since matrix multiplication counts walks, each term of P () corresponds to a length n + 1 s-t walk. For xj 2 fx1 ; : : : ; xn g, the exponent is the number of occurrences of vertex j in the walk. The exponent of xn+1 is the di erence between (n + 1)M and the sum of edge distances in the walk. A term of the form (12) x1 xn xn(n+1+1)M ?L corresponds to a tour of length L. Hence, the TSP solution is the minimum L 2 f0; : : : ; (n + 1)M g such that n+1)M ?L) P () > 0 D1 Dn Dn((+1
If there is no such L, then there are no tours, so the solution is 1.
(13)
6
4 Formula, Algorithm, and Analysis To compute (13), rst choose values for the nite-di erence parameters. For example, let uj = 1 and vj = 0 for all j 2 f1; : : : ; n + 1g. Let k = (n + 1)M ? L. Expand the nite di erences to produce the formula: k X ) P () = 1 (n?x1 ?:::?x )+(k?x +1 ) P (x) D1 Dn Dn(k+1 k! x2f1;0g f0;:::;kg xn+1 (?1) (14) Computing this formula requires O(2n nM ) evaluations of P (x). Each evaluation requires O(n3) operations. (This can be reduced to O(nE ), where E is the number of edges, by using a dynamic programming procedure similar to the one in [6].) We need to nd the minimum L 2 f0; : : : ; (n + 1)M g such that the corresponding formula is positive. Note that the values of P (x) are the same in all formulas. To avoid redundant computation, evaluate the formulas together, one x assignment at a time. Keep a vector of running totals, one for each formula, indexed by L 2 f0; : : : ; (n + 1)M g. For each x assignment, compute P (x) and add the appropriate multiple of it to each vector entry. The algorithm has time complexity O(2nn4 M ). The algorithm requires O(n2 ) space to store the problem instance, O(n) space to compute P (x), and O(nM ) space to store the running totals of formulas (14). n
n
n
7
5 An Implementation Issue { Large Integers The previous analysis ignores the fact that very large integers may occur in the computation. The absolute value of P (x) can be nearly nn [(n + 1)M ](n+1)M . The running total of formula (14) may be even larger. Even for problems of modest size, numbers may occur that are too large to be stored using the integer types available on most computers. We propose two methods to circumvent this problem. The rst solution is to implement storage and operations for larger integers in software. The time required to run the algorithm may increase substantially, since each arithmetic operation in the algorithm may require several operations in the computer. Given a problem instance, use oating point arithmetic to compute an upper bound for the following upper bound on P ().
Pmax = P (1; : : : ; 1; (n + 1)M )
(15)
The number of bits needed to store this number is no greater than 1 + log Pmax. Another bit is needed for the sign. So 2 + log Pmax bits suce to store the integers used to compute P (x). Note that the following expression is an upper bound for a running total of the sum in formula (14).
Tmax = 2n
(n + 1)M (n + 1)M=2 Pmax
(16)
Hence, 2 + log Tmax bits suce to store each running total. For an alternative solution, observe that although we work with large numbers, in the end we only wish to know whether the sum in (14) is nonzero. Compute the formula using arithmetic modulo some number b1 . If the result is nonzero, then the formula itself is nonzero. If the result is zero, then the formula is either zero or a positive multiple of b1 . Now compute the formula using arithmetic modulo some number b2 that has no common divisor with b1 greater than one. If the result is nonzero, then the formula itself is nonzero. If both results are zero, then the formula is either zero or a positive multiple of b1 b2 . Let b1 ; : : : ; bm share no divisors greater than one, and let b1 bm > Tmax. Compute the formula using arithmetic modulo b1 ; : : : ; bm . If any result is nonzero, then the formula is nonzero. If all results are zero, then the formula is zero. To implement each modulo arithmetic operation, the computer can perform the operation using normal arithmetic, then perform the mod operation on the result. To avoid over ow when multiplication is performed, each number bi should be no greater than the square root of the maximum integer allowed by
8 the computer. Hence, b1 ; : : : ; bm can be generated by the following sequence, where p1 ; : : : ; pm are the rst m primes.
p
bi = max pk such that pki MAXINT k >0 i i
i
i
(17)
9
6 Conclusion We have developed an algorithm for the traveling salesman problem by applying nite di erences to a generating function, producing a formula to count minimum length tours. Although the formula involves large integers, we have shown that the algorithm can be implemented using a smaller range of integers with modulo arithmetic. The algorithm has several free parameters, including the nite-di erence parameters and the bases for modulo arithmetic. One opportunity for future research is to nd settings for these parameters that allow reductions in computation. The proper choice of nite-di erence parameters allows computational reductions for a similar problem { the 0-1 matrix permanent [3]. There is a framework for the development of reductions in [1], Ch. 4. Finite-di erence formulas for multilinear terms have been used in algorithms for several problems, including counting paths and cycles by length [2], sequencing, bin packing, and deadlock avoidance [1]. In some of these algorithms, nite-di erence formulas for non-multilinear terms have been used to reduce computation for problem instances with symmetry. The algorithm developed in this paper is the rst use of a nite-di erence formula with an indicator variable to measure the quantity being optimized. This technique may prove useful for other optimization problems.
10
7 Acknowledgements We thank Dr. Mani Chandy for his inspiration. He sparked the results in this paper by proposing that nite di erence techniques could be used for optimization as well as counting. We also thank Rajit Manohar for his advice regarding the use of prime numbers in computation.
11
References [1] E. Bax, Finite-di erence algorithms for counting problems, CalTech-CSTR-97-23. (PhD Thesis) [2] E. Bax and J. Franklin, A nite-di erence sieve to count paths and cycles by length, Inform. Process. Lett., 60 (1996) 171-176. [3] E. Bax and J. Franklin, A permanent formula with many zero-valued terms, Inform. Process. Lett., 63 (1997) 33-39. [4] R. Bellman, Dynamic programming treatment of the traveling salesman problem, J. Assoc. Comput. Mach., 9 (1962) 61-63. [5] M. R. Garey and D. S. Johnson, Computers and Intractability { A Guide to the Theory of NP-Completeness, W. H. Freeman and Company, New York, 1979. [6] R. M. Karp, Dynamic programming meets the principle of inclusion and exclusion, Oper. Res. Lett., 1 (2) (1982) 49-51.
