1
An efficient version of Schoning’s algorithm applied for ONE-IN-THREE SAT Cristian Dumitrescu Independent mathematician, Kitchener, Canada.
Abstract. In this article I describe an efficient, randomized algorithm that finds a solution to the ONEIN-THREE SAT problem (when one exists) in polynomial time with high probability. Keywords. The Satisfiability Problem, ONE-IN-THREE SAT, Markov chain, random walk with absorbing barriers, NP-complete.
1. Introduction We will consider the ONE-IN-THREE SAT problem, a variant of 3SAT for which every clause has exactly one literal in the clause true (instead of at least one). In [1] it is proved that ONE-IN-THREE SAT is NP-complete. Proposition 1. ONE-IN-THREE SAT is NP-complete. Proof. For the proof see [1], chapter 9, page 207, problem 9.5.3. Lemma 1 (see [1], lemma 11.2, page 247). If X is a random variable taking nonnegative integer values, 1 then for any k > 0 the probability prob{X ≥ k · E(X)} ≤ , where E(X) denotes the expected value k of X. Proof. Let 𝑝𝑖 be the probability that 𝑋 = 𝑖 . Then we have: E(X) = ∑i i · pi = ∑i≤k ·E(X) i · pi + ∑i>k ·E(X) i · pi > k · E(X) · prob{X > k · E(X)}. From the relation above the lemma follows. In this article I will present an efficient probabilistic algorithm that solves ONE-IN-THREE SAT in polynomial time with high probability.
2. The presentation of the algorithm Definition 1. Given a truth assignment for the n variables in a ONE-IN-THREE SAT instance, the dual assignment is simply obtained by flipping the truth value for each variable (from 0 to 1 and from 1 to 0). Definition 2. Given a truth assignment for the n variables in a ONE-IN-THREE SAT instance, we have two types of flips, the 0 → 1 flip (when we change the corresponding variable so that a literal changes its truth value from 0 to 1), and the 1 → 0 flip (when we change the corresponding variable so that a literal changes its truth value from 1 to 0). We are now ready to present the algorithm. This algorithm is a modified version of Schoning’s algorithm.
2
Modified Schoning Algorithm. Input: a formula in ONE-IN-THREE SAT 3-CNF with n variables. Guess an initial assignment 𝒂 ∈ {𝟎, 𝟏}𝒏 for the n variables, uniform at random. Repeat 𝑪 · 𝒏𝟐 times: Randomly choose one type of flip to perform. If the formula is satisfied by the dual assignment: stop and accept. If the formula is satisfied by the actual assignment: stop and accept. If there are no literals left that allow the type of flip chosen: stop and reject. Choose one literal at random (from any clause of the expression) that allows the type of flip chosen, and flip its truth value. There are a few differences between this algorithm and Schoning’s algorithm (see [2]) and the present algorithm. We focus on the overall number of possible flips, not just on one clause at a time. This is, in fact, a generalization of Schoning’s algorithm. In the next section I will outline a proof that in a sufficiently large number of steps that does not exceed a quadratic polynomial of n, the algorithm will find a satisfying solution (if it exists) with high probability. For an example, see A4.
3. Analysis of the algorithm We consider a 3SAT problem with n variables. We define as usual the Hamming distance from an actual truth assignment for the n variables to a satisfying assignment (if it exists, and if there are more than one just choose one). The Hamming distance is the number of variables that have to be flipped in order to get to the satisfying assignment. This Hamming distance will be contained in the set {0, 1, 2, … … . . n}, and will also represent the state of the associated Markov chain (this method has also been used by Schoning). In fact, we will be dealing with one dimensional random walks with two absorbing barriers, but we may also refer to them as Markov chains.. We are ready to state the theorem. Theorem 1. For all (including the hardest) 3SAT problems (after the reduction to a ONE-IN-THREE SAT instance), the algorithm presented in section 2 will find a satisfying assignment or its dual (finding its dual is equivalent to finding a satisfying assignment) in polynomial time (quadratic time) with high probability. For sufficiently large constant C (and by running the algorithms several times), we can make the probability that the algorithm will fail (meaning that it will report that the expression is unsatisfiable when in fact it is) with probability arbitrarily small. Proof. We convert the 3SAT problem that we have into a ONE-IN-THREE SAT problem. For details, see A1. We have two types of flips that appear in the algorithm (also see A3). The 0 → 1 flip. 2 In this case the Hamming distance increases by 1 with probability . 3 1
In this case the Hamming distance decreases by 1 with probability . 3
The 1 → 0 flip. 1 In this case the Hamming distance increases by 1 with probability . 3 2
In this case the Hamming distance decreases by 1 with probability . 3
This happens because in the context of ONE-IN-THREE SAT, a satisfying clause has exactly one
3
literal with the truth value 1 and two literals with the truth value 0 (see A3 for details). 1
We note that in the algorithm, both the 0 → 1 flips and the 1 → 0 flips are chosen with probability . 2 That means that overall, in the inner loop of the algorithm, the Hamming distance will increase with 1 2 1 1 1 probability · + · = . Also we see that overall, the Hamming distance will decrease also with 2 1
3 1
2 1
3 2
2 1
probability · + · = . We have here a symmetric (unbiased) random walk with two absorbing 2 3 2 3 2 barriers, when the Hamming distance is 0 (when the satisfying assignment is found), or at Hamming distance n (when the dual of a satisfying assignment is found). We know that for a one dimensional, perfectly symmetric random walk with two absorbing barriers (and with n states), the expected lifetime is quadratic in n, and that is the most symmetric, balanced case (and this is a well known result, I will not give the proof here). From lemma 1 we see that the probability that the absorption time (seen a s a random variable) is higher than quadratic (when a solution exists) is vanishingly small. The problem can also be formulated in terms of the expected duration of the game for a version of the gambler ruin problem (for an unbiased game), but I will not emphasize that here (for details see A2). In any case, we have quadratic time until absorption with arbitrarily high probability. Let’s assume that we start with a 3SAT problem with 𝑛′ variables and 𝑚′ clauses (we emphasize this case because 3SAT problems are most studied). We write 𝑚′ = 𝛼 · 𝑛′ , where 𝛼 is a small positive constant. For the most difficult problems 𝛼 is around 4.267. The equivalent ONE-IN-THREE SAT problem will have 𝑛′ + 4 · 𝑚′ variables and 3 · 𝑚′ clauses (I assume that the reader is familiar with the reduction of a 3SAT problem into ONE-IN-THREE SAT, this is a well known procedure, see A1). In other words, the equivalent ONE-IN-THREE SAT problem will have (4 · 𝛼 + 1) · 𝑛′ variables and 3 · 𝛼 · 𝑛′ clauses (we understand that we take the integer part of these expressions, but for simplicity we will omit that). That means that the constant C must be greater than (4 · 𝛼 + 1)2 (for the most difficult problems). We note that the optimum value of the constant (for a given 3SAT instance) depends on ratio between the number of clauses and the number of variables of the expression. By taking the constant C large enough in the algorithm, and if we run the algorithm several times, we can make the probability of failure as small as we want. QED. Observation 1. We must be careful and see what happens when there are no literals left that allow the chosen type of flip to be performed, for example if all literals have the truth value 0 (or all 1) in the current assignment. That is not likely to happen for the interesting problems where we have variables connected to both positive literals (the associated variable itself) and negative literals (the negation of the associated variable). Also this situation appears with vanishing probability, and by running the algorithm many times we can solve this problem.
4. Discussion and conclusions For general implications, related to efficiently solving NP – complete problems, see [3]. An interesting application is related to the problem of automated theorem proving using an efficient algorithm for NP – complete problems. The impact of this type of algorithm in mathematics, cryptography, science in general is hard to estimate. Also note that a derandomization of these algorithms could lead to interesting results. In a few words, this is a very important results with extremely important applications in any field of activity.
Aknowledgements. Credit must be given where credit is due. Without Schoning’s algorithm this type of algorithm would not exist. Schoning’s algorithm hits the satisfying assignment in at most exponential time (in the number of variables). The trick here is to take into consideration the dual assignment, focus on the ONE-IN-THREE SAT problem, and focus on the two types of flips possible. I also emphasize that Professor Fortnow’s book (reference [3]) is a great source of inspiration and motivation that led me to seriously consider this problem. Also I emphasize that both Professor Schoning and Professor Fortnow
4
corrected some errors that I had in some previous versions of this paper, and for that I am very grateful, but that does not mean that they endorse this version of my paper.
Appendix A1. The reduction of 3SAT into ONE-IN-THREE SAT. For each clause (𝑥˅𝑦˅𝑧) in the 3SAT expression, we add 4 new variables a,b,c,d and replace the clause with the conjunction of the three new clauses (┐𝑥˅𝑎˅𝑏)˄(𝑦˅𝑏˅𝑐)˄(┐𝑧˅𝑐˅𝑑). If our original problem had n’ variables and m’ clauses , then this transformation will produce a ONE-IN-THREE SAT instance with 𝑛′ + 4 · 𝑚′ variables and 3 · 𝑚′ clauses. It can be proved that the transformed ONE-IN-THREE SAT instance has a solution under the ONE-IN-THREE SAT conditions (truth assignment) iff the original 3SAT instance has a solution under 3SAT conditions. A2. Symmetric random walks ,absorption time and the gambler ruin problem. Suppose that we start 1 with d dollars and make a sequence of bets. For each bet we win 1 dollar with probability , and we 2
1
lose 1 dollar with probability . We quit if either we go broke or when we reach n dollars (when we 2 win 𝑛 − 𝑑 dollars). Let S be the number of steps until we hit the boundary, that is, until we win or lose. Let 𝐸𝑑 be the expectation of S when we start with d dollars. That is, 𝐸𝑑 is the expected time to win or lose when we start with d dollars. Then we have 𝐸𝑑 = 𝑑 · (𝑛 − 𝑑) . In an unbiased game, we expect to play a number of steps equal to the product of the amount we are willing to lose times the amount we want to win (the gambler ruin problem appears in any standard text ). Translated to our problem, we start at a distance d from a solution and we decrease or increase this distance by one unit with 1 probability . We stop when we hit the solution 𝑑 = 0 , or when we hit its dual 𝑑 = 𝑛 . The duration of 2 the random walk until absorption is at most quadratic in n. A3. In the proof of theorem 1 we wrote that: The 0 → 1 flip. 2 In this case the Hamming distance increases by 1 with probability . 3 1
In this case the Hamming distance decreases by 1 with probability . 3
The 1 → 0 flip. 1 In this case the Hamming distance increases by 1 with probability . 3 2
In this case the Hamming distance decreases by 1 with probability . 3
In the following we will justify these relations. We consider the clause (𝑥˅𝑦˅𝑧) with the literals x,y,z being positive or negative (variables or the negation of variables). We assume that the satisfying assignment for the whole expression exists and for these three literals takes the values (1,0,0), in other words 𝑥 = 1, 𝑦 = 0, 𝑧 = 0. We consider the 1 → 0 flip. We have a few cases to consider. Case 1. The actual (or current) assignment sets the clause to a clause with one literal true. Subcase 1.1. The actual assignment sets the clause to (1,0,0). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝1 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 0. Subcase 1.2. The actual assignment sets the clause to (0,1,0). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝2 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1. Subcase 1.3. The actual assignment sets the clause to (0,0,1). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝3 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1. 1
From symmetry each of these three subcases will occur with probability . Then in this case, the 3 overall probability of decreasing the Hamming distance to the solution by 1 is 𝑝𝑐𝑎𝑠𝑒 1 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1 1 1 2 · 0+ · 1+ · 1= . 3
3
3
3
5
Case 2. The actual (or current) assignment sets the clause to a clause with two literals true. Subcase 2.1. The actual assignment sets the clause to (1,1,0). In this case the probability of decreasing the Hamming distance to the solution by 1 1 is 𝑝1 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = . 2 Subcase 2.2. The actual assignment sets the clause to (1,0,1). In this case the probability of decreasing the Hamming distance to the solution by 1 1 is 𝑝2 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = . 2 Subcase 2.3. The actual assignment sets the clause to (0,1,1). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝3 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1. 1
From symmetry each of these three subcases will occur with probability . Then in this case, the 3 overall probability of decreasing the Hamming distance to the solution by 1 is 𝑝𝑐𝑎𝑠𝑒 2 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1 1 1 1 1 2 · + · + · 1= . 3
2
3
2
3
3
Case3. The actual (or current) assignment sets the clause to a clause with three literals true (1,1,1). In this case the probability of decreasing the Hamming distance to the solution by 1 is 2 𝑝𝑐𝑎𝑠𝑒 3 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = . 3
We now assume again that the satisfying assignment for the whole expression exists and for these three literals takes the values (1,0,0), in other words 𝑥 = 1, 𝑦 = 0, 𝑧 = 0, but we consider the 0 → 1 flip. We have again three cases to consider. Case 1. The actual (or current) assignment sets the clause to a clause with one literal true. Subcase 1.1. The actual assignment sets the clause to (1,0,0). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝1 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 0. Subcase 1.2. The actual assignment sets the clause to (0,1,0). In this case the probability of decreasing the Hamming distance to the solution by 1 1 is 𝑝2 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = . 2 Subcase 1.3. The actual assignment sets the clause to (0,0,1). In this case the probability of decreasing the Hamming distance to the solution by 1 1 is 𝑝3 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = . 2
1
From symmetry each of these three subcases will occur with probability . Then in this case, the 3 overall probability of decreasing the Hamming distance to the solution by 1 is 𝑝𝑐𝑎𝑠𝑒 1 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1 1 1 1 1 1 · 0+ · + · = . 3
3
2
3
2
3
Case 2. The actual (or current) assignment sets the clause to a clause with two literals true. Subcase 2.1. The actual assignment sets the clause to (1,1,0). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝1 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 0. Subcase 2.2. The actual assignment sets the clause to (1,0,1). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝2 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 0. Subcase 2.3. The actual assignment sets the clause to (0,1,1). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝3 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1 . 1
From symmetry each of these three subcases will occur with probability . Then in this case, the 3
6 overall probability of decreasing the Hamming distance to the solution by 1 is 𝑝𝑐𝑎𝑠𝑒 2 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1 1 1 1 · 0+ ·0+ · 1= . 3
3
3
3
Case3. The actual (or current) assignment sets the clause to a clause with no literals true (0,0,0). In this case the probability of decreasing the Hamming distance to the solution by 1 is 𝑝𝑐𝑎𝑠𝑒 3 (𝑑𝑒𝑐𝑟𝑒𝑎𝑠𝑒 𝐻) = 1 . 3
A similar analysis can be made in order to calculate the probability of increasing the Hamming distance for each case and subcase. We note that we took into account the fact that the probabilities of hitting each subcase for every case considered (when we consider the random walk on the assignment hypercube) are equal, and this happens for reasons of symmetry. We note (for example) that a variable that appears in many positive literals (with truth value 1) in many clauses, has a slightly better chance to be chosen for a 1 → 0 flip as compared to a variable that appears in only a few places. Also a variable that appears in many positive literals (with truth value 0) in many clauses, has a slightly better chance to be chosen for a 0 → 1 flip as compared to a variable that appears in only a few places. Similarly for variables appearing in negative literals. Our random walk is not a pure random walk on the assignment hypercube. It looks like that, but when choosing the variables involved in flips we take into account the structure of the problem. If we consider the ensemble of paths the random walk will take (for a quadratic number of steps), and we pick a certain path, then with probability close to one the frequencies with which the subcases discussed above appear (when we run the algorithm) will be equal. In other words, the frequency with which the actual assignments (1,0,0), (0,1,0), (0,0,1) (for example) appear will be the same (related to a given clause). Similarly for (1,1,0), (1,0,1), (0,1,1) . Some variables will be involved in flips more than others, but the relative configuration (related to a given clause) will appear in the same proportion. This statement is not hard to prove. A4. An example. We will consider a very simple example, in order to clarify the dynamics of the algorithm. We start with the 3SAT expression (𝑥˅𝑦˅𝑧)˄(┐𝑥˅┐𝑦˅┐𝑧) . We consider the transformed ONE-IN-THREE SAT expression (┐𝑥˅𝑎1 ˅𝑏1 )˄(𝑦˅𝑏1 ˅𝑐1 )˄(┐𝑧˅𝑐1 ˅𝑑1 )˄(𝑥˅𝑎2 ˅𝑏2 )˄(┐𝑦˅𝑏2 ˅𝑐2 )˄(𝑧˅𝑐2 ˅𝑑2 ) . We now follow the algorithm. We start with the random truth assignment 𝑥 = 0, 𝑦 = 0, 𝑧 = 0, 𝑎1 = 1, 𝑏1 = 1, 𝑐1 = 0, 𝑑1 = 1, 𝑎2 = 0, 𝑏2 = 0, 𝑐2 = 1, 𝑑2 = 1 . Our ONE-IN-THREE SAT expression will look like this: (1˅1˅1)˄(0˅1˅0)˄(1˅0˅1)˄(0˅0˅0)˄(1˅0˅1)˄(0˅1˅1) . Now we randomly choose a type of flip to perform. We assume that we chose the 1 → 0 flip. We randomly choose the location within the expression where to perform this flip. We assume that we chose the location as indicated in bold in the expression (1˅1˅1)˄(0˅1˅0)˄(1˅0˅𝟏)˄(0˅0˅0)˄(1˅0˅1)˄(0˅1˅1) . We flip the truth value of the variable 𝑑1 to 𝑑1 = 0 . The expression will become (1˅1˅1)˄(0˅1˅0)˄(1˅0˅0)˄(0˅0˅0)˄(1˅0˅1)˄(0˅1˅1) . We randomly choose again the type of flip to perform. We assume that we chose the 1 → 0 flip. We randomly choose the location within the expression where to perform this flip. We assume that we chose the location as indicated in bold in the expression (1˅1˅1)˄(0˅1˅0)˄(1˅0˅0)˄(0˅0˅0)˄(1˅0˅1)˄(0˅1˅𝟏) . We flip the truth value of the variable 𝑑2 to 𝑑2 = 0 . The expression will become (1˅1˅1)˄(0˅1˅0)˄(1˅0˅0)˄(0˅0˅0)˄(1˅0˅1)˄(0˅1˅0) . We randomly choose again the type of flip to perform. We assume that we chose the 1 → 0 flip. We randomly choose the location within the expression where to perform this flip. We assume that we chose the location as indicated in bold in the expression (1˅1˅𝟏)˄(0˅1˅0)˄(1˅0˅0)˄(0˅0˅0)˄(1˅0˅1)˄(0˅1˅0) . We flip the truth value of the variable 𝑏1 to 𝑏1 = 0 (we note that it appears in two locations). The expression will become (1˅1˅0)˄(0˅0˅0)˄(1˅0˅0)˄(0˅0˅0)˄(1˅0˅1)˄(0˅1˅0) . We randomly choose again the type of flip to perform. We assume that we chose the 0 → 1 flip. We randomly choose the location within the expression where to perform this flip. We assume that we chose the location as indicated in bold in the expression (1˅1˅0)˄(0˅0˅0)˄(1˅0˅0)˄(𝟎˅0˅0)˄(1˅0˅1)˄(0˅1˅0) . We flip the truth value of the variable x to 𝑥 = 1 (we note that it appears in two locations). The expression will become (0˅1˅0)˄(0˅0˅0)˄(1˅0˅0)˄(1˅0˅0)˄(1˅0˅1)˄(0˅1˅0) . We randomly choose again the type of flip to perform. We assume that we chose the 0 → 1 flip. We randomly choose the location within the expression where to perform this flip. We assume that we chose the location as indicated in bold in the expression
7 (0˅1˅0)˄(𝟎˅0˅0)˄(1˅0˅0)˄(1˅0˅0)˄(1˅0˅1)˄(0˅1˅0) . We flip the truth value of the variable y to y = 1 (we note that it appears in two locations) . The expression will become (0˅1˅0)˄(1˅0˅0)˄(1˅0˅0)˄(1˅0˅0)˄(0˅0˅1)˄(0˅1˅0) . We found the satisfying assignment 𝑥 = 1, 𝑦 = 1, 𝑧 = 0, 𝑎1 = 1, 𝑏1 = 0, 𝑐1 = 0, 𝑑1 = 0, 𝑎2 = 0, 𝑏2 = 0, 𝑐2 = 1, 𝑑2 = 0 for the ONE-IN-THREE SAT expression considered. Observation 2. The interesting problems will have variables that are connected to both positive and negative literals within the expression. That means that for the problems of interest, any truth assignment will guarantee that we have both 0’s and 1’s within the expression, so both flips will be possible at any given time. For the interesting problems, the algorithm will never get blocked. References (in the order in which they appear mentioned in the article): [1]. C.H. Papadimitriou, “Computational Complexity “, Addison - Wesley Publishing Company, Inc., 1994. [2]. Uwe Schoning, “ A probabilistic Algorithm for k-SAT and Constraint Satifaction Problems“, Research Supported by the ESPRIT Basic Research, 1991. [3]. L. Fortnow, “ The Golden Ticket, P, NP, and The Search For The Impossible “, Princeton University Press, 2013.
Cristian Dumitrescu, 119 Young St., Ap. 11, Kitchener, Ontario N2H 4Z3, Canada.
Email:
[email protected] [email protected] Tel : (519) 574-7026
Cristian Dumitrescu. Biographical notes. I was born in 1964 in Romania. In 1988 I graduated the University of Bucharest, Faculty of Mathematics with a BSc. In Mathematics. I worked as a high school teacher of mathematics and computer programmer for various companies and software houses in Romania, UK and Canada. I have been a Canadian citizen since 1998. In the last 20 years I have been working in a different field of activity, but I have always kept close to my heart mathematics and physics.
8
