Filtering algorithms and Dual Encoding of non binary ...

International Conference on Intelligent Information and Network Technology (IC2INT’13)

Filtering algorithms and Dual Encoding of non binary Constraint Satisfaction Problems Bouhouch Adil

Loqman Chakir

ELQadi Abderahim

Team TIM High School of Technology Moulay Ismail University, Meknes, Morocco

Team TIM High School of Technology Moulay Ismail University, Meknes, Morocco

Team TIM High School of Technology Moulay Ismail University, Meknes, Morocco

[email protected]

[email protected]

[email protected]

ABSTRACT A constraint satisfaction problem (CSP) involves the assignment of values to variables which are subject to a set of constraints. It’s known that any non-binary Constraint Satisfaction Problem can be transformed into an equivalent binary CSP using the dual encoding (DE). Therefore, we can use the well-established binary techniques to solve or to reduce it. In this paper, we provide a systematic comparison empirically of the performances of the filtering algorithms on the large and hard non binary CSPs. In particular, we analyze the potential benefits of applying filtering algorithm on the non binary CSP compared to arc consistency on the dual encoding. Finally, some computational experiments solving the non binary CSPs are shown.

General Terms Optimization under constraint programming , Artificial Intelligence.

filtering can be used alone to reduce the size of problem or to work in parallel with solving methods. In this paper we will attempt to answer the question: which of Non-binary filtering and binary filtering is useful to combine with a binary solver as the first pre-processing, here we choose a binary version of Maintaining Arc Consistency (MAC) method based on AC3 algorithm, this method is considered to be among the most efficient generic approaches for the solution of CSP instances. MAC algorithm explores the search space depth-first [2] backtracks when dead-ends occur, and enforces (generalized) arc consistency after each taken decision (variable assignment or value refutation) during search. In this work we will study three approaches: 

Keywords Constraint satisfaction problems, dual encoding, binary filtering, non-binary constraints.

1. INTRODUCTION Many problems from the real world can be represented as constraint satisfaction problems. For example, spatial and temporal planning, qualitative and symbolic reasoning, diagnosis, scheduling, hardware design and verification, realtime systems and robot planning, etc... In general, a CSP is a problem composed of a finite set of variables, each of which has almost a finite domain of values, and a set of constraints. In the practices, we can easily convert any non-binary CSP into an equivalent binary one. There are two well known transformations, the dual transformation [1] and the Hidden Variable Transformation [1]; therefore we can exploit stronger binary resolving methods or binary filtering (Arc consistency) to solve non binary CSPs. Techniques to solve CSP can be classified into those complete and incomplete methods. Complete methods explore a search space systematically and provide all solutions or non-feasibility of the problem, while incomplete methods provide just an acceptable solution. Recently, new heuristics methods based on neural network have shown a high performance [13] [14]. The





The first approach (DE+AC3+MAC) is divided into three steps: The first one concern converting the non binary CSP problem into binary CSP using dual encoding (DE). The second step involves applying Arc consistency (AC3) to reduce the obtained binary CSP. The later step concerns applying the MAC to solve the obtained binary CSP. See Fig 3. The second approach (DFC+DE+MAC) is also divided into three steps: The first concerns applying a Domain filtering consistency (DFC) to reduce the non binary CSP [12]. The second step involves converting the resulting CSP into binary CSP using dual encoding (DE). The later step concerns applying the MAC to solve the obtained dual encoding problem. See Fig 3. The third approach (DFC+DE+AC3+MAC) concerns combining the first and second approaches.

This paper is organized as follows. In section 2, we provide a formulation of Constraint Satisfaction problem. In section 3, we describe the conversion of non binary CSP problem to equivalent binary problem. In section 4, we describe the filtering algorithms. The experimental results are presented in section 5.

2. GENERAL APPROACHES FOR SOLVING CSP The constraint satisfaction problem can be formulated as a triple (X;D;C) where:  X = {X1,....., XN } is a set of N variables.  D = {D1,....., DN } where each Di is a set of di possible values for Xi.

1

International Conference on Intelligent Information and Network Technology (IC2INT’13)

 C = {C1,....., Cm } is a set of constraints. Each constraint Ci involves an ordered a subset of variables of X. This subset of variables is the scope of Ci and is denoted by var(Ci). The arity of constraint is the number of variables in its scope variables. A binary CSP is a problem where all its constraints either are unary or binary constraints. A non-binary CSP is a problem which includes at least one non-binary Constraint. A solution to a CSP is a complete assignment that satisfies all constraints. In some problems, the goal is to find all such assignments. Finding a solution to a CSP is an NP-complete problem. On one hand, we can guess the assignments of all variables and it is not difficult to check whether all the constraints are satisfied given these assignments. So it is in NPHard. On the other hand, the satisfiability problem SAT, which is known to be NP-hard, can be encoded as a CSP. Many approaches have been developed for solving these problems. One simple approach to solve CSPs is the generate-and-test paradigm GT. In this approach, each possible combination of the variables is systematically generated and then tested to see if it satisfies all the constraints. The first combination that satisfies all the constraints is the solution. The number of combinations considered by this method is the size of the Cartesian product of all the variable domains. Thus, the time complexity of this approach is exponential in the number of variables. Empirically, this method performs very poorly to find first solution. The backtracking algorithm BT is a central algorithm for solving constraint satisfaction problems . It employs a depth-first search in order to instantiate variables and a back-track mechanism when dead-ends occur. Many works have been devoted to improve its forward and backward phases by introducing lookahead and look-back schemes [12]. However, the run-time complexity of backtracking algorithm for most nontrivial problems is still exponential. One of the reasons for this poor performance is that the backtracking paradigm suffers from thrashing (search in different parts of the space keeps failing for the same reason). Another drawback of the standard backtracking algorithm is that it has to perform redundant work. Other search algorithms for classical CSPs include: Forward Checking, Partial Lookahead, Full Lookahead, and Really Full Lookahead. They are introduced and their performances are compared in [14].

3. NON-BINARY TO BINARY TRANSFORMATIONS

c

each constraint C into a dual variable Vi , over a domain Di i i

built by supported tuples of original constraint C i . We refer to the variables of the original non-binary CSP as ordinary variables noted xi with i=1,2,...,n, n is the number of problem variables. Then, we link each pair of variables Vi and sharing

( var(Ci )  var(C j )  1 ),

at

last with

one a

ordinary

binary

requires that the pair tuples

  Dici

j and   D j values

'

c

assigned to their scopes dual variables must have the same value of shared ordinary variables x k  var(Ci )  var(C j ) : τ[ var (C )  var (C )]  τ' [ var (C )  var (C )] i j i j

Consider the following example with six variables with 0-1 domains, and four constraints: c1: x + x + x = 1 c2: x - x + x = 1 , c3 : x 4 + x 5 - x 6  1 , 1 3 4 1 2 6 and c4 : x 2 + x 5 - x 6 = 0 .The Dual transformation of this problem is shown in Figure 1. Figure 1 shows the DE for this problem. In the DE we have four dual variables Vi, i = 1, 2, 3, 4. The domains of these variables are the tuples that satisfy the respective constraint. For example, the dual variable V2 associated with the second constraint has the domain {(0, 0, 1), (1, 0, 0), (1, 1, 1)}. There are binary compatibility constraints between dual variables whose corresponding non-binary constraints share one or more original variables. For example, there is a constraint between V1 and V2 (denoted by R11) because the corresponding non-binary constraints of V1 and V2 share original variable x1.

Fig 1 : Example of dual encoding graph transformation

Dual Variable Encoding (DE) transformation is one of two ways to translate a non-binary CSP to binary one, we have chosen DE on which the level of consistency archived is higher than Hidden Variable Encoding. Dual Variable Encoding consists converting

V j ( i  j ),

value; that means, to be satisfied, the binary Ci , j dual constraint

variable

compatibility

constraint Ci , j . This constraint restricts the dual variables to

Noting that CSP (X;D;C) takes O(n + nd + mr) space, where n denotes the number of variables, d the size of the largest domain, m the number of constraints, and r the arity of the largest constraint scheme in (X;D;C). For the dual transformations of an original (non-binary) formulation, we can also specify its size in terms of the parameters of (X;D;C). In the worst case, the dual transformation (V;D;C) takes O(m+ mdr + m2) space. Thus, the space required by dual transformations can grow exponentially with the augmentation of constraints arity in the original formulation; therefore, a filtering process can reduce search space by changing constraints relation structures. For DFC+DE scenario, just a valid tuples of constraints relations found by filtering pre-process will be migrated to the domain of Dual Variables.

tuples in which the original shared variables take the same

2

International Conference on Intelligent Information and Network Technology (IC2INT’13)

4. PROBLEM REDUCTION Filtering algorithms allow the simplification of the CSP before or during the search of solutions. The most used methods reduce the problem into an equivalent CSP while conserving the arcconsistency property and exactly all the solutions. Actually, filtering algorithms remove the values which obviously can not belong to the solution and thus reduce the search space. Although filtering can proves the unsatisfiability of the problem, if the CSP is not consistent. Definitions: 

Let X j  var(Ci ) .

A

 

vD X j

value

is

consistent with a constraint Ci if   rel(Ci ) such that  [ X j ]  v and  is valid. In this case we say that  is a support of v in Ci . 

A constraint Ci is Arc Consistent if for each variable X j  var(Ci ) , v  DX j  , there exists a support for v in Ci .



A CSP (X,D,C) is arc consistent if there is no empty domain in D and all constraints in C are Arc Consistent.

CSP Problem Dual Variable Encoding Translation

Filtering CSP via AC3

of a variable can be found in, at least, a support of any constraint. We say that a variable of a constraint satisfaction problem is arcconsistent with another one if each of its admissible values is consistent with some admissible value of the second variable, for binary CSPs there are many stronger algorithms for arc consistency (AC). Arc consistency on the dual transformation is tighter than arc consistency on the original formulation, which itself is equivalent to arc consistency on the hidden transformation [11]. AC can be enforced in a binary CSP with O(ed2) optimal worstcase time complexity, since in the DE of a CSP with e k−ary constraints there are at most e(e − 1)/2 binary constraints (when all pairs of dual variables share one or more original variables). Therefore, we can enforce AC in the DE of k−ary CSP with O(e2d2k) in worst-case complexity. This is significantly more expensive compared to the O(edk) complexity bound of GAC in the non-binary representation

4.2 Filtering on Non-binary CSPs For binary constraints, many domain filtering consistencies have been proposed and evaluated, including inverse and singleton consistencies [3, 4, 5]. The situation is very different in the case of non-binary constraints; there are a limited number of consistencies that are stronger than Generalized Arc consistency which have been developed, including relational consistency [6], pairwise consistency [7], and hyper m-consistency [8]. However, these are typically not domain filtering. In addition, algorithms that enforce them typically have a high time complexity. In [9] authors study the following domain filtering consistencies: restricted pairwise consistency (RPWC), relational path inverse consistency (rPIC), and max restricted pairwise consistency (maxRPWC). For our filtering algorithm on original non-binary CSP, we have chosen a maxRPWC-3 algorithm.

End

 i  1,..., N  D(Vi )  ø

Yes

No Solution

NO

MAC Solver

All Constraints are satisfied

False

True Solution

End

a Fig 2 :DE+AC3+MAC Schema Solver Non-binary Arc constituency algorithm (maxRPWC-3)

4.1 Arc consistency in Dual Encoding The constraint satisfaction problems (CSP) are usually represented as graphs, where nodes correspond to variables and edges to constraints. As arc consistency is one of the basic properties of (CSP), it guarantees that any value of the domain

Where Algorithm DFcons takes as input a (non-binary) CSP (X,D,C), a specified domain filtering local consistency DFC and a parameter current-var, and enforces DFC (X,D,C),. The parameter current-var is set to −1 if the algorithm is used

3

International Conference on Intelligent Information and Network Technology (IC2INT’13)

standalone (e.g. for pre-processing a problem with a given consistency). Otherwise if the algorithm is applied during search (e.g. to maintain a given consistency), current-var is the currently assigned variable. DFcons uses a list Q (that can be implemented as a stack or as a queue) of constraints to propagate value deletions.

Revise Function

Non Binary CSP

Filtering CSP via DFC End

 i  1,..., N  D( X i )  ø

No Solution

Yes

NO Dual Variable Encoding Translation MAC Solver

All Constraints are satisfied

generated CSPs. Simulation has been done with the following Machine characteristics: I5 core(TM) 2,35GHz processor and limitation of memory to 2GO .to generate random problems, we use a randomly generator based model extended Model B as it is described in [10]. To summarize this generation method, a random non-binary CSP is defined by the following five input parameters:  n : number of variables  d : domain size  k: arity of the constraints  p: density percentage of the generated graph, which determines the number of constraint.  t: the tightness of each constraint, is a ratio between the number of effective support tuples and the max we can generate for this class (dk) In the following, a class of non-binary CSPs will be denoted by a tuple of the form < n, d, p, k,t >. All constraints are ternary. For simulation, we generate 50 instances for each t value, and we calculate the average of run time of each instance in the same class of t value. In Figure 4 the algorithms are performed on the class, which has m = 129 constraint. We note that the three algorithms have almost the same behaviour execution time. So, for the number of visited nodes, Figure 5 shows that the number of visited nodes by DE + MAC and DFC+ DE +AC3+MAC are less than DFC + MAC, this is due to the power of the AC3 on one hand ,on the other hand DFC algorithm can’t eliminate all invalid tuples . This distinction is clearly obvious in the instances of class, with a d=15 and m=18 constraints, see Figure 6 and Figure 7. As a summary, the filtering on non-binary original problem combined with a filtering on binary Dual Encoding does not decrease the total time achieved by solver, because filtering of non-binary problem facilitates AC3 binary filtering task. To enrich our study, we choose to run resolving methods on some real classes downloaded from [15] and with replacing MAC implementation with a known Choco java solver [20], the result is shown in Table 1. The first column shows the resolution of non-binary CSP solved directly by default Choco Solver, while the second column shows the resolution of binary DE transformation of non-binary instances. In the other columns we have a filtering time and resolution time separated. We notice for aim class the resolution of binary CSPs translations are faster than the resolution of original non-binary ones, but in class TSP (Travelling Salesman Problem) we observe the inverse. As For the crossword instances, they are solved in the filtering stage.

False

True Solution

End

a Fig 3: DFC+DE+MAC Schema Solver

5. Experimentation and results To evaluate the performance of proposed resolving methods and hybrid filtering method which combine them (DFC+DE+AC3+ MAC), we run some preliminary experiments on the randomly generated problems. we compare the run times and visited nodes during the resolution of dual encoding of random ternary

4

International Conference on Intelligent Information and Network Technology (IC2INT’13)

Fig 6: run times over instances of class

Fig 4 run times over instance of class

Fig 5: Visited Nodes by MAC algorithm on the class

Fig 7: visited Nodes by MAC algorithm on the class

Table 1. Run times for resolving some non-binary instances from real class Choco

DE+Choco DE+AC3+choco

Instances

DFC+DE+choco

AC3

solver

DFC

Solver

DFC+DE+AC3+choco DFC

AC3

solver

aim-100-1-6-sat-1_ext

3185

188

75

0

226

0

229

0

0

aim-100-1-6-sat-3_ext

>300000

102

14

0

26

185

37

15

0

dubois-20_ext

35552

5710

3

6105

20

5980

21

2

6258

dubois-23_ext

312422

45178

2

45522

13

45362

11

1

44901

crossword-m1c-lex-vg14-14_ext

47

157

45

67

67

0

crossword-m1c-lex-vg14-15_ext

14

37

2

17

17

0

70 16

0 0

0 0

crossword-m1c-lex-vg14-16_ext

16

35

2

6

6

0

tsp-20-68_ext

64

40759

3046

4813

11911

2066

7 11931

0 223

0 1004

tsp-20-8_ext

34

339116

9596

1578

25752

1624

27561

967

1549

5

International Conference on Intelligent Information and Network Technology (IC2INT’13)

6. Conclusion In this paper, we have studied the performance of filtering algorithms: Domain filtering for non-binary CSP and a binary filtering on its Dual Encoding in order to choose one of them. We have found that the filtering algorithm on the binary problems is faster than non-binary problems. Nevertheless, the domain filtering algorithm DFC can solve the problem in some cases and reduces the size of the problem in other cases. The obtained results are promising and encourage us to combine a native binary solver with filtering of non-binary CSP, which can resolve the problem when the CSP is inconsistent, before generating the dual problem which is greedy in terms of time and memory, especially if the solver use constraints in extension form. In the future, we will try to improve filtering on non-binary CSP in way to prune more supports relations of constraints and combine it with our binary solver [16] [17].

[12] Dechter, R. Frost, D. 2002. Backjump-based backtracking for constraint satisfaction problems. Artificial Intelligence, 136, pp. 147-188. [13] Ettaouil, M. Loqman, C. 2009. A New Optimization Model for Solving the Constraint Satisfaction Problem. Journal of Advanced Research in Computer Science. Vol. 1, Issue. 1, pp. 13-31. [14] Ettaouil, M. Loqman, C. Haddouch, K. Hami , Y. 2013. Maximal Constraint Satisfaction Problems solved by Continuous Hopfield Network. Journal Wseas Transactions on Computers, Vol. 12, Issue. 2, pp. 26-40. [15] http://www.cril.univ-artois.fr/~lecoutre/benchmarks.html) an CSPs Benchmerk. [16] A CSP java API http://choco.mines-nantes.fr/.

7. REFERENCES [1] Samaras , N. Stergiou, K. 2011. Binary Encodings of Nonbinary Constraint Satisfaction Problems: Algorithms and Experimental. J. Artif. Intell. Res. Volume 24, pages 641684 . [2] Sabin, D. and Freuder, E.C. 1994. Contradicting conventional wisdom in constraint satisfaction. In Proceedings of CP’94, pp. 10–20. [3] Debruyne, R. Bessière, C. 2001. Domain filtering consistencies, J. Artif. Intell. Res., 14, pp. 205–230. [4] Freuder, E. Elfe, C. 1996. Neighborhood inverse consistency preprocessing, in: Proceedings of AAAI’96 pp. 202–208. [5] Verfaillie, D. Martinez, C. Bessière, 1999. A generic customizable framework for inverse local consistency, in: Proceedings of AAAI’99, pp. 169–174. [6] Van Beek, P. Dechter, R. 1995. On the minimality and global consistency of row-convex constraint networks, JACM, 42(3), pp. 543–561. [7] Janssen, P. Jégou, P. Nouguier, B. Vilarem, M.C. 1989.A filtering process for general constraint satisfaction problems: Achieving pairwise consistency using an associated binary representation, in: Proceedings of IEEE Workshop on Tools for Artificial Intelligence, 1989, pp. 420–427. [8]

Jégou, P. 1993. On the consistency of general constraint satisfaction problems, in: Proceedings of AAAI’93, pp. 114–119.

[9] Bessiere, C. Stergiou, K. Walsh, T. 2008. Domain filtering consistencies for non-binary constraints, Artificial Intelligence, 172, pp. 800–822. [10] Bessière, C. Meseguer, P. Freuder, E. C. Larrosa, J. 1999. On Forward Checking for Non-binary Constraint Satisfaction. In Proceedings of CP’99, pp. 88–102. [11] Bacchus, F. Chen, X. van Beek, P. Walsh, T. 2002. Binary vs. Non-Binary Constraints, Artificial Intelligence, 140 (12), pp. 1–37.

6