On the Dynamic Detection of Interchangeability in Finite Constraint Satisfaction Problems Amy M. Beckwith Department of Computer Science and Engineering, 115 Ferguson Hall, University of Nebraska-Lincoln, Lincoln NE 68588-0115
[email protected]
Abstract. We investigate techniques to discover and exploit problem structure in the form of interchangeability. We enhance the performance of backtrack search with forward-checking (FC-BT) for finding all solutions to a finite Constraint Satisfaction Problem(CSP) with dynamic interchangeability detection and dynamic variable ordering. We evaluate these strategies both theoretically and empirically.
1
Motivation
It is widely acknowledged that real-world problems exhibit an intrinsic nonrandom structure that makes most instances ‘easy’ to solve. Such a structure is difficult to replicate ‘faithfully’ in instances on which we execute and evaluate algorithms intended to solve real-world problems. While it seems presumptuous to include readily in the model a particular structure or characteristic of a problem (e.g., pigeon-hole principle), it is questionable whether one can expect the validity and significance of experiments conducted on randomly generated problems to continue to hold in practical settings. Our goal is to design techniques that can benefit from the intrinsic structure of an instance of a real-world problem without restricting ourselves to the declared structure of a particular class of problems. One solution to this difficult dilemma is to develop general methods that dynamically uncover and exploit special structures embedded in a given instance of a problem to enhance the performance of problem solving. One such method is the computation of interchangeability and symmetry relations among the entities of the problem. Although it is likely that the computation of such relations is intractable in the most general case, several approximations have been proposed, such as weakening the relations or localizing their computation. In our research, we propose to integrate interchangeability detection mechanisms with backtrack search, investigate and evaluate the effectiveness of this integration, and demonstrate its utility under the most adverse conditions (i.e., random problems generated with no embedded structure and puzzles known to be particularly resistant to interchangeability detection techniques).
2
Introduction
We focus on problems that can be modeled as Constraint Satisfaction Problems because this paradigm is flexible enough to represent many challenging problems in engineering, computer science, and management. A finite Constraint Satisfaction Problem (CSP) is defined as P=(V, D, C); where V={V1 , V2 , . . ., Vn } is a set of variables, D={DV1 , DV2 , . . ., DVn } is the set of their corresponding domains (the domain of a variable is a set of possible values), and C a set of constraints that specifies the acceptable combinations of values for variables. A solution to the CSP is the assignment of a value to each variable such that all constraints are satisfied. The question is to find one or all solutions. We study CSPs with finite domains and binary constraints (i.e., they apply to two or fewer variables), but intend to extend our work beyond these two restrictions. Since a general CSP is NP-complete, it is usually solved by backtrack search, which is an exponential procedure. We enhance this basic backtrack search through the identification and exploitation of structure in the problem instance. This structure is in the form of symmetries. In particular, we make use of a type of symmetry called interchangeability, which was introduced and categorized in [Freuder 1991]. We limit our investigations to interchangeability among the values in the domain of one given variable. Interchangeability between two values for a variable exists, in either a local or a global environment, if the values can be substituted for one another without affecting the environment. Once interchangeable values in a variable are detected, they can be replaced by one representative of the bundle, thus reducing the size of the initial problem. In this work, we concentrate on mechanisms for uncovering and exploiting interchangeability while finding all solutions to a CSP. Two such techniques have already been proposed in the literature: NIC reported in [Haselb¨ ock 1993] and CPR reported in [Hubbe and Freuder 1989]. We propose a new method to search with interchangeabilities based on the joint discrimination tree (JDT). The discrimination tree was introduced by [Freuder 1991] as an efficient way to detect interchangeability among the values of one variable, and generalized by [Choueiry and Noubir 1998] to encompass two or more variables. We use the repeated computation of the JDT during search to find dynamic interchangeable sets (DNPI) and exploit the JDT to perform implicitly forward checking during search. NIC can be computed in a pre-processing step prior to search and provides static bundling of the solution space. CPR and DNPI are computed during search and provide dynamic bundling, which yields a better compaction of the solution space. We assess the performance of this new dynamic bundling strategy both theoretically and empirically. Then we investigate the combination of bundling strategies with variable ordering heuristics, and show empirically the utility of combining dynamic bundling with dynamic variable ordering using the simple least-domain heuristic. This paper is organized as follows. Section 3 gives an overview of work already completed: Section 3.1 discusses the utility of interchangeability for searching for all solutions, and Section 3.2 discusses the combination of interchangeability
with variable ordering heuristics. These two aspects are discussed in [Choueiry and Beckwith 2001] and [Beckwith and Choueiry 2001], respectively. Section 4 concludes this paper and lists some of the activities on our research agenda.
3
Work completed so far
At these early stages of our research on exploiting interchangeability, we first show that interchangeability can be used to enhance the performance of the familiar backtracking search with forward checking (FC-BT) [Haralick and Elliott 1980] by dynamically bundling the solution space. Then we demonstrate that this bundling process can be advantageously combined with one standard improvement to backtrack search, namely dynamic variable ordering [Bacchus and van Run 1995]. 3.1
Interchangeability to enhance search
In [Choueiry and Beckwith 2001], we introduce a forward checking strategy with dynamic bundling (FC-DNPI), which computes interchangeability at the instantiation of each variable, during search. Because instantiations restrict the domain of the instantiated variables to the assigned values, interchangeabilities that did not exist before search began may present themselves during search. We show that the bundling done by FC-DNPI is equivalent to that of done by CPR. Then we compare the three bundling strategies (one static and two dynamic) in terms of standard criteria: number of nodes visited, number of constraint checks and number of generated bundles. We also give CPU time. We establish theoretically and demonstrate empirically that dynamic bundling is always beneficial to the search process in terms of constraints checked and nodes visited, and that static bundling is usually beneficial, but not always (for example, in puzzles, static bundling may significantly add to the number of constraint checks). The results in terms of comparisons between the various strategies are illustrated in Figure 1. These results hold for all static or dynamic variable orderings, provided search computes all solutions and the initial variable ordering is the same across all strategies. Number of Solution Bundles Number of Constraints Checks Number of Nodes Visited > FC-NIC > > > FC-BT FC-NIC FT-BT FC-CPR = FC-DNPI FC-NIC FC-CPR > FC-DNPI FC-BT > FC-CPR = FC-DNPI
Fig. 1. Comparing bundling strategies.
Theorem 1. For the number of nodes visited (NV), the following orders hold: NV(FC-BT) ≥ NV(FC-NIC) ≥ NV(FC-DNPI) and NV(FC-CPR) ≥ NV(FC-DNPI). Theorem 2. For the number of constraints checked (CC), the following orders hold: CC(FC-BT) ≥ CC(FC-CPR)= CC(FC-DNPI). However, CC(FC-NIC) is comparable to neither CC(FC-BT) nor CC(FC-DNPI).
Theorem 3. For the number of solution bundles generated (SB), the following total order holds: SB(FC-BT) ≥ SB(FC-NIC) ≥ SB(FC-CPR) = SB(FC-DNPI)
Additionally, we verify each of these results empirically by running experiments on a battery of random problems and puzzles that further demonstrate their usefulness. 3.2
Combining dynamic bundling with ordering heuristics
In [Beckwith and Choueiry 2001], we investigate the combination of dynamic bundling with ordering heuristics. For that purpose, we combine: – three ordering heuristics (i.e., static least domain (sld), dynamic least domain (dld), and promise heuristic) and – three search strategies (i.e., no-bundling (FC-BT), static bundling (FC-NIC) and dynamic bundling(FC-DNPI)). These combinations yield the nine search algorithms shown in Figure 2.
Bundling
Ordering None Static
Dynamic
Static
Dynamic variable
Dynamic variable-value
FC-BT-sld FC-NIC-sld FC-DNPI-sld
FC-BT-dld FC-NIC-dld FC-DNPI-dld
FC-BT-promise FC-NIC-promise FC-DNPI-promise
New searches
Fig. 2. Nine search algorithms tested.
We develop strategies to allow the combination of bundling and dynamic variable and/or value ordering, and use these strategies to perform search. Because dynamic ordering, by its nature, does not allow us to maintain the same orders of variables and/or values across the strategies, theoretical comparisons cannot be made. Instead, we conduct tests on randomly generated problems and puzzles, and compare the strategies with respect to the three criteria cited above and CPU time. From our observations of the empirical data, we report first that dynamic bundling is always worthwhile. This is not a surprising result. The data shows that even in problems known to not contain interchangeability (such as puzzles), FC-DNPI never expands more nodes or checks more constraints than FC-BT. Further, the bundling is always better (equal, in puzzles, where no bundles are possible). The CPU-time in FC-DNPI is always much better than FC-BT, and almost always better than FC-NIC. This supports the theoretical claims made above. We also note that dynamic ordering (dld) is almost always better than static ordering (sld), both for bundled and non-bundled search. However, we notice that promise, though designed to visit a minimal number of nodes, performs a very large number of constraint checks which result in consistently poor performance, especially in terms of CPU-time. Simple dynamic least domain with our dynamic bundling strategy yields the most competitive results.
3.3
Contributions
The contributions of these two works include: 1. We show how to exploit the JDT to compute interchangeabilities dynamically (DNPI) during backtrack search, yielding a better bundling of the solution space than NIC [Haselb¨ ock 1993], and using no more constraint-checks than forward checking (FC-BT). 2. We provide an adaptation of the backtrack-search procedure to allow dynamic variable-value orderings with interchangeability. 3. We establish theoretically and demonstrate empirically order relations between the FC-BT and the (static/dynamic) bundling algorithms with respect to three criteria that assess the search effort and the ‘compaction’ of the solution space, and with respect to CPU-time. 4. We show the additional value of ordering variables dynamically (rather than statically) during search, and show that the improvement due to dynamic bundling is greater than the improvement from dynamic variable ordering.
4
Conclusions and Future research
We propose a new search procedure (FC-DNPI) based on the dynamic computation of interchangeability. We discuss the relations between this new strategy and the known FC-BT and FC-NIC procedures. We compare these strategies theoretically and empirically in terms of three criteria for assessing the performance of search and show that our strategy can never cause any degradation even when no bundling is possible. Further, we demonstrate that ordering strategies and bundling mechanisms are orthogonal but perfectly compatible processes for improving the performance of search. The former allows a better navigation in the search space and the latter shrinks its size. We demonstrate that both are successful in making search run faster, and we propose a combination that we prove empirically to be worthwhile. Our investigations can be continued in a number of directions: – All the results reported here hold for finding all solutions. We are currently investigating to what extent they hold for finding a single, or a maximal bundle [Lesaint 1994], alternatively, a pre-specified number of bundles. – Design a generator of random problems (such as the one described in [Freuder and Sabin 1995]) that models interchangeability and controls its occurrences in a problem instance, then conduct tests on these problems to demonstrate the significance of the utility of our techniques. – Demonstrate that dynamic bundling remains competitive when integrated to search strategies that are based on maintaining arc-consistency (MAC). – We are currently working on interchangeability for non-binary constraints and hope to report results in the (near) future. – We firmly believe that the use of interchangeability should be extended to CSPs with continuous domains, especially CSPs with monotonic constraints, functional constraints, and what we call pseudo-functional constraints, which are constraints that can be represented by block-diagonal binary matrices.
– Since these techniques, while harmless, cannot significantly improve the solving of puzzle-like problems, it is tempting to integrate DNPI with the symmetry-detection algorithm described in [Benhamou 2000], although computing those symmetries is known to be equivalent to graph isomorphism [Crawford 1992; Benhamou 2000].
Acknowledgments This work is directed by Berthe Y. Choueiry.
References Bacchus, Fahiem and Run, P.van 1995. Dynamic Variable Ordering in CSPs. In Principles and Practice of Constraint Programming, CP’95. Lecture Notes in Artificial Intelligence #976, Springer Verlag. 258–275. Beckwith, Amy M. and Choueiry, Berthe Y. 2001. Effects of Dynamic Ordering and Bundling on the Solution Space of Finite Constraint Satisfaction Problems. In Submitted to CP’2001. Benhamou, Beliad 2000. Symmetry and Dominance in Constraint Satisfaction Problems (Draft). Research Report TR-353, LIM, Centre de Math´ematiques et Informatique de Marseille. Choueiry, Berthe Y. and Beckwith, Amy M. 2001. Techniques for Bundling the Solution Space of Finite Constraint Satisfaction Problems. In Submitted to CP’2001. Choueiry, Berthe Y. and Noubir, Guevara 1998. On the Computation of Local Interchangeability in Discrete Constraint Satisfaction Problems. In Proc. of AAAI-98, Madison, Wisconsin. 326–333. Crawford, James M. 1992. A theoretical analysis of reasoning by symmetry in first-order logic (extended abstract). In Working Notes of the Workshop on Tractable Reasoning, AAAI-92, San Jose, CA. Freuder, Eugene C. and Sabin, Daniel 1995. Interchangeability Supports Abstraction and Reformulation for Constraint Satisfaction. In Symposium on Abstraction, Reformulation and Approximation, SARA’95, Ville d’Esterel, Canada. Freuder, Eugene C. 1991. Eliminating Interchangeable Values in Constraint Satisfaction Problems. In Proc. of AAAI-91, Anaheim, CA. 227–233. Haralick, Robert M. and Elliott, Gordon L. 1980. Increasing Tree Search Efficiency for Constraint Satisfaction Problems. Artificial Intelligence 14:263–313. Haselb¨ock, Alois 1993. Exploiting Interchangeabilities in Constraint Satisfaction Problems. In Proc. of the 13 th IJCAI, Chamb´ery, France. 282–287. Hubbe, Paul D. and Freuder, Eugene C. 1989. An Efficient Cross Product Representation of the Constraint Satisfaction Problem Search Space. In Proc. of AAAI-92, San Jose, CA. 421–427. Lesaint, D. 1994. Maximal Sets of Solutions for Constraint Satisfaction Problems. In Proc. of the 11 th ECAI, Amsterdam, The Netherlands. 110–114.
