Combining SAT Solvers On Discrete Resources

Combining SAT Solvers On Discrete Resources Yanik Ngoko University of Yaounde I, Cameroon LIG, Grenoble University, France [email protected] ABSTRACT We are interested in this work in solving efficiently a set of instances of the SAT problem using several solvers such as to minimize the total execution time. We propose two new approaches based on the usage of a representative benchmark of SAT instances. In the first approach, we improve a problem introduced in [7, 4] such as to reduce the number of unsolved SAT instances. The second approach extends a naive solution for combining SAT solvers in a parallel context in introducing the selection of a convenient set of SAT solvers. We study the theoretical properties of our approaches and realize many experiments using a benchmark of SAT instances. The obtained results show that it is interesting to combine SAT solvers for reducing the number of unsolved SAT instances and the mean execution time required to solve a set of SAT instances.

KEYWORDS: Parallel SAT, porfolio, algorithms selection .

1. INTRODUCTION 1.1. Description of the context and motivation We are interested in this work by the resolution of hard computational problems like the satisfiability problem (SAT). There exist many solvers that have been developed for this problem. It is also well-established that there is no solver which dominates the others on unrestricted SAT instances [13]. In this paper, we study how in a parallel context with discrete resources, one can combine efficiently SAT solvers. Our objective is to minimize the execution time of a set of SAT instances. We propose several approaches inspired by portfolio problem, introduced in the field of finance many years ago [5]. This problem can be informally recalled as follows: given a set of opportunities,

Denis Trystram LIG, Grenoble University, France [email protected]

an amount of possible investments on the set of opportunities and the payoff obtained when investing an amount on each opportunity, what is the best amount of investment to make on each opportunity in order to maximize the sum of the payoffs? This scheme can be adapted to algorithms using the algorithm portfolio approach [2, 1, 7, 4]. A portfolio of algorithms is a collection of different algorithms (or the same algorithm with different parameters) running in an interleaved scheme. We consider the following variant of the portfolio problem: The input is a set of m ressources and a set of k algorithms A = {a1 , . . . , ak } (that are SAT solvers). For any instance I of SAT, we are looking for a Pk vector (S1 , . . . , Sk ) with i=1 Si ≤ m and 0 ≤ Si ≤ m such that if we execute each algorithm ai , 1 ≤ i ≤ k on I with Si resources and stop the execution when an algorithm finds a solution, we obtain the shortest execution time. The portfolio of algorithms generalizes the problem of selecting of the best SAT solver. Morever the interest to allocate for every instance to solve the adequate resources on more than one algorithm is that we may not know which is the best suited SAT solver for each instance before actually solving it. Our proposed approaches aim at improving those introduced in [7, 4] from the observation that they do not focus on the necessity to provide an answer for all the considered instances.

1.2. Related works There exist many studies for the selection and combination of algorithms, and in particular SAT Solvers. In [13], the authors describe a selection process (Satzilla) for SAT which generalizes one proposed in [11]. This approach allows to win many prizes in SAT competitions1 Satzilla considers two main elements, namely, a distribution and a set of features characterizing the target SAT instances. In [12] a reinforcement learning approach is proposed to select branching rules on DPLL based solver for SAT. The works done in [13, 11, 12] are all based on machine learning techniques. There exist also some alternative works on 1 http://www.satcompetition.org

SAT based on algorithm portfolio. In [8], the authors study how to interleave the execution of different heuristics in order to reduce the execution time of a set of instances and applied to solve SAT. In [7, 3, 4] the authors study how resources can be shared for SAT solvers such as to efficiently solve instances closed to those of a predefined benchmark. In [10], the authors propose a dynamic algorithm portfolio model in which the allocation of resources to SAT solvers can be changed during the execution of an instance.

1.3. Contributions In this paper we propose two models for building a portfolio of SAT solvers given a benchmark of SAT instances. The first model has been suggested in [4] and is based on a general problem (discrete Resource Sharing Scheduling Problem, dRSSP ) that we improve for the case of SAT. The improvement that we propose here is based on the fact that SAT is a decision problem on which we have to provide a response of its satisfiability. Thus, we adapt dRSSP such as to improve the number of SAT instances on which we are able to give a response on its satisfiability. The second model is built from a specific portfolio in which the difference between the number of resources assigned to each algorithm ai is at most 1. We propose an approach to reduce the total execution time required to solve all the instances in this model. These results are finally assessed by using instances extracted from actual execution time of existing SAT solvers.

1.4. Organization of the paper The rest of the paper is organized as follows. In Section 2, we present the online algorithm portfolio model that we consider in the paper to combine SAT solvers and we present the general approach (called CBAL) that we will use. In Section 3, we consider the CBAL approach when there exists a benchmark of instances and propose an algorithm based on the discrete Resource Sharing Problem (dRSSP ) introduced in [4]. Then, we analyze its application in the case of SAT, We finally propose an approach to take into account the observed limits. In considering a naive solution for the portfolio problem, we propose in Section 4 another algorithm to combine SAT solvers when there is a benchmark of instances. Finally, some experimental results for proposed algorithms are presented and discussed in Section 5 and we conclude in Section 6.

2. COMBININIG SAT SOLVERS ONLINE 2.1. The online algorithm portfolio problem The online portfolio of algorithms in a parallel context can be described as an adversarial game with N successive rounds. At the beginning of the game, the adversary fixes

a cost C(ai , Ij , l) > 0 for each algorithm i ∈ {1, . . . , k}, instance j ∈ {1, . . . , N } and resource l ∈ {0, . . . , m}. At the beginning of round j, the player chooses a vecPk j tor S j = (S1j . . . , Skj ) such that i=1 Si ≤ m and j Si ∈ {0, . . . , m}. After this choice, he gets in return the associated payoff C(S j , Ij ) = min C(ai , Ij , Sij ). When 1≤i≤k

chosing S j , the player has no knowledge about the fixed cost C(ai , Ij , l), 1 ≤ i ≤ k, 0 ≤ l ≤ m . The objective of the player is to minimize the sum of observed payoffs PN j j=1 C(S , Ij ). In this problem, the value C(ai , Ij , l) corresponds to the execution time of ai , on Ij with l resources. In a sequential context, the general problem in algorithm portfolio consists in building a function σ : [1, ∞[−→ A which at each time t gives the algorithm σ(t) which should be run such that if starting the execution of any instance I of SAT at time 0, we follow the execution as indicated by σ and stop when an algorithm finds a solution. In this way, we obtain the shortest execution time. We will consider in this problem that when giving more resources to an algorithm, the execution time decreases proportionnally (linear cost assumption). Let C(ai , Ij ) denotes the execution cost of algorithm ai on m resources (C(ai , Ij ) = C(ai , Ij , m)). The linear cost assumption formally states that that on m − m′ resources, the execum tion cost of ai is equal to m−m ′ C(ai , Ij ). We will also assume in what follows that m > k. This later assumption indicates that there are enough resources to give at least one resource to each algorithm. In what follows, we present a general approach for this problem.

2.2. The Constant Rebalanced Algorithm The Constant Rebalanced Algorithm (CBAL) is a well used approach for solving portfolio problem in finance. It consists mainly to choose a fixed amount of investment that we apply for all opportunities [5]. In a parallel context with discrete resources, this approach can be considered in choosing a fixed allocation of discrete resources and applying it for the resolution of each instance. Given a set of algorithms A and a set of instances I, one interesting property here is that the optimal solution using the CBAL is always at least as good as the best algorithm aopt ∈ A executed on the set of instances in I. A particular case using the CBAL approach consists in considering the M A (Mean Allocation) algorithm [4] which given a resource decomposition m = qk + r (where 0 ≤ r < k) allocates q + 1 resources to r algorithms ai , 1 ≤ i ≤ k and q resources to the k − r remaining ones. We start by executing all the algorithms on the allocated resources, we stop them as soon as an algorithm finds a solution. The necessary synchronization to inform

the different algorithms that one algorithm has completed its execution can efficiently be implemented in a sharedmemory context. Proposition 2.1. M A is a 2k − 1 competitive algorithm for the online portfolio problem. Proof Let C ∗ (I) be the shortest execution time of an instance I with an algorithm executed with the set of all the resources. Since we allocate at least q, (m = qk + r) resources to each algorithm (and then also to the one which solves I on all resources with the cost C ∗ (I)), then I is ∗ solved with a cost at least equal to C(I) = m q C (Ij ) = r ∗ k + q C (Ij ). Since r is at most equal to k − 1 and q is at least equal to 1, we have : C(I) ≤ (2k − 1)C ∗ (I). Since C ∗ (I) is a lower bound of the execution time of I with a portfolio, which concludes the proof. One can notice that it is hard to obtain a better approximation ratio than the M A algorithm if we do not have more informations on the problem. Indeed, given a CBAL where an algorithm a∗i has less than ⌈ m k ⌉ resources, one can easily remark that on a sequence of instances I = {I1 , . . . , In } 1 where for each Ij , C(a∗i , Ij ) < min ∗ m .C(ai , Ij ) the i=1,i6=i

M A algorithm is better. Let us consider a CBAL approach for the online portfolio problem, but now, we suppose that we have a benchmark which is representative of the set of online instances to solve. This assumption has often been made in different machine learning based SAT approach as [13, 12]. In what follows, we will present two approaches for combining algorithms based on a Benchmark. The first model is based on the dRSSP problem presented in [4]. The second model improves the M A algorithm by taking into account the benchmark of instances. Let consider the differents algorithms ai as candidate algorithms and the execution time of a portfolio of algorithms on an instance as its execution cost on this instance. We start by presenting the first model.

3. COMBINING ALGORITHMS USING dRSSP The dRSSP (discrete Resource Sharing Scheduling Problem) can be described as follows: Instance: A finite set of instances I = {I1 , . . . , In }, a finite set of candidate algorithms A = {a1 , . . . , ak }, a set of m identical resources, a cost C(ai , Ij , p) ∈ R+ for each Ij ∈ I, ai ∈ A and p ∈ {1, . . . , m}, a real value T ∈ R+ . Question: Is there a vector S = (S1 , . . . , Sk ) with Pk S i=1 Si ≤ m such that Pi n ∈ {0, . . . , m} and 0 < min {C(a , I , S )|S > 0} ≤T ? i j i i j=1 1≤i≤k

In this problem, C(ai , Ij , p) is the execution cost of the candidate algorithm ai , on Ij with p resources. Assumimg the linear cost assumption, we have the rela-

tion C(ai , Ij , p) = m p C(ai , Ij ), where C(ai , Ij ) = C(ai , Ij , m). dRSSP can be used to define a CBAL in assuming that for the set of online instances to solve, we have a benchmark of instances I. We build a solution S of dRSSP using these instances and our candidate algorithms for solving an online instance (we do not know which one), we execute each algorithm ai with Si resources. The interest of this sharing model is that in practice if the benchmark of instances is representative over all the possible instances, we may have a better mean execution cost than using the M A. The representativeness of the benchmark means that in the set of whole instances to solve, there are many instances that are those in the benchmark or that have ”similar” execution cost on different algorithms than those in the benchmark.

3.1. Analysis of dRSSP Complexity results and algorithms Theorem 3.1. dRSSP with linear cost assumption is NPComplete and it is not approximable unless P = NP [4]. The proof is based on a reduction to the vertex cover problem. We refer the reader to [4] for the details. In [7], an exact algorithm is provided for the continuous version of dRSSP where a non integer fraction of resources can be allocated to a candidate algorithm. In [8], a time sharing version of this problem is presented and a 4-approximation algorithm is provided. In [4], many approximation schemes are provided for dRSSP using an oracle based approach. We will now make some general remarks for dRSSP in the case of the SAT problem.

Considerations on the dRSSP for the case of SAT For SAT, the solution of dRSSP can be used to define a CBAL in taking the candidate algorithms as parallel SAT solvers. However it is important to notice that SAT solvers are generally built using a Branch and Bound approach that in practice is executed with a cutoff time. The cutoff time globally indicates the maximum amount of time led to the Branch and Bound algorithm to provide a solution. In considering the usage of a such cutoff in dRSSP , one can notice that on an instance Ij to solve, the resource sharing can be built such that when executing an instance Ij of the benchmark the solver i∗ such that {C(ai∗ , Ij , Si∗ ) = min {C(ai , Ij , Si )|Si > 0} stops its execution due to the 1≤i≤k

cutoff time. It then leads that we may not provide a solution on the satisfiability of Ij . This observation raises the question of fixing the cutoff time in a portfolio of SAT solvers. One can argue that we should fix a large cutoff to face this

situation. However, let us notice that if we fix a small cutoff time, one can take advantage of the portfolio execution that comprises several algorithms whose behaviour is not the same depending on the instances. In what follows, we propose a policy to fix cutoff time.

3.2. A policy to set cutoff time We suggest the following approach to fix the cutoff time. On the set of all the resources, we execute all candidate algorithms with a global cutoff time equal to a fixed value B. If a candidate algorithm has a proportion m′ of resources, we execute it with a local cutoff time equal to m.B m′ . One can notice that this policy respects the linear cost assumption previously defined. When a candidate algorithm has more resources, its cutoff time is smaller than when it has less resources. With the previous policy, one can notice that in a CBAL built with an exact solution of dRSSP , the previous observation still holds. Morever, on an instance Ij and a defined resource sharing, a solver can stop its execution without providing a solution on the satisfiability even if there exists a solver which can provide a solution. We propose in the next section some improvements based on the previous observations.

3.3. Discrete Resource Sharing with cutoff Definition Being given a porfolio of SAT solvers, one can notice that when running an instance I, if after the end of execution of a solver, we stop the other solvers only if this solver provided a solution on the satisfiablity of I, we will then obtain a response on the satisfiability of this instance if there is a solver in the portfolio with at least one resource which can give it. To improve the dRSSP , we will consider this later approach here. We suppose that when a candidate algorithm stops its execution on an instance, we first verify that it gave a solution. If this is not the case, we continue the execution of the other candidate algorithms until we get a response on the satisfiability. For the instance Ij to solve, let consider that the candidate algorithm aσ(j) is such that ∀i, C(aσ(j) , Ij ) < C(ai , Ij ). Let also suppose that at least one resource has been allocated to it. With the previous approach, if we do not consider when stopping a candidate algorithm, the time necessary to ensure that it found a solution, we can say that to solve the instance Ij , we will take C(a ,Ij ) . This later observation at most a time equal to m Sσ(j) σ(j) leads us to define a bi-criteria version of dRSSP . On one criteria, we target the minimization of the sum of execution time of the first candidate algorithm. But as we said, this candidate algorithm may not provide a solution on the

instance to solve. On the second criteria, we target the minimization of the upper bound that we can get in waiting that a candidate algorithm gives an answer on the satisfiability of an instance. This problem can be defined as follows: discrete Resource Sharing Problem with cutoff dRSSPc Instance: A finite set of instances I = {I1 , . . . , In }, a finite set of candidate algorithms A = {a1 , . . . , ak }, a set of m identical resources, a cost C(ai , Ij ) ∈ R+ for each Ij ∈ I and ai ∈ A, a real value T1 ∈ R+ , a set of value k real value {W1 , . . . , Wk }, a real value T2 . Question: Is there a vector S = (S1 , . . . , Sk ) with Pk Si ∈ {1, . . . , m} and ≤ m such that i=1 Si Pn C(ai ,Ij ) Wk W1 min { |S > 0} ≤ T and i 1 j=1 Si S1 +· · ·+ Sk ≤ 1≤i≤k

T2 Pn In this later problem, Wi = j=1|σ(j)=i C(aσ(j) , Ij ).

Algorithms

To solve dRSSPc , we assume that at least two resources are assigned to each candidate algorithm ai . This implies that m ≥ 2k. Let us also assume without loss of generality that m is even. Theorem 3.2. If there exists an α-approximation for the dRSSP , there exists a (3α, 4) approximation for dRSSPc on the zenith point 2 . Proof Let consider the Bicriteria Resource Sharing Algorithm (BRSA) which proceeds as follows: It solves the dRSSP on m resources using the α-approximation. and obtain a sharing vector S1 . In considering the following non linear program U P P : Wk 1 minimize W S 1 + · · · + Sk s.t S1 + . . . Sk ≤ m Si ≥ 2, ∀i Si ∈ [0, m] P n Where Wi = j=1|σ(j)=i C(aσ(j) , Ij ), it solves it using a convex nonlinear solver. Given a solution to the dRSSP , it retrieves to each algorithm ai , ⌈ S2i ⌉ resources and obtain a Pk vector of resources S 1 such that i=1 Si1 ≤ m 2 resources. Given a solution to U P P it divides the number of resources of each candidate algorithms by 2 and we set the number of resources to its integer part. It then obtain a vector S 2 Pk such that i=1 Si2 ≤ m 2 . It then consider as solution to the dRSSPc the vector S where Si = Si1 + Si2 . Since in transforming the solution of the dRSSP one multiplies its objective function by at most 3 and by at most 4 in the case of the U P P (2 in dividing the set of resources and 2 in taking the integer part), we obtain the proof. dRSSPc is an approach to take into account the fact that we need to get a response on the satisfiability. Its interest is 2 If (f ∗ , f ∗ ) are the costs of the two objectives function on the zenith 1 2 point then for any solution of dRSSPc such that the objective functions have the costs (f1 , f2 ), we have f1 ≤ f1∗ and f2 ≤ f2∗

that given an instance, the difference of resources between the candidate algorithms which provide a response on the satisfiability and the first candidate algorithms which stops its execution may be shorter than in the dRSSP case. This observation is also valid when using the M A algorithm. However this later algorithm does not perform well when we have many candidate algorithms. In what follows, we will propose an approach to select a subset of convenient candidate algorithms for Mean allocation.

4. IMPROVING THE MA WITH ALGORITHM SELECTION 4.1. Observations If there are too many candidate algorithms, the resources attributed to each algorithm by M A will be reduced. However, in some cases, it is not necessary to take into account all the different candidate algorithms. The following table illustrates this situation. In this example, we can notice that Table 1. Execution time on three candidate algorithms and 6 resources with B = 100 C(.) a1 a2 a3

I1 100 2 4

I2 100 3 1

I3 4 100 100

each algorithm is the best on the set of the 6 resources on one instance. Thus, using M A, the resolution of the three instances has a cost of 21. However, in taking only 2 of the 3 candidate algorithms (for instance a1 and a2 ) this cost is reduced to 18. Morever, taking only those two algorithms is enough for solving all the instances. Thus, there exists a subset of algorithms that can provide a satisfiability response on the set of all the instances. The interest to have a such set is that when applying M A on it, we benefit from having less algorithms. We present now how to proceed to reduce the set of algorithms.

4.2. The Algorithm Subset Selection Problem Given a benchmark of SAT and a cutoff time, let suppose that on the set of all the resources, there is at least one candidate algorithm which can give a response on the satisfiability of a benchmark instance. To reduce the set of candidate algorithms, we suggest to select a subset such that for all instances, the sum of the execution times of the selected candidate algorithms is minimized and on each instance, at least one algorithm can provide a response on its satisfiability.

The main problem of this selection can then be presented as follows: Algorithm Subset Selection Problem (ASSP ) Instance: A finite set of instances I = {I1 , . . . , In }, a finite set of candidate algorithms A = {a1 , . . . , ak }, a set of m identical resources, a cost R(ai , Ij ) ∈ {0, 1} for each Ij ∈ I, ai ∈ A, k integer values w1 , . . . wk and a real value T. Question: Is there a subset Ac ∈ A such that P k c i=1 {wi |ai ∈ A } ≤ T and for each Ij ∈ A there exc ist aij ∈ A and R(aij , IP j) = 1 ? n In this problem, wi = j=1 C(ai , Ij ), R(ai , Ij ) = 1 if algorithm ai gives a response on the satisfiability of Ij with the cutoff time policy, 0 otherwise.

4.3. Complexity and algorithms Theorem 4.1 ASSP is NP complete Proof First, considering a set Ac , it is easy to check in time O(nk) if ∀Ij , ∃ai ∈ Ac such that R(ai , Ij ) = 1 and Pk c i=1 {wi |ai ∈ A } ≤ T . It is also easy to verify that ASSP can be reduced to the minimum set cover problem. Indeed, being given a universe U = {u1 , . . . , un }, subsets {X1 , . . . , Xk } with Xi ∈ U , costs c1 , . . . , ck such that we are looking for Pk a set cover X c ∈ {X1 , . . . , Xk } with i=1 {ci |Xi ∈ X c } < T s, we build an ASSP problem with the following data: T = T s and wi = ci , 1 ≤ i ≤ n. A = {a1 , . . . , ak } where each ai corresponds to a set Xi ∈ {X1 , . . . , Xk }. I = {I1 , . . . , In } where each Ij corresponds to an element uj of U . R(ai , Ij ) = ( 1 if uj (corresponding to Ij ) belongs to Xi . We then 0 otherwise. solve this problem and if we obtain as solution the set Ac , we take the set X c = {Xi s.t ai ∈ Ac } as a solution to the minimum set cover problem. Let consider an instance of the minimum set Cover problem for which the corresponding ASSP has a solution Ac . One can notice that Pk c c i=1 {ci |Xi ∈ X } ≤ T . The set X is a set cover since if there was uj ∈ U with uj ∈ / Xi , ∀Xi ∈ X c then Ac would not have been a solution for ASSP since there will exist no algorithm ai ∈ Ac such that R(ai , Ij ) = 1. On the other hand, if there exists a set cover X c such that the total cost does not exceed T , then Ac = {ai s.t Xi ∈ X c } would have been a solution for ASSP . The previous reduction shows that ASSP can be taken as a particular description of the set cover problem. Indeed, in taking, the universe as the set of instances and algorithms as a subset of instances defined by the fact that the algorithm solves them in a cost shorter than the cutoff time, we have to solve the minimum set cover problem. It then leads

that any approximation algorithms used for the set cover problem can be used here. We then deduce the following result: Theorem 4.2 There exists a ln(n) + 1 approximation algorithm for the ASSP . Proof The result is straghtforward by using the greedy approximation algorithm provided for the minimum set cover problem in [6].

5. Experimental results To validate our algorithms, we run several experiments using a SAT database (SatEx [9]3 ) which gives the CPU time for all the executions of a set of 23 heuristics (SAT solvers) and a benchmark of 1303 instances for SAT on a sequential machine.

5.1. Brief description of the benchmark The 1303 instances of the SatEx database are issued from many domains where the SAT problem are encountered. Some of them are: Logistic planning, formal verification of microprocessors, Scheduling. Some instances are also issued from many challenging benchmarks for SAT which are used in one of the most popular annual SAT competition 4

5.2. SAT solvers The SatEx database contains three main SAT solvers families. These are: • The DLL family with the following algorithms: asat, csat, eqsatz, nsat, sat − grasp, posit, relsat, sato, sato − 3.2.1, satz, satz − 213, satz − 215, zchaf f • The DP family with : calcres, dr, zres, • The randomized DLL family with : ntab, ntab−back, ntab − back2, relsat − 200 It contains also some other algorithms: heerhugo, modoc, modoc − 2.0.

5.3. Experiments plan To assess our algorithms, we run experiments with the set of solvers and instances given by the SatEx database. We assumed an online stteing where the instances to solve belong to the benchmark (SatEx database). The objective is to observe the mean execution time that we have on instances of the benchmark in computing our different portfolios. In the experiments, we used the asymptotic scheme 3 http://www.lri.fr/˜simon/

satex/satex.php3

4 http://www.satcompetition.org

M AG2 (x) proposed in [4] to solve dRSSP . Given k candidate algorithms and a value x, this scheme produces a k/x approximation of the optimal solution where each candidate algorithm has at least one resource. In our experiments, we run x = 2 and x = 3 and we only present results for x = 2 since the execution costs of built portfolio on instances with x = 3 were very similar. We modified the M AG2 algorithm such as to continue the execution of SAT solvers if stopped ones do not give a response on the satisfiability of the instance being solved. For the BRSA in our experiments, we supposed that each SAT solvers should have at least one resource. Since in this algorithm we have to solve a dRSSP , we used here the M AG2 (2) algorithm. We also modified the U P P defined in Section for the BRSA such as to give at least one resource to each solver. We did two series of experiments that we are described below. • In the first series of experiments (Experiments 1) we assumed to have 100 resources and we set successively the cutoff time to 1000, 5000 and 10000. Given a SAT solver ai and an instance Ij of the SatEx database, let us denote by cpu(ai , Ij ) the execution cost of the algorithm ai applied on instance Ij as indicate in SatEx. For each SAT solver ai , we took its execution cost on 100 resources as equal to cpu(ai , Ij ). If this execution cost was greater than the chosen cutoff value, we set it to this later value. • In the second series (Experiments 2), the number of resources was reduced to 50 and we proceed as in Experiments 1. In all these experiments, we assumed the linear cost assumption. In Figure 1, we present on 100 resources the total execution cost necessary to solve benchmark instances of SAT solvers combined in portfolio. In the total execution cost we do not include those of instances which can not been solved by any SAT solvers. Algorithms executed here are: The Mean allocation algorithm (M A), the Mean Allocation with selection of candidate algorithms following the ASSP approach (M AASSP ), M AG2 and BRSA. Let us notice that BRSA performs better than the others. All the algorithms (M A, M AASSP , M AG2 , BRSA) were executed until they provide a response on the satisfiability of an instance. One can also notice in Figure 1 that making solvers selection improves the quality of the Mean Allocation Algorithm. This is due to the fact that the selection of SAT solvers considerably reduces the number of algorithms in the portfolio. For instance, taking B = 10000 or B = 5000, we just used here 6 SAT solvers instead of 23 for M A. Before using ASSP to select SAT solvers, we noticed that there were some instances that can not been

solved by any SAT solvers. We have eliminated these instances in the selection of a subset of SAT solvers. 1e+06

Total execution Cost

MA MAASSP MAG2 BRSA

100000

10000 1000

5000 Cutoff

10000

Allocation algorithm is improved when selecting a convenient subset of SAT solvers. In table 2, we summarize the number of unsolved instances that we obtained on Experiments 1 (the same numbers were obtained on Experiments 2). We also add the number of unsolved instances of the best individual SAT solvers on the execution time (here the Zchaf f solver). As it can be noticed, combining multiple solvers is interesting to reduce the number of unsolved instances. In Figure 3, we present the execution time obtained by our algorithms. These times are obtained on a an Intel(R) Core(TM)2 Duo CPU 2.26 GHz /2GO. of memory. These results show that the BRSA algorithm has a similar execution time than M AG2 (2). The execution times of these algorithms however are greater than those of the M A algorithm with the selection of a subset of solvers.

Figure 1. Total execution cost on 100 resources Table 2. Number of unsolved instances Algorithms/cutoff MA M AASSP M AG2 BRSA Zchaf f

1e+06

Total execution cost

MA MAASSP MAG2 BRSA

100000

10000 1000

5000 Cutoff

10000

Figure 2. Total execution cost on 50 resources

1000 MA MAASSP MAG2 BRSA

Execution time

100

10

1

0.1 50

100 Total resources number

1000 12 12 12 12 24

5000 11 11 11 11 23

10000 8 8 8 8 23

6. CONCLUSION In this paper we introduced two models to combine efficiently SAT solvers. These models are all based on the usage of a benchmark of instances for SAT. The first model extends the dRSSP introduced in [4] to combine in such a way to take into account the cutoff time. The second model extends a naive solution for combining SAT solvers in portfolio in introducing the selection of a convenient set of SAT solvers. Following the theoretical analysis, some experiments have been conducted to study in detail the adequation of our approaches to the cutoff time. These results are preliminary, we propose a continuation to combine SAT solvers in a more realistic setting without the linear cost assumption. We also propose to consider the possibility to characterize the distribution of incoming online instances to be able to take into account the potential non exhaustivity of the benchmark.

Figure 3. Execution time

REFERENCES Figure 2 reports the total execution time on 50 resources. As in Experiments 1, the BRSA algorithm has the shortest mean execution cost. One can notice also that the Mean

[1] C. Gomes and B. Selman, ”Algorithm portfolios”, Artificial Intelligence Journal , 43-62, 2001.

[2] B. Huberman, R. Lukose and T. Hogg, ”An economics approach to hard computational problems”, Science, 275: 5154, 1997. [3] M. Petrik and S. Zilberstein, ”Learning static parallel portfolios of algorithms”, Annals of Mathematics and Artificial Intelligence, pages 85-106, 2006 [4] Marin Bougeret et al., ”Combining Multiple Heuristics on Discrete Resources”, Accepted for publication in Internationnal Conference on Parallel and distributed Systems, 2009. [5] Allan Borodin and Ran El-Yaniv, ONLINE COMPUTATION AND COMPETITIVE ANALYSIS, Cambridge University Press, 2005. [6] Dorit Hochbaum(Editor), APPROXIMATION ALGORITHMS FOR NP-HARDS PROBLEMS, PWS Publishing Company, Boston, MA, 1997. [7] T. Sayag, S. Fine and Y. Mansour, ”Combining multiple heuristics”, Proceedings of the 23rd International Symposium on Theoretical Aspects of Computer Science, pages 242-253, 2006. [8] M. Streeter, D. Golovin and S. Smith, ”Combining multiple heuristics online”, Proceedings of the 22rd Conference on Artificial Intelligence, pages 1197-1203, 2007. [9] Laurent Simon and and Philippe Chatalic, ”SATEx: a Webbased Framework for SAT Experimentation”, Proceedings of the Workshop on Theory and Applications of Satisfiability Testing (SAT2001), Boston University, Massachusetts, USA, June 14th-15th 2001. Online Publication. [10] M. Gagliolo and J. Schmidhuber, ” Learning dynamic algorithm portfolios”, Annals of Mathematics and Artificial Intelligence, 47(3-4):295-328, 2007. [11] K. Leyton-Brown et al, ”Boosting as a metaphor for algorithm design”, In Proceeding of International Conference on Principles and Practice of Constraint Programming, pages 899-903, 2003. [12] M. G. Lagoudakis and M. L. Littman, ”Learning to select branching rules in the DPLL procedure for satisfiability”, Electronic Notes in Discrete Mathematics, pages 344-359, 2001. [13] Lin Xu et al, ”SATzilla: Portfolio-based Algorithm Selection for SAT”, Journal of Artificial Intelligence Research, pages 565-606, 2008. [14] Harry Markowitz, ”The early history of portfolio theory: 1600-1960”, Financial Analysts Journal, 55 (4), pages 5-16, 1999.