Integrating Artificial Neural Networks and Constraint Logic Programming

2 downloads 0 Views 258KB Size Report
the Tangram puzzles 15] and the N-queens problem 21] with satisfactory results. ..... H (the head) is an atom, ~G (the guard) is a conjunction, possibly empty, of ...
Integrating Arti cial Neural Networks and Constraint Logic Programming J.H.M. Lee and V.W.L. Tam Department of Computer Science The Chinese University of Hong Kong Hong Kong Email: fjlee,[email protected] Phone: +852 609 8426 Fax: +852 603 5024

Abstract

Many real-life problems belong to the class of constraint satisfaction problems (CSP), which are NP-complete, and some NP-hard, in general. When the problem size grows, it becomes dicult to program solutions and to execute the solution in a timely manner. In this paper, we present a novel constraint logic programming language PROCLANN that is expected to handle CSP in the order of hundreds of thousands of variables and constraints. The syntax of PROCLANN is similar to that of Flat GHC. PROCLANN uses the standard goal reduction strategy as frontend to generate constraint and an ecient backend constraint-solver based on arti cial neural network . PROCLANN retains the simple and elegant declarative semantics of constraint logic programming. Its operational semantics is probabilistic in nature but it possesses the soundness and probabilistic completeness results. A novelty of PROCLANN is that while it is a committed-choice language, PROCLANN supports non-determinism, allowing the generation of more than one answer to a query. An initial prototype implementation of PROCLANN is constructed using Prolog and C, with higher eciency than the state of the art implementation in CLP. In addition, the constraint-solver of PROCLANN is amenable to hardware implementation and ecient implementation on massively parallel computers due to the inherent data parallelism in arti cial neural networks. Our benchmarking results supports that PROCLANN has high potential for solving very large-scale industrial scheduling, cutting-stock, temporal and spatial reasoning, etc.

1

1 Motivation and Related Work The problem at hand is that of constraint satisfaction problems (CSP) de ned in the sense of Mackworth [16], which can be stated brie y as follows: We are given a set of variables, a domain of possible values for each variable, and a conjunction of constraints. Each constraint is a relation de ned over a subset of the variables, limiting the combination of values that the variables in this subset can take. The goal is to nd a consistent assignment of values to the variables so that all the constraints are satis ed simultaneously. CSP's occur in all walks of industrial applications, such as scheduling, binpacking, resource allocation, etc. A typical problem consists of variables and constraints in the thousands. Rapid growth of modern industrial sectors and higher degree of automation suggest the appearance of very large-scale CSP, in the order of hundreds of thousand of variables and constraints, in the foreseeable future. There are two concerns in tackling very large-scale CSP's: programming and ecient execution . The former is well addressed by logic programming in Prolog. The relational nature of constraints and the declarativeness of CSP's make Prolog an ideal language to specify CSP. The basic backtracking tree-search execution strategy of Prolog, however, usually produces unacceptable performance on a computer even on medium size problems, as the tree contains too many possibilities to be exhaustively searched. Van Hentenryck [25] introduces consistency techniques [16, 17] into logic programming so that constraints are used actively to prune search space a priori. The framework is realized in the CHIP language [7], which has been successfully applied to solving such industrial applications as car sequencing [4], disjunctive scheduling, graph coloring, and rmware design [5], just to name a few. A full account of its applications can be found in [5, 25, 6, 18]. CHIP's execution mechanism is still based on tree search and backtracking, which are key barriers to the scaling up of the language to tackle very largescale CSPs. The eciency concern remains. A key component of a CLP system is the constraint-solver. For various constraint domains, many classical constraint-solving techniques exist but the techniques are all algorithmic in nature and cannot escape from the curse of NP-completeness. Rabin [19] was one of the rst to recognize the need and propose randomized or probabilistic algorithms for concrete algorithmic problems in computer science although the idea can be traced back to Monte 2

Carlo methods [2, 10]. Most arti cial neural network (ANN) models fall into the probabilistic algorithm camp. There has been recent interest in applying ANN to optimization and CSP. For examples, Hop eld network, the Boltzmann Machine and the elastic network have been used in the traveling salesman problems [12, 1, 8], the Tangram puzzles [15] and the N-queens problem [21] with satisfactory results. Wang and Tsang [26, 22] proposed GENET, a generic ANN model, for solving general CSP's with binary constraints. A massively parallel implementation of GENET may attain a theoretical speedup in the order of 108 over sequential heuristic search. Wang and Tsang [27] also proposed a cascadable VLSI design for GENET. ANN, while ecient, is dicult to program. Translating a CSP into a neural network is often a tedious and error-prone task. This is where logic programming can help. We propose to integrate constraint logic programming and ANN. A CSP is speci ed as a constraint logic program. The logic programming part of the execution mechanism generates the corresponding neural work of the CSP, which is then submitted to the backend ANN-based constraint-solver for further scrutiny. This way we obtain the best of both worlds: an easy to program language with high eciency. There has been other attempts [9, 20], with di erent motivations, in the amalgamation of logic programming and ANN. These approaches are all translational, in which a set of logical formula is translated into a neural net and theorem proving becomes an energy minimization process. Our proposal represents a radically di erent approach, which aims at tight coupling of logical deduction and neural network. We propose a new logical inference system which is based on ANN while the usual logic programming semantics is retained. The paper is organized as follows. Section 2 reviews the GENET model and shows how GENET can be improved and adapted to incremental execution. The syntax and semantics of the PROCLANN language is presented in section 3. Soundness and probabilistic completeness results are shown and the notion of probabilistics non-determinism is introduced. In section 4, we describe the architecture of the PROCLANN prototype implementation and its comparison to CHIP using the N-Queens problems for benchmarking. We summarize our contributions and shed light on future work in section 5.

3

2 An ANN-Based Constraint-Solver In this section, we describe a CSP solver that is based on ANN. The language framework that we shall present later is not tied to any particular ANN model but the chosen model has to satisfy the following criteria: 1. (Network uniqueness) Every CSP (or constraint network) can be translated into a unique network topology in the model. 2. (Probabilistic Completeness) If a CSP has solution , then there exists non-zero probability that the corresponding neural network has  as answer. We say that the neural network model is probabilistically complete . 3. (Incrementality) The model must be amenable to ecient incremental execution. Any ANN model that satis es the above properties can be used in our framework. We have chosen the GENET model [26, 22] to demonstrate the feasibility of our proposal. In the following, we give a brief introduction to GENET and show how we adapt the execution model of GENET to an incremental version. Our statistics shows that the incremental version is at least as ecient as the \batch" version. At appropriate places, we explain why GENET satis es the three criteria.

2.1 GENET

GENET [26, 22] is a generic neural network simulator that can be used to solve general CSPs with nite domains. It generates a sparsely connected network for a given CSP with constraints according to the following ways :  each domain variable in the CSP is represented by a collection (i.e. a cluster) of nodes;  each node i is used to represent a value in the domain and has two attributes: state Si and input Ii ;  the state Si is either 1 for on or 0 for o ;  connections among the nodes are constructed according to the constraints with only inhibitory connection between incompatible nodes, the weight of the connection between nodes i and j is denoted by Wij which is always a negative integer and initially given the value -1; 4

 the input to each node i is a weighted P sum of all its connected nodes' states, i.e. input to node Ii = kj ;j 6 i Wij  Sj . =1 =

It is easy to check from this de nition that GENET satis es the network uniqueness requirement. To illustrate how a network is constructed for a CSP, let us take a simple but tight binary CSP as example. Assume there are ve nite domain variables Z1 to Z5 , all with domain d = f1; 2; 3g. Let Vi denotes the value assigned to Zi . There are two constraints: (1) Vi + Vi+1 is even and (2) either V1 = 2 or V5 = 2. Figure 1 (a) is the constraint network of the example as

Z1 even(Z1 ,Z2 ) f1; 2; 3g Z1 = 2 or Z5 = 2 Z5 f1; 2; 3g even(Z4 ,Z5 )

f1; 2; 3g Z

2

even(Z2 ,Z3 )

f1; 2; 3g Z

3

Z4 f1; 2; 3g

-1 Cluster 1 1 -1 1 -1 1 -1 1 -1 2 3

-1

2 3

-1

2

-1

3

3 -1

even(Z3 ,Z4 )

(a) Constraint Network of a tight CSP

-1

2 3 -1

(b) Network Structure of the CSP

Figure 1: The constraint network and the GENET network of a tight CSP. de ned in the sense of Mackworth [16]. Figure 1 (b) is the corresponding network topology in GENET. Each column of 3 nodes represents a cluster for each domain variable. And each connection is initially set to -1. The cross inhibitory connections are constructed according to constraint 1. For example, when V1 = 2 and V2 = 1, their sum is 3 (odd) which violates constraint 1. So, the second node in rst column and rst node in second column is connected up. For constraint 2, consider when V1 = 1, V5 should then be 2. In order to ful ll this requirement, node 1 of rst cluster should 5

2

-1

send inhibitory e ect to the rst and third node of Z5 . The other two connections are constructed similarly.

2.1.1 Dynamics of GENET At any moment, only one node in a cluster is on . We say that the variable as represented by the cluster is assigned the value as represented by the label of the on-node. For the purpose of our discussion, we de ne a network state to be an assignment of values to each of the clusters (variables). A solution state is a consistent assignment of values to the variables so that none of the constraints are violated. Dynamics of a neural network concerns how the network changes states and/or connection weights and/or other system parameters before it reaches the solution state(s). Dynamics of GENET can be realized in its network convergence procedure as follows : 1. Initially one node in each cluster is randomly selected to be on. 2. Every node then calculates its input. 3. In each cluster, node with the maximum input will be turned on. To avoid chaotic or cyclic wandering of network states, if there are several nodes with maximum inputs, node that is already on will remain on. Otherwise, GENET will just randomly select one out of them. This step is called the state update rule . 4. When the network settles in a stable state (i.e. with all on-nodes unchanged), GENET checks to see if all the on-nodes have zero input. If so, the state represent a solution to the CSP and the simulator stops with success . Otherwise, the state represents a local maximum . 5. If the network settles in a local maximum of cluster input(s), a heuristic learning rule will update the all connection weights by Wij := Wij ? Si  Sj : 6. Goto step 2. We can understand the network convergence procedure from an energy maximization perspective. Let N (S ) be the set of on-nodes of state S . We de ne the energy of state S by X I = X X W S ; E (S ) = ij j i i2N (S )

i2N (S ) j 2Ci

6

where Ci is the set of all nodes connected to node i. Equivalently the energy of a state is the sum of on-nodes' inputs. The energy is a non-positive quantity since (1) the initial weights are negative and (2) the heuristic learning rule only decreases the weights. In the state convergence step, the state update rule chooses a node with maximum input from within a cluster. In general the energy of the new state is higher than that of the old one. At worst, the energy remains the same. The maximum possible energy is zero, which happens only when all on-nodes' inputs are zero. That means that no two on-nodes are connected to each other. Since only incompatible nodes are connected, a zero-energy state represents a solution. Therefore, GENET works by maximizing the energy of the network. A network may be trapped in a local maxima, whose energy is less than zero. In this case, the heuristic learning rule will be applied to alter the weight of the arcs. The net e ect of the alteration is that the energy of the current state is lowered with respect to its neighbouring state, thereby destroying the current state's local maximality. We state without proof the following theorem, which demonstrates that GENET is probabilistically complete. The proof of theorem 2.1 can be found in [3]. Theorem 2.1: Assume that the randomizer is fair. If  is a solution of a CSP P , then there exists non-zero probability that GENET will settle in a solution state that corresponds to .

2.2 Incremental GENET

The original GENET operates in the \batch" mode: the entire network must be constructed before computation starts. In CLP systems, however, adding new constraint to an existing solvable collection is a primitive and frequent step in the operation. Therefore, an ecient incremental version of GENET is necessary. A naive but ecient incremental GENET is as follows. initialize the network to the empty network while there are more increments of constraints to add add the next increment to the network by creating nodes and arcs execute GENET

endwhile

Incrementality originates from the re-use of the weights of the arcs, which was computed using the heuristic learning rule in previous cycle. In other 7

words, the network is trained while it is being built incrementally. Before we construct incremental GENET, we have in fact modi ed the way node inputs are calculated. This modi cation leads to improvement in GENET's performance. We then compare the original GENET, the improved GENET, and the incremental GENET on the N-Queens problem in gure 2 1200 1000 800

Original GENET 3 New GENET + Incremental GENET 2

3

Time 600 in sec. 400 200 + 03 2 10

+ 3 2 20

+ 3 2

30

3 + 2

40

3 + 2

3 + 2

3 + 2

50 60 70 No. of Queens

3 + 2 80

3 + 2

+ 2

90

100

Figure 2: Comparison of GENET, Improved GENET, and Incremental GENET. The graphs were generated by executing the three versions of GENET on 10- to 100-Queens on a SUN SPARCstation 10 model 30. For each test case, the measurement recorded is an average over ten runs. As expected, the improved GENET represents a speedup over GENET. The result also shows that incremental GENET is at least as ecient, if not slightly better, than the improved GENET.

3 The PROCLANN Language

In this section, we describe the PROCLANN (PROgramming in Constraint Logic with Arti cial Neural Network) language. PROCLANN is a committed8

choice language with a syntax similar to that of Flat GHC [23, 24]. This design decision is justi able by the fact that goal reduction is used only to inject constraints into a neural network but not to perform searching. The operational semantics of PROCLANN is similar to that of the CLP scheme [13, 14] but the use of an ANN-based constraint-solver is made explicit in the PROCLANN Computation Model , which is both sound and complete. The probabilistic nature of PROCLANN leads to the notion of probabilistic completeness . One distinguished feature of PROCLANN is that while PROCLANN is a committed-choice language, it is non-deterministic so that multiple answers to the same query can be generated; hence the name probabilistic non-determinism .

3.1 Syntax and Declarative Semantics

Since we are interested in CSP, we need the notion of domains and domain variables [25], abbreviated as d-variable . A domain is a nite set of constants. Each d-variable has the form X d , where d is the associated domain of X d . To di erentiate d-variables from Herbrand variables, we call the latter h-variables . The constraint domain D in consideration de nes a set C of primitive constraints . PROCLANN only supports binary constraints. As speci ed in the CLP scheme [13, 14], the equality constraint #=1 is included in C . An atom is any standard atomic formula whose predicate symbol is di erent from those of the primitive constraints. A clause in PROCLANN has the form ~ B1 ; : : : ; Bn : (n  0) H :- G~ | C; where 1. H (the head ) is an atom, G~ (the guard ) is a conjunction, possibly empty, of system predicates, C~ (the constraints ) is a conjunction, possibly empty, of atomic constraints, and B1 ; : : : ; Bn (the body ) is a conjunction of atoms; 2. G~ contains no d-variables; 3. C~ contains only d-variables and ground terms; The \:-" symbol is read as if , and the commas and the \|" symbol are read as conjunction. If a clause has no guard, the \|" can be omitted. Thus 1

All PROCLANN constraints are preceded by the # symbol.

9

a clause has the same declarative reading as a Horn clause with domain variables [25]. A program is a set of clauses. A query in PROCLANN is simply a clause without the head and the guard. The concepts of truth, model, and logical consequences of PROCLANN are equivalent to those of CHIP [25]. We conclude this subsection by a PROCLANN program for nqueen(N,L) :- length(L,N), L in 1..N, constraint(L). length([],0). length([H|T],N) :- N > 0 | N1 is N-1, length(T,N1). constraint([]). constraint([X|Y]) :- safe(X,Y,1), constraint(Y). safe(X,[],Nb). safe(X,[H|T],Nb) :- noattack(X,H,Nb), Newnb is Nb + 1, safe(X,T,Newnb). noattack(X,Y,Nb) :- X #\= Y, X #\= Y+Nb, Y #\= X+Nb.

Figure 3: A PROCLANN program for the N-Queens problem. the N-Queens problem, as shown in gure 3. The built-in predicate in/2 declares that each element of L is a list of d-variables with domain f1; : : : ; ng. The \#\=" predicate stands for the disequality constraint. The program is almost identical to the CHIP version, except that there is no need to perform explicit labelling operation using the CHIP indomain/1 predicate.

3.2 Operational Semantics

The introduction of d-variables prompts for modi cation of the uni cation algorithm. Similar to CHIP [25], we have three cases to consider in the new de nition. 1. If a h-variable X is uni ed with a d-variable Y e , uni cation succeeds and the binding fX=Y e g is generated. 10

2. If two d-variables X d and Y e are uni ed, uni cation succeeds and the constraint X d #= Y e is generated. No binding is generated. 3. If a d-variable X d is uni ed with a constant c, uni cation succeeds and the constraint X d #= Y fcg is generated. Again no binding is generated. Therefore, the output of the uni cation is a set of equality constraints C~u and a set  of bindings of the form X=t, in which X is a h-variable. We de ne the PROCLANN Computation Model in the following. Let P be a PROCLANN program and Qi a query. Qi+1 is derived from Qi = ~ B1 ; : : : ; Bj ; : : : ; Bq , where C~ is a conjunction of primitive constraints, if C; the following conditions are satis ed: 1. Bj is the selected atom in Qi ;

2. there exists a clause \H :- G~ | C~ 0 ; A1 ; : : : ; An :" (n  0) in P such that (a) H is uni able with Bj generating substitution i and constraint set C~u, and (b) the execution of the guard G~ succeeds; 3. the corresponding neural network of C~00 = C~ [ C~u [ C~ 0 converges to a solution state in nite amount of time; 4. Qi+1 = C~00 ; B1 ; : : : ; Bj ?1 ; A1 ; : : : ; An ; Bj +1 ; : : : ; Bq . A derivation is a, possibly in nite, sequence of queries Q = Q0 ; Q1 ; Q2 ; : : : such that Qi+1 is derived from Qi generating substitution i+1 . A derivation is successful if it is nite with m steps and the last query contains no atoms. In this case, the constraint accumulated in the last goal is called the answer constraint and the associated neural network is called the answer network . A computed answer of a successful derivation is the union of a solution state N of the answer network N and the composition of substitutions  = 1    m restricted to the h-variables in Q. There is no notion of failed derivations in PROCLANN.

3.3 Soundness and Completeness of the PROCLANN Computation Model

The PROCLANN computation model is very similar to the operational semantics of the CLP scheme, except that the use of neural network model is 11

made explicit. It follows that the PROCLANN computation model is sound . The completeness result requires the notion of probabilistic completeness . In the following, we state without proof the soundness and completeness of the PROCLANN computation model. Theorem 3.1: (Soundness of PROCLANN Computation Model) Let P be a PROCLANN program. If a query Q has a successful derivation with computed answer , then P j= 8(Q); where 8(F ) means the universal closure of the formula F .

Theorem 3.2: (Probabilistic Completeness of PROCLANN Computation Model) Let P be a PROCLANN program, Q a query,  = d [ h a substitution, where d contains only bindings for d-variables and h contains only those for h-variables. If P j= 8(Q), then there exist a successful derivation for Q with answer network N such that  there is a non-zero probability that N will converge into the solution state d , and  if h0 is the computed answer of the successful derivation restricted to h-variables, then there exist substitution  such that h = h0 . Theorem 3.2 reviews the probabilistic nature of PROCLANN to the fullest extent. It means that if the CSP speci ed by the program is solvable, then there always exists a successful derivation and the answer net obtained has non-zero probability to converge to any solution of the CSP.

3.4 Probabilistic Non-determinism

PROCLANN is a committed-choice language but yet it supports both don't care and don't-know non-determinisms, allowing the generation of multiple answers to a query. A PROCLANN computation can be divided into two parts: network generation and solution generation. Don't-care nondeterminism is exhibited in the rst part, in which the program is used as a network generator using goal reduction on guarded clauses. The neural network, while being built up incrementally, is also subject to network training. A successful derivation leads to an answer network. Don't-know non-determinism originates from the second part of the execution, in which 12

the answer network is further executed using the network convergence procedure. As in the case of Prolog, PROCLANN users can request for another answer after obtaining the rst one, and so on. Upon request, PROCLANN only has to re-start the second part of the execution. We shall show in the next section that the solution generation part is relatively inexpensive with respect to the overall cost of execution. The implication is that we can generate subsequent answers at low cost. In conventional logic programming languages, the cost of getting the subsequent answers grows exponentially in general. Last but not least, answers are produced randomly; hence the name probabilistic non-determinism .

4 Prototype Implementation To demonstrate the feasibility of our proposal, we have built a prototype of PROCLANN using Quintus Prolog Release 3.1 and C. We compare our implementation with Cosytec CHIP version 4.0.1 using the N-Queens problem again. All benchmarking was performed on a SUN SPARCstation 10 model 30.

4.1 Architecture

Incremental GENET (I-GENET) is implemented as a Quintus Prolog foreign C object with two interface functions: addConstraints(L) and addDomain(X,D), where L is a list of constraints in textual form, X is a domain variable with domain D represented as list. The addDomain/2 predicate instructs I-GENET to add the d-variable X D into the network by creating a cluster of jDj nodes. The goal reduction part of PROCLANN is performed naturally by Prolog. Constraints are passed to I-GENET in a derivation step via the addConstraints/1 predicate. The following is the revised syntax of the noattack/3 predicate in our prototype language: noattack(X,Y,Nb) :- addConstraints( [X #\= Y, X #\= Y+Nb, Y #\= X+Nb] ).

There is no Prolog coding involved. The foreign C object consists of around 1000 lines of C code.

13

4.2 Benchmarking and Comparison with CHIP

The N-Queens problem is suitable benchmark for serious computer programs since the number of queens can be increased without limit. The PROCLANN N-Queens program is shown in gure 3. The CHIP counterpart is adopted from the one that comes with the Cosytec package. Note that the rst-fail principle [11] is used in labeling in the CHIP program. PROCLANN execution can be divided into two parts: network building and solution nding by ANN. The former consists of creation of clusters of nodes (when the in/2 predicate is encountered) and making inhibitory connections between nodes (during a derivation step). The latter is the time that GENET takes to converge into a solution state. Figure 4 shows the breakup of the total processing time into these two components. The 7000 Total processing time 3 3 6000 Network building time + 3 + + 5000 Solution nding time 2 3 4000 + Time in sec.3000 3 + 2000 3 + 3 + 2 1000 + 3 2 2 + 3 + 3 2 + 2 + + + + + 2 2 2 2 2 0 3 2 3 2 3 2 3 2 3 2 3 0 20 40 60 80 100 120 140 160 No. of Queens Figure 4: Breakdown of PROCLANN Execution Time. time unit is in second. Due to the probabilistic nature of PROCLANN, the recorded time for PROCLANN represents average of ten runs. In making inhibitory connections, a form of consistency checking is performed. Thus PROCLANN spends considerable time in building the network. The time for exercising the network convergence procedure is small. Therefore, it is relatively inexpensive for PROCLANN to obtain subsequent answers after 14

the rst one. Figure 5 re ects a naive comparison between PROCLANN and CHIP. The recorded time for CHIP is the time taken to obtain the rst solution. As can be seen from the graph, the PROCLANN time grows with the number of queens as expected. The curve for CHIP exhibits irregularity, with a sudden peak at 90-Queens. At 149- and 150-Queens, CHIP did not return in 24 hours and we aborted the execution. PROCLANN works at 149Queens but 150-Queens exceeds the capacity of the Prolog engine part of PROCLANN. The trend of the CHIP time is shown clearer in gure 6, 9000 8000 7000 6000 5000 Time in sec.4000 3000 2000 1000 0

+

PROCLANN 3 CHIP +

3 3

0

3

3

3

3 3 3 + + + + + + 3 + + + 3 + 3 + 3 + 3 + 3 3 20

40

60 80 100 No. of Queens

120

140

160

Figure 5: A First Comparison between PROCLANN and CHIP. which has the 90-Queens anomaly removed. We explain the irregularity in the CHIP time as follows. Similar to Prolog, CHIP employs a depth rst search strategy to explore the derivation tree of a query. The order of branches in a derivation depends on the order of clauses in the program. If the rst solution is more to the \left" of the associated tree, then CHIP can locate the solution in less time. A fairer comparison is for CHIP to compute the \median" solution out of the entire solution space of a particular N-Queens problem. We are not aware of any general formula to compute the number of solutions of an N-Queens 15

7000 6000 5000 4000 Time in sec.3000 2000 1000 0

PROCLANN 3 CHIP +

3

0

3 + 3 + + + 3 + 3 + 3 + 3 + 3 3 20

40

3

3

3

3

3

+ + + + + 60 80 100 120 140 No. of Queens

160

Figure 6: Ignoring the 90-Queens Anomaly. problem. A good approximation is to use half of the CHIP time to nd all solutions to an N-Queens problem using the built-in bagof/3 predicate. Since the search space of the N-Queens problem grows exponentially with N , we can only perform benchmarking between 4- to 12-Queens for CHIP within a reasonable response time. The result is shown in gure 7.

5 Concluding Remarks The contribution of this paper is ve-fold. First, we identify that arti cial neural network models can be used as backend constraint-solver of a CLP system. Any ANN model that satis es the network uniqueness, the probabilistic completeness, and the incrementality criteria can be used for the purpose. We study the GENET model in particular and show how it can be adapted to incremental execution without sacri cing eciency. Second, we de ne the PROCLANN language and its computation model. Since PROCLANN is not tied to any speci c ANN model, it can be regarded as a general theoretical framework for the integration of constraint logic programming and arti cial neural network. Third, PROCLANN retains important 16

60

+

50

PROCLANN 3 CHIP +

40 Time 30 in sec. 20 10

+

+ 3 3 3 3 3 3 3 3 3 3 + 3 + 3 + 3 + 3 + 3 + 3 03 4 6 8 10 12 14 16 18 20 No. of Queens

Figure 7: An Alternative Comparison between PROCLANN and CHIP. semantic properties of logic programming, namely soundness and completeness. The latter is probabilistic. Fourth, we introduce non-determinism into a committed-choice language. This is a novel feature of PROCLANN. Fifth, a prototype implementation of PROCLANN is built and its eciency compares favourably over a commercial CHIP implementation. The prototype was constructed in only two man-months. There are still many optimization possibilities. PROCLANN is also amenable to massively parallel implementation, which will yield a higher degree of speedup. There are a few directions for future work. First, the inherent data parallelism in GENET suggests implementation of PROCLANN on massively parallel SIMD computers. The idea is to execute goal reduction at the frontend of the SIMD machine. Constraints generated will be passed to the incremental GENET running in the massively parallel backend. Second, the GENET model supports only binary constraints. An ad hoc non-binary example for the car-sequencing problem is presented in [26]. It should be interesting to check if the model can be generalize for non-binary constraints. Third, we use the GENET model only as a case study. Other ANN models should be investigated. Fourth, the PROCLANN computation model is 17

independent of the constraint domain although the PROCLANN language is tailored to nite domain constraint-solving. The suitability of other constraint domains for PROCLANN are topics of further studies.

References [1] E.H.L. Aarts and J.H.M. Korst. Boltzmann machines for traveling salesman problems. European Journal of Operational Research, 39:79{ 95, 1989. [2] K. Binder and D.W. Heermann. Monte Carlo Simulation in Statistical Physics. Springer-Verlag, 1988. [3] L.W. Chan and J.H.M. Lee. On the dynamics of GENET. Technical report, Department of Computer Science, The Chinese University of Hong Kong, 1993. (in preparation). [4] M. Dincbas, H. Simonis, and P. Van Hentenryck. Solving the carsequencing problem in constraint logic programming. In Proceedings of the European Conference on Arti cial Intelligence, pages 290{295, 1988. [5] M. Dincbas, H. Simonis, and P. Van Hentenryck. Solving large combinatorial problems in logic programming. Journal of Logic Programming, 8:75{93, 1990. [6] M. Dincbas, P. Van Hentenryck, H. Simonis, A. Aggoun, and T. Graf. Applications of CHIP to industrial and engineering problems. In Proceedings of the 1st International Conference on Industrial & Engineering Applications of Arti cial Intelligence & Expert Systems, pages 887{ 892, 1988. [7] M. Dincbas, P. Van Hentenryck, H. Simonis, A. Aggoun, T. Graf, and F. Berthier. The constraint logic programming language CHIP. In Proceedings of the International Conference on Fifth Generation Computer Systems (FGCS'88), pages 693{702, Tokyo, Japan, December 1988. [8] R. Durbin and D. Willshaw. An analogue approach to the traveling salesman problem using an elastic net method. Nature, 326:689{691, 1987. 18

[9] P. Eklund and F. Klawonn. Neural fuzzy logic programming. IEEE Transactions on Neural Networks, 3(5):815{818, September 1992. [10] J. Hammersley and D.C. Handscomb. Monte Carlo Methods. Chapman and Hall, Inc., 1964. [11] R.M. Haralick and G.L. Elliot. Increasing tree search eciency for constraint satisfaction problems. Arti cial Intelligence, 14:263{313, 1980. [12] J.J. Hop eld and D. Tank. \Neural" computation of decisions in optimization problems. Biological Cybernetics, 52:141{152, 1985. [13] J. Ja ar and J-L. Lassez. Constraint logic programming. Technical Report 72, Department of Computer Science, Monash University, Clayton, Victoria, Australia, 1986. [14] J. Ja ar and J-L. Lassez. Constraint logic programming. In Proceedings of the 14th ACM POPL Conference, pages 111{119, Munich, January 1987. [15] O azer K. Solving tangram puzzles : A connectionist approach. International Journal of Neural Systems, 8(5):603{616, 1993. [16] A.K. Mackworth. Consistency in networks of relations. AI Journal, 8(1):99{118, 1977. [17] B.A. Nadel. Constraint satisfaction algorithms. Computational Intelligence, 5:188{224, 1984. [18] M. Perrett. Using constraint logic programming techniques in container port planning. ICL Technical Journal, 7(3):537{545, May 1991. [19] M.O. Rabin. Probabilistic algorithms. In J.F. Traub, editor, Algorithms and Complexity, Recent Results and New Directions, pages 21{39. Academic Press, 1976. [20] T.J. Reynolds, H.H. Teh, and B.T. Low. Neural logic programming. In Proceedings of the 2nd International IEEE Conference on Tools for A.I., pages 485{491, 1990. [21] O. Shagrir. A neural net with self-inhibiting units for the n-queens problem. International Journal of Neural Systems, 8(3):249{252, 1993. 19

[22] E.P.K. Tsang and C.J. Wang. A generic neural network approach for constraint satisfaction problems. In G Taylor, editor, Neural Network Applications, pages 12{22. Springer-Verlag, 1992. [23] K. Ueda. Guarded Horn Clause. PhD thesis, University of Tokyo, Tokyo, Japan, 1986. [24] K. Ueda. Guarded horn clause: A parallel logic programming language with the concept of a guard. Technical Report TR-208, Institute of New Generation Computer Technology, Tokyo, Japan, February 1987. [25] P. Van Hentenryck. Constraint Satisfaction in Logic Programming. The MIT Press, 1989. [26] C.J. Wang and E.P.K. Tsang. Solving constraint satisfaction problems using neural networks. In Proceedings of the IEE 2nd Conference on Arti cial Neural Networks, pages 295{299, 1991. [27] C.J. Wang and E.P.K. Tsang. A cascadable VLSI design for GENET. In Proceedings of the Oxford VLSI Workshop, 1992.

20