Logic Program Synthesis as Problem Reduction Using ... - CiteSeerX

Automated Software Engineering, 8, 167–193, 2001 c 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. °

Logic Program Synthesis as Problem Reduction Using Combining Forms ANDREAS HAMFELT [email protected] Department of Information Science, Division of Computer Science, Uppsala University, Box 513, S-751 20 Uppsala, Sweden JØRGEN FISCHER NILSSON [email protected] NIKOLAJ OLDAGER [email protected] Department of Information Technology, Technical University of Denmark, DK-2800 Lyngby, Denmark

Abstract. This paper presents an approach to inductive synthesis of logic programs from examples using problem decomposition and problem reduction principles. This is in contrast to the prevailing logic program induction paradigm, which relies on generalization of programs from examples. The problem reduction is accomplished as a constrained top-down search process, which eventually is to reach trivial problems. Our induction scheme applies a distinguished logic programming language in which programs are combined from elementary predicates by means of combinators conceived of as problem reduction operators including list recursion forms. The operator form admits inductive synthesis as a top-down piecewise composition of semantically meaningful program elements according to the compositional semantics principle and with appeals neither to special generalization mechanisms nor to alternative forms of resolution and unification, or predicate invention. The search space is reduced by subjecting the induction process to various constraints concerning syntactical form, modes, data types, and computational resources. This is illustrated in the paper with well-modedness constraints with the aim of synthesising well-moded, procedurally acceptable programs. Preliminary experiments with the proposed induction method lead us to tentatively conclude that the presented approach forms a viable alternative to the prevailing inductive logic programming methods applying generalization from examples. Keywords: predicate combinators, recursion operators, synthesis by composition and specialization, inductive synthesis, program schemata

It is important to observe from the very beginning that the logic of all known attempts at a so-called logic of induction is exactly the same as the logic of that process of thought which is called deductive reasoning [. . . ]. G.H. v. Wright: The Logical Problem of Induction, Oxford, 1941, 1957. 1.

Introduction

Inductive Logic Programming (ILP) is concerned with the construction of logic programs from sample program results through a formal generalization procedure (Lavrac and Dzeroski, 1994; Muggleton, 1992). ILP appeals to the logical view on induction according to which induction consists in generalizing specific facts to universally quantified sentences according to formal principles originating in Plotkin (1969). Thus the notion of

168

HAMFELT ET AL.

generalization is central to induction and therefore also to ILP. This has given rise to antiunification concepts, which seek the most specific generalization of terms, in contrast to the most general specification in ordinary unification. It has also more recently spurred the development of special inverse resolution procedures (Muggleton, 1996; Nienhuys-Cheng and de Wolf, 1997) and particular predicate invention mechanisms, see Flener and Yilmaz (1999) for a recent comprehensive survey. The logical approach to induction has proved useful in machine learning but seems still to be problematic for inducing logic programs with recursive data types such as lists. Induction of logic programs with recursive data types is notoriously difficult because generalization from examples tends to lead to overly general and non-terminating recursion forms. Our approach to this problem is—in the vein of schema-guided approach to induction—to propose a set of fixed recursion forms intended to cover common recursion structures applied in logic programming with lists. This paper focusses on induction of pure logic programs for list processing at the level of basic text book examples. In order to apply these recursion forms this paper as a follow up on Hamfelt and Fischer Nilsson (1999a, b) advances a novel approach to induction of logic programs called COMBINDUCE, which neither appeals to special generalization mechanisms nor to alternative forms of resolution. Our approach to induction of logic programs rests on a problem reduction conception in which the program is to be constructed using appropriate combining forms conceived of as problem reduction operators including list recursion forms. The problem reduction is accomplished as a constrained top-down search process, which eventually is to reach trivial problems. This approach is thus distinguished by absence of generalization mechanisms and in its ideal form also of explicit predicate invention. In our earlier proposal METAINDUCE Hamfelt and Fischer Nilsson (1994), induction of recursive programs was addressed by suggesting that the program structure to be induced has to comply with given recursion schemes of universal nature. The induction process then essentially reduces to synthesis of auxiliary predicates applied as parameters or arguments to the recursion schemes. The auxiliary predicates in simpler cases would be defined by nonrecursive clauses formed by a least general generalization procedure from the examples. In more complex cases the whole induction procedure may be invoked recursively to invent a new recursively defined auxiliary predicate. The present approach, COMBINDUCE, also draws on recursion schemes but differs from METAINDUCE in applying a novel variable-free combinatory form of logic programs, COMBILOG, introduced in seminal form in Hamfelt and Fischer Nilsson (1994, 1997) and being fully developed in Hamfelt and Nilsson (1999a), Hamfelt et al. (submitted). A key tenet in COMBINDUCE is the superiority of the variable-free operator form of programs for ILP in which recursion schemes are added as “super”-operators. Moreover, a key aspect of the inductive synthesis system presented herein is adoption of well-modedness constraints which drastically reduce the search space of candidate programs. In addition this constraint makes it possible to discard negative examples. The paper is organized as follows. Section 2 reviews the fundamentals of logic programs, Section 3 describes the combinatory form of logic programs. Section 4 presents a metalogic program interpreter for combinatory logic programs. This interpreter forms the basis for the induction of programs introduced in Section 5. Sections 6 and 7 explain how to impose

LOGIC PROGRAM SYNTHESIS

169

constraints on the synthesis in the form of well-modedness conditions. Finally Section 8 summarizes the capabilities of the described approach and compares with some well known contemporary approaches to inductive logic programming. 2.

Logic program predicates and relations

We consider here pure logic programs in the form of definite clauses ¡ ¢ ¡ ¢ ¡ ¢ p t01 , . . . , t0n 0 ← q1 t11 , . . . , t1n 1 ∧ · · · ∧ qβ tβ1 , . . . , tβn β devoid of extra-logical and non-monotonic features such as “negation as failure” and “cut”. The collection of γ defining definite clauses for a predicate p can be put in the form p( XE ) ←

γ ^ βi _ i

qi j (tEi j )

(1)

j

where XE is a tuple of distinct variables and tEi j are tuple terms conforming with the arity of the predicates qi j . The elimination of left hand side non-variable terms proceeds by appealing to a distinguished clause id(X, X ) through rewritings (cf. e.g. Petorossi and Proietti, 1998) which do not affect the declarative semantics. This rewriting is exemplified below. Without essential loss of generality it is assumed that compound terms are list terms so that the Herbrand universe is generated by the available constants and the list constructor. Moreover, goal clauses are assumed to be of the form ← p( XE ). A logic program serves to specify and compute (in part or in full) the relational denotations of the predicates introduced in the program. Consider a logic program with Herbrand universe H constituted by the ground terms which can be formed by the available constant and function symbols. In this program the relational extension of an n-ary predicate is an n-ary relation comprising a subset of the n-ary universal relation {ht1 , . . . , tn i | ti ∈ H } being the n-fold Cartesian product H n of the Herbrand universe H . For a logic program P consisting of definite clauses we introduce the notation [[ p]]nP for the relational extension of an n-ary predicate p. In logic programming theory this extension is defined via the notion of logical consequence as: [[ p]]nP = {ht1 , . . . , tn i ∈ H n | P |= p(t1 , . . . , tn )}. It is a fundamental and well-known result in logic program theory that this relation coincides with the corresponding set of ground atomic formulas p(t1 , . . . , tn ) which can be proved by means of SLD-resolution (Apt and Marchiori 1994). When there is no source of confusion we omit the index P in the expression [[ p]]nP . Likewise n is also often omitted. 3.

Compositional logic programming

In the tradition of compositional (“structured”) programming and stepwise program development programs are composed from specifications in a top-down problem decomposition

170

HAMFELT ET AL.

manner, according to which a larger program is obtained by systematically piecing together smaller programs by means of combining forms. In functional programming a combining form such as, say, functional composition is naturally understood as a higher order function taking functions as arguments and yielding a function as result, cf. e.g. Reade (1989). Similarly, in the context of logic and logic programming new predicates (model-theoretically conceived of as relations or truth-valued functions) can be formed by composition of basic predicates or program defined predicates by means of combining forms functioning as higher-order predicates. In a recent paper (Hamfelt et al., submitted) we pursue the compositional principle in the context of logic programming by proposing a variable-free combinator form of definite clauses termed COMBILOG. This means that in COMBILOG clauses of the form (1) are replaced by combinatorial defining clauses p←ϕ where ϕ is a ground combinatory predicate expression of the form hctermi ::= hpidi | hcombi(hctermi{, hctermi}∗ ) where h pidi is a predicate identifier and hcombi is one of the predicate operators to be introduced below. 3.1.

Elementary relations

In the approach to logic programming based on combining forms a key issue is the identification of an appropriate set of elementary relations analogously to identification of elementary functions in functional programming. By contrast, in traditional logic programming based on definite clauses no set of such elementary relations is identified since clauses offer rather unconstrained means of building formulas utilizing terms. Since we adhere to the data type of lists in the present context, in COMBILOG we introduce the following set of elementary predicates with accompanying relational extensions as shown in Table 1. It is assumed that the Herbrand universe H is denumerably infinite by presence among the constants of the empty list together with free list formation. Table 1.

Elementary predicates. Name

Arity

Extension relation

Cardinality

Clauses

true

0

{hi}

1

true()

false

0

{}

0

–

constc

1

{hci}

1

const(c)

id

2

{hx, xi | x ∈ H }

∞

id(X, X )

cons

3

{hx, y, [x|y]i | x, y ∈ H }

∞

cons(X, Y, [X |Y ])

LOGIC PROGRAM SYNTHESIS

171

Table 1 shows the elementary predicates used presently. The last column shows the corresponding defining clauses in ordinary definite logic program form. List construction and list destruction are accomplished by the cons predicate. The denotations of the elementary predicates are as follows [[true]]0 = [[false]]0 = [[constc ]]1 = [[id]]2 = [[cons]]3 =

{hi} {} {hci} for all relevant constants c {ht, ti ∈ H 2 | t ∈ H } {ht 0 , t 00 , [t 0 | t 00 ]i ∈ H 3 | t 0 , t 00 ∈ H }

Using these elementary predicates the append predicate with the recursive definition append([], X, X ) append([X |T ], Y, Z ) ← append(T, Y, U ) ∧ cons(X, U, Z ) can be reshaped in the form (1) as append(X, Y, Z ) ← (const[] (X ) ∧ id(Y, Z )) ∨ (cons(F, T, X ) ∧ append(T, Y, U ) ∧ cons(F, U, Z )).

(2)

In the present conception of logic programming the result of a logic program is an nary relation which can be conceived theoretically to be formed by repeated filtering, permutation, and composition from the given elementary relations stated above. This view conforms with formation of relations in relational database algebra as well as the formation of functions by combining forms in functional programming languages (Bird and de Moor, 1997). Accordingly the next step is to devise an appropriate system of operations on relations. 3.2.

Relational operators and predicate expressions

In this section we identify a set of relational operators which is to replace in COMBILOG the definite clauses with an accompanying goal clause in ordinary logic programming. The relations considered are all subsets of the universal relations of assorted degrees. The most obvious transformation to perform on a single relation may be permutation and projection. Permutation and projection of an n-ary relation r with respect to an index list [µ1 , . . . , µm ] where µ j ∈ {1..n}, is defined as {htµ1 , tµ2 , . . . , tµm i|∃ht1 , . . . , tn i ∈ r }. This covers as a special case inversion of a binary relation r (usually written r ^ ). It turns out that it is feasible to generalize this manipulation operation with an expansion option leading to what we call the make operator (Hamfelt and Nilsson, 1999a; Hamfelt et al., submitted), defined as follows in clause notation adopted as part of the metalanguage ¡ ¢ make[µ1 , . . . , µm ](P) X µ1 , . . . , X µm ← P(X 1 , . . . , X n )

172

HAMFELT ET AL.

with accompanying set theoretical definition [[make[µ1 , . . . , µm ]( p)]]m = {htµ1 , . . . , tµm i ∈ H m | ∃ht1 , . . . , tn i ∈ [[ p]]n } or more explicitly [[make[µ1 , . . . , µm ]( p)]]m = {ht10 , . . . , tm0 i ∈ H m |∃ht1 , . . . , tn i ∈ [[ p]]n s.t. for i = 1..m if µi ≤ n then ti0 = tµi } where the requirement that µ j ∈ {1..n} is abandoned. Thus the make operator is to be understood as a generic operator parameterised by the index list [µ1 , . . . , µm ]. The 1-ary universal relation is now achieved by predicate expression make[1](true), the 2-ary universal relation by make[1, 2](true), and the n-ary universal relation by make[1, 2, . . . , n](true), the latter conforming with the unit clause p(X 1 , . . . , X n ). Actually the identity relation id introduced as an elementary relation can be produced by make[1, 1](true). With id available, however, from now on it is assumed that the indices in the index list are distinct. This operator and some of the following operators are inspired by the predicate functors introduced in Quine (1971), see also Bacon (1985), and adapted to definite clauses in the present proposal. Basic COMBILOG predicate expressions consist of an elementary predicate optionally prefixed with the make operator conforming with the following syntactical rules: hbasicpredi ::= helempredi | make[i 1 , . . . , i n ](helempredi) helempredi ::= true | false | constc | id | cons Incidentally our notion of basic predicate conforms with the notion of a basic function in recursive function theory. 3.3.

COMBILOG programs

COMBILOG programs consist of predicate definitions of the form p←ϕ where ϕ is either a predicate identifier (elementary relation or user-introduced) or a compound predicate expression comb(ϕ1 , . . . , ϕm ), where comb is a combinator (relational operator as make or the ones to be introduced below) and ϕi are recursively predicate expressions. The relational denotation of the term comb(ϕ1 , . . . , ϕm ) is determined compositionally as a relation-valued function Fcomb of the relational denotations [[ϕi ]] of the subterms ϕi [[comb(ϕ1 , . . . , ϕm )]] = Fcomb ([[ϕ1 ]], . . . , [[ϕm ]]).

(3)

LOGIC PROGRAM SYNTHESIS

173

Actually this compositionality takes form of a tuple valued metalanguage function [[comb(ϕ1 , . . . , ϕm )]] = { f comb (t1 , t2 , . . . , tm ) | ti ∈ [[ϕi ]]}. As a simple example the unit clause head([X | ], X ) via the rewriting head(L , X ) ← cons(X, T, L) cons(X, T, [X | T ]) in COMBILOG is defined by head ← make[3, 1](cons). The relational denotation can be derived as follows using the above definitions: [[head]] = [[make[3, 1](cons)]] = {ht3 , t1 i | ∃t2 ht1 , t2 , t3 i ∈ [[cons]]} = {ht3 , t1 i | ∃t2 (ht1 , t2 , t3 i ∈ {hx, y, [x | y]i | x, y ∈ H })} = {h[x | y], xi | x, y ∈ H } This calculation is not intended as a suggestion for executing COMBILOG programs, let alone for the fact that it is not viable for recursively defined predicates. Recursively defined predicates call for extension of the compositional semantics with a fix point semantics as elaborated in Hamfelt et al. (submitted). Since in the present context we focus on list processing programs, explicit recursive definitions are replaced by distinguished recursion combinators in the form of fold combinators for lists as introduced below. In Section 4 is presented a metalogic programming environment for execution of COMBILOG programs using ordinary SLD-resolution. This metainterpreter also forms the backbone of the COMBINDUCE system. 3.4.

Logical operators

In COMBILOG besides the manipulation operator make we introduce the two combinators and(P, Q)(X 1 , . . . , X n ) ← P(X 1 , . . . , X n ) ∧ Q(X 1 , . . . , X n ) or(P, Q)(X 1 , . . . , X n ) ← P(X 1 , . . . , X n ) ∨ Q(X 1 , . . . , X n ) with accompanying set theoretical definitions [[and(ϕ, ψ)]]n = [[ϕ]]n ∩ [[ψ]]n [[or(ϕ, ψ)]]n = [[ϕ]]n ∪ [[ψ]]n which are also understood to be generic in terms of the assumed common arities of predicate expressions ϕ and ψ. In Hamfelt et al. (submitted) is proved combinatory completeness of COMBILOG with these three operators (admitting explicit recursive definitions) relative to definite clauses, which implies that COMBILOG is as expressive as definite clause logic.

174

HAMFELT ET AL.

Using these combinators introduced so far append becomes append ← or(and(make[1, 2, 3](const[] ), make[3, 1, 2](id)), make[1, 2, 3](and(make[3, 4, 5, 1, 2, 6](cons), and(make[4, 2, 5, 6, 1, 3](append), make[4, 5, 3, 1, 6, 2](cons))))). Obviously this operator form is infeasible. Therefore, in the next section we introduce appropriate list recursion operators, which in common cases admit compact and comprehensible formulations. 3.5.

List recursion operators

As counterparts of the foldright and foldleft higher order functionals well-known in functional programming the above operators are now extended by similar relational operators foldright and foldleft defined as follows foldr(P, Q)(Y, [], Z ) ← Q(Y, Z ) foldr(P, Q)(Y, [X |T ], W ) ← foldr(P, Q)(Y, T, Z ) ∧ P(X, Z , W ) foldl(P, Q)(Y, [], Z ) ← Q(Y, Z ) foldl(P, Q)(Y, [X |T ], W ) ← P(X, Y, Z ) ∧ foldl(P, Q)(Z , T, W ) with accompanying set theoretical definitions S∞ [[foldri (P, Q)]]3 [[foldr(P, Q)]]3 = i=0 3 [[foldr0 (P, Q)]] = {hy, [], zi ∈ H 3 | hy, zi ∈ [[Q]]2 } [[foldri+1 (P, Q)]]3 = {hy, [t1 | t2 ], wi ∈ H 3 | ∃z ∈ H s.t. hy, t2 , zi ∈ [[foldri (P, Q)]]3 ∧ ht1 , z, wi ∈ [[P]]3 } and S∞ [[foldri (P, Q)]]3 [[foldl(P, Q)]]3 = i=0 3 [[foldl0 (P, Q)]] = {hy, [], zi ∈ H 3 | hy, zi ∈ [[Q]]2 } [[foldli+1 (P, Q)]]3 = {hy, [t1 | t2 ], wi ∈ H 3 | ∃z ∈ H s.t. ht1 , y, zi ∈ [[P]] ∧ hz, t2 , wi ∈ [[foldli (P, Q)]]3 }. For instance the relational append predicate from Section 3.1 in the fold-extended COMBILOG becomes append ← make[2, 1, 3](foldr(cons, id)) where make[2, 1, 3] swaps the two first arguments (since foldr differs from append in that the second argument, not the first, is the recursion parameter). The alternative backward use of append with only third argument being ground calls for a variant fold combinator to be introduced in Section 7 below.

LOGIC PROGRAM SYNTHESIS

3.6.

175

Pure COMBILOG programs

In analogy to the fold functionals in functional programming the present fold operators may be conceived of as higher order predicates accepting two predicate arguments and yielding a predicate. In Hamfelt and Nilsson (1997) we conduct an extended analysis of the applicability of these operators in the context of logic programming. This study concludes that most common list programs can be expressed in terms of the above fold operators as a replacement for explicit recursive definitions. It is also discussed how the expressivity of clauses is compromised when explicit unconstrained recursive definitions are replaced by combinators. For the case of fold expressible logic programs the predicate expression ϕ consists solely of operators and names of elementary predicates. That is to say, user defined predicates are discarded and the entire program becomes one predicate expression ϕ as exemplified with the append definition above. A program consisting of a single predicate expression is called a pure COMBILOG program. Thus a subprogram identified by a predicate in definite clauses, here becomes an anonymous subexpression of ϕ, in so far as the program is expressible in terms of the available combinators. 3.7.

The join operator

For the sake of shorter COMBILOG programs (a heuristic concern in the synthesis) we introduce a further operator, which is reminiscent of the equi-join operator known from relational data base algebra. It is defined here as follows: join[i, j](P, Q)(X 1 , . . . , X m , Y1 , . . . , Y j−1 , Y j+1 , . . . , Yn ) ← P(X 1 , . . . , X m ) ∧ Q(Y1 , . . . , Y j−1 , X i , Y j+1 , . . . , Yn ) with accompanying set theoretical definition [[join[i, j](ϕ, ψ)]] = {hx1 , . . . , xm i_ hy1 , . . . , y j−1 i_ hy j+1 , . . . , yn i | hx1 , . . . , xm i ∈ [[ϕ]] ∧ hy1 , . . . , yn i ∈ [[ψ]] ∧ xi = y j }. The positive index numbers i and j are assumed not to exceed the arity m of ϕ and n of ψ, respectively. The arity of the relation resulting from the join is m + n − 1. It is seen that this join operator is a generalization of the well-known composition operator for two binary relations. As such it may be considered the COMBILOG counterpart of the functional composition ‘◦’ in functional programming. As a consequence of the combinatorial completeness theorem in Hamfelt et al. (submitted) this operator is strictly redundant since it can be replaced by an expression comprising the already introduced operators including and. However, it admits more elegant COMBILOG programs in frequent cases. For instance the unit clause make unit list(X, [X |[]])

176

HAMFELT ET AL.

via the flattened reformulation make unit list(X, Y ) ← const[] (Z ), cons(X, Z , Y ) in COMBILOG first becomes make[1, 3](and(make[2, 1, 3](const[] ), cons)). Reformulation with the join operator yields make[2, 3](join[1, 2](const[] , cons)). This example also illustrates the flattened clause form a` la DATALOG underlying COMBILOG in which constant and compound terms are replaced by invocation of the predefined elementary predicates, as explained systematically in Hamfelt et al. (submitted). 4.

Execution of COMBILOG programs

This section explains how COMBILOG programs in the form of the above expressions can be executed using SLD-resolution by embedding the COMBILOG expressions as terms in a metalogic programming environment consisting of definite clauses. This operational semantics complements the relational semantics of Section 3. The metalogic interpreter comprises an apply predicate, whose first argument is a COMBILOG predicate expression being represented as a metalanguage term, and whose second argument is the list of object language arguments. The apply predicate expresses application of the first argument predicate to the subsequent arguments. This means that the atomic formula p(t1 , . . . , tn ) is replaced by apply( p, [t1 , . . . , tn ]). A COMBILOG program expression can be represented directly as a logic program ground term. For instance, the above append program becomes make([2, 1, 3], (foldr (cons, id))). The clauses for the apply predicate can be conceived of as a non-ground metainterpreter where object language predicates and term argument lists are kept in separate arguments. For each combinator comb taking m predicate expression arguments and producing an n-ary predicate there are defining clauses of the principal form apply(comb(P1 , . . . , Pm ), [X 1 , . . . , X n ]) ← apply(P1 , [X 11 , . . . , X 1n ]) ∧ · · · ∧ apply(Pm , [X m1 , . . . , X mn ]) ∧ apply comb([X 1 , . . . , X n ], [X 11 , . . . , X 1n ], . . . , [X m1 , . . . , X mn ]) reflecting the compositional principle in (3). The auxiliary predicate apply comb is specific to comb. In particular for the elementary predicates true, constc , id, cons, which may be conceived of as combinators taking no predicate arguments (i.e., m = 0) the following apply clauses are introduced: apply(true, []) apply(const(c), [c]) for each relevant constant c including []

LOGIC PROGRAM SYNTHESIS

177

apply(id, [X, X ]) apply(cons, [X, T, [X | T ]]). The defining clauses for and, or and make are as follows: apply( and(P, Q), [X 1 ]) ← apply(P, [X 1 ]) ∧ apply(Q, [X 1 ]) apply( and(P, Q), [X 1 , X 2 ]) ← apply(P, [X 1 , X 2 ]) ∧ apply(Q, [X 1 , X 2 ]) .. .

apply( and(P, Q), [X 1 , . . . , X n ]) ← apply(P, [X 1 , . . . , X n ]) ∧ apply(Q, [X 1 , . . . , X n ]) apply( or(P, Q), [X 1 ]) ← apply(P, [X 1 ]) ∨ apply(Q, [X 1 ]) apply( or(P, Q), [X 1 , X 2 ]) ← apply(P, [X 1 , X 2 ]) ∨ apply(Q, [X 1 , X 2 ]) .. .

apply( or(P, Q), [X 1 , . . . , X n ]) ← apply(P, [X 1 , . . . , X n ]) ∨ apply(Q, [X 1 , . . . , X n ]) apply( make([µ1 , . . . , µm ], P), [X µ1 , . . . , X µm ]) ← apply(P, [X 1 , . . . , X n ])

up to some fixed maximum arity m and n. In particular the last clause is generic in n as well as in µ1 , . . . , µm . For instance, for the combinator make[3, 1] applied to a ternary predicate there is the actual clause: apply(make([3, 1], P), [X 3 , X 1 ]) ← apply(P, [X 1 , X 2 , X 3 ]). For foldr there are the clauses apply(foldr(P, Q), [Y, [], Z ]) ← apply(Q, [Y, Z ]) apply(foldr(P, Q), [Y, [X | T ], W ]) ← apply(foldr(P, Q), [Y, T, Z ]) ∧ apply(P, [X, Z , W ]). For join there are the clauses apply( join([i, j], P, Q), [X 1 , . . . , X m , Y1 , . . . , Y j−1 , Y j+1 , . . . , Yn ]) ← apply(P, [X 1 , . . . , X m ]) ∧ apply(Q, [Y1 , . . . , Y j−1 , X i , Y j+1 , . . . , Yn ]). The apply predicates constitute an abstract machine for execution of combinatory programs in the metalogic programming environment as an adaptation of a technique originating in Warren (1982) and pursued in Hamfelt and Nilsson (1997), Hamfelt et al. (submitted). A COMBILOG program constituted by predicate expression ϕ is invoked by the goal clause ← apply(ϕ, [X 1 , . . . , X m ]). As an example invocation of append, say, with the goal clause ← append([a],[b], Z) in the

178

HAMFELT ET AL.

metalogic program setting with COMBILOG takes the form: ← apply(make([2, 1, 3], (foldr(cons, id)), [[a], [b], Z ]) ← apply(foldr(cons, id), [[b], [a], Z ]) ← apply(foldr(cons, id), [[b], [], Z ]), apply(cons, [a, Y, Z ]) ← apply(id, [[b], Y ]), apply(cons, [a, Y, Z ]) ← apply(cons, [a, [b], Z ]) where the syntax of the predicate expressions has been adapted to the term form. In Hamfelt et al. (submitted) is stated and proved a theorem that the proof theoretical semantics obtained via apply is equivalent with the compositional relational semantics from Section 3. The key principle in the present approach to inductive synthesis is to utilize the invertibility of the apply predicate goal ← apply(P, [t1 , . . . , tm ]) where t1 , . . . , tm is a program example and P is to be instantiated to a program. For instance, in principle the above append predicate can be synthesized by the goal clause ← apply(P, [[], [], []]), apply(P, [[a], [b], [a, b]]). 5.

Towards induction of programs from examples

We now consider induction of COMBILOG programs from example program results. The objective is to construct a syntactically least COMBILOG predicate expression ϕ whose relational extension [[ϕ]] comprises the given examples. This means that given the suite of test example n-tuples e1 , e2 , . . . , es , is sought a syntactically smallest ϕ such that s [ {ei } ⊆ [[ϕ]]. i=1

The inductive synthesis is carried out as a problem reduction search process in which the combinators are conceived of as problem reduction operators and the elementary predicates act as trivial problems. The synthesis may be conceived simplistically initially as a generateand-test process conforming with the goal clause ← gen(P) ∧

s ^

apply(P, ei )

i

where a predicate gen is to generate a succession of suitable program candidates as instantiations ϕ of the variable P.

LOGIC PROGRAM SYNTHESIS

179

In principle our synthesis as an amelioration of the simplistic scheme is conducted by posing instead a goal clause ←

s ^

syn(P, ei )

i

now applying a synthesis predicate syn (integrating generation and testing) of the following recursive divide-and-conquer form syn(comb(P1 , . . . , Pm ), Ex) ← apply comb(Ex, Ex1 , . . . , Exm ) ∧ syn(P1 , Ex1 ) ∧ · · · ∧ syn(Pm , Exm ). This is recognized as a declaratively equivalent version of the apply clause scheme of Section 4, where the synthesis predicate syn uses apply in an inverted manner, with the first argument being output argument. 5.1.

Induction of elementary programs

The elementary programs of Section 3.1 are synthesized with the following clause syn(P, Ex) ← apply(P, Ex) referring to the apply definitions in Section 4. This clause demonstrates our inductive synthesis approach in a seminal form, with the first argument being a variable to be instantiated to the combinator covering all the available examples. Thus the syn predicate is capable of synthesising from a single example a program consisting of an elementary predicate, so far. During the course of the induction the combinator terms being synthesized may be nonground, with the variables representing unknown (sub)combinator terms. Thus our induction procedure proceeds by specializing (instantiating) in a top-down fashion partially specified combinatory program forms guided by the given program result test suite. 5.2.

Synthesis with the make operator

In order to explain synthesis with the make operator let us first consider the case of, say, make[3, 2] applied to a ternary predicate P. This synthesis uses the clause syn(make([3, 2], P, [X 3 , X 2 ])) ← syn(P, [X 1 , X 2 , X 3 ]). In the general case of synthesis with the make operator is to be considered all instantiations of the index list Il up to some maximum arity. syn(make(Il, P), Ex) ← apply make(Il, Ex, E x1) ∧ syn(P, Ex1) This clause synthesizes a predicate expression make(Il, ϕ), where proposals for the index list Il (with accompanying permutations of tuples in Exlist giving Exl1) are produced successively by apply make in an operational understanding of the clause.

180

HAMFELT ET AL.

Let us assume a maximum arity of three. The possible combinations for apply make conforming with a 3-ary predicate argument to make can then be made explicit by the unit clauses: apply apply apply apply apply apply apply apply apply apply apply apply apply apply apply 5.3.

make([1], [X 1 ], [X 1 , X 2 , X 3 ]) make([2], [X 2 ], [X 1 , X 2 , X 3 ]) make([3], [X 3 ], [X 1 , X 2 , X 3 ]) make([1, 2], [X 1 , X 2 ], [X 1 , X 2 , X 3 ]) make([1, 3], [X 1 , X 3 ], [X 1 , X 2 , X 3 ]) make([2, 1], [X 2 , X 1 ], [X 1 , X 2 , X 3 ]) make([2, 3], [X 2 , X 3 ], [X 1 , X 2 , X 3 ]) make([3, 1], [X 3 , X 1 ], [X 1 , X 2 , X 3 ]) make([3, 2], [X 3 , X 2 ], [X 1 , X 2 , X 3 ]) make([1, 2, 3], [X 1 , X 2 , X 3 ], [X 1 , X 2 , X 3 ]) make([1, 3, 2], [X 1 , X 3 , X 2 ], [X 1 , X 2 , X 3 ]) make([2, 1, 3], [X 2 , X 1 , X 3 ], [X 1 , X 2 , X 3 ]) make([2, 3, 1], [X 2 , X 3 , X 1 ], [X 1 , X 2 , X 3 ]) make([3, 1, 2], [X 3 , X 1 , X 2 ], [X 1 , X 2 , X 3 ]) make([3, 2, 1], [X 3 , X 2 , X 1 ], [X 1 , X 2 , X 3 ]).

Synthesis of basic COMBILOG programs

Let us consider the synthesis of a basic predicate expression (cf. Section 3.2) for the tail predicate yielding the tail of a non-empty list, given say the two examples [[a, b], [b]] and [[a], []]. Synthesis of tail is initiated with a goal clause ← syn(P, [[a, b], [b]]) ∧ syn(P, [[a], []]) where the synthesized predicate expression make[3, 2](cons) obtains as binding to P in the following way. Attempts to instantiate P with an elementary program fail since the only elementary combinator of arity 2 is identity, which fails with the first example. Invocation of the clauses for make will eventually reach apply make([3, 2], [X 3 , X 2 ], [X 1 , X 2 , X 3 ]) yielding the derived subproblem non-ground example [ , [b], [a, b]]. Recursive invocation of syn with this example is satisfied by the elementary predicate cons. The result of the synthesis of tail then becomes make[3, 2](cons) since this instantiation of P also succeeds with the second example. We have now illuminated the principles of our induction method by specifying COMBINDUCE for the class of basic COMBILOG programs. In the actual implementation of the synthesizer COMBINDUCE the syn predicate is equipped with additional arguments constraining the admissible COMBILOG programs by means of well-modedness and data type restrictions as described below. The next step is to extend the considered language with the remaining operators. However before we consider induction with additional operators in Section 6 we discuss

LOGIC PROGRAM SYNTHESIS

181

constraints to be imposed on the induced programs in order to eliminate undesired solutions. For example the universal relation make[1, 2](true) is obtained as an undesired solution to tail. Such spurious solutions are not ruled out simply by omitting true since for instance the universal binary relation can be obtained also as make[2, 3](const([])). 6.

Constraints on the synthesis

The above outlined synthesis method when applied in an unconstrained manner, may propose as solution to the synthesis of an n-ary predicate ϕ simply the universal relation make[1, 2, . . . , n](true) since [[ϕ]] ⊆ [[make[1, 2, . . . , n](true)]] = H n . For this reason, and furthermore in order to constrain the search process heuristically as much as possible, various constraints are imposed on the admissible predicate expressions concerning syntactical form, modes, data types, and resources as to be explained in the following subsections. These constraints are studied here as an alternative to use of negative examples for ruling out unwanted solutions such as the universal relation since we have no principled way of suggesting negative examples. 6.1.

Constraints on the form of expressions

It is observed that iterated applications of make can always be contracted into a single application of make. Moreover identity combinator versions of make may be omitted. The or operator is left out of consideration through the following analysis: The logical or operator distributes over make, and, and foldr/l. For instance the following equations are available as rewrite rules on predicate expressions ¯ 1 ), make[µ](P ¯ 2 )) make[µ](or(P ¯ 1 , P2 )) = or(make[µ](P foldr(or(P1 , P2 ), Q) = or(foldr(P1 , Q), foldr(P2 , Q)) foldr(P, or(Q 1 , Q 2 )) = or(foldr(P, Q 1 ), foldr(P, Q 2 )). The distributivity property implies that any predicate expression ϕ can be rewritten to the semantically equivalent ϕ1 or ϕ2 or . . . or ϕk , where k > 0 and ϕ1 , . . . , ϕk are or-free. Therefore we consider separately the induction of the subexpressions ϕi with the pertinent examples. This does not imply that programs are generally to be induced one clause at a time, since there are still logical disjunctions present within the definitions of the fold operators. Certain other combinatory programs are precluded from synthesis such as and(P, P), as justified by the identity and(P, P) = P. As a final kind of constraint ensuring that a shortest program is obtained, and at the same time preventing the induction process from running astray, we propose that the induction search process be carried out as an iterative staged search with successive increasing limits on the size of the predicate expression resulting from the induction. Readers familiar with the notion of well-moded logic programs can skip the rest of this section without loss of continuity.

182 6.2.

HAMFELT ET AL.

Constraints on the modes and types of expressions

The motivation for introducing modes has already been stated above. In addition we suggest introduction of data type constraints in order to further limit the search space in the induction process. Thus mode and data type constraints are to be checked in the course of inducing a predicate expression. The notion of well-moded logic programs is described e.g., in Apt and Marchiori (1994). Let us recall the fundamentals before applying it to the combinatory form of definite clauses. Each predicate argument can be associated with either an input tag denoted ‘+’ or an output tag denoted ‘−’. The input/output tags called modes are subject to the so called well-modedness constraints for each predicate. The mapping from argument position number to the mode tag is called a mode list. A fixed mode list can be assumed for each predicate in a program without essential loss of generality. Assuming a left-most computation rule a definite clause logic program is well-moded iff every clause of it is, and a clause is well-moded iff • every variable occurring in an input position of a body atom occurs in an input position of the head, or in an output position of body atom to the left of the considered one, • every variable occurring in an output position of the head occurs in an input position of the head, or in an output position of a body atom. An atomic query is well-moded iff input positions are ground terms. In well-moded programs variables at output positions in the head and input positions in the body must appear at other positions. Thus the search space of acceptable programs is reduced. In addition, modes aid choosing the procedurally right variants of recursion forms cf. Hamfelt and Fischer Nilsson (1996), such as choosing between procedural variants of foldr as discussed in Section 7.4. The theory of well-moded programs established for definite clause logic programs carries over to COMBILOG via the clausal definitions of operators. Thus we demand that the synthesized combinator programs be well-moded. However, a single mode list for each combinator is too restrictive. Therefore the operators are overloaded with admissible mode tuples called valences. Table 2 shows admissible valences for the elementary operators. The description of valences is compactified by introducing the mode tag ‘∗’ meaning optionally either + or −. Using this convention Table 2 condenses to Table 3. Table 2.

Admissible valences of the elementary operators. Operator

Valences

true

[]

false

[]

const

[−], [+]

id

[+, −], [−, +], [+, +]

cons

[+, +, −], [+, −, +], [+, +, +], [−, +, +], [−, −, +]

183

LOGIC PROGRAM SYNTHESIS Table 3.

Compactified valences of the elementary operators. Operator

6.3.

Valences

true

[]

false

[]

const

[∗]

id

[+, ∗], [∗, +]

cons

[+, +, ∗], [∗, ∗, +]

Well-Modedness of make

Consider the predicate expression make[µ1 , . . . , µm ](ϕ n ). Assume that the subterm ϕ n has a valence [v1 , . . . , vn ]. The valence [w1 , . . . , wm ] of make[µ1 , . . . , µm ](ϕ n ) relative to [v1 , . . . , vn ] is determined by the following constraints wi = vµi or wi = +, if µi ∈ {1, . . . , n} and wi = + if µi 6∈ {1, . . . , n} and vi = − if i 6∈ {µ1 , . . . , µm }. 6.4.

Well-modedness of foldr

Generally speaking the valences of combinator terms are determined as functions of valences of the predicate argument terms. In Table 4 is shown the resulting valences for foldr in terms of the admissible valences for P and Q. For foldr there are initially 23+2+3 = 256 combinations of valences but only 18 of them are well-moded. These 18 admissible valences condense to Table 4 using the ∗-tag. Table 4.

Well-moded combinations of valences of the foldr(P, Q) operator. Operator

P-valence

Q-valence

foldr (P, Q)-valence

foldr

[∗, ∗, −]

[∗, −]

[+, +, −]

[∗, ∗, −]

[−, −]

[−, +, −]

[−, ∗, −]

[∗, −]

[+, −, −]

[−, ∗, −]

[−, −]

[−, −, −]

184

HAMFELT ET AL.

Table 5. And-valence for the i’th position. P-valence

Q-valence

and (P, Q)-valence

∗

∗

+

−

∗

−

Termination is guaranteed by requiring that the list recursion parameter be ground. This rules out the two last lines in Table 4. Furthermore it is required that the arguments of P take uniform valences in the unfoldings, and similarly for Q. 6.5.

Well-modedness of and

As is apparent from the clausal definition of and the valences for the i’th positions are determined independently of valences of the other argument positions as stated in Table 5. 6.6.

Data type constraints on programs

In addition to the well-modedness constraints described above one can introduce also data type well-formedness constraints on programs. In the present case of list data types the types for the elementary predicates would be as in Table 6. Here τ is a type variable. The table assumes a simple type system (subject to further elaboration) comprising individuals i, lists of individuals list(i), lists of lists of individuals list(list(i)) etc. 7.

Induction of well-moded programs with recursion

This section considers induction of non-trivial list processing programs, i.e., programs calling for recursively defined predicates. As already indicated in previous sections the considered recursion forms are to be expressed via fold operators. Table 6.

Datatypes for elementary predicates. Name

Types

true

[]

false

[]

constc

[i]

constnil

[list(τ )]

id

[τ, τ ]

cons

[τ, list(τ ), list(τ )]

LOGIC PROGRAM SYNTHESIS

185

In the considered version of COMBILOG an operator term to be synthesized is to obey the following grammar: hctermi ::= const([]) | id | make unit list | cons | foldr(hctermi, hctermi) | foldl(hctermi, hctermi) make[i 1 , . . . , i n ](hctermi) | and(hctermi, hctermi) The operator make unit list is introduced as an elementary predicate with the purpose of enhancing the performance; make unit list has the same well-moded combinations of valences as the id operator. A more restrictive grammar might be introduced to reduce the search space. Below is shown a version of the synthesizer from Section 5 with mode well-formedness constraints imposed through auxiliary predicates make valence and check valence. Moreover, the available input/output examples are aggregated into a list in the below formulations. This is in order to dispose of an intermediate program solution in case it fails on one of the given examples, thus reducing the amount of back-tracking. The second parameter in the syn predicate contains the list of valences imposed on candidate programs. For the elementary operators there are syn(const(C), Val, Exl) ← check valence(Val, [V1]) ∧ test examples(const(C), Exl) syn(id, Val, Exl) ← check valence(Val, [[+, V1], [V2, +]]) ∧ test examples(id, Exl) syn(cons, Val, Exl) ← check valence(Val, [[+, +, V1], [V2, V3, +]]) ∧ test examples(cons, Exl) syn(make unit list, Val, Exl) ← check valence(Val, [[+, V1], [V2, +]]) ∧ test examples(make unit list, Exl) test examples(P, [E x]) ← apply(P, Ex) test examples(P, [Ex1, Ex2|Exl]) ← apply(P, Ex1) ∧ test examples(P, [Ex2|Exl]). The auxiliary predicate test examples tries each of the available examples with the apply predicate as defined in Section 4. For the apply predicate we add the unit clause apply(make unit list, [X, [X ]]). For the make operator we have syn(make(Il, P), Val, Exl) ← make valence(Il, Val, V al1) ∧ make example(Il, Exl, E xl1) ∧ syn(P, Val1, Exl1).

186

HAMFELT ET AL.

Actually, in the COMBINDUCE prototype, make is split internally into three more fundamental operators: permute, expand and project. The auxiliary predicate make valence carries out necessary permutations as specified by the index list Il. Synthesis of the above example tail is initiated with a goal clause, say, ← syn(P, [+, −], [ [[a, b], [b]], [[a], []] ]) where the synthesized predicate expression make[3, 2](cons) obtains as binding to P. The previously obtained universal relation solution in Section 5.3 is now ruled out by the wellmodedness constraint. 7.1.

Synthesis with fold

The predicate invocation syn(fold(P, Q), . . .) traverses the example list and synthesizes a program for Q in the case of an empty second component list in an example triple and synthesizes P in the case of a non-empty second component list in the example triple. One observes that the predicate unfolds examples according to the second component list structure (being ground) following the structure of the apply evaluation of fold. For the foldr operator we have syn(foldr(P, Q), Val, [[Y, [], W ]]) ← foldr valence(Val, Val1, Val2) ∧ syn(Q, Val1, [[Y, W ]]) syn(foldr(P, Q), Val, [[Y, [], W ]|Exl]) ← foldr valence(Val, Val1, Val2) ∧ syn(Q, Val1, [[Y, W ]]) ∧ syn(foldr(P, Q), Val, Exl) syn(foldr(P, Q), Val, [[Y, [X |T ], W ]|Exl]) ← foldr valence(Val, Val1, Val2) ∧ syn(foldr(P, Q), Val, [[Y, T, Z ]|Exl]) ∧ syn(P, Val 2, [[X, Z , W ]]). The foldr valence predicate implements Table 4. In the special cases where the second components throughout in the examples are empty lists, only Q is instantiated, thus nonground synthesized programs may arise. This situation is remedied by supplying the synthesizer with additional, appropriate examples. 7.2.

Synthesis with and

For the and operator we have syn(and(P, Q), Val, Exl) ← and valence(Val, Val1, Val2) ∧

LOGIC PROGRAM SYNTHESIS

187

syn(P, Val1, Exl) ∧ syn(Q, Val2, Exl) where the predicate and valence implements Table 5. 7.3.

Combination of make and and

Often combinations of make and and call for local relaxations on the well-modedness constraints. This is because the and operator insists on identical arguments of its operands in constrast to the conjunction in definite clauses. As an example consider the execution of the earlier mentioned operator expression for make unit list program in the interpreter with added valences: ← apply[+,−] (make([1, 3], and(make([2, 1, 3], const([])), cons)), [a, X ]). ← apply[+,−,−] (and(make([2, 1, 3], const([])), cons), [a, Y, X ]). ← apply[+,−,−] (make([2, 1, 3], const([])), [a, Y, X ]) ∧ apply[+,+,−] (cons, [a, Y, X ]). ← apply[−] (const([]), [Y ]) ∧ apply[+,+,−] (cons, [a, Y, X ]). ← apply[+,+,−] (cons, [a, [], X ]). ← The subexpression make[2, 1, 3](const ([])) is not well-moded in isolation; still wellmodedness is obtained in the above context. An ad hoc solution to this problem is the introduction of auxiliary super-operators such as join functioning as replacement for the above solution program for make unit list as shown in Section 3.7. Another solution (adopted in the prototype) amounts to explicitly listing the different combinations of operators where non-well-modedness in isolation is allowed according to the well-modedness constraints. 7.4.

Sample syntheses of recursive programs

Induction of the append program may be initiated with the two examples [[], [], []], and [[a], [b], [a, b]] as in the following goal clause for intended ordinary forward use of append ← syn(P, [+, +, −], [[[], [], []], [[a], [b], [a, b]]]) giving the recursive predicate P = make[2, 1, 3](foldr(cons, id)). Synthesis of the append program used backwards calls for introduction of variant operators cf. Hamfelt and Fischer Nilsson (1996), foldr reverse and foldl reverse, defined by

188

HAMFELT ET AL.

swapping the atoms in the bodies as in apply(foldr reverse(P, Q), [Y, [], Z ]) ← apply(Q, [Y, Z ]) apply(foldr reverse(P, Q), [Y, [X |T ], W ]) ← apply(P, [X, Z , W ]) ∧ apply(foldr reverse(P, Q), [Y, T, Z ]) with analogous clauses in the synthesizer. Synthesis of the inverted append initiated with ← syn(P, [−, −, +], [[[], [], []], [[a], [b], [a, b]]]) gives P = make[2, 1, 3](foldr reverse(cons, id)). As our next example we consider the synthesis of a program for reversal of lists. ← syn(P0, [+, −], [[[a, b], [b, a]]]). In the course of inducing the na¨ıve reverse solution the operator term make[2, 3](foldr(P, Q)) is to be introduced. However, the basic recursive predicates instantiated from this predicate expression are not well-moded in isolation since the first argument to foldr remains a logical variable never to be used. Thus the general problem is that the fold schemes are too general in certain applications, thereby locally violating the well-modedness requirements. One solution to this technicality would be to revise the well-modedness theory so that it accepts non-well-moded arguments being irrelevant to the program result. However, for the sake of the exposition at present is assumed existence of a specialization of foldr conforming with the definition foldr2(P, Q) = make[2, 3](foldr(P, Q)). Below we highlight steps in the synthesis of the na¨ıve reverse program. Eventually the search process instantiates P0 to foldr2(P1, Q1), giving the following goal clause ← syn(foldr2(P1, Q1), [+, −], [[[a, b], [b, a]]]). We proceed by successively unfolding the example list according to the foldr scheme. ← syn(Q1, [−], [[Z 1]]) ∧ syn(P1, [+, +, −], [[b, Z 1, Z ]]) ∧ syn(P1, [+, +, −], [[a, Z , [b, a]]]). The arity of Q1 is 1, thus we get the instantiation Q1 = const([]) and Z 1 = [].

LOGIC PROGRAM SYNTHESIS

189

Next, the search process instantiates P1 to foldr(P2, Q2) reducing the goal clause to ← syn(Q2, [+, −], [[b, Z ]]) ∧ syn(foldr(P2, Q2), [+, +, −], [[a, Z , [b, a]]]). Eventually, we get the instantiation Q2 = make unit list and Z = [b] ← syn(foldr(P2, make unit list), [+, +, −], [[a, [b], [b, a]]]). Unfolding the example list gives ← syn(make unit list, [+, −], [[a, Z ]]) ∧ syn(P2, [+, +, −], [[b, Z , [b, a]]]). Because of well-modedness of make unit list, we get Z = [a] and P2 = cons ← syn(cons, [+, +, −], [[b, [a], [b, a]]]). As solution is obtained the basic recursive predicate P0 = foldr2(foldr(cons, make unit list), const([])) corresponding to the well-known na¨ıve solution to the reversal problem. This is an example of a successful induction of a non-trivial program with two nested recursions without recursive predicates in the background knowledge. Thus the example illustrates predicate invention, namely invention of the inner recursive predicate performing the unit list concatenation append(X, [Y ], Z ). The program for reversal of a list has a well-known, more elegant, version, the non-na¨ıve reverse program, also known as reverse accumulate: reverse acc(X, Y ) ← foldl(cons, id)([], X, Y ) In COMBILOG we explicitly need to describe the introduction of a new variable and its instantiation to the empty list. Thus as solution to the above reversal example is also obtained the following program as a binding to P0: P0 = make[2, 3](and(make[1, 2, 3](const([])), foldl(cons, id))) Note that this combination of and and make again introduces local non-well-modedness. The reverse accumulate program uses only one recursion operator, illustrating that by use of suitable recursion operators, more complicated problems may turn out to have rather simple solutions.

190

HAMFELT ET AL.

Table 7.

Induced sample programs. Program

Aux. predicates

CPU-time (secs.)