subsumption for structural matching

-subsumption for structural matching Luc De Raedt Department of Computer Science, Katholieke Universiteit Leuven Celestijnenlaan 200A, B-3001 Heverlee, Belgium e-mail : [email protected] Peter Idestam-Almquist Department of Computer and Systems Sciences, Stockholm University Electrum 230, S-164 40 Kista, Sweden e-mail : [email protected] Gunther Sablon Department of Computer Science, Katholieke Universiteit Leuven Celestijnenlaan 200A, B-3001 Heverlee, Belgium e-mail : [email protected] October 18, 1996 Abstract

Structural matching, originally introduced by Steven Vere in the mid-seventies, was a popular subeld of inductive concept-learning in the late seventies and early eighties. Despite various attempts to formalize and implement the notion of \most specic generalisation" of two productions, several problems remained. These include using background knowledge, the non-uniqueness of most specic generalisations, and handling in-equalities. We show how Gordon Plotkin's notions of \least general generalisation" and \relative least general generalisation" dened on clauses can be adapted for use in structural matching such that the remaining problems disappear. Dening clauses as universally quantied disjunctions of literals and productions as existentially quantied conjunctions of literals, it is shown that the lattice on clauses imposed by -subsumption is order-isomorphic to the lattice on productions needed for structural matching. The mapping also shows the relation between the older work on structural matching and the fashionable inductive logic programming paradigm.

1 Introduction In the early seventies, Gordon Plotkin 13, 14], introduced a notion of -subsumption between di erent clauses (i.e., universally quanti ed disjunctions of literals). In 1971, Plotkin 15] 1

extended -subsumption into relative -subsumption with regard to a background theory. Now, -subsumption and relative -subsumption are the basic notions of generality employed by many inductive logic programming (ILP) systems, cf., e.g., Golem 12], Claudien 2], FOIL 16]. In the mid seventies, inspired by Plotkin, Steven Vere 18]) studied the problem of generality between productions (i.e., existentially quanti ed conjunctions of literals). Vere's work was intended to form a sound basis for generalizing scenes described by conjunctions of facts (such as in the blocks world). The interest in representing examples as sets of ground facts (i.e., interpretations) has recently witnessed a revival with the PAC-learning results for the nonmonotonic ILP setting (cf. 3] ). However, whereas Plotkin used a rst-order language including constants, functors and variables, Vere did not allow for functors. Furthermore, whereas Plotkin de ned -subsumption using classical substitutions, which replace variables by terms, Vere employed inductive substitutions, replacing terms (at speci ed places) by variables. As a consequence, Plotkin's -subsumption induces a complete lattice on (equivalence classes of) clauses, but Vere's notion of structural matching does not. One of the remaining problems with Vere's framework is that the most speci c generalisation of two productions is not necessarily unique. This in turn leads to combinatorial problems when searching the generalisation space, since several generalisations (and paths through the search space) may have to be considered. The work by Vere was later continued by Hayes-Roth and McDermott 6], Ganascia and Kodrato 4], Vrain and Kodrato 20] and Gemello 5], also with the aim of taking background knowledge into account. However, none of these extensions resulted in a framework having the same power and properties as Plotkin's -subsumption, which explains why structural matching is less popular than ILP. In this note, we adapt Plotkin's original notions of -subsumption and relative -subsumption for use to Vere's problem. In particular, we explicitly give an isomorphism between Plotkin's approach and structural matching. Our main contribution is to show that the two approaches can be considered equivalent on a di erent representation. As a consequence, Plotkin's results and algorithms directly map to structural matching, thereby providing a better understanding of structural matching, solving some of its problems, and extending its application scope. At a more general level, the mapping between the representations and operations employed in structural matching and ILP sheds new light on the relation between these two techniques. We hope it may help to revive interest in structural matching and to understand how existing algorithms and systems from ILP may be mapped onto structural matching. The paper is organised as follows: in Section 2, we introduce the logical concepts needed in Section 3, we briey summarize the main characteristics of -subsumption and "least general generalisation" in Section 4, we outline how this notion can be mapped onto structural matching in Sections 5 and 6, we generalize the results of Sections 3 and 4 to relative -subsumption and "relative least generalisation" nally, in Section 7, we conclude.

2 Logic programming concepts We rst outline some standard logic programming concepts (see (Lloyd 87) for more details). 2

A rst-order alphabet is a set of variables, functor symbols, and predicate symbols. An atom p(t1 : : : tn) is a predicate symbol p followed by a bracketed n-tuple of terms. A positive literal is an atom, a negative literal is a negated atom. A term t is a variable V or a functor symbol f (t1 : : : tn) immediately followed by a bracketed n-tuple of terms. Constants are functor symbols of arity 0.

Denition 1 A clause is a nite disjunction of literals closed under universal quantication 8(l1 _ : : : _ ln). The clausal language DA given by a rst-order alphabet A consists of the set of all clauses that can be constructed from the symbols in A. An example clause D1 is, e.g., 8X (:block(X ) _ black(X )), stating that all blocks are

black.

Denition 2 A production is a nite conjunction of literals closed under existential quantication 9(l1 ^ : : : ^ ln). The production language CA given by a rst-order alphabet A consists of the set of all productions that can be constructed from the symbols in A. An example production C1 is, e.g., 9X (block(X ) ^ :black(X )), stating that there is a

block which is not black. The results and discussion in this paper will be independent of the choice of rst-order alphabet A, and we will therefore for simplicity in the following write D and C instead of DA and CA. For the purposes of this paper, a formula F will be a production or a clause. We will denote the set of literals in a formula F by set(F ). E.g., the example clause D1 above corresponds to the set set(D1) = f:block(X ) black(X )g. Also, we introduce two one-to-one mappings, which will be used further on.

Denition 3 The mapping FC : C 7! D maps every production in C to its negation in D. The mapping FD : D 7! C maps every clause in D to its negation in C . This means that and

FC (9(l1 ^ : : : ^ ln)) = 8(:l1 _ : : : _ :ln)

FD (8(l1 _ : : : _ ln)) = 9(:l1 ^ : : : ^ :ln ): The reader may notice that FC ;1 = FD and FD ;1 = FC , since ::l = l for any literal l. A substitution = fV1=t1 : : : Vn =tn g is an assignment of terms to variables. Applying a substitution to a term, literal, or formula F yields the instantiated term, literal, or formula F , where, for every 1  i  n, all occurences of the variables Vi in F are simultaneously replaced by the corresponding terms ti.

3

-subsumption

By now we can introduce Plotkin's notion of -subsumption.

Denition 4 A clause D1 -subsumes a clause D2 if and only if there exists a substitution such that set(D1) set(D2). We write D1 D D2 to denote that D1 -subsumes D2. 3

For example, father(X,Y) _ : parent(X,Y) _ : male(X) -subsumes father(jef,paul) _ : parent(jef,paul) _ : parent(jef,ann) _ : male(jef) _ : female(ann) with = fX / jef, Y / paul g. The notion of least general generalisation of a set of clauses is then de ned as follows:

Denition 5 A clause G is a generalisation of a set of clauses S = fD1 : : :  Dn g if and only if, for every 1  i  n, G D Di . A generalisation G of a set of clauses S is a least general generalisation (lggD ) of S if and only if, for every generalisation G0 of S , G0 D G. The operation of computing an lggD underlies many ILP systems. Plotkin 13] has investigated the structure imposed by -subsumption. He has proven the following key properties:

Implication. If D1 -subsumes D2, then D1 j= D2. The opposite does not hold for selfrecursive clauses: let D1 = p(f (X )) _ :p(X ) and D2 = p(f (f (Y ))) _ :p(Y ) then D1 j= D2 but D1 does not -subsume D2. Therefore deduction using -subsumption is

not equivalent to implication among clauses. This also motivated research on inverting implication by Idestam-Almquist 7], Muggleton10], Lapointe and Matwin 8]. This property also shows that -subsumption is a meaningful structure on D for ILP systems. Indeed, typical ILP systems employ a set of positive examples P (represented by ground clauses) and aim at deriving a hypothesis H (represented by a set of clauses) that implies the facts, i.e., H j= P . Because of the implication property, this aim will be ful lled when all clauses in P are -subsumed by clauses in H . Equivalence. There exist di erent clauses that are equivalent under -subsumption, e.g., parent(X,Y) _ : mother(X,Y) _ : mother(X,Z) -subsumes parent(X,Y) _ : mother(X,Y) and vice versa. Because two clauses equivalent under -subsumption are also logically equivalent (i.e., by implication), learning systems should generate at most one clause of each equivalence class. For an extended discussion of equivalence, see Maher 9]. Reduction. To get around this problem, Plotkin de ned equivalence classes of clauses, and showed that there is a unique representative (up to variable renamings) of each equivalence class, which he named the reduced clause. A clause D is reduced if and only if there exists no literal l such that (D ;flg) and D are equivalent under -subsumption. Learning systems can get around the problem of equivalent clauses when working with reduced clauses only. An algorithm to reduce clauses directly follows from the de nition of a reduced clause: repeatedly delete a literal from a clause and check whether the resulting clause is -subsumed by the original clause if it is, apply the corresponding substitution, omit duplicate literals, and repeat the procedure if no literal can be deleted in this way, conclude that the clause is reduced. Lattice. The set of reduced clauses forms a lattice, i.e., any two reduced clauses have a unique lubD (least upper bound, which is the lggD ) and a unique glbD (greatest lower bound).The least general generalisation operator is the basic operation of many speci c to general ILP systems, e.g., Golem 12], Clint 1], ITVS 17] applied on ILP. 4

Plotkin has also given a procedure to compute the lggD of two clauses: the lggD of the terms f (s1 : : :  sn) and f (t1 : : : tn) is f (lggD (s1 t1) : : : lggD (sn  tn)). The lggD of the terms f (s1  : : : sn) and g(t1 : : : tm), with f 6= g, is the variable v, where v represents this pair of terms throughout. The lggD of two atoms p(s1 : : : sn) and p(t1 : : : tn) is p(lggD (s1 t1) : : : lggD (sn tn)), the lggD being unde ned when the sign or the predicate symbols are unequal. Finally, the lggD of two clauses D1 and D2 is then _ _ lgg (l  l ): D 1 2 l1 2set(D1 ) l2 2set(D2 )

For example, the lggD of father(tom,ann) _ : parent(tom,ann) _ : male(tom) _ : female(ann) and father(jef,paul) _ : parent(jef,paul) _ : male(jef) _ : male(paul) is father(X,Y) _ : parent(X,Y) _ : male(X) _ : male(Z). The equivalent reduced clause is father(X,Y) _ : parent(X,Y) _ : male(X). Innite Ascending Chains. There exist in nite strictly ascending chains in the above lattice, e.g., h(X1 ) _ : p(X1 X2) h(X1 ) _ : p(X1 X2) _ : p(X2  X3) h(X1 ) _ : p(X1 X2) _ : p(X2  X3) _ : p(X3 X4) :::

Innite Descending Chains. There also exist in nite strictly descending chains in the lattice, e.g., h(X1) _ : p(X1 X2) _ : p(X2  X1 ) h(X1) _ : p(X1 X2) _ : p(X2  X3 ) _ : p(X3 X4) _ : p(X4  X1 ) h(X1 ) _ : p(X1 X2 ) _ : p(X2  X3 ) _ : p(X3  X4 ) _ : p(X4 X5) _ : p(X5 X6 ) _ : p(X6 X7) _ : p(X7 X8) _ : p(X8 X1) :::

All clauses in both these in nite series -subsume the clause h(X) _ : p(X,X) and are -subsumed by the clause h(X) _ : p(X,Y).

4 Structural matching In this section, we map Plotkin's notion of -subsumption onto productions. We will show that the resulting structure is order-isomorphic to -subsumption on clauses. As a result the algorithms and results by Plotkin can be directly mapped to productions. Denition 6 A production C1 -subsumes a production C2 if and only if there exists a substitution such that set(C1) set(C2). We will write C1 C C2 to denote that C1 -subsumes C2. For example, 9X Y (circle(X ) ^ red(X ) ^ circle(Y ) ^ large(Y )) -subsumes circle(a) ^ red(a) ^ large(a) with substitution = fX=a Y=ag. A straightforward property of C , which could be used as an alternative de nition, is: 5

Lemma 1 Let C1 and C2 be two productions. Then, C1 C C2 if and only if FC (C1) D FC (C2). Let us now show that -subsumption imposes a meaningful structure on C . This follows from :

Lemma 2 Let C1 and C2 be two productions. If C1 C C2, then C2 j= C1. Proof: This will be proven as a special case of Lemma 3.

2

In structural matching examples are ground (i.e.,variable free) productions. Ground productions closely correspond to logical interpretations (which can also be represented by a conjuction of true and false ground facts). When using examples that are interpretations, Lemma 2 shows that -subsumption is doing precisely what is expected: all interpretations that are a model for a production C2 (i.e., that are covered by C2) are also a model for every production C1 that -subsumes C2. Thus, -subsumption provides a genuine generality relation on C . It is straightforward to see that the structure imposed by -subsumption on C is exactly the same as that imposed on D. This is because both productions and clauses are mapped to sets of literals when reasoning about -subsumption. Formally, we have: Theorem 1 The ordered set (C  C ) is order-isomorphic to the ordered set (D D ). Proof: We have to show that there is a one-to-one mapping from C to D, which is monotone. The mapping FC clearly is one-to-one. It is monotone because for any productions C1 and C2, C1 C C2 if and only if FC (C1) D FC (C2) (Lemma 1). 2 As a consequence, all results proven by Plotkin and all algorithms on clauses apply to productions as well. For learning algorithms the most important results are concerned with reduced descriptions and least general generalisations. We do not provide detailed descriptions of the resulting algorithms as they are essentially the same as Plotkin's, due to the uniform set representation. However, we can now easily de ne lggC in terms of lggD and the mappings FC and FD because of Theorem 1.

Corollary 1 Let C1 and C2 be two productions. Then, lggC (C1 C2) = FD (lggD (FC (C1) FC (C2))): Let us also illustrate that the resulting structure on the search space solves some of the remaining problems with structural matching.

Example Consider the following descriptions of scenes (from 6]). C1 = circle(a) ^ red(a) ^ large(a) C2 = circle(b) ^ circle(c) ^ red(b) ^ green(c) ^ small(b) ^ large(c) 6

G1 = 9X (circle(X ) ^ red(X )) G2 = 9X (circle(X ) ^ large(X )) G = 9X Y (circle(X ) ^ large(X ) ^ circle(Y ) ^ red(Y )) Hayes-Roth and McDermott's algorithm would return both G1 and G2 as maximally speci c generalisations of C1 and C2. As a consequence, their system has to explore two di erent paths through the search space. Similar problems exist for the framework of Steven Vere 18]. In contrast, according to our notion of generality G is the unique least general generalisation of C1 and C2. Furthermore, G is reduced. 3 Another problem solved adopting the ordered set (C  C ) is that it enables to handle function symbols.

5 Relative -subsumption Plotkin 15] extended the notion of -subsumption to -subsumption relative to a theory (i.e., a nite set of clauses).

Denition 7 A clause D1 -subsumes a clause D2 relative to a theory T , denoted D1 DT D2, if and only if there exists a C-derivation of a clause R from D1 and T such that R D D2. A C-derivation of R from D1 and T is a resolution derivation of R from D1 and T in which the clause D1 is used exactly once. Our de nition slightly di ers from Plotkin's de nition in Plotkin's de nition a resolution derivation is a C-derivation if and only if the clause D1 is used at most once. Hence, our notion of relative -subsumption di ers from Plotkin's w.r.t. clauses that logically follow from the theory only. With our de nition a clause is a lggD of a set of clauses if and only if it is a rlggD of this set of clauses w.r.t. the empty theory. This is not the case with Plotkin's de nition. However, the di erence between the two de nitions is not important since clauses that follow from the theory only are usually not considered in inductive learning. Note that if a clause D1 -subsumes a clause D2, then it also -subsumes D2 w.r.t. a theory T , by choosing R = D1.

Denition 8 A clause G is a relative generalisation of a set of clauses S = fD1 : : : Dn g w.r.t. a theory T if and only if, for every 1  i  n, G DT Di . A relative generalisation G of a set of clauses S w.r.t. a theory T is a relative least general generalisation (rlggDT ) of S if and only if, for every relative generalisation G0 of S , G0 DT G.

The idea of relative -subsumption is that one can take into account a background theory when learning. Relative -subsumption has similar properties as -subsumption.

Implication. If a clause D1 -subsumes a clause D2 relative to a theory T , then T fD1 g j= D2 . This property shows that relative -subsumption imposes a meaningful structure on D for ILP systems in the presence of a background theory. Indeed, typical ILP systems 7

employ a set P of positive examples (represented by ground de nite clauses) and a theory T , and aim at deriving a hypothesis H that implies the clauses , i.e., T H j= P . Because of the implication property, the aim of ILP will be ful lled when all clauses in P are -subsumed by clauses in H relative to T . Equivalence, Reduction and Lattice. Because relative -subsumption is directly derived from -subsumption, the structure (D DT ) is very similar to (D D ). Indeed, Plotkin 15] also de nes notions of equivalence, reduction, and relative least general generalisation w.r.t. a theory T . Unfortunately, the theoretical results for generalisation relative to a theory are not as good as for generalisation without respect to a theory. In particular, in general there does not exist an rlggDT of a set of clauses w.r.t. a theory. This is due to the fact that a clause by de nition is a nite disjunction of literals, while a relative least general generalisation may include an in nite number of literals. However, whenever there exists an rlggDT it is unique up to equivalence. The rlggDT of a set of de nite clauses fD1 : : : Dn g w.r.t. a theory T can be computed in the following way 11] 1. For each clause Di (1  i  n) collect all ground unit clauses that are derivable from T and the complement Di of Di, and construct a clause Di0 that is the complement of this set of unit clauses. Then the rlggDT of fD1 : : : Dn g w.r.t. T can be found by computing the lggD of fD10  : : : Dn0 g.

6 Relative Structural Matching Kodrato and Ganascia 4], and Vrain 20] have all given de nitions of structurally matching two productions in the presence of background knowledge, thereby building on Vere's original framework. However, similarly as Vere's original framework, these techniques su er from a number of problems, which include the fact that there may be multiple relative most speci c generalisations, and diculties in handling functors. The question thus arises as to whether it is possible to map Plotkin's structure of relative -subsumptions onto the set of productions. One way to de ne relative subsumption for productions would be to map Plotkin's de nition directly onto productions. As this would involve modifying the proof-procedure, this would be rather complicated. Therefore, we will rather upgrade the alternative de nition of -subsumption for production given in Lemma 1. This results in the following de nition :

Denition 9 A production C1 -subsumes a production C2 relative to a theory T , denoted C1 CT C2, if and only if FC (C1) DT FC (C2). First note that in this de nition T is also a clausal theory. Second, if T is empty, this de nition again reduces to -subsumption for productions. Let us again show that this structure is meaningful for concept-learning when using interpretations or ground productions as examples and a clausal background theory:

Lemma 3 Let C1 and C2 be productions and T a theory. If C1 CT C2 then T ^ C1 j= C2. 1

At present we are trying to prove that the property also holds for full clausal logic.

8

Proof: C1 CT C2 i FC (C1) DT FC (C2) only if (because of the relation of DT to implication) T ^ FC (C1) j= FC (C2) only if T ^ :FC (C2) j= :FC (C1) only if T ^ C2 j= C1 (because :FC (C ) = C for all productions C ).

2

This lemma reduces to Lemma 2 when T is empty. Again it is straightforward to show that

Theorem 2 (C  CT ) and (D DT ) are order-isomorphic for all theories T . And again this allows us to compute the rlggC , whose de nition is analogous, using rlggD .

Corollary 2 Let C1 and C2 be two productions and T a theory. Then rlggCT (C1 C2) = FD (rlggDT (FC (C1) FC (C2))): This also means that the rlggCT inherits all the properties of the dual rlggDT . Using this operation again solves many of the problems with the operations proposed by Vere 18, 19], Kodrato and Ganascia 4] and Vrain 20], in particular, uniqueness, use of functors and also handling inequalities. To handle inequalities, it suces to add the clauses di erent(a b) and di erent(a c), etc. for all di erent combinations of known constants to the theory. To illustrate the use of background knowledge we take the following example of Kodrato and Ganascia 4]. The example also illustrates that the use of functors is possible.

Example Consider C1 = red(a) ^ red(b) ^ shape(b square), and C2 = red(c) ^ shape(d square). The background knowledge T contains the \color hierarchy": color(X,red) _ : red(X). color(X,green) _ : green(X). color(X,yellow) _ : yellow(X). Also known is the fact that each object has a color. Kodrato and Ganascia express this as: 8X 9Y : color(X Y ). In clausal form we can make use of a Skolem functor col=1, and express this as the clause color(X,col(X)) in T . Finally T contains the above mentioned clauses that express that di erent constants represent di erent objects. The production red(X ) ^ red(Y ) ^ square(Z ) ^ di erent(X Z ) is an lggC of the two productions C1 and C2. The \reduced" form of this production is: G1 = red(X ) ^ square(Z ) ^ di erent(X Z ). The production red(X )^red(Y )^square(Z )^di erent(X Z )^color(U V )^color(W Z ) is an rlggCT of C1 and C2 with respect to the given background knowledge. The 9

\reduced" form of this production w.r.t. -subsumption is: G2 = red(X ) ^ square(Z ) ^ di erent(X Z ) ^ color(W Z ). Kodrato and Ganascia obtain G1 and G2 by structurally matching C1 and C2 in two di erent ways. Whereas they call these two possible generalisations between which one cannot choose \because they have an equal number of variables", the relationship of structural matching with Plotkin's work reveals that G1 subsumes G2 w.r.t. -subsumption. This means that G2 is a more speci c generalisation, which could therefore be preferred over G1. Note that, unlike in structural matching, this has the advantage that no further heuristics (e.g., the counting of variables) is needed, since the relative least general generalisation is unique modulo reduction. Furthermore, whereas structural matching explicitly records all variable bindings and controls the introduction of new variables heuristically, Plotkin's algorithm deals with all these problems in a uniform way. 3

7 Conclusion We have shown the order-isomorphisms between Plotkin's framework and the framework of structural matching. This isomorphism implies that the results obtained in one framework can be transferred to the other. This in turn leads to an improved structure on the space of productions for use in concept-learning. Finally, we hope that this work will stimulate further cross-fertilization between structural matching and ILP. Some challenging remaining research issues include: can we adapt rlgg based ILP systems, such as Muggleton and Feng's Golem for use in structural matching?

Acknowledgements Luc De Raedt is supported by the Belgian National Fund for Scienti c Research. Peter Idestam-Almquist is supported by the Swedish Research Council for Engineering Sciences (TFR). Luc De Raedt and Peter Idestam-Almquist are also supported by the ESPRIT Basic Research Project No. 6020 on Inductive Logic Programming.

References 1] L. De Raedt. Interactive Theory Revision: an Inductive Logic Programming approach, Academic Press, 1992. 2] L. De Raedt and M. Bruynooghe. A theory of clausal discovery. In Proceedings of the 13th International Joint Conference on Articial Intelligence, pages 1058{1063. Morgan Kaufmann, 1993. 3] L. De Raedt and S. Dzeroski. First order jk-clausal theories are PAC-learnable. Articial Intelligence, 70:375{392, 1994. 10

4] J.G. Ganascia and Y. Kodrato . Improving the generalization step in learning. In R.S Michalski, J.G. Carbonell, and T.M. Mitchell, editors, Machine Learning: an articial intelligence approach, volume 2, pages 215{241. Morgan Kaufmann, 1986. 5] R. Gemello, F. Mana, and L. Saitta. Rigel: An inductive learning system. Machine Learning, 6:7{35, 1991. 6] F. Hayes-Roth and J. McDermott. An interference matching technique for inducing abstractions. Communications of the ACM, 21:401{410, 1978. 7] P. Idestam-Almquist. Generalisation under implication using or-introduction. In Proceedings of the 6th European Conference on Machine Learning, volume 667, pages 56{64. Lecture Notes in Arti cial Intelligence, 1993. 8] S. Lapointe and S. Matwin. Sub-uni cation: a tool for ecient induction of recursive programs. In Proceedings of the 9th International Workshop on Machine Learning. Morgan Kaufmann, 1992. 9] M.J. Maher. Equivalences of logic programs. In Proceedings of Third International Conference on Logic Programming, Berlin, 1986. Springer. 10] S. Muggleton. Inverting implication. Articial Intelligence, 1994. To appear. 11] S. Muggleton. Inductive Logic Programming. New Generation Computing,8:295-317, 1991. 12] S. Muggleton and C. Feng. Ecient induction of logic programs. In Proceedings of the 1st conference on algorithmic learning theory, pages 368{381. Ohmsma, Tokyo, Japan, 1990. 13] G. Plotkin. A note on inductive generalization. In Machine Intelligence, volume 5, pages 153{163. Edinburgh University Press, 1970. 14] G. Plotkin. Automatic Methods of Inductive Inference. PhD thesis, Edinburgh University, 1971. 15] G. Plotkin. A further note on inductive generalization. In Machine Intelligence, volume 6, pages 101{124. Edinburgh University Press, 1971. 16] J.R. Quinlan. Learning logical de nitions from relations. Machine Learning, 5:239{266, 1990. 17] G. Sablon, L. De Raedt, and M. Bruynooghe. Iterative Versionspaces. Articial Intelligence, 69:393-409, 1994. 18] S.A. Vere. Induction of concepts in the predicate calculus. In Proceedings of the 4th International Joint Conference on Articial Intelligence, pages 282{287. Morgan Kaufmann, 1975. 11

19] S.A. Vere. Induction of relational productions in the presence of background information. In Proceedings of the 5th International Joint Conference on Articial Intelligence, Morgan Kaufmann, 1977. 20] C. Vrain. Ogust: A system that learns using domain properties expressed as theorems. In Y. Kodrato and R.S. Michalski, editors, Machine Learning: an articial intelligence approach, volume 3, pages 360{381. Morgan Kaufmann, 1990.

12