Knowledge Representation, Search Problems and Model ... - CiteSeerX

1

Knowledge Representation, Search Problems and Model Expansion DAVID G. M ITCHELL AND E UGENIA T ERNOVSKA

ABSTRACT. Arguments for logic-based knowledge representation often emphasize the primacy of entailment in reasoning, and traditional logic-based formulations of AI tasks were frequently in terms of entailment. More recently, practical progress in satisfiability-based methods has encouraged formulation of problems as model finding. Here, we argue for the formalization of search problems, which abound in AI as well as other areas, as a particular form of model finding called model expansion. An important conceptual part of this proposal is the formalization of the problem instance as a structure, rather than as a formula. Adopting this view leads naturally to taking descriptive complexity theory as the starting point for developing a theory of languages for representing search problems. We explain the formalization of search as model expansion, and the reasons we consider it an appropriate basis for such a theory. We emphasize the role of model expansion for first order logic, with extensions, in specifying NP search problems, and describe the formalization of arithmetic in this context.

This paper is dedicated to Hector J. Levesque.

1

Introduction

Search Problems as Model Finding A classical argument in favour of explicit logic-based knowledge representation invokes the primacy of entailment in reasoning, and the fact that deduction can be automated because of the correspondence between the semantic notion of entailment and the syntactic notion of proof. As important as this is, not all reasoning involves deduction, and indeed it seems more natural to view search problems, in which we are asked to exhibit an object satisfying certain properties, as model finding rather than theorem proving. In this paper, we expand on a proposal first made by the authors in [Mitchell and Ternovska 2005], toward a logicbased theory of languages for representing search problems. Traditional AI practice frequently cast problem solving as entailment, or theorem proving, as in the early work of Green [1969]. There, a user axiomatizes the claim that a solution exists, and a theorem prover is used to show the claim is true. As in much Prolog-based problem solving, a side effect of proving the theorem is construction of a term which describes a solution. Selman, Levesque and Mitchell [1992], in the paper which introduced the GSAT algorithm for SAT, argued that many AI problems traditionally cast as theorem proving should be re-formulated as model finding, concluding that in certain cases “model finding has a clear advantage over theorem proving and may lead us to AI methods that scale up more gracefully in practice”. This has turned out to be true, and perhaps more generally than suggested in that paper.

David G. Mitchell and Eugenia Ternovska

Satisfiability-based techniques have since become widely successful and influential, providing computational engines for a variety of reasoning tasks. However, they did not initially offer much help with representation. For example, normal practice in SAT-based solving involves explicitly designing and implementing a reduction from the problem at hand to SAT, with each CNF formula describing the solutions for a particular instance of the problem. In 1999, three papers introduced logic programming-based languages for representing problems, with a distinct separation of problem description and individual instances. Cadoli et al [1999] proposed an extension to Datalog programs, called NP-Spec. Niemela [1999] and Marek and Truszczynski [1999] independently proposed representing search problems as normal logic programs under the stable model semantics, the approach now known as answer set programming (ASP). As described in [Marek and Truszczynski 1999], a specification for search problem Π consists of a normal logic program PΠ , and an instance I is represented by a set of atoms edbΠ (I). A solution to I is obtained by finding a stable model of the program PΠ ∪ edbΠ (I). Thus [Marek and Truszczynski 1999] point out the importance of separation of specification from data (instance), but both are formulas (logic programs), and the distinction is in a sense one of convention. In [Mitchell and Ternovska 2005], the authors proposed the logical task of model expansion as a formalization of search problems upon which to develop a theory of languages for specifying search problems (and systems to support these languages in practice). An important conceptual element of this proposal is the formal distinction of instances and specifications in which instances are taken to be structures. In the theory of computing, a search problem is a binary relation on strings, defining the solutions for each instance. The strings are assumed to be encodings of objects which, viewed abstractly, and in the language of logic, are structures. Taking instances to be structures allowed us to depart from the use of Herbrand models, which are at the heart of the logic programming-based methods mentioned above. From a knowledge representation point of view, taking instances as structures amounts to committing to what some part of the world is like, whereas the primary value of using formulas is to allow multiple models, that is, to give approximate descriptions of the world. The abstraction of search problems as model expansion brings us directly to descriptive complexity theory, the study of expressiveness of languages in terms of the computational complexity of classes of structures definable in a language [Immerman 1999] (see also [Gr¨adel 2007]). Knowing the expressiveness of a language, in this sense, is essential to good implementation and to appropriate selection of a language for a given task. Brachman and Levesque, in their paper “Expressiveness and tractability in knowledge representation and reasoning” [1987], demonstrated the inherent tradeoff between complexity and expressiveness in KR languages. While such a tradeoff was well established in theoretical work, the paper served as a wake-up call to AI researchers. Later, Levesque and Lakemeyer [2001] emphasized the interplay between representation and reasoning and the importance of the “study of how knowledge can at the same time be represented as comprehensively as possible and be reasoned with as effectively as possible.” Taking descriptive complexity theory as central in the study of languages provides a basis for developing languages with controlled expressiveness, to support effective representation and reasoning. The Theorems of Cook and Fagin Cook’s theorem [Cook 1971], which states that every problem in the complexity class NP can be reduced in polynomial time to propositional satisfiability (SAT), suggested the

Search Problems as Model Expansion

following scheme for solving NP search problems (also known as problems in FNP): 1. For each problem Π, implement a polytime reduction f to SAT; 2. Given an instance of Π, use a SAT solver to find a satisfying assignment for f (Π), and thus a solution for Π, if there is one. For some time this scheme was widely considered an idea of purely theoretical interest. However, progress in building and applying SAT solvers and related technology, such as solvers for ASP and satisfiability modulo theories (SMT, see e.g. Barret et al [2008]), has been so significant that researchers have begun to refer to NP as “the new tractable”. Fagin’s theorem [Fagin 1974] says that the problems which can be axiomatized in the existential fragment of second order logic (∃SO) are exactly those in NP. This result shows that the first step above, designing and implementing a reduction, can be replaced with declarative modelling. That is, rather than implement a reduction from Π to SAT, we may axiomatize Π with an ∃SO formula φΠ . An instance of Π is a structure A, and deciding if A has a solution is then equated with checking if A |= φΠ . Since this is in NP, it can be done by reduction to SAT. In particular, we can find solutions to A by grounding: 1. Produce a ground formula Gnd(φΠ , A) representing the solutions to A. 2. Use a SAT solver to find a satisfying assignment for Gnd(φΠ , A), if there is one. It is easy to verify that there are grounding algorithms which, for every fixed formula φ, produce the grounding Gnd(φ, A) in time polynomial in the size of A. An implementation of such an algorithm provides a universal automated reduction to SAT for problems in NP. That is, for each fixed φΠ , the unary function Gndφ (A) = Gnd(φ, A) is essentially a polytime reduction from Π to SAT. The theorems by Cook and Fagin suggest a conceptually elegant general method for solving NP search and decision problems, a fact which has certainly been observed by others. Our proposal amounts to taking this method seriously as the starting point for a general theory underlying new and existing languages and systems for representing and solving search problems. There are many foundational and practical issues between the idea and its realization, and also many issues in development and use of actual systems. In this paper, we describe early steps in developing the theory. In the Section 2, we discuss the formalization of search as model expansion, and in particular NP-search as model expansion for first order logic. In Sections 3 and 4, we discuss several extensions of FO, which bring us closer to practical languages for representing NP search problems. Section 5 discusses a number of implemented systems for representing and solving search problems, and Section 6 briefly concludes the paper.

2

Search Problems as Model Expansion

We begin this section with a review of some necessary definitions. A vocabulary is a set τ of relation and function symbols, each with an associated arity. Constant symbols are zeroary function symbols. A structure A for vocabulary τ (or, τ -structure) is a tuple containing a universe or domain A, and a relation (function) for each relation (function) symbol of τ . If R is a relation symbol of vocabulary τ , the relation corresponding to R in a τ -structure A is denoted RA . For example, we write A A A A = (A; R1A , . . . RnA , cA 1 , . . . ck , f1 , . . . fm ),

David G. Mitchell and Eugenia Ternovska

where the Ri are relation symbols, fi function symbols, and constant symbols ci are 0-ary function symbols. For a formula φ, we write vocab(φ) for the collection of exactly those function and relation symbols which occur in φ. Let σ and τ be vocabularies, with σ ⊂ τ , and let A be a σ-structure. A τ -structure B is an expansion of A to τ if A = B (i.e., their domains are the same), and for every relation symbol R and in σ, RA = RB and for every function symbol f in σ, f A = f B . Definition: Model Expansion (MX) For any logic L, (e.g., FO, SO, FO(ID), FO(LFP), an ASP language, etc.,) the L model expansion problem, L-MX, is: Instance: A pair hφ, Ai, where 1) φ is an L formula, and 2) A is a finite σ-structure, for vocabulary σ ⊆ vocab(φ). Problem: Find a structure B such that 1) B is an expansion of A to vocab(φ), and 2) B |= φ. The vocabulary ε := vocab(φ) \ σ, of symbols not interpreted by the structure, is called the expansion vocabulary. Symbols of the expansion vocabulary are second-order free variables. MX, Fixed Formula Setting We consider two natural settings for MX, the combined setting, in which we take both the formula and the structure as input, and the fixed formula setting, in which we consider a fixed formula and take structures as inputs. We will briefly discuss the combined setting below, but for most of what follows we focus on the fixed formula setting. In this setting, we consider a fixed formula vocabulary σ, and take only a σ-structure as input. The setting corresponds to the notion of data complexity as defined in [Vardi 1982]. The fixed formula setting provides a natural formalization of search problems. Here, φ is a problem description or specification, and a problem instance is a finite σ-structure A. The formula specifies the relationship between an instance and its solutions. Finding a solution amounts to finding a structure B which is an expansion of A to the vocabulary of φ, and satisfies φ. A solution for instance A is given by the interpretations of the expansion vocabulary in structure B. EXAMPLE 1 (K-Colouring as FO MX, formula fixed). The problem is: Given a graph and a set K of colours, find a proper K-colouring of the graph. We will have two sorts, vertices V and colours K. The instance vocabulary is σ := {E}. The expansion vocabulary ε := {colour}, where colour is a function from verticies to colours. The formula is: φ := ∀x∀y (E(x, y) ⊃ (colour(x) 6= colour(y))), where colour is a free second-order variable. An instance is a structure A = (V ∪ K; E A ), and a solution is an interpretation for colour such that A

}| { z (V ∪ K; E A , colourB ) |= φ. | {z } B

Search Problems as Model Expansion

Assume a standard encoding of languages as classes of structures as done in descriptive complexity (see, e.g., [Libkin 2004]). FO model expansion can define exactly the same classes of structures that ∃SO can, which immediately implies the following theorem. THEOREM 2 ([Fagin 1974]). The first-order model expansion problem in the data complexity setting captures NP. The property means that FO model expansion is in NP, and moreover that every (search) problem in NP can be represented as FO MX, with a fixed formula. Thus, FO MX is a universal framework for representing NP-search problems. Combined with an automated reduction from FO model expansion to SAT, which may be provided by a grounding algorithm, we have a universal method for modelling and solving NP-search problems. While FO MX captures NP, for practical purposes it must be extended with features to make representation of complex domains convenient. In [Mitchell and Ternovska 2005], we proposed using FO augmented with non-monotonic inductive definitions, as described in Section 3. In other work, we extend FO with arithmetic, including aggregate operators, as described in Section 4. It can be seen that model expansion is the task underlying a wide range of practical tools for solving search problems. For example, we have shown that MX underlies the highlevel specification language E SSENCE (see [Mitchell and Ternovska 2008]), although the logic required to have the same expressive power as unrestricted E SSENCE is much more expressive than FO. REMARK 3. Since FO MX can specify the same problems that ∃SO can, one may question the need for the term MX. We use it because we are interested in MX for a variety of logics other than FO, and because for each logic L there are several tasks of interest. The Importance of Capturing Complexity Classes for Declarative Programming We claimed, in [Mitchell and Ternovska 2005], that an important property for a specification language is capturing a complexity class. To illustrate, if a language L captures NP, we know that: (a) L can express every problem in NP – which gives the user an assurance of universality of the language for the given complexity class, (b) no more than NP can be expressed – thus solving can be achieved by universal polytime transformation to an NP-complete problem, for example by polytime grounding to SAT. The capturing property is of clear practical importance. This does not mean that specification languages should necessarily have restricted power, but that we should understand which fragments of an expressive language have restricted power, and understand the possible consequences of using more expressive fragments. While many users would not be familiar with complexity theory and its practical implications, they could still be provided with fragments of specification languages corresponding to different complexity classes, and advised to use a “safe” fragment when possible. Generalizing to Other Complexity Classes Results analogous to Fagin’s theorem followed for other complexity classes, most notably for P [Immerman 1982; Vardi 1982; Livchak 1983]. As with Fagin’s theorem and NP, these results show that logics can be seen as modelling languages for problems in corresponding

David G. Mitchell and Eugenia Ternovska

complexity classes, and provide the basis for universal solving methods for problems in these classes. The following properties for MX in the data complexity setting are immediate from the definition of MX and results in [Gr¨adel 1992] and [Stockmeyer 1977]. ◦ On ordered structures, FO universal Horn MX captures P, ◦ On ordered structures, FO universal Krom MX captures NL, ◦ Π1k−1 MX captures the ΣP k level of the Polynomial Hierarchy. Notice the requirement of ordered structures above. Ordered structures are those with a total ordering on domain elements. For precise details, see [Ebbinghaus and Flum 1995] or [Libkin 2004]. Intuitively, the order is necessary to mimic the computation of a Turing machine on a tape. While for NP the order can be represented by an existentially quantified second-order variable, it is conjectured that there is no logic capturing the class P on arbitrary structures. The conjecture is due to Gurevich, and the positive resolution of this conjecture (i.e., a proof that no such logic exists) would imply P6= NP. For further discussion and references see [Immerman 1999; Libkin 2004; Gr¨adel 2007]. MX, Combined Setting In this setting, both formula φ and the structure A are part of the instance. The setting corresponds to the notion of combined complexity [Vardi 1982]. Here, we may still have a formula which is fixed for all instances, but an instance will consist of a formula together with a finite structure (which may be just the universe). The expansions must satisfy the conjunction of the two formulas. We have the following. THEOREM 4. First-order model expansion, in the combined setting, is NEXPTIMEcomplete. This property is equivalent to a result in [Vardi 1982], and can be shown by an easy reduction from Bernays-Sch¨onfinkel satisfiability or combined complexity of ∃SO over finite structures, as described in [Mitchell and Ternovska 2005]. Combined setting is very useful for knowledge representation when one needs to represent a part of an instance by a formula. This is especially convenient when non-determinism is involved. In [Mitchell, Ternovska, Hach, and Mohebali 2006], we give an example of a planning problem represented as MX in the combined setting. Model Expansion in the Context of Related Tasks Model expansion is closely related to the more studied tasks of model checking (MC) and finite satisfiability. For a given logic L, 1. Model Checking (MC): given (A, φ), where φ is a sentence in L and A is a finite structure for vocab(φ), does A  φ? 2. Model Expansion (MX): given (A, φ), where φ is a sentence in L, A is a finite σstructure and σ ⊆ vocab(φ), is there B for vocab(φ) which expands A and B  φ? 3. Finite Satisfiability: given a sentence φ in L, is there a finite A for vocab(φ) such that A  φ? In model checking, the entire structure is given and we ask if the structure satisfies the formula; in model expansion part of a structure is given and we ask for an expansion satisfying the formula; in finite satisfiability we ask is any finite structure satisfies the formula. Thus,

Search Problems as Model Expansion

for any logic L, the complexity of model expansion — in both fixed formul and combined cases — lies between that of the other two tasks: MC(L) ≤ MX(L) ≤ Satisfiability(L). In the case of FO, we have the following. In the fixed formula (data complexity) setting FO MC is in AC0 : capturing AC0 requires extending the logic, for example to FO(