Logical Characterizations of Complexity Classes 1 Introduction 2

Logical Characterizations of Complexity Classes Iain A. Stewart Department of Mathematics and Computer Science Leicester University Leicester LE1 7RH, U.K. e-mail : [email protected]

1 Introduction Prior to 1974, results in nite model theory had appeared only sporadically, e.g., Trakhtenbrot's Theorem [74] and the 0-1 law for rst-order logic [23]: there had been no real concerted research eort. The main reason for this lack of development was that many of the fundamental results of model theory, e.g., the Completeness and Compactness Theorems, fail when restricted to nite structures (see [27]); and consequently the nite case was deemed by many to be not particularly interesting. However, Fagin's characterization [19] of the complexity class NP as the class of problems de nable in existential second-order logic exhibited a striking link between the computational complexity of a problem and whether that problem could be de ned in some logic; and it was this link between computational complexity and nite model theory which stimulated the explosive growth in the latter subject that we have witnessed since the late seventies. It is my intention in these notes to examine this link in more detail; and especially the logical characterization of complexity classes. These notes and the references therein are by no means intended to be exhaustive: our aim is merely to highlight some important results, to hint at some of the proof techniques employed and to provide pointers to other sources. A general reference for basic de nitions and more information is [17]. Throughout, all our structures are nite of size at least 2, a structure of size n always has universe f0; 1; : : : ; n ? 1g and our signatures consist of relation and constant symbols (unless otherwise stated).

2 Beyond rst-order logic: second-order logic From a complexity-theoretic point of view, the expressive power of rst-order logic, FO, is somewhat limited: it is an easy task to show that any problem (i.e., isomorNotes for lectures presented at the Ninth European Summer School in Logic, Language and Information , Aix-en-Provence, August 1997. 

1

phism closed set of structures over some signature) de nable in rst-order logic can be accepted by a logspace deterministic Turing machine, i.e., it is in the complexity class L. Furthermore, there are problems in L that are not de nable in rst-order logic. For example, a simple Ehrenfeucht-Frasse game [17, Chapters 1 & 2] yields that the problem, PARITY, over the empty signature and which consists of all those structures with an even number of elements, is not de nable in rst-order logic. In order to try and relate logical expressibility with computational complexity, which is our goal, we need somehow to increase the expressive power of rst-order logic. A natural consideration is second-order logic , SO. Whereas in rst-order logic, quanti cation is over the elements of a structure's domain, in second-order logic we can quantify over sets of elements - and even over sets of pairs, triples, and so on, of elements. For example, consider the following sentence 2 SO over the empty signature: 9A9F (8x9y(F (x;y) ^ 8z(F (x; z) ) y = z)) ^8x8y8z((F (x; z) ^ F (y; z)) ) x = y) ^8x8y(((A(x) ^ F (x; y)) ) :A(y)) ^ ((:A(x) ^ F (x; y)) ) A(y)))): Here, A is a new relation symbol of arity 1, F is a new relation symbol of arity 2 and the quanti cation is interpreted as \there exists a set A of elements and there exists a set F of ordered pairs of elements such that: : :" (of course, \8A : : :", for example, would be interpreted as \for every set A of elements: : :"). A structure (over the empty signature) satis es if and only if its universe has even size. Consequently, allowing second-order quanti cation does allow us to describe more problems than are describable using rst-order logic. In fact, from a complexity-theoretic point of view, some \dicult" problems can be described in SO. For example, let 2 = hE i, where E is a binary relation symbol. We can equate any 2-structure S with an undirected graph via \there is an edge (u; v) in S if and only if E (u; v) or E (v; u) holds in S ". The problem 3COL over 2 is de ned as consisting of all graphs that can be properly 3-coloured (that is, one of 3 distinct colours can be assigned to each vertex so that if any two vertices are joined by an edge then they must be coloured dierently), and the following second-order sentence describes 3COL: 9R9W 9B (8x((R(x) _ W (x) _ B (x)) ^ :(R(x) ^ W (x)) ^:(R(x) ^ B (x)) ^ :(W (x) ^ B (x))) ^8x8y((E (x;y) _ E (y; x)) ) (:(R(x) ^ R(y)) ^ :(W (x) ^ W (y)) ^:(B (x) ^ B (y))))); where R, W and B are new relation symbols of arity 1. Recall that the problem 3COL is well-known to be NP-complete via logspace reductions (see [22]). In a seminal result, Fagin proved what is now known as Fagin's Theorem [19]. Theorem 1 (Fagin's Theorem) [19] A problem is in NP if and only if it can be de ned in existential second-order logic. 2

Existential second-order logic , 11 , is the fragment of SO where every formula consists of an existentially quanti ed pre x of (new) relation symbols followed by a rst-order formula.

Proof (Rough sketch ) One half of Fagin's Theorem, namely that every problem de nable in 11 is in NP, is straight-forward: one simply \guesses" relations (i.e.,

a polynomial number of bits) for the existentially quanti ed relation symbols in a sentence of 11 and then veri es deterministically in polynomial-time that the rstorder part of the sentence holds with respect to these guessed relations and the structure in hand. The other half is where the essence of the proof lies. Let M be an O(nk ) non-deterministic Turing machine. Such a machine can always be taken to operate in two phases: rst, there is the guessing phase where a string of 0's and 1's is non-deterministically guessed (on the work-tape); and second, there is the checking phase where the machine operates deterministically but uses the guessed string of symbols (the time taken is summed over the two phases). The guesses of the machine M can be encoded as a sequence of relations. For example, suppose we wanted to encode 2n3 guessed symbols, where n is the size of our input structure. We would use two relations R1 and R2 of arity 3 via: the rst guessed symbol is 1 if and only if R1(0; 0; 0) holds; the second guessed symbol is 1 if and only if R1(0; 0; 1) holds; ::: the n3th guessed symbol is 1 if and only if R1(n ? 1; n ? 1; n ? 1) holds; the (n3 + 1)th guessed symbol is 1 if and only if R2(0; 0; 0) holds; the (n3 + 2)th guessed symbol is 1 if and only if R2(0; 0; 1) holds; ::: the 2n3th guessed symbol is 1 if and only if R2(n ? 1; n ? 1; n ? 1) holds. Next, we encode the contents of M 's work-tape using new relations. For example, suppose that M has time complexity nk . Then the contents of the work-tape at every instant can be represented using 2 relation symbols T1 and T2 of arity 2k via: at time instant t 2 f0; 1; : : : ; n ? 1gk , work-cell u 2 f0; 1; : : : ; n ? 1gk contains 0 if neither T1(t; u) nor T2(t; u) holds; at time instant t 2 f0; 1; : : : ; n ? 1gk , work-cell u 2 f0; 1; : : : ; n ? 1gk contains 1 if both T1(t; u) and T2(t; u) hold; at time instant t 2 f0; 1; : : : ; n ? 1gk , work-cell u 2 f0; 1; : : : ; n ? 1gk contains the blank symbol if exactly one of T1(t; u) and T2(t; u) holds (here, we identify, for example, the set of tuples f0; 1; : : : ; n ? 1gk with the set f0; 1; : : : ; nk ? 1g). Similarly, the state and the head positions of M can be encoded using new relations. Finally, we devise an existential second-order sentence to say that there exists a set of relations corresponding to the non-deterministic guesses and the instantaneous descriptions of M so that the guessed relations \ t together" to form a description 3

of an accepting computation of M on some input structure (there is an analogy here with the proof of Cook's Theorem: see, e.g., [22]). We have been rather slapdash in implicitly assuming that the universe of our input structure is linearly ordered in the canonical way. What we should really say is that there exists a linear ordering on the universe of our input structure and that the least element of this linear ordering is 0, the successor of 0 is 1, and so on. It should be clear that we do not need a speci c linear ordering and that any linear ordering will do. We can de ne a linear ordering by existentially quantifying a new binary relation symbol and saying, in rst-order logic, that this binary relation is a linear order (we include this remark in this sketch proof as linear orders play an important role in the logical characterization of complexity classes, as we shall see later on). For each positive natural number i, let 1i (resp. 1i ) denote those problems de nable by sentences of SO consisting of a quanti ed pre x p of relation symbols, followed by a rst-order sentence, where the rst quanti ed relation symbol of p is quanti ed by 9 (resp. 8) and there are i ? 1 alternations of quanti ers as one works down p (so, for example, a problem in 13 can be de ned by a sentence of the form: 9R1;1 : : : 9R1;k1 8R2;1 : : : 8R2;k2 9R3;1 : : : 9R3;k3 ; where each Ri;j is a new relation symbol and is rst-order). Let pi denote the ith complexity class of the Polynomial Hierarchy , PH (see, e.g., [22]).

Theorem 2 [71]

For each positive natural number i, a problem is in pi if and only if it can be de ned by a sentence of 1i .

Hence, the problems de nable in SO are identical to those in the Polynomial Hierarchy, i.e., SO captures PH. Fagin's Theorem came about after Fagin's work on Asser's Problem : \Is the set of spectra closed under complementation?", where a spectrum is the set of cardinalities of the nite models of a rst-order sentence (this problem was rst posed by Asser in 1955). Fagin's research led him to consider generalized spectra , i.e., problems de nable in existential second-order logic. Whilst Asser's Problem remains unresolved, it does have links with complexity theory.

Theorem 3 [19, 37] A set of natural numbers is a spectrum if and only if it can be solved in NEXPTIME (which consists of those decision problems accepted by a non-deterministic Turing machine in O(2kn ) time, for some constant k). Consequently, Asser's Problem has a positive solution if and only if NEXPTIME = co-NEXPTIME (and the Generalized Asser Problem : \Is the class of generalized spectra closed under complementation?", has a positive solution if and only if NP = co-NP). 4

Note that Fagin's Theorem presents the possibility that model-theoretic techniques might be used to solve complexity-theoretic questions. Indeed, there is a (generalized Ehrenfeucht-Frasse) game (see [20]) which captures de nability in 11, and consequently there is the potential for proving that problems (in particular, problems in co-NP) are not de nable in 11, and so are not in NP. However, the combinatorics behind playing such games is prohibitively complicated at present. In the context of complementation, progress has been made regarding the monadic fragment of 11, mon11 (or mon NP); that is, where all quanti ed relation symbols in any formula are necessarily unary (the problem 3COL, for example, is in mon11). Using generalized Ehrenfeucht-Frasse games, Fagin [20] proved that mon11 is not closed under complementation by showing that the problem CONNECTIVITY, consisting of all connected graphs, is not de nable in mon11 but is de nable in mon11 (or mon co-NP).

Some related research

The logic 11 has a normal form, called Skolem Normal Form : every problem de nable in 11 can be de ned by a sentence of the form:

9R1 : : : 9Ra8x1 : : : 8xb9y1 : : : 9yc ; where each Ri is a relation symbol, each xi and yj are rst-order variables and is rst-order quanti er-free. The Skolem Normal Form for 11 has been used as the basis for a systematic analysis of optimization problems from the point of view of descriptive complexity [38, 49, 50]. Normal forms for 12 have been studied in [18]. The de nability of problems in fragments of monadic second-order logic has been studied to some depth, e.g., [3, 10, 16, 21, 47, 52, 69], and there are strong links between de nabliity in monadic SO on strings and nite automata and formal language theory (see [72]).

3 Beyond rst-order logic: xed points

Whilst SO (resp. 11) captures PH (resp. NP), from a complexity-theoretic viewpoint we have \jumped over" important complexity classes such as L, NL and P in our move from FO ( L) to SO and 11. The question remains as to how we might logically capture these complexity classes. Essentially, we need some \second-order construct" with which we can extend FO but which does not (appear to) give us the full power of SO or even 11. Looking at FO from a computational point of view, it is apparent that there is no mechanism for recursion. One way of introducing recursion is to build xed points. For example, let  be some signature, let R be some k-ary relation symbol not in  and let '(x) 2 FO( [ hRi) (i.e., '(x) is a rst-order formula over  [ hRi)) be such that the variables of the k-tuple x constitute the free variables of '. Moreover, 5

suppose that the following condition holds: for every S 2 STRUCT() (where STRUCT() is the set of nite structures over ) and for every pair of k-ary relations R1  R2 over jS j (the universe of S ), we have that: fu 2 jS jk : 'S (u; R1) holdsg  fu 2 jS jk : 'S (u; R2) holdsg: Then we say that ' is monotone . Monotone formulae, such as ' above, can be used to build xed points. Retaining the above notation, let R0 be the empty k-ary relation (over jS j). For all i  0 de ne: Ri+1 = fu 2 jS jk : 'S (u; Ri ) holdsg: It is easy to show that Ri  Ri+1, for all i  0, and consequently ' must have a xed point , namely Rj where j is the least index such that Rj = Rj+1 . Let us call this xed point the least xed point (as it coincides with least xed point of ': see, e.g., [17]) and let us denote this least xed point by LFP0[x; R; 'S ]. For example, consider the following formula '(x; y) 2 FO(2 [ hRi), where R is a binary relation symbol: x = y _ E (x; y) _ E (y; x) _ 9z((E (x; z) _ E (z; x)) ^ (R(z; y) _ R(y; z))): For S 2 STRUCT(2) and R0 the empty binary relation, de ne R1 = f(u; v) 2 jS j2 : 'S (u; v; R0) holdsg = f(u; v) 2 jS j2 : there is a path in the graph S of length at most 1 from u to vg; R2 = f(u; v) 2 jS j2 : 'S (u; v; R1) holdsg = f(u; v) 2 jS j2 : there is a path in the graph S of length at most 2 from u to vg; ::: Ri+1 = f(u; v) 2 jS j2 : 'S (u; v; Ri) holdsg = f(u; v) 2 jS j2 : there is a path in the graph S of length at most i + 1 from u to vg; ::: Hence, for every u; v 2 jS j, (u; v) 2 LFP0[(x; y); R; 'S ] if and only if there is a path in the graph S from u to v. Thus, because we have already seen that CONNECTIVITY is not de nable in FO, adding the capability to build least xed points yields an increase in expressive power. The logic (LFP0)[FO] is FO augmented with the operator LFP0. In more detail, (LFP0)[FO] is the closure of FO under the usual rst-order constructs and also applications of the operator LFP0 to monotone formulae. For example, CONNECTIVITY is de ned by the following sentence of (LFP0)[FO]: 8u8vLFP0[(x; y); R; '](u; v) 6

(note that LFP0[(x; y); R; '] describes a binary relation and u and v instantiate that relation). In general, a formula ' in an application of LFP0 of the form: LFP0[x; R; '](y1; y2; : : :; yk ) may have other free variables apart from those of x. These other free variables are free in the resulting formula, as are y1; y2; : : :; yk , whereas the variables of x are now bound. Note that we can apply rst-order constructs to formulae involving applications of the operator LFP0 (as we did to de ne CONNECTIVITY, above) and also re-apply the operator LFP0 to such formulae as long as the formula to which LFP 0 is applied is a monotone formula . One might be inclined now to investigate the expressibility of the logic (LFP0)[FO] in comparison with FO, 11 and SO; but let's look at this logic more closely. It is reasonable to expect that any bona de logic should be such that its formulae can be recursively enumerated; that is, we should be able to systematically list all well-formed formulae of the logic. However, Ajtai and Gurevich [4] showed that it is undecidable as to whether an arbitrary formula of FO is monotone or not (even when we allow in nite structures), and consequently the formulae of the logic (LFP0)[FO] are not recursively enumerable. But this observation does not force us to dispense with our basic approach of building xed points. What we need is some (decidable) syntactic restriction which forces formulae to be monotone, but which is not so severe as to unduly limit the expressive powers of the resulting logic. Let  be some signature, let R be some k-ary relation symbol not in  and let '(x) 2 FO( [ hRi) be such that the variables of the k-tuple x constitute the free variables of '. Moreover, suppose that every occurrence of the relation symbol R in ' is positive , i.e., does not appear within the scope of a negation sign. It is easy to show that any such positive formula is necessarily monotone. Consequently, we de ne the logic (LFP)[FO] just as we did before except that the operator LFP can only be applied to positive formulae (positive with respect to the relation symbol R over which the operator LFP is being applied). The formulae of the logic (LFP)[FO] are now clearly recursively enumerable. The sentence above de ning CONNECTIVITY is a sentence of (LFP)[FO] and so even with our syntactic restriction, we can still de ne problems that are not de nable in FO. But how does the de nability of the logic (LFP)[FO] compare with that of the logic (LFP0)[FO]? Has our syntactic restriction decreased the expressive power of (LFP0)[FO]? Gurevich and Shelah [28] showed that the logics (LFP)[FO] and (LFP0)[FO] are equally expressive, i.e., that restricting the syntax of (LFP0)[FO] as above does not restrict the expressive power. This is especially interesting when one compares it with Ajtai and Gurevich's result [4] that there are monotone formulae of FO that are not logically equivalent to any positive formula (this result does not hold when we consider the class of all structures, nite and in nite). But we are straying from our goal of examining the logical characterization of complexity classes. We introduced xed point operators in the hope that we might 7

logically capture complexity classes \between" L and NP. The question remains as to which, if any, complexity class is captured by the logic (LFP)[FO]. It turns out that the logic (LFP)[FO] does indeed capture a complexity class; but only in the presence of additional \built-in" relations. A built-in successor relation in any logic is a binary relation symbol, s, that is interpreted on any structure of size n as the relation f(0; 1); (1; 2); : : : ; (n ? 2; n ? 1)g, along with two constant symbols, 0 and max, which are always interpreted as 0 and n ? 1, respectively. Note that in the presence of a built-in successor relation, a sentence of some logic might not de ne a problem, i.e., a set of structures that is closed under isomorphism. Such sentences are not of interest to us: we are only ever interested in sentences of logics that de ne problems. For example, the following sentence of FO with a built-in successor relation: 9x(s(0; x) ^ E (0; x)) does not de ne a problem as, for example, isomorphic graphs of size 3 do not necessarily agree on this sentence. On the other hand, the following sentence of (LFP)[FO] with a built-in successor relation de nes the problem PARITY: LFP[(x; y); R; '(x; y)](max; 0); where R is a relation symbol of arity 2 and '(x; y) is de ned as: (x = 0 ^ y = 0) _ 9z(s(z; x) ^ R(z; 0) ^ y = max) _ 9z(s(z; x) ^ R(z; max) ^ y = 0)) (intuitively, the built-in successor relation is only used to \move through" the structure and does not refer to the elements of the structure in an \explicit fashion"). When a built-in successor relation is present in some logic such as (LFP)[FO], we denote this fact by writing (LFP)[FOs].

Theorem 4 [32, 75] A problem is in P if and only if it can be de ned by a sentence of the logic (LFP )[FO s]. Proof (Rough sketch ) As with the proof of Fagin's Theorem, one half of the proof is relatively easy. Let be any problem de nable in (LFP)[FOs ]. We proceed

by induction on the symbolic complexity (i.e., the length) of the de ning sentence. (Note that a sentence is built from formulae and so to apply the inductive hypothesis, we need to add extra constant symbols to our underlying signature so as to cater for free variables in the constituent formulae. This creates no problems.) The only non-trivial case is when is de ned by a sentence which is built from a formula ' by a single application of the operator LFP. The induction hypothesis implies that given some structure S , we can determine in polynomial time (in the size of S ) as to whether the formula ' holds in S when the free variables and relation symbols of 8

' are given some prescribed values. Hence, by computing each iteration of the least xed point process in turn (using the polynomial time algorithm alluded to above), we can determine whether the sentence holds in some structure (note that there is only a polynomial number of iterations to compute). Conversely, let M be some deterministic polynomial-time Turing machine. We use a relation R of appropriate arity to encode an instantaneous description of M , similarly to the proof of Fagin's Theorem except that we \squeeze" the encoding into one relation. For example, suppose that we have two relations, R1 and R2, of arity 2. We can encode these relations using a relation R of arity 3 as follows: R(x; y; z) holds

if and only if either R1(x; y) holds and z = 0 or R2(x; y) holds and z = max:

We then build a formula ' 2 (LFP)[FOs] to which we apply the operator LFP (with the external relation symbol being R) so that: (a) if R0 is the empty relation then R1 (built using R0 and ') is an encoding of the initial instantaneous description of M (on some input) (b) each iteration Ri in the computation of the least xed point is an encoding of the ith instantaneous description of M (on some input) (c) termination of the computation of M on some input is signalled by Ri+1 being identical to Ri , for some i; and acceptance is signalled by Ri(max; max; : : :; max) holding. Consequently, the problem accepted by M can be de ned by a sentence of the form: LFP[x; R; '](max; max; : ::; max); and the result follows. In order to circumvent the technicality of only allowing the operator LFP to be applied to positive formulae (with respect to the external relation symbol), the inductive xed point operator , IFP, is often used. The operator IFP can be applied to any formula and the semantics ensures that a xed point exists. Let  be some signature, let R be some k-ary relation symbol not in  and let '(x) 2 FO( [ hRi) be such that the variables of the k-tuple x constitute the free variables of '. Then for any S 2 STRUCT(), the inductive xed point IFP[x; R; 'S ] is de ned as follows: Ri+1 = fu 2 jS jk : either Ri (u) holds or 'S (u; Ri ) holdsg; for each i  0,and IFP[x; R; 'S ] = Rj , where j is the least index such that Rj = Rj+1. The logic (IFP)[FO] is built as was (LFP)[FO] except that there is no restriction on how the operator IFP is applied; as, with the above semantics, a xed point always exists. The following is a simple corollary of Theorem 4. 9

Corollary 5

A problem is in P if and only if it can be de ned by a sentence of the logic (IFP )[FO s ].

In fact, even in the absence of a built-in successor relation the logics (LFP0)[FO], (LFP)[FO] and (IFP)[FO] are equally expressive [28]; although Chandra and Harel [9] showed that there are problems in P that are not expressible in any of these logics. Also related to LFP is the partial xed point operator , PFP. As with the operator IFP, the operator PFP can be applied arbitrarily, with semantics de ned as follows. Let  be some signature, let R be some k-ary relation symbol not in  and let '(x) 2 FO( [ hRi) be such that the variables of the k-tuple x constitute the free variables of '. Then for any S 2 STRUCT(), the partial xed point PFP[x; R; 'S ] is de ned as follows:

Ri+1 = fu 2 jS jk : 'S (u; Ri ) holdsg; for each i  0, and PFP[x; R; 'S ] = Rj , where j is the least index such that Rj = Rj+1 , if such a j exists, and the empty relation otherwise. Note that we need not have that Ri  Ri+1, for all i  0, for a xed point to exist: it may be the case that the Ri's do not grow monotonically yet the process still terminates in a xed point. The logic (PFP)[FO] is as expected. The proof of the following theorem is similar to that of Theorem 4 (except that the iterative process of computing the xed point, if it exists, might consist of an exponential number of iterations; although it can still be completed in polynomial space).

Theorem 6 [1, 75] A problem is in PSPACE if and only if it can be de ned by a sentence of the logic (PFP )[FO s]. Again, in the absence of the built-in successor relation our complexity-theoretic characterization fails: Chandra and Harel [9] showed that there are problems in PSPACE that are not de nable in (PFP)[FO]: in fact, the problem PARITY is not de nable in (PFP)[FO]. However, even though there are very basic problems, in a complexity-theoretic sense, which are not de nable in logics such as (LFP)[FO] and (PFP)[FO], Abiteboul and Vianu [2] exhibited a remarkable relationship between the complexity-theoretic question \Is P equal to PSPACE?" and the purely logical question \Is every problem de nable in (PFP)[FO] also de nable in (LFP)[FO]?".

Theorem 7 [2] P = PSPACE if and only if every problem de nable in (PFP )[FO ] is also de nable in (LFP) [FO]. 10

Related research

Many of the inexpressibility results concerning logics such as (LFP)[FO] and (PFP)[FO] are inherited from the facts that both of these logics are fragments of bounded variable in nitary logic L!1! (introduced by Barwise [7]), and L!1! has a zero-one law [39]. Other inductive operators have been introduced so as to provide logical characterizations of complexity classes. For example, the context-sensitive transitive closure operator was developed in [61] so as to capture complexity classes ranging from L up to PSPACE (again, in the presence of a built-in successor relation). This context-sensitive transitive closure operator is essentially a cross between the inductive operators of this section and the transitive closure operator of the next.

4 Beyond rst-order logic: generalized quanti ers De ne the signature 2++ to consist of a binary relation symbol E and two constant symbols C and D. Structures over 2++ can be equated with digraphs with two distinguished vertices in the natural way. De ne the problem TC over the signature 2++ as: TC = fS 2 STRUCT(2++) : there is a directed path in the digraph described by E from vertex C to vertex Dg: We can extend FO using an operator corresponding to the problem TC as follows. The logic (TC)[FO] is de ned as the closure of FO under the usual rst-order operations and also under applications of the operator TC, where if ' 2 (TC)[FO] has free variables those of the k-tuples x and y and the m-tuple z, for some k and m, then the formula de ned as: TC[x; y'(x; y; z)](u; v); where u and v are k-tuples of (not necessarily distinct) variables and constant symbols is a formula of (TC)[FO]: the free variables of are those variables of the tuples z, u and v. We interpret a formula such as as follows. Let  be the underlying signature of ', let S be a -structure and let a be some assignment of an element of jS j to every free variable of . Build the digraph (S ) whose vertex set is jS jk and where there is an edge from vertex p to vertex q if and only if the formula ' holds in S where the variables of x are given the values p, the variables of y are given the values q and the variables of z are given their values from the assignment a. Let the source (resp. sink) of the digraph (S ) be the vertex obtained by interpreting the tuple u (resp. v) using the assignment a (and possibly S if u or v contains constant symbols). Then holds in S for the assignment a if and only if there is a directed path in the digraph (S ) from the source to the sink. 11

Adding the operator TC to the logic FO to obtain the logic (TC)[FO] essentially amounts to adding to FO the capability of computing the transitive closure of some (de nable) digraph. For example, the following sentence of (TC)[FO] over the signature 2 describes the problem CONNECTIVITY:

8x8yTC[u; v(E (u; v) _ E (v; u))](x; y); and, as mentioned earlier, this problem is not de nable in FO. Hence, extending FO with a transitive closure operator does increase the expressibility. Note that there is nothing special about extending FO with an operator corresponding to the problem TC: any problem could have been used (the syntax is essentially the same although, of course, the semantics will depend on the operator). The basic construction of extending FO with some operator corresponding to some problem has its roots in model theory when Lindstrom [42] and Mostowski [45] extended FO with generalized quanti ers . Operators such as TC are equivalent to uniform sequences of generalized quanti ers or vectorized generalized quanti ers (see, e.g., [17]). We know from complexity theory that the problem TC is complete for NL via logspace reductions [51]. This hints that there might be a closer relationship between the logic (TC)[FO] and the complexity class NL. Immerman [33] shed much light on this relationship by capturing NL by the fragment TC[FOs ] of (TC)[FOs] (so there is a built-in successor relation available), where TC[FOs] is de ned by insisting that any application of the operator TC in any formula must not appear within the scope of a negation sign, i.e., it must appear positively .

Theorem 8 [33] A problem is in NL if and only if it can be de ned in the logic TC  [FO s ]. Proof (Rough sketch ) The proof that any problem de nable in TC[FOs] is in NL

is a straightforward induction on the symbolic complexity of the de ning sentence. As with the proof of Fagin's Theorem and Theorem 4, building a sentence to describe a machine computation is where the ingenuity lies. Let be some problem in NL, and let M be some nondeterministic logspace Turing machine accepting . An instantaneous description of M on some input structure of size n consists of the contents of the work-tape, the state and the values of the workhead and the input-head (we insist that the input is presented to M on a separate read-only input tape), and can be encoded by a constant number, k say, of variables taking values from f0; 1; : : : ; n ? 1g. Also, given a k-tuple of variables whose values are drawn from f0; 1; : : : ; n ? 1g, it is possible to build a formula ' 2 TC[FOs] with two free variables, u and v, say, such that '(u; v) holds (for some values of u and v) if and only if the uth bit of the binary representation of v is a `1'. Using this formula, we can build another formula of TC[FOs ] which describes those k-tuples of variables which encode legitimate instantaneous descriptions of M ; and we can also build yet another formula of TC[FOs] which describes those pairs of k-tuples which encode 12

instantaneous descriptions, ID1 and ID2, say, of M and for which it is possible for M to move from ID1 to ID2 in one move. We can also arrange our encoding scheme so that the k-tuple (0; 0; : : : ; 0) corresponds to the initial instantaneous description of M and (max; max; : : :; max) corresponds to the nal accepting instantaneous description of M . A nal application of the TC operator can be used to describe that M , when started in its initial instantaneous description, can eventually reach its nal accepting instantaneous description; and the resulting sentence of TC[FOs] that describes is of the form: TC[x; y'](0; 0; : : :; 0; max; max; : : :; max) where ' is, of course, a formula of TC[FOs ] and where both x and y are k-tuples of variables. It is worth pointing out that the built-in successor relation is needed in the construction so as to cater for the moves of a Turing machine (input- or work-) head one cell to the left or one cell to the right. Note that if Immerman had proved that the logic (TC)[FOs] captures NL then he would have solved the long-standing open problem of whether NL is closed under complementation. As it happens, about a year later, in a celebrated result, Immerman did show that (TC)[FOs] captures NL [34]; and so NL is closed under complementation (a result also proven independently by Szelepcsenyi [73]).

Theorem 9 [34] A problem is in NL if and only if it can be de ned in the logic (TC ) [FO s]. Corollary 10 [34, 73] TC [FO s] = (TC )[FO s] and NL = co-NL. In the absense of the built-in successor relation, the logic (TC)[FO] has a zeroone law (it is a fragment of L!1! ), and so Theorems 8 and 9 fail: for example, PARITY 62 (TC)[FO]. Numerous other complexity classes have been similarly captured by extending FOs with appropriate uniform sequences of generalized quanti ers. Before we present some of these characterizations, we require additional de nitions. Let the signature 2;2 consist of two binary relations symbols, P and N . A structure S over 2;2 can be equated with a conjunction of Boolean clauses via:  if P (i; j ) holds in S then literal Xi is in clause Cj  if N (i; j ) holds in S then literal :Xi is in clause Cj . Let the signature 3++ consist of the relation symbol R of arity 3 and the constant symbols C and D. A path system is a set of vertices together with a ternary relation R de ning accessibility rules via: 13

 if x and y are accessible (where maybe x = y) and R(x; y; z) holds then z is

accessible. There is a source vertex, which initially is the only accessible vertex, and the set of accessible vertices is built up by repeated applications of the accessibility rules. Clearly, we can equate a 3++-structure with a path system complete with a source and a sink. De ne the problems: DTC = fS 2 STRUCT(2++ ) : there is a deterministic path in the digraph described by E from vertex C to vertex Dg PS = fS 2 STRUCT(3++ ) : the sink D is accessible from the source C in the path system described by Rg HP = fS 2 STRUCT(2++ ) : there is a Hamiltonian path in the digraph described by E from vertex C to vertex Dg SAT = fS 2 STRUCT(2;2) : the c.n.f. Boolean formula S is satis ableg HEX = fS 2 STRUCT(2++ ) : Player 1 can always win the game of Hex on the graph described by E with source C and sink Dg: A deterministic path in some digraph is a path such that every vertex of the path, except possibly the last, has out-degree one in the digraph. The game of Hex is played on a graph G = (V; E ), with a source C and a sink D, as follows. Player 1 begins by placing a blue pebble on some unpebbled vertex of V n fC; Dg. Player 2 then places a red pebble on some unpebbled vertex of V n fC; Dg. Player 1 then places a blue pebble on some unpebbled vertex of V n fC; Dg; and so on until all the vertices of V n fC; Dg have pebbles on them. Player 1 wins the game if there is a path of blue pebbled vertices from C to D, otherwise Player 2 wins.

Theorem 11 (i ) L = (DTC )[FO s] = DTC [FO s]. ([33]) (ii ) P = (PS )[FO s] = PS [FO s]. ([61]) (iii ) NP = HP [FO s]. ([11, 56]) (iv ) LNP = SAT [FO s] = (SAT )[FO s] = (HP )[FO s ] = 3COL[FO s ] = (3COL)[FO s ]. ([53, 59, 60]) (v ) PSPACE = (HEX )[FO s ] = HEX [FO s ]. ([43, 44]) We have seen logical characterizations of complexity classes both in the presence and absence of the built-in successor relation, and in the previous section we have seen how some logical characterizations of complexity classes fail in the absence of the built-in successor relation. An immediate question therefore arises regarding the exact role of the built-in successor relation in the logical characterization of 14

complexity classes. As regards complexity classes \below" NP, such as L, NL and P, there is no known characterization of any such complexity classes in the absence of built-in relations such as the successor relation ; and it is one of the most important open questions in nite model theory as to whether such a characterization exists (see, e.g., [48] for more details). The complexity class NP can be regarded as a threshold, in the above respect, since some logical characterizations of NP fail without the built-in successor relation yet some, like Fagin's Theorem, hold even in its absence.

Theorem 12 (i ) NP = HP [FO ]. ([11, 66]) (ii ) Let be some problem that is closed under sub-structures. Then the logic ( )[FO] has a zero-one law ; and so, in particular, the problem PARITY can not be de ned in this logic. ([14]) So, for example, by applying Theorem 12, the logical characterizations in Theorem 11 involving the operators PS, 3COL and HEX all fail in the absence of the built-in successor relation.

Related research

Immerman [30, 31] captured NL, P and PSPACE in terms of uniform sequences of rst-order sentences where bounds are placed on the number of variables used and the lengths of the sentences. Immerman [33] used a symmetric transitive closure operator STC to capture symmetric logspace, NSYMLOG ( rst de ned in [41]): STC[FOs] = NSYMLOG. It has recently been shown that NSYMLOG is closed under complementation [46]; and so STC[FOs] = (STC)[FOs ]. Immerman [35] captured parallel complexity classes based on the CRAM model in terms of iterated rst-order sentences. Immerman and Landau [36] captured the complexity class DET and some classes contained within L using rst-order logic (with built-in relations) augmented with appropriate operators. Barrington, Immerman and Straubing [8] studied circuit classes within the complexity class NC1 and characterized them in various ways using uniform sequences of generalized quanti ers and also uniform sequences of rst-order formulae. An alternative method of capturing complexity classes such as L, NL and P is to consider fragments of SO obtained, for example, by restricting the rst-order pre x and the quanti er-free rst-order formula of sentences in Skolem Normal Form [26]. Amalgamating inductive operators and uniform sequences of generalized quanti ers has been considered so as to capture complexity classes such as PNP [25, 63]. The class of problems in NP which are monotone , was characterized by extending FOs with a uniform sequence of generalized quanti ers and also as a fragment of 15

existential second-order logic [62, 67]. Some logical characterizations of the class of monotone problems in P were examined in [40]. The possibility of extending FO (and other logics without built-in relations) with a ( nite or in nite) sequence of generalized quanti ers and so capture P has been considered. For example, it has been shown that given an in nite sequence of generalized quanti ers whose arities are bounded by some natural number, there are problems in P which are not de nable in ( )[FO] [15, 29]. Gottlob [24] examined the general relationship between LC and ( )[FOs ], where C is some complexity class and is some problem complete for C (via appropriate reductions). Dawar, Gottlob and Hella [13] have shown that if (HP)[FO] = LNP then certain exponential Boolean hierarchies collapse (a circumstance which is thought unlikely): in fact, they prove the above result for any NP problem and not just HP.

5 Normal forms So far, we have established logical characterizations of a variety of complexity classes. It turns out that all of the logics involved have normal forms which, rst, tell us that the expressive power of the particular logic is identical to that of a proper fragment, and, second, that certain problems are complete for the complexity class involved, not just via (traditional) logspace and polynomial time reductions but via extremely restricted logical translations. Let us return to the logic (TC)[FOs ]. As we saw earlier, Immerman proved that the class of the problems de nable in this logic is, in fact, the complexity class NL, and that (TC)[FOs ] = TC[FOs]. In fact, (TC)[FOs ] collapses to an even more restricted fragment, and in order to prove this we proceed with a process of what amounts to\quanti er and connective elimination". Consider the sentence  de ned as: TC[x1; y1'1](0; max) ^ TC[x2; y2'2](0; max); where jx1j = jy1j = jx2j = jy2j = k (and 0 and max are k-tuples). For any structure S over the underlying signature of , S j=  if and only if there is a path in the digraph, H1, described by 'S1 (x1; y1) from vertex 0 to vertex max, and there is a path in the digraph, H2, described by 'S2 (x2; y2) from vertex 0 to vertex max. Let H be the digraph obtained by taking disjoint copies of H1 and H2 and including the additional edge from vertex max of H1 to vertex 0 of H2. Let the \source" of H be the vertex 0 of the copy of H1 and the \sink" of H be the vertex max of the copy of H2 . Then S j=  if and only if there is a path in H from the source to the sink. Now, it turns out that we can actually describe H by a \quanti er-free combination", , of '1 and '2: (u; x; v; y) = (u = 0 ^ v = 0 ^ '1(x; y)) _ (u = max ^ v = max ^ '2(x; y)) _(u = 0 ^ x = max ^ v = max ^ y = 0); 16

where x and y are k-tuples of variables. (the digraph described by (u; x; v; y) has additional isolated vertices when compared with H but this is of no consequence). Consequently:

S j=  if and only if S j= TC[(u; x); (v; y) (u; x; v; y)](0; max): In particular, if '1 and '2 are quanti er-free then is quanti er-free. A similar construction suces to eliminate the quanti er 8 from a sentence  of the form: 8zTC[x; y'(x; y; z)](0; max): By xing the value of the variable z, '(x; y; z) describes some digraph Gz when interpreted in some appropriate structure S of size n, say, and:

S j=  if and only if each of the digraphs Gz ; for z = 0; 1; : : : ; n ? 1; has a path from its vertex 0 to its vertex max: Let G be the digraph obtained by \stringing together" the digraphs G0; G1; : : : ; Gn?1 as we did in the previous paragraph with H1 and H2 (i.e., by joining the vertex max of Gi to the vertex 0 of Gi+1 , for each i = 0; 1; : : : ; n ? 2), with the source of G being de ned as the source of G0 and the sink as the sink of Gn?1 . Then:

S j=  if and only if there is a path from the source to the sink in G; and, moreover, G can be described by a formula which is a quanti er-free combination of ' and other atomic formulae. Thus, if ' is quanti er-free then the problem de ned by  is also de ned by a sentence of the form: TC[x0; y0 (x0; y0)](0; max); where is quanti er-free. By considering in turn each of the ( nite number of) ways a formula of the logic TC[FOs] can be built up from its constituent sub-formulae and using induction, we can eventually obtain the following normal form result for (TC)[FOs].

Theorem 13 [33] Any problem in NL can be described by a sentence of (TC )[FO s ] of the form : TC [x; y'(x; y)](0; max); where ' 2 FO s is quanti er-free, jxj = jyj = k, for some k, and 0 (resp. max) is the constant symbol 0 (resp. max) repeated k times.

Let the fragment (TC)i[FOs ] of the logic (TC)[FOs] consist of those formulae in which the operator TC is nested no more than i times, with the logic TCi[FOs] de ned analogously. 17

Corollary1 14 NL = TC [FO s].

Whilst Theorem 13 details a very strong normal for the logic (TC)[FOs], it also tells us more about the complexity-theoretic properties of the problem TC. Let

be any problem in NL, and suppose that is over the signature . By Theorem 13,

can be de ned by a sentence of (TC)[FOs] of the form: TC [x; y'(x; y)](0; max); where ' is quanti er-free. That is, given any instance of , i.e., any structure S 2 STRUCT(), 'S (x; y) describes a digraph (S ), with source 0 and sink max, such that S 2 if and only if ((S ); 0; max) 2 TC. This mirrors the notion of a polynomial time reduction, for example, in complexity theory except that instead of having a polynomial time algorithm to reduce one problem to another, we have a quanti er-free rst-order formula to describe one problem in terms of another. Of course, any rst-order formula can be computed in polynomial time (and even in logspace). Consequently, our normal form result for the logic (TC)[FOs] yields as an immediate corollary that the problem TC is complete for NL not just via logspace reductions but via quanti er-free rst-order translations .

Corollary 15

The problem TC is complete for NL via quanti er-free rst-order translations . In fact, Immerman proved yet more. Let ' 2 FOs(), for some signature , be of the form: _f ^ : i 2 I g; i i for some nite index set I , where: (i) each i is a conjunction of logical atomic relations and their negations, and no symbol of  appears in any i; (ii) each i is atomic or negated atomic; (iii) if i 6= j then i and j are mutually exclusive. Then ' is a quanti er-free projection . Quanti er-free projections are quanti er-free rst-order formulae of a very restricted form. What Immerman proved was the following. Theorem 16 [33] Any problem in NL can be described by a sentence of (TC )[FO s ] of the form : TC [x; y'(x; y)](0; max); where ' is a quanti er-free projection, jxj = jyj = k, for some k, and 0 (resp. max) is the constant symbol 0 (resp. max) repeated k times. Hence, the problem TC is complete for NL via quanti er-free projections.

18

It is worth spending some time on quanti er-free projections in order to appreciate just how restricted they are and to see how dicult it is to imagine a more restricted notion of reduction under which problems remain complete for well-known complexity classes. Suppose that 1 and 2 are problems over 2, for example (and so they are digraph problems), and suppose further that there is a quanti er-free projection from 1 to 2. That is, there is a quanti er-free projection '(x; y), where jxj = jyj = k, such that for any structure S 2 STRUCT(), S 2 1 if and only if the digraph (S ) described by 'S (x; y) is in 2. As ' is a quanti er-free projection, the following hold regarding S and (S ). (i) If jS j = n then j(S )j = nk , for all n. (ii) Fix n. Then there is a digraph Gn whose vertex set is f0; 1; : : : ; n ? 1gk and some of whose edges are labelled with literals from: fE (u; v); :E (u; v) : u; v 2 f0; 1; : : : ; n ? 1gk g such that: for every S 2 STRUCT(2) of size n, S 2 1 if and only if the digraph obtained from Gn by omitting all edges whose label, if it exists, evaluates to \false" when interpreted in S is such that it is in 2. (iii) The in nite sequence of digraphs fGn g is uniform (in a very strong sense): for example, there certainly exists a logspace algorithm which given n, outputs a description of Gn . Essentially, the edges in the target digraph (S ) depend on the existence of at most one edge in the source digraph S . It is hard to see how to make quanti erfree projections more restricted and still obtain complete problems for well-known complexity classes. Many of the other logics de ned earlier have normal forms resulting in completeness results via logical translations such as quanti er-free projections.

Theorem 17 (i ) L = (DTC )[FO s ] = DTC 1[FO s ] and the problem DTC is complete for L via quanti er-free projections. ([33]) (ii ) P = (PS )[FO s] = PS 1[FO s] and the problem PS is complete for P via quanti er-free projections. ([61]) (iii ) NP = HP [FO ] = HP 1[FO ] and the problem HP is complete for NP via quanti er-free projections without successor. ([11, 66]) (iv ) NP = 3COL1[FO s] = SAT 1[FO ], the problem 3COL is complete for NP via quanti er-free projections and the problem SAT is complete for NP via quanti er-free projections without successor. ([53, 64]) (v ) PSPACE = (HEX )[FO ] = HEX 1[FO ] and the problem HEX is complete for PSPACE via quanti er-free projections. ([6])

19

Whilst Theorem 17 details some very strong completeness results, quanti er-free projections are suciently restrictive that some \traditional" complete problems no longer remain complete via quanti er-free projections. Theorem 18 [5] There are problems which are complete for NP via quanti er-free rst-order translations (and so logspace reductions) but which are not complete for NP via quanti erfree projections. Whereas the traditional view of the class of NP-complete problems is that they are all, in some sense, identical, when one considers completeness via quanti er-free projections one can begin to \tell them apart". Another criterion for distinguishing NP-complete problems is that some remain complete via quanti er-free projections, say, in the absence of the built-in successor relation (e.g., HP) whereas some don't (e.g., 3COL). Yet another criterion for distinguishability is that (assuming NP 6= co-NP) the expressibility of rst-order logic when augmented with uniform sequences of generalized quanti ers corresponding to NP-complete problems varies (e.g., HP[FOs ] = NP yet 3COL[FOs] = LNP). All of these criteria for distinguishing complete problems only arise when one takes a descriptive approach to complexity theory. Let us end by showing how a particular normal form result of Theorem 17 leads to a simple proof of Fagin's Theorem. By Theorem 17(iv), any problem 2 NP can be de ned by a sentence of the logic 3COL[FOs] of the form: 3COL[x; y'(x; y)]; where ' 2 FOs is quanti er-free and where jxj = jyj = k. As we saw earlier, the problem 3COL can be de ned in 11 by the sentence: 9R9W 9B (8x((R(x) _ W (x) _ B (x)) ^ :(R(x) ^ W (x)) ^:(R(x) ^ B (x)) ^ :(W (x) ^ B (x))) ^8x8y((E (x; y) _ E (y; x)) ) (:(R(x) ^ R(y)) ^ :(W (x) ^ W (y)) ^:(B (x) ^ B (y))))) Hence, can be de ned by the sentence: 9R09W 09B 0(8x((R0(x) _ W 0(x) _ B 0(x)) ^ :(R0(x) ^ W 0(x)) ^:(R0 (x) ^ B 0(x)) ^ :(W 0(x) ^ B 0(x))) ^8x8x(('(x; y) _ '(x; y)) ) (:(R0(x) ^ R0(y)) ^ :(W 0(x) ^ W 0(y)) ^:(B 0(x) ^ B 0(y))))); where R0, B 0 and W 0 are new relation symbols of arity k. The built-in successor relation and the two associated constants can be de ned in 11, and so the result follows. (It should be remarked that Fagin's Theorem is not used in the proof of Theorem 17(iv).) 20

Related research

Descriptive complexity theory gives rise to new methods for proving the completeness of some problem via logical translations [58] (the method of quanti er elimination has been sketched above). Other problems have been shown to be complete for various complexity classes (including NSYMLOG and the classes pi and pi of the Polynomial Hierarchy) via quanti er-free projections (see, e.g., [36, 54, 55, 57, 68]). Complete problems for complexity classes consisting of monotone problems via monotone quanti er-free projections , have been given in [40, 62, 67]. The apparent anomaly that HP[FOs] = NP and 3COL[FOs] = LNP, yet both logics are formed by augmenting FOs with operators corresponding to NP-complete problems has been investigated in [65]. Yet another criterion concerning the distinguishability of NP-complete problems has been given in [70]. This criterion involves de nability in bounded-variable in nitary logic.

References [1] S. Abiteboul and V. Vianu, Datalog extensions for database queries and updates, J. Comput. System Sci. 43 (1991) 62{124. [2] S. Abiteboul and V. Vianu, Generic computation and its complexity, Proc. 23rd ACM Ann. Symp. on Theory of Computing (1991) 209{219. [3] M. Ajtai and R. Fagin, Reachability is harder for directed than for undirected nite graphs, J. Symbolic Logic 55 (1990) 113{150. [4] M. Ajtai and Y. Gurevich, Monotone versus positive, J. Assoc. Comput. Mach. 34 (1987) 1004{1015. [5] E. Allender, J. Balcazar and N. Immerman, A rst-order isomorphism theorem, Lecture Notes in Computer Science Vol. 665, Springer-Verlag (1993) 163{174. [6] A. Arratia-Quesada and I.A. Stewart, Generalized Hex and logical characterizations of polynomial space, Inform. Process. Lett. to appear (1997). [7] J. Barwise, On Moschovakis closure ordinals, J. Symbolic Logic 42 (1977) 292{ 296. [8] D.A.M. Barrington, N. Immerman and H. Straubing, On uniformity within NC, J. Comput. System Sci. 41 (1990) 274{306. [9] A. Chandra and D. Harel, Structure and complexity of relational queries, J. Comput. System Sci. 25 (1982) 99{128. 21

[10] S.S. Cosmadakis, Logical reducibility and monadic NP, Proc. 34th IEEE Ann. Symp. on Foundations of Computer Science (1993) 52{61. [11] E. Dahlhaus, Reduction to NP-complete problems by interpretations, Lecture Notes in Computer Science Vol. 171, Springer-Verlag (1984) 357{365. [12] A. Dawar, Generalized quanti ers and logical reducibilities, J. Logic Computat. 5 (1995) 213{226. [13] A. Dawar, G. Gottlob and L. Hella, Capturing relativized complexity classes without order, Technical Report Nr. 96/105, Christian Doppler Lab, Technische Univ. Wien (1996). [14] A. Dawar and E. Gradel, Generalized quanti ers and 0-1 laws, Proc. 10th IEEE Ann. Symp. on Logic in Computer Science (1995) 54{64. [15] A. Dawar and L. Hella, The expressive power of nitely many generalized quanti ers, Informat. Computat. 123 (1995) 172{184. [16] M. de Rougemont, Uniform de nability on nite structures with successor, Proc. 16th ACM Ann. Symp. on Theory of Computing (1984) 409{417. [17] H.-D. Ebbinghaus and J. Flum, Finite Model Theory , Springer (1995). [18] T. Eiter, G. Gottlob and Y. Gurevich, Normal forms for second-order logic over nite structures, and classi cation of NP optimization problems, manuscript (1995). [19] R. Fagin, Generalized rst-order spectra and polynomial-time recognizable sets, Complexity of Computation, SIAM-AMS Proceedings Vol. 7 (ed. R.M. Karp) (1974) 43{73. [20] R. Fagin, Monadic generalized spectra, Zeitschr. f. math Logik und Grundlagen d. Math 21 (1975) 89{96. [21] R. Fagin, L. Stockmeyer and M.Y. Vardi, On monadic NP vs. monadic co-NP, Inform. Computat. 120 (1995) 78{92. [22] M.R. Garey and D.S. Johnson, Computers and Intractability: a Guide to the Theory of NP-Completeness , W.H. Freeman and Co. (1979). [23] Y.V. Glebskii, D.I. Kogan, M.I. Liogonki and V.A. Talanov, Range and degree of realizability of formulas in the restricted predicate calculus, Cybernetics 5 (1969) 142{154. [24] G. Gottlob, Relativized logspace and generalized quanti ers over nite structures, Proc. 10th Ann. IEEE Symp. on Logic in Computer Science (1995) 65{78. 22

[25] E. Gradel, On logical descriptions of some concepts in structural complexity theory, Lecture Notes in Computer Science Vol. 440, Springer-Verlag (1990) 163{175. [26] E. Gradel, Capturing complexity classes by fragments of second-order logic, Theoret. Comput. Sci. 101 (1992) 35{57. [27] Y. Gurevich, Toward logic tailored for computational complexity, Lecture Notes in Mathematics Vol. 1104, Springer-Verlag (1984) 175{216. [28] Y. Gurevich and S. Shelah, Fixed-point extensions of rst-order logic, Ann. Pure Appl. Logic 32 (1986) 265{280. [29] L. Hella, Logical hierarchies in PTIME, Inform. Computat. bf 129 (1996) 1{19. [30] N. Immerman, Number of quanti ers is better than number of tape cells, J. Comput. System Sci. 22 (1981) 384{406. [31] N. Immerman, Upper and lower bounds for rst order expressibility, J. Comput. System Sci. 25 (1982) 76{98. [32] N. Immerman, Relational queries computable in polynomial time, Inform. Control 68 (1986) 86{104. [33] N. Immerman, Languages that capture complexity classes, SIAM J. Comput. 16 (1987) 760{778. [34] N. Immerman, Nondeterministic space is closed under complementation, SIAM J. Comput. 17 (1988) 935{938. [35] N. Immerman, Expressibility and parallel compexity, SIAM J. Comput. 18 (1989) 625{638. [36] N. Immerman and S. Landau, The complexity of iterated multiplic-ation, Inform. Computat. bf 116 (1995) 103{116. [37] N.D. Jones and A.L. Selman, Turing machines and the spectra of rst-order formulas with equality, J. Symbolic Logic 39 (1974) 139{150. [38] Ph.G. Kolaitis and M.N. Thakur, Logical de nability of NP optimization problems, Inform. Computat. 87 (1994) 321{353. [39] Ph.G. Kolaitis and M.Y. Vardi, In nitary logic and 0-1 laws, Inform. Computat. 98 (1992) 258{294. [40] C. Lautemann, T. Schwentick and I.A. Stewart, On positive P, Proc. 11th Annual IEEE Conference on Computational Complexity , IEEE Computer Society Press (1996) 162{170. 23

[41] H. Lewis and C. Papadimitriou, Symmetric space bounded computation, Theoret. Comput. Sci. 19 (1982) 161{188. [42] P. Lindstrom, First order predicate logic with generalized quanti ers, Theoria 32 (1966) 186{195. [43] J.A. Makowsky and Y.B. Pnueli, Oracles and quanti ers, Lecture Notes in Computer Science Vol. 832, Springer-Verlag (1994) 189{222. [44] J.A. Makowsky and Y.B. Pnueli, Computable quanti ers and logics over nite structures, in: Quanti ers: Generalizations, Extensions and Variants of Elementary Logic (M. Krynicki, M. Mostowski and L.W. Szczerba eds.), Kluwer Academic Publishers, to appear (1996). [45] A. Mostowski, On a generalization of quanti ers, Fund. Math. 44 (1957) 12{36. [46] N. Nisan and A. Ta-Shma, Symmetric logspace is closed under complement, Proc. 27th Ann. ACM Symp. on Theory of Computing , (1995) 140{146. [47] J. Nurmonen, On winning strategies with unary quanti ers, J. Logic Computat. 6 (1996) 779{798. [48] M. Otto, Bounded Variable Logics and Counting , Springer-Verlag (1997). [49] A. Panconesi and D. Ranjan, Quanti ers and approximation, Theoret. Comput. Sci. 107 (1993) 145{163. [50] C.H. Papadimitriou and M. Yannakakis, Optimization, approximation and complexity classes, J. Comput. Systems Sci. 43 (1991) 425{440. [51] W.J. Savitch, Relationships between nondeterministic and deterministic tape complexities, J. Comput. System Sci. 4 (1970) 177{192. [52] T. Schwentick, On Winning Ehrenfeucht Games and Monadic NP , Ph.D. Thesis, Johannes Gutenberg-Universitat Mainz Tech. Rep. Nr. 3/95. [53] I.A. Stewart, Comparing the expressibility of languages formed using NPcomplete operators, J. Logic Computat. 1 (1991) 305{330. [54] I.A. Stewart, Complete problems involving boolean labelled structures and projection translations, J. Logic Computat. 1 (1991) 861{882. [55] I.A. Stewart, Complete problems for symmetric logspace involving free groups, Inform. Process. Lett. 40 (1991) 263{267. [56] I.A. Stewart, Using the Hamiltonian path operator to capture NP, J. Comput. Systems Sci. 45 (1992) 127{151. 24

[57] I.A. Stewart, The complexity of restricted versions of the generalized word problem in free groups, Int. J. Alg. Computat. 2 (1992) 221{236. [58] I.A. Stewart, Methods for proving completeness via logical reductions, Theoret. Comput. Sci. 118 (1993) 193{229. [59] I.A. Stewart, Logical characterizations of bounded query classes I: logspace oracle machines, Fund. Informat. 18 (1993) 65{92. [60] I.A. Stewart, Logical characterizations of bounded query classes II: polynomialtime oracle machines, Fund. Informat. 18 (1993) 93{105. [61] I.A. Stewart, Context-sensitive transitive closure operators, Ann. Pure App. Logic 66 (1994) 277{301. [62] I.A. Stewart, Logical descriptions of monotone NP problems, J. Logic Computat. 4 (1994) 337{357. [63] I.A. Stewart, Incorporating generalized quanti ers and the least xed point operator, Lecture Notes in Computer Science Vol. 832, Springer-Verlag (1994) 318{333. [64] I.A. Stewart, On completeness for NP via projection translations, Math. Systems Theory 27 (1994) 125{157. [65] I.A. Stewart, Monotonicity and the expressibility of NP operators, Math. Log. Quart. 40 (1994) 132{140. [66] I.A. Stewart, unpublished manuscript (1994). [67] I.A. Stewart, Complete problems for monotone NP, Theoret. Comput. Sci. 145 (1995) 147{157. [68] I.A. Stewart, Completeness for path-problems via weak logical reductions, Inform. Computat. 121 (1995) 123{134. [69] I.A. Stewart, Regular subgraphs in graphs and rooted graphs, and de nability in monadic second-order logic, Math. Logic Quart. 43 (1997) 1{21. [70] I.A. Stewart, Logics with zero-one laws that are not fragments of boundedvariable in nitary logic, Math. Logic Quart. to appear (1997). [71] L. Stockmeyer, The polynomial time hierarchy, Theoret. Comput. Sci. 3 (1976) 1{22. [72] H. Straubing, Finite Automata, Formal Logic and Circuit Complexity , Birkhauser (1994). 25

[73] R. Szelepcsenyi, The method of forced enumeration for nondeterministic automata, Acta Informat. 26 (1988) 279{284. [74] B.A. Trakhtenbrot, Impossibility of an algorithm for the decision problem on nite classes, Doklady 70 (1950) 569{572. [75] M. Vardi, Complexity of relational query languages, Proc. 14th ACM Ann. Symp. on Theory of Computing (1982) 137{146.

26