Parsing by matrix multiplication generalized to Boolean grammars$

Parsing by matrix multiplication generalized to Boolean grammarsI Alexander Okhotin1 Department of Mathematics and Statistics, University of Turku, Turku FI–20014, Finland

Abstract The well-known parsing algorithm for context-free grammars due to Valiant (“General context-free recognition in less than cubic time”, Journal of Computer and System Sciences, 10:2 (1975), 308–314) is analyzed and extended to handle the more general Boolean grammars, which are context-free grammars augmented with conjunction and negation operators in the rules. The algorithm reduces construction of a parsing table to computing multiple products of Boolean matrices of various sizes. Its time complexity on an input string of length n is O(BMM(n) log n), where BMM(n) is the number of operations needed to multiply two Boolean matrices of size n × n, which is O(nω ) with ω < 2.373 as per the current knowledge. A parse tree can be constructed in time MM(n) logO(1) n (where MM(n) is the complexity of multiplying two integer matrices), by applying a known efficient procedure for determining witnesses for Boolean matrix multiplication. The algorithm has a succinct proof of correctness and is ready to be implemented. Key words: Boolean grammars, conjunctive grammars, context-free grammars, matrix multiplication, parsing

1. Introduction Context-free grammars are the universally accepted mathematical model of syntax, and their status is well justified. On the one hand, their expressive means are natural, in the sense that whatever they define is intuitively seen as the syntax of something. On the other hand, they can be implemented in a variety of efficient algorithms, including a straightforward cubic-time parser, as well as many practical parsing algorithms working much faster in special cases. The main idea of the context-free grammars is inductive definition of syntactically correct strings. For example, a grammar S → aSb | ε represents a definition of the following form: a string has the property S if and only if either it is representable as awb for some string w with the property S, or it is the empty string. Note that the vertical line in the above grammar is essentially a disjunction of two syntactical conditions. Boolean grammars, introduced by the author [14], are an extension of the context-free grammars, which maintains the main principle of inductive definition, but allows the use of any Boolean operations to combine syntactical conditions in the rules. This significantly increases the expressive power of the model [8, 7, 13, 14]. At the same time, Boolean grammars inherit the basic parsing algorithms from the context-free grammars, including the Cocke–Kasami–Younger algorithm [14] along with its variant for unambiguous grammars [17], the Generalized LR [15], as well as the linear-time recursive descent [16]. For more information about Boolean grammars, an interested reader is directed to a recent survey paper [19]. The straightforward upper bound on the complexity of parsing for Boolean grammars is the same as for ordinary context-free grammars: that is, O(n3 ), where n is the length of the input string [14]. However, I A preliminary version of this paper, entitled “Fast parsing for Boolean grammars: a generalization of Valiant’s algorithm”, was presented at the 14th International Conference on Developments in Language Theory (DLT 2010) held in London, Ontario, Canada on August 17–20, 2010, and its extended abstract appeared in the conference proceedings [18]. Email addresses: [email protected] (Alexander Okhotin) 1 Supported by the Academy of Finland under grant 257857. Preprint submitted to Elsevier July 11, 2013

for ordinary grammars, there also exists an asymptotically faster parsing algorithm due to Valiant [26]: this algorithm computes the same parsing table as the simple Cocke–Kasami–Younger algorithm, but does so by offloading the most intensive computations into calls to a Boolean matrix multiplication procedure. The latter can be efficiently implemented in a variety of ways. Given two n × n Boolean matrices, a straightforward calculation of their product uses n3 conjunctions and (n − 1)n2 disjunctions. An improved
n3 algorithm by Arlazarov et al. [3] reduces the number of bit operations to O log n , which is achieved by pre-computing products of all bit vectors of length log n with certain submatrices. Another much more sophisticated combinatorial algorithm for Boolean matrix multiplication, due to R. Williams [28], operates in
3 time O logn2 n . An asymptotically more significant acceleration is obtained by using fast algebraic algorithms for multiplying n×n numerical matrices, such as Strassen’s [25] algorithm that requires O(n2.81 ) arithmetical operations, or the method of Coppersmith and Winograd [6], which, with the recent improvements by V. Vassilevska Williams [27], achieves theoretical running time O(n2.373 ). These algorithms can be applied to multiplying n × n Boolean matrices by calculating their product in the ring of residues modulo n + 1 [1]. Whatever method for Boolean matrix multiplication is used in Valiant’s algorithm, as long as matrices are multiplied in time BMM(n), context-free parsing in time O(BMM(n) log n) is obtained. Valiant’s result has inspired some further interesting studies. A total rethinking of the algorithm was undertaken by Rytter [23], who presented its divide-and-conquer approach in terms of parse tree partitions. Bened´ı and S´ anchez [4] implemented the algorithm in the case of stochastic grammars, where Strassen’s matrix multiplication can be applied most efficiently, and reported a noticeable performance increase. An extension of the algorithm for context-free grammars over partially computable alphabets was developed by Bertoni et al. [5] and works in time Θ(BMM(nα )), where α > 1 is a constant determined by the commutativity relation between the symbols. Rajasekaran and Yooseph [22] extended Valiant’s algorithm to the family of tree-adjoining grammars (which generalize context-free grammars by allowing inner rewriting of parse trees), and established a recognition algorithm working in time Θ(BMM(n2 )), improving over the Θ(n6 )-time direct algorithm. A reverse reduction of parsing to matrix multiplication was first discovered by Satta [24], who demonstrated that any algorithm for constructing a full parsing table for tree-adjoining q grammars in time O(|G|p ·nq ) leads to an O(n2p+ 6 )-time algorithm for Boolean matrix multiplication. Satta’s idea was later applied by Lee [12] to show that context-free parse table construction in time O(|G|p · nq ) q similarly implies BMM(n) = O(n2p+ 3 ). Taking a closer look at Valiant’s algorithm, one can see that first the entire grammar is encoded in a certain algebraic structure—basically, a semiring with non-associative multiplication—then the notion of a transitive closure of a Boolean matrix is extended to matrices over this semiring, so that the desired parsing table could be obtained as a closure of this kind, and finally it is demonstrated that such a closure can be efficiently computed using Boolean matrix multiplication. This approach essentially relies on having two operations in a grammar, concatenation and union, which give rise to the product and the sum in the semiring. Because of that, Valiant’s algorithm in its original presentation cannot be applied to Boolean grammars. This paper aims at rewriting Valiant’s algorithm to make it work in the more general case of Boolean grammars. It is shown that using matrices over a special semiring as an intermediate abstraction is in fact unnecessary, and that matrix multiplication is actually needed to compute the concatenations only, while Boolean logic can be evaluated separately. Furthermore, the proposed algorithm maintains one fixed data structure, the parsing table, and whenever the matrix is to be cut as per Valiant’s divide-and-conquer strategy, the new presentation of the algorithm only distributes the ranges of positions in the input string among the recursive calls. This leads to an improved version of the algorithm, which, besides being applicable to a larger family of grammars, is also better understandable than the original Valiant’s algorithm, has a succinct proof of correctness and is ready to be implemented. The resulting improved understanding of how exactly a large bulk of concatenations can be represented by Boolean matrix multiplication has already been used in a recent paper by Okhotin and Reiwießner [21] to obtain a fast recognition algorithm for input strings over a one-symbol alphabet. That algorithm uses Boolean convolution as the underlying algebraic problem, and utilizes fast Fourier transform to calculate it efficiently. 2

Following a brief introduction to Boolean grammars, given in Section 2, this paper sets off by presenting the simple cubic-time recognition algorithm (Section 3), both in the usual set-theoretic notation and on the level of bit operations. Next, an alternative order of evaluating the operations in this algorithm is illustrated on a small example, where the calculation of the parsing table for a 5-symbol string can use products of 2 × 2 Boolean matrices. The next Section 4 presents the desired algorithm for constructing the parsing table, in which the evaluation of logically independent bits in the parsing table is totally reordered to maximize the use of Boolean matrix products. This yields the known Valiant’s algorithm, though in the form applicable to Boolean grammars; the algorithm is accompanied with a proof of correctness and an analysis of complexity. Some simple observations on implementing the algorithm are presented in the following Section 5. The question of constructing a parse tree is handled in Section 6. While the running time is not as issue in the case of ordinary context-free grammars, where, given a parsing table, a tree can be constructed in time O(n2 ), in the case of Boolean grammars, this direct approach leads to cubic time. This question is solved by augmenting the parsing table with the data on factorizations of substrings, and modifying the main algorithm to construct such a table, with ordinary Boolean matrix multiplication replaced with the problem of computing integer witnesses for matrix multiplication. Applying the known algorithm of Alon and Naor [2] for constructing those witnesses in time MM(n) logO(1) n, where MM(n) is the number of operations needed to multiply two integer matrices of size n × n, yields the desired efficient procedure for constructing parse trees. The next Section 7 extends the algorithm to a generic procedure working over abstract structures. As compared with Valiant’s [26] semirings without associativity, the abstract structures assumed in this paper are a little less restrictive, which allows them to capture parsing for Boolean grammars. In Section 8, the given generic algorithm is applied to stochastic grammars (with disjunction only), and it is also applicable to such more general models as fuzzy Boolean grammars [9] and stochastic conjunctive grammars [29]. One more application of the generic algorithm is given in the last Section 9, which extends fast parsing to the well-founded definition of Boolean grammars due to Kountouriotis et al. [11]. 2. Boolean grammars Let Σ be a finite nonempty set used as an alphabet, let Σ∗ be the set of all finite strings over Σ. For a string w = a1 . . . a` ∈ Σ∗ with ai ∈ Σ, the length of the string is denoted by |w| = `. The unique empty string of length 0 is denoted by ε. Denote by Σ6` the set of all strings over Σ of length up to ` > 0. For a string w ∈ Σ∗ and for its every partition w = uv, u is a prefix of w and v is its suffix ; furthermore, for every partition w = xyz, the string y is a substring of w. Any subset of Σ∗ is a language over Σ. The basic operations on languages are concatenation K · L = { uv | u ∈ K, v ∈ L } and Boolean set operations: union K ∪ L, intersection K ∩ L, and complementation L. Boolean grammars are a family of formal grammars in which all these operations can be explicitly specified. Definition 1. [14] A Boolean grammar is a quadruple G = (Σ, N, R, S), where Σ is a finite set of symbols in the language being defined; N is the set of symbols representing the syntactic properties defined by the grammar, which are historically called nonterminal symbols; R is a finite set of rules of the form A → α1 & . . . & αm & ¬β1 & . . . & ¬βn ,

(1)

where m > 1, n > 0, αi , βi ∈ (Σ ∪ N )∗ ; S ∈ N is the initial symbol, representing the set of strings defined by the grammar. Let |G| denote the length of the description of the grammar, measured by the number of symbols. If negation is not allowed, that is, m > 1 and n = 0 in every rule, then the resulting grammars are known as conjunctive grammars [13]. If conjunction is also prohibited, and thus every rule must have m = 1 and n = 0, then the (ordinary) context-free grammars are obtained. The intuitive semantics of a Boolean grammar are fairly clear: a rule (1) specifies that every string that satisfies each of the conditions αi and none of the conditions βi is therefore generated by A. However, formalizing this definition has proved to be rather nontrivial in the general case. In the case of conjunctive 3

grammars (including ordinary context-free grammars), the semantics can be equivalently defined by a least solution of language equations and by term rewriting. The definition by language equations carries on to Boolean grammars of the general form as follows. A grammar is interpreted as a system of language equations in variables N , in which the equation for each A ∈ N is  \ m n [ \ βj (2) A= αi ∩ A→α1 &... & αm & ¬β1 &... & ¬βn ∈R

i=1

j=1

In general, such a system may have no solutions (as in the equation S = S corresponding to the grammar S → ¬S) or multiple solutions (with S = S being the simplest example). The easiest definition of Boolean grammars given below dismisses such systems as ill-formed, and considers only systems with a unique solution. That unique solution defines the language generated by the grammar. The actual condition imposed by the definition is stronger than just solution uniqueness. Definition 2 ([14]). Let G = (Σ, N, R, S) be a Boolean grammar and consider the associated system of language equations (2). Assume that for every number ` > 0 there exists a unique vector of languages (. . . , LC , . . .)C∈N with LC ⊆ Σ6` , such that a substitution of LC for C, for each C ∈ N , turns every equation (2) into an equality modulo intersection with Σ6` . Then G complies to the semantics of a strongly unique solution, and, for every A ∈ N , the language LG (A) is defined as the component LA of the unique solution of this system. The language generated by the grammar is L(G) = LG (S). This fairly rough restriction ensures that the membership of a string in the language depends only on the membership of shorter strings, which is essential for grammars to represent inductive definitions. An alternative definition of the semantics of Boolean grammars, which interprets them in terms of three-valued logic, will be presented in Section 9. Example 1 ([13]). Consider the following conjunctive grammar, which generates the language { an bn cn | n > 0 }, and its associated system of language equations. S A B C D

→ → → → →

AB & DC aA | ε bBc | ε cC | ε aDb | ε

S A B C D

= = = = =

AB ∩ DC ({a}A) ∪ {ε} ({b}B{c}) ∪ {ε} ({c}C) ∪ {ε} ({a}D{b}) ∪ {ε}

The rules for the nonterminals A, B, C and D form an ordinary context-free grammar, and they define LG (AB) = { ai bn cn | i, n > 0 } and LG (DC) = { am bm cj | j, m > 0 }. Then the conjunction in the rule for S specifies the strings satisfying both conditions given by AB and DC: L(AB) ∩ L(DC) = { ai bj ck | j = k and i = j } = { an bn cn | n > 0 }. Putting negation over one of the conjuncts in the rule for S allows defining another language not represented by any ordinary context-free grammar. Example 2. The following Boolean grammar, presented along with the associated system of language equations, generates the language { am bn cn | m, n > 0, m 6= n }: S A B C D

→ → → → →

AB & ¬DC aA | ε bBc | ε cC | ε aDb | ε

S A B C D

= = = = =

AB ∩ DC ({a}A) ∪ {ε} ({b}B{c}) ∪ {ε} ({c}C) ∪ {ε} ({a}D{b}) ∪ {ε}

The two concatenations AB and DC are the same in Example 1. Then the propositional connectives in the rule for S specify the following combination of the conditions given by AB and DC: L(AB) ∩ L(DC) = { ai bj ck | j = k and i 6= j } = { am bn cn | m, n > 0, m 6= n }. 4

S → AB & DC

(a)

A →aA

B →bBc

S → AB & ¬DC

(b)

D →aDb

A→aA

C →cC

B→bBc

A→aA A →ε

B →ε

D →ε

C →ε A →ε

a

b

c

a

a

B→ε

b

c

Figure 1: Parse trees (a) for the conjunctive grammar in Example 1, and (b) for the Boolean grammar in Example 2.

Whenever a Boolean grammar generates a string, it defines one or more parse trees for that string [14]. These are, strictly speaking, finite acyclic graphs rather than trees, and they represent parses of a string according to the positive conjuncts in the rules. Let G = (Σ, N, R, S) be a grammar. A parse tree of a string w = a1 . . . a` from a nonterminal A ∈ N has ` ordered leaves labelled with a1 , . . . , a` , and the rest of the nodes are labelled with rules from R. Each node of the tree corresponds to a substring ai+1 . . . aj , and if it is labelled with a rule A → α1 & . . . & αm & ¬β1 & . . . & ¬βn , then it has exactly |α1 | + . . . + |αm | sons corresponding to the symbols in the positive conjuncts. For each nonterminal B ∈ N in each αt , the corresponding son is labelled with some rule for B, while for each symbol a ∈ Σ in each αt , the son is a leaf labelled with a. Furthermore, for each conjunct αt , the corresponding |αt | subtrees together encompass the entire substring ai+1 . . . aj , and in this way the tree represents m parses of the same substring. The tree has a unique root labelled with a rule for A. A parse tree for the conjunctive grammar in Example 1 is given in Figure 1(a). Negative conjuncts are not represented in parse trees. For instance, parse trees defined by the grammar in Example 2, such as the one in Figure 1(b), have the same form as the trees of an ordinary context-free grammar with a rule S → AB. Occasionally, a parse tree may contain multiple parses of the same substring with respect to the same nonterminal symbol. For this, one can either reuse a single parse subtree, connecting its root with multiple inbound arcs, or have separate parses of the same string instead. A parse tree is said to be expanded, if separate parses are used in every case, so that every node labelled with a rule has a unique inbound arc. If parse subtrees are always reused, that is, if for every nonterminal A and for every substring of w there is at most one node labelled with a rule for A that corresponds to this substring, then the parse tree is called condensed. A condensed tree of w contains at most |N | · 21 |w|(|w| + 1) + |w| nodes. On the other hand, expanded parse trees can be much larger for some grammars. Example 3. The following grammar defines the language a+ , and the size of expanded parse trees is expo-

5

(a)

(b)

S → AS & BS

S →AS & BS

S → AS & BS

S →AS & BS

A

B

a

A

S →a

B

a

A

a

B

B

A

a

S

S A

a

S → AS & BS

B

S

S

a

Figure 2: (a) Condensed and (b) expanded parse trees for the grammar in Example 3. The labels A, B and S abbreviate the rules A → a, B → a and S → a.

nential in the length of the string. S → AS & BS S→a A→a B→a Figure 2 illustrates the condensed parse tree and the expanded parse tree of w = aaa. To conclude this brief introduction into Boolean grammars, consider that every Boolean grammar can be transformed to an equivalent grammar G = (Σ, N, R, S) in binary normal form [14], in which every rule in R is of the following form. A → B1 C1 & . . . & Bm Cm & ¬D1 E1 & . . . & ¬Dn En & ¬ε

(m > 1, n > 0, Bi , Ci , Dj , Ej ∈ N )

A→a

(a ∈ Σ)

S → ε, The rule of the latter kind is allowed only if S is never used in right-hand sides of any rules. The known general transformation to the binary normal form causes an exponential blowup in the size of the grammar, although it is not known whether this blowup is necessary. All parsing algorithms considered in this paper assume a grammar in this normal form. 3. Simple cubic-time parsing Let G = (Σ, N, R, S) be a Boolean grammar in binary normal form, let w = a1 . . . an be an input string. The simple cubic-time parsing algorithm constructs the parsing table T ∈ (2N )n×n , with each element Ti,j with 0 6 i < j 6 n representing the set of nonterminals that generate the substring between the positions i + 1 and j: Ti,j = { A ∈ N | ai+1 . . . aj ∈ LG (A) }. The elements of this table can be computed inductively on the length j − i of the substring, beginning with the elements Ti−1,i , each depending only on the symbol ai , and continuing with larger and larger substrings, until the element T0,n is computed. The properties of one-symbol strings are determined directly: Ti−1,i = { A | A → ai ∈ R }. 6

For each substring ai+1 . . . aj of length at least 2, consider first all concatenations BC, with B, C ∈ N , into which this substring can be factorized: Pi,j = { (B, C) | B, C ∈ N, ai+1 . . . aj ∈ LG (BC) }. This set of pairs can be calculated as follows, Pi,j =

j−1 [

Ti,k × Tk,j ,

k=i+1

where the position k represents a cutting point of the string ai+1 . . . aj into two substrings: ai+1 . . . ak generated by B and ak+1 . . . aj generated by C, and the data on those shorter substrings are acquired from the sets Ti,k and Tk,j , which must have been computed before. Then, using the set Pi,j , one can determine the set of nonterminals generating the substring in question as Ti,j = f (Pi,j ), where the function f : 2N ×N → 2N , defined by f (P ) = { A | ∃A → BC ∈ R : (B, C) ∈ P } for ordinary context-free grammars, and by f (P ) = {A | ∃A → B1 C1 & . . . & Bm Cm & ¬D1 E1 & . . . & ¬Dm0 Em0 ∈ R : (Bt , Ct ) ∈ P and (Dt , Et ) ∈ / P for all t} in the general case of Boolean grammars, represents the logic in the rules of the grammar. This computation is implemented in Algorithm 1. Algorithm 1 (Extended Cocke–Kasami–Younger [13, 14]). Let G = (Σ, N, R, S) be a Boolean grammar in binary normal form. Let w = a1 . . . an , where n > 1 and ai ∈ Σ, be an input string. For all 0 6 i < j 6 n, let Ti,j be a variable ranging over subsets of N , and let Pi,j be a variable ranging over subsets of N × N. 1: for i = 1 to n do 2: Ti−1,i = { A | A → ai ∈ R } 3: for ` = 2 to n do 4: for i = 0 to n − ` do 5: let j = i + ` 6: let Pi,j = ∅ 7: for all k = i + 1 to j − 1 do 8: Pi,j = Pi,j ∪ (Ti,k × Tk,j ) 9: Ti,j = f (Pi,j ) 10: accept if and only if S ∈ T0,n In total, there are Θ(n2 ) elements, and each of them, on average, takes Θ(n) operations to compute, which results inSa cubic time complexity. The most time-consuming operation in Algorithm 1 is computing j−1 the sets Pi,j = k=i+1 Ti,k × Tk,j . If each Cartesian product is computed individually, as it is done in line 8 of the above algorithm, then spending linear time for almost each Pi,j is unavoidable. The idea behind fast parsing is to rearrange the order of these operations, so that much of the work could be represented as Boolean matrix multiplication. To see how the work done by Algorithm 1 is related to matrix multiplication, consider the following adaptation of the notion of matrix product to matrices with subsets of N as elements. 7

Figure 3: How products of submatrices of T contribute to calculating submatrices of P : (a) in the basic algorithm; (b) in Example 4.

Definition 3. For any numbers m, `, n > 1, let X ∈ (2N )m×` and Y ∈ (2N )`×n be two matrices with subsets of N as elements. Their product X × Y is a matrix Z ∈ (2N ×N )m×n , with Zi,j =

` [

Xi,k × Yk,j .

k=1

Such a product can be represented as a product of |N |2 pairs of Boolean matrices as follows. For every BC B, C ∈ N , consider the Boolean matrix Z BC , where Zi,j denotes the membership of the pair (B, C) in BC Zi,j . Then the Boolean matrix Z is exactly the product X B × Y C of the Boolean matrix X B , which represents the membership of B in the elements of X, with the similarly defined Boolean matrix Y C . In the terminology of Definition 3, lines 6–8 of Algorithm 1 actually multiply a 1 × (` − 1) submatrix of T by a (` − 1) × 1 submatrix of T , as illustrated in Figure 3(a). The result is a 1 × 1 matrix, that is, a single set Pi,j ⊆ N × N . Thus, matrix products used in Algorithm 1 are always products of row vectors with column vectors. In order to use fast matrix multiplication, one should somehow rearrange the total bulk of operations calculated in line 8, for all applicable i, j, k, so that large blocks of Cartesian products would be calculated together as products of square matrices. The below example demonstrates such a rearrangement for input strings of length 5, which allows using a product of two 2 × 2 submatrices of T . Example 4. Let w = a1 a2 a3 a4 a5 be an input string, and consider the partially constructed parsing table depicted in Figure 3(b), with Ti,j constructed for 0 6 i < j 6 3 and for 2 6 i < j 6 5, that is, for the substrings a1 a2 a3 and a3 a4 a5 , as well as for their substrings. Then the following product of matrices of sets        T0,2 T0,3 T2,4 T2,5 (T0,2 × T2,4 ) ∪ (T0,3 × T3,4 ) (T0,2 × T2,5 ) ∪ (T0,3 × T3,5 ) X0,4 × = = T1,2 T1,3 T3,4 T3,5 (T1,2 × T2,4 ) ∪ (T1,3 × T3,4 ) (T1,2 × T2,5 ) ∪ (T1,3 × T3,5 ) X1,4   P P0,5 defines partial data for the following four elements of the table of pairs: P0,4 . First of all, each of P 1,4 1,5 these four elements satisfies Xi,j ⊆ Pi,j . In particular, the set X1,4 is exactly P1,4 . The set X0,4 lacks the Cartesian product T0,1 × T1,4 that should be in P0,4 by definition, and thus does not take into account the factorization a1 ·a2 a3 a4 ; actually, the set T1,4 is not yet known at this point, and hence the calculation of that Cartesian product has to be delayed. The element X1,5 is symmetrically incomplete, as it lacks the Cartesian product T1,4 ×T4,5 corresponding to the factorization a2 ·a3 a4 a5 . Finally, X0,5 misses two Cartesian products, T0,1 × T1,5 and T0,4 × T4,5 , which can be handled only using the not yet available elements T0,4 and T1,5 . In total, this matrix product computes 8 Cartesian products out of the 12 needed for these four elements of P , and the computation could then proceed with calculating the remaining four Cartesian products. Already in this small example, using one product of 2 × 2 matrices requires changing the order of computation of the elements {Ti,j }: the elements T0,3 and T2,5 need to be calculated before T1,4 . Furthermore, 8

X0,5 X1,5



the subsequent computation should be arranged to take care of the four remaining factorizations, which also must be considered in a specific order, evaluating Ti,j = f (Pi,j ) for the appropriate entries at the appropriate time. In the next section, the whole algorithm will be restated as a recursive procedure, which arranges the computation so that as much work as possible is offloaded into products of the largest possible matrices. 4. Parsing reduced to matrix multiplication Let w = a1 . . . an be an input string. For the time being, assume that n + 1 is a power of two, that is, the length of the input string is a power of two minus one; this restriction can be relaxed in an implementation, which will be discussed in the next section. The algorithm uses the following data structures. First, there is an (n + 1) × (n + 1) table T with Ti,j ⊆ N , as in Algorithm 1, and the goal is to set each entry to Ti,j = { A | ai+1 . . . aj ∈ L(A) },

for all 0 6 i < j 6 n.

The second table P has elements Pi,j ⊆ N × N , each corresponding to the value of P computed by Algorithm 1 in the iteration (` = j − i, i). The target value is Pi,j = { (B, C) | ai+1 . . . aj ∈ L(B)L(C) },

for all 0 6 i < j 6 n.

All entries of both tables are initialized to empty sets, and then are gradually filled by the following two recursive procedures: • The first procedure, compute(`, m), constructs the correct values of Ti,j for all ` 6 i < j < m, as illustrated in Figure 4(a). • The other procedure, complete(`, m, `0 , m0 ), is defined for ` < m 6 `0 < m0 , where m − ` = m0 − `0 is a power of two. Its four arguments actually specify a submatrix of T containing all elements Ti,j with ` 6 i < m and `0 6 j < m0 , as shown in Figure 4(b). The procedure assumes that the elements Ti,j are already constructed for all i and j with ` 6 i < j < m, as well as for all i, j with `0 6 i < j < m0 ; these are the dark grey triangles in Figure 4(b). It is furthermore assumed that for all ` 6 i < m and `0 6 j < m0 , the current value of Pi,j is Pi,j = { (B, C) | ∃k (m 6 k < `0 ) : ai+1 . . . ak ∈ L(B), ak+1 . . . aj ∈ L(C) }, which is a subset of the intended value of Pi,j . This means that whenever a substring ai+1 . . . aj is in LG (BC) because of a partition with a middle point between m and `0 , the pair (B, C) must already be in Pi,j , as illustrated in the figure. Under these assumptions, complete(`, m, `0 , m0 ) constructs Ti,j for all ` 6 i < m and `0 6 j < m0 , which is shown in Figure 4(b). The algorithm is going to calculate products of square submatrices of T , putting the results to submatrices of P , and these operations shall be represented in the algorithm using the following notation. Given an n × n matrix X and a quadruple of numbers A = (`, m, `0 , m0 ), as in the arguments for the procedure complete(), denote by XA the (m − `) × (m0 − `0 ) submatrix of X formed by its rows from ` to m − 1 and its columns from `0 to m0 − 1. A product of matrices of sets X, Y ∈ (2N )m×m is a matrix Z ∈ (2N ×N )m×m , defined by Zi,j =

m [

Xi,k × Yk,j ,

k=1

as in the earlier Definition 3. Elementwise union of two matrices Z, Z 0 ∈ (2N ×N )m×m is denoted by b where Zbi,j = Zi,j ∪ Z 0 . Z ∪ Z 0 = Z, i,j 9

(a)

0

0

(b)

m

0

m

0

B

i

m ' m

m'

' BC

C

k j

m' n-1

n-1

Figure 4: (a) The submatrix of T calculated by compute(`, m); (b) The pre-conditions of complete(`, m, `0 , m0 ) and the submatrix of T it calculates.

Algorithm 2 (Parsing through matrix multiplication). Let G = (Σ, N, R, S) be a Boolean grammar in binary normal form. Let w = a1 . . . an , where n > 1 and ai ∈ Σ, be an input string; assume that n + 1 is a power of two. Main procedure: 1: compute(0, n + 1) 2: Accept if and only if S ∈ T0,n Procedure compute(`, m): 3: if m − ` > 4 then 4: compute(`, `+m 2 ) 5: compute( `+m 2 , m) `+m `+m 6: complete(`, 2 , 2 , m)

/* see Figure 5(a) */

Procedure complete(`, m, `0 , m0 ), which requires m − ` = m0 − `0 : 7: 8: 9: 10: 11: 12:

13: 14: 15: 16: 17: 18: 19: 20:

if m − ` = 1 and m = `0 then T`,`+1 = { A | A → a`+1 ∈ R } else if m − ` = 1 and m < `0 then T`,`0 = f (P`,`0 ) else if m − ` > 1 then `0 +m0 `+m `+m 0 0 0 `0 +m0 0 `0 +m0 denote B = (`, `+m 2 , 2 , m ), B = ( 2 , m, ` , 2 ), C = ( 2 , m, ` , 2 ), 0 0 0 0 0 `+m 0 ` +m `+m ` +m `+m ` +m0 0 0 D = (`, 2 , ` , 2 ), D = ( 2 , m, 2 , m ), E = (`, 2 , 2 , m0 ), as illustrated in Figure 5(b). complete(C) PD = PD ∪ (TB × TC ) complete(D) PD0 = PD0 ∪ (TC × TB0 ) complete(D0 ) PE = PE ∪ (TB × TD0 ) PE = PE ∪ (TD × TB0 ) complete(E)

10

(a) 0

0

+m 2

m

n-1

(b) 0

0

+m 2

m

'

'+m' 2

m'

+m 2

m +m 2

m n-1

' '+m' 2

m' n-1 Figure 5: (a) Matrix partition in compute(`, m); (b) matrix partition in complete(`, m, `0 , m0 ).

The partitions of the matrix in compute() and complete() are illustrated in Figure 5. Note that m − ` is a power of two in each call to compute() and to complete(), and accordingly, if the input string is of length 2k − 1, then the algorithm multiplies submatrices of size 1 × 1, 2 × 2, 4 × 4, and so on up to 2k−2 × 2k−2 . Furthermore, both m and ` are always divisible by m − `, and hence all 2k−i × 2k−i submatrices being multiplied are aligned over a 2k−i -step grid. Lemma 1. Let ` < m 6 `0 < m0 , where m − ` = m0 − `0 is a power of two, and assume that Ti,j = { A | ai+1 . . . aj ∈ L(A) } for all i and j with ` 6 i < j < m, as well as for all i, j with `0 6 i < j < m0 . Furthermore, assume that, for all ` 6 i < m and `0 6 j < m0 , Pi,j = { (B, C) | ∃k (m 6 k < `0 ) : ai+1 . . . ak ∈ L(B), ak+1 . . . aj ∈ L(C) }. Then complete(`, m, `0 , m0 ) returns with Ti,j = { A | ai+1 . . . aj ∈ L(A) } for all ` 6 i < m and `0 6 j < m0 . Proof. Induction on m − `. Basis I: m − ` = 1 and m = `0 . Then the algorithm has to handle a single element T`,`+1 , and this element is correctly computed in line 8. Basis II: m−` = 1 and m < `0 . Again, there is only one element to compute, and the current value of P`,`0 is { (B, C) | ∃k (` < k < `0 ) : a`+1 . . . ak ∈ L(B), ak+1 . . . a`0 ∈ L(C) } = { (B, C) | a`+1 . . . a`0 ∈ L(B)L(C) }. Then line 10 of complete() computes f (P`,`0 ) = { A | a`+1 . . . a`0 ∈ L(A) } and thus sets T`,`0 correctly. Induction step. Let ` < m 6 `0 < m0 with m−` = m0 −`0 > 1, and assume that complete(`1 , m1 , `2 , m2 ) works correctly for m1 − `1 = m2 − `2 < m − `. Consider the computation of complete(`, m, `0 , m0 ), which begins with the submatrices A, A0 , A00 , A000 , B and B 0 of T already computed, as in Figure 4(b). 0 `0 +m0 The first call to complete( `+m 2 , m, ` , 2 ) in line 13 requires that the current value of each Pi,j with 0 0 ` +m `+m 0 6 i < m and ` 6 j < (that is, in the C-submatrix of P ) is { (B, C) | ∃k (m 6 k < `0 ) : 2 2 ai+1 . . . ak ∈ L(B), ak+1 . . . aj ∈ L(C) }, which is true by the assumption. It also requires that the A0 and A00 -submatrices of T are known, which is also true. Then, by the induction hypothesis, this call to complete() determines all values of T in the submatrix C. 0 0 Next, the matrix product in line 14 adds to each Pi,j with ` 6 i < `+m and `0 6 j < ` +m (in the 2 2 `+m submatrix D) all pairs (B, C) with ai+1 . . . ak ∈ L(B), ak+1 . . . aj ∈ L(C) and 2 6 k < m. Taking into account that all such pairs with m 6 k < `0 were already there by the assumption, Pi,j now contains these pairs for all k with `+m 6 k < `0 . The A- and A00 -submatrices of T are known. Then the induction 2 0 `0 +m0 hypothesis is applicable to the subsequent call to complete(`, `+m 2 ,` , 2 ) in line 15, and so it computes all values of T in the D-submatrix. 11

0

0

` +m Symmetrically, the next lines 16–17 compute all Ti,j with `+m 6 j < m0 , that is, the 2 6 i < m and 2 0 submatrix D . 0 0 and ` +m 6 j < m0 (that is, in the E-submatrix At this moment, each element Pi,j with ` 6 i < `+m 2 2 of P ) contains all pairs (B, C) with ai+1 . . . ak ∈ L(B), ak+1 . . . aj ∈ L(C) and m 6 k < `0 . The subsequent `0 +m0 0 line 18 adds to each Pi,j all pairs with `+m 2 6 k < m, and line 19 adds all pairs with ` 6 k < 2 . With these additions, each Pi,j contains all pairs (B, C) satisfying ai+1 . . . ak ∈ L(B) and ak+1 . . . aj ∈ L(C) for 0 0 `+m `0 +m0 0 some `+m 6 k < ` +m 2 2 . All conditions necessary to call complete(`, 2 , 2 , m ) are now fulfilled, and, 0 0 `+m by the induction hypothesis, line 20 constructs all elements Ti,j with ` 6 i < 2 and ` +m 6 j < m0 . 2 0 This is the last remaining submatrix E, and now Ti,j is computed for all ` 6 i < m and ` 6 j < m0 , which completes the proof.

Lemma 2. The procedure compute(`, m), executed on any such ` and m that m − ` is a power of two, returns with Ti,j = { A | ai+1 . . . aj ∈ L(A) } for all ` 6 i < j < m. Proof. Induction on m − `. The base case is m−` = 2, in which the procedure compute() makes no recursive calls to itself, and line 6 calls complete(`, ` + 1, ` + 1, ` + 2). The pre-conditions for calling complete() are met, because there exist no elements Ti,j with ` 6 i < j < ` + 1 or with ` + 1 6 i < j < ` + 2, and no numbers k with ` + 1 6 k < ` + 1. Hence, by Lemma 1, this call results in T`,`+1 correctly computed (which will actually be done in line 8). Since this is the only element Ti,j with ` 6 i < j < m = ` + 2, the lemma holds for this case. If m−` > 4, then compute() will first call itself twice to compute the values of Ti,j for all ` 6 i < j < `+m 2 `+m and for all `+m 6 i < j < m. Then, when complete(`, `+m 2 2 , 2 , m) is called, the condition on Ti,j in Lemma 1 is satisfied. The second condition of the lemma is that each Pi,j contains all pairs (B, C) corresponding to some k with `+m 6 k < `+m 2 2 , and since there are no such values of k, this condition is satisfied as well. Therefore, Lemma 1 is applicable to this call, and it asserts that Ti,j will be correctly set for all i and j satisfying ` 6 i < `+m and `+m 2 2 6 j < m, which are all the remaining values of i and j. In order to estimate the running time of the algorithm, it would be sufficient to use the well-known general solutions of recurrence relations. However, in order to understand what the algorithm actually does, it is more useful to determine, exactly how many times the procedures compute() and complete() are called for subproblems of each size, and how many products of matrices of each size get computed. Lemma 3. Let the input string be of length 2k − 1. Then, in the computation of the main procedure, i. for each i ∈ {0, . . . , k − 1}, compute(`, m) with m − ` = 2k−i is called exactly 2i times, ii. for each i ∈ {1, . . . , k − 1}, complete(`, m, `0 , m0 ) with m − ` = 2k−i is called exactly 22i−1 − 2i−1 times, iii. for each i ∈ {2, . . . , k}, products of submatrices of size 2k−i × 2k−i are calculated exactly 22i−1 − 2i times. Proof. The first claim is proved by an obvious induction on i, and the proof can be safely omitted. Turning to the second claim, the induction here is also rather simple, and proceeds as follows. For succinctness, the phrase “complete() of size s” shall refer to any calls to the procedure complete(`, m, `0 , m0 ) with m − ` = s. Then, as the base case, i = 1, a call to complete() of size 2k−1 is made only once, from the top-level compute(0, 2k ), and accordingly 22i−1 − 2i−1 = 21 − 20 = 1. For the induction step, assume that complete() of size 2k−i is called 22i−1 − 2i−1 times, and consider the calls to complete() of size 2k−i−1 . First, complete() of size 2k−i−1 is called 4 times from each instance of complete() of size 2k−i , and secondly, it is called once from each instance of compute(`, m) with m − ` = 2k−i . In total, this sums up to 4 · (22i−1 − 2i−1 ) + 1 · 2i = 22(i+1)−1 − 2(i+1)−1 calls, as claimed. Finally, each call to complete() of size 2k−(i−1) calculates four products of 2k−i × 2k−i matrices. In total, products of matrices of size 2k−i × 2k−i are calculated 4 · (22(i−1)−1 − 2i−2 ) = 22i−1 − 2i times. 12

According to these calculations, the time spent on matrix multiplication dominates the running time, which leads to the following estimation of the algorithm’s complexity. Theorem 1. Given a Boolean grammar G in binary normal form and a string of length
n, Algorithm 2 constructs the parsing table T for this grammar and this string in time O |G|·BMM(n) log n , where BMM(n) is the time needed to
multiply two n×n Boolean matrices. Assuming that BMM(n) = n2+Ω(1) , the complexity is Θ |G| · BMM(n) . Proof. Assume that n = 2k − 1. The correctness of the algorithm is asserted by Lemma 2, according to which, the call to compute(0, 2k ) in line 1 of the main procedure calculates all Ti,j with 0 6 i < j < 2k . If n + 1 is not a power of two, then one can construct the parsing table for a padded string, and then use only its relevant entries. In order to estimate the running time, by Lemma 3, it is sufficient to sum up the time used for matrix multiplication. Let BMM(n) = nω · f (n), where ω > 2 and f (n) = no(1) . According to Lemma 3(iii), for every i ∈ {2, . . . , k}, products of matrices of size 2k−i × 2k−i are calculated 22i−1 − 2i times, and each of them requires multiplying C = O(|G|) actual Boolean matrices of size 2k−i × 2k−i . The total number of operations is thus estimated as follows:

C

k X

(22i−1 − 2i )BMM(2k−i ) 6 C

i=2

k X

22i−1 BMM(2k−i ) = C

i=2

= C · 2ωk

k X

k X

22i−1 2ω(k−i) f (2k−i ) =

i=2

2(2−ω)i−1 f (2k−i ) 6 C · 2ωk f (2k )

i=2

k X

2(2−ω)i−1 = C · BMM(2k )

i=2

k X

2(2−ω)i−1 .

i=2

It remains to estimate the sum. Under the assumption that ω > 2, it is bounded by a constant as a convergent geometric series, k ∞ X X 22(2−ω) , 2(2−ω)i 6 2(2−ω)i = 1 − 22−ω i=2 i=2

and therefore, the total number of operations is O |G| · BMM(2k ) = O |G| · BMM(n) . On the other hand, if ω = 2, then k X i=2

2(2−ω)i =

k X

1 = k − 1,

i=2



leading to the upper bound O |G| · BMM(2k ) · k = O |G| · BMM(n) log n . 5. Notes on implementation The restriction on the length of the string requiring it to be a power of two minus one is convenient for the algorithm’s presentation, but it would be rather annoying for any implementation. This restriction can be circumvented as follows. Let w = a1 . . . an be an input string of any length n > 1. The algorithm shall construct a table of size (n + 1) × (n + 1), yet while doing so, it will imagine a larger table of size rounded up to the next power of two. Whenever a subroutine call is concerned entirely with the elements beyond the edge of the table, this call is skipped. Any calculations of matrix products with one of the matrices split by this edge are adjusted to multiply rectangular submatrices fitting into the table. In the main procedure, the call to compute(0, n + 1) in line 1 shall be replaced with 1: compute(0, 2dlog2 (n+1)e )

13

The procedure compute(`, m) may now be called for a number m pointing beyond the end of the string, and it will split this range of positions into two halves as usual. The subsequent recursive calls to compute(`, m) may have the entire range of positions beyond the end of the string, in which case there is nothing to compute. Accordingly, the procedure compute(`, m) is modified to begin with a conditional statement ensuring that n > ` + 1, and returning immediately if it does not hold. Similar changes are made to the procedure complete(), which needs to be invoked only if the second range of positions contains at least one input symbol. For this purpose, complete(`, m, `0 , m0 ) shall begin with testing that n > `0 , returning otherwise. Finally, the calls to the matrix multiplication procedure will now also occasionally refer to submatrices lying partially or completely beyond the (n + 1) × (n + 1) matrices T and P . If one of the matrices being multiplied is completely beyond the end of the string, this product need not be computed. If it is only partially beyond, then it is sufficient to multiply only the portions of the matrices that fit into the m−` (n + 1) × (n + 1) area. For instance, consider the matrix product TD × TB0 in line 19, in which the m−` 2 × 2 m−` m−` 0 0 `0 +m0 0 submatrix D beginning at (`, ` ) is multiplied by the 2 × 2 submatrix B beginning at (` , 2 ). 0 0 Assume that ` +m 6 n < m0 . Then the second matrix B 0 does not entirely fit into the (n + 1) × (n + 1) area, 2
0 0 `0 +m0 and the algorithm shall multiply D by the m−` rectangular matrix beginning at (`0 , ` +m 2 × n−
2 2 ). 0 0 `0 +m0 Their product is also a matrix of size m−` , which is placed in P beginning at (`, ` +m 2 × n− 2 2 ). If 0 0 n is strictly less than ` +m 2 , then this product is not computed at all. Similar modifications apply to all matrix products computed by the algorithm. Another question concerns the data structures for the algorithm. In general, not everything mentioned in the theoretical presentation of the algorithm would need to be computed for an actual grammar. First assume that the grammar is ordinary context-free, that is, uses disjunction only. In this case, whenever a pair (B, C) is added to Pi,j , it will eventually make all nonterminals A with a rule A → BC be added to Ti,j ; and if there are no such nonterminals, then there is no need to consider the pair (B, C). Accordingly, the data structure P is not needed, and all matrix multiplication procedures can output their result directly into the appropriate elements of T . If the grammar is conjunctive or Boolean, then it is indeed necessary to use P , yet only for the rules involving multiple conjuncts. Simple rules with a unique positive conjunct can be treated in the simplified way described above, with all matrix products being directly flushed into T . If there exists a rule A → BC & . . . with at least two conjuncts, or any rule A → ¬BC & . . ., then all data about the pair (B, C) need to be stored in P as described in the algorithm. These data shall be used in the calculation of f , which takes into account all conjuncts in each complex rule involving conjunction or negation. With these observations made, the following data structures naturally come to mind: A • For each nonterminal A ∈ N , an (n + 1) × (n + 1) upper-triangular Boolean matrix T A , with Ti,j representing the membership of A in the set Ti,j . All matrix products computed in the algorithm have some submatrices of this matrix as the arguments.

• For every such pair (B, C) ∈ N × N that occurs in any multiple-conjunct rule A → BC & . . . or is negated in any rule A → ¬BC & . . ., the algorithm shall maintain an (n + 1) × (n + 1) upper-triangular Boolean matrix P BC . The algorithm was implemented by Okhotin and Reitwießner [20] on a computer equipped with a graphics processing unit (GPU), which is used to accelerate the computation of Boolean matrix products. 6. Constructing a parse tree The algorithm has so far been defined as a recognizer, which determines whether the string is generated by the grammar or not. If the string is found to be generated by the grammar, one would typically be interested in obtaining a parse tree of this string. How can this algorithm be modified to construct such a tree? 14

At the first glance, since all the necessary information is in the parsing table Ti,j , one can first build this table by Algorithm 2, and then use another procedure to construct the parse tree according to this table. For the case of ordinary context-free grammars, this latter procedure is well-known [10], and it can be straightforwardly generalized to Boolean grammars, as presented in Algorithm 3. Algorithm 3 (Simple parse tree construction). Let G = (Σ, N , P , S) be a Boolean grammar in binary normal form, let w = a1 . . . an with n > 1 and ai ∈ Σ be an input string, and let the sets Ti,j = { A | ai+1 . . . aj ∈ LG (A) } be available for all 0 6 i < j 6 n. Then, for every substring a`+1 . . . am and for every nonterminal A ∈ T`,m generating this substring, the following procedure constructs an expanded parse tree of a`+1 . . . am from A. Procedure parse(A, `, m): 1: if m − ` = 1 then 2: return tree with root A → am connected to the leaf am 3: else 4: for all rules A → B1 C1 & . . . & Br Cr & ¬Br+1 Cr+1 & . . . & ¬Bs Cs & ¬ε ∈ R do 5: integer p[1..s] 6: for t = 1 to s do 7: for k = ` + 1 to m − 1 do 8: if Bt ∈ T`,k and Ct ∈ Tk,m then 9: p[t] = k 10: if p[t] is initialized for all t ∈ {1, . . . , r} and for none of t ∈ {r + 1, . . . , s} then 11: Create a node τ labelled A → B1 C1 & . . . & Br Cr & ¬Br+1 Cr+1 & . . . & ¬Bs Cs & ¬ε 12: for t = 1 to r do 13: Add descendant parse(Bt , `, p[t]) to τ 14: Add descendant parse(Ct , p[t], m) to τ 15: return τ This obvious generalization of a well-known procedure maintains its running time, which is proportional to n · t, where n is the length of the input and t is the number of nodes in the resulting tree. For ordinary context-free grammars, t = Θ(n), and hence the time complexity of this step is Θ(n2 ), which is smaller than the time spent constructing the parsing table. However, for conjunctive and Boolean grammars, this upper bound no longer holds. First of all, Algorithm 3 as it is constructs an expanded parse tree, which, as shown in Example 3, may be of exponential size. One can straighforwardly modify the algorithm to construct a condensed parse tree using the method of memoization: so that every time the function parse() builds a parse subtree for some nonterminal and substring, it remembers a pointer to this subtree, and any subsequent calls to parse() with the same arguments shall directly return this pointer. However, even with this change, a condensed parse tree may be quadratic in size, and hence Algorithm 3 requires cubic time in the worst case. The bottleneck of Algorithm 3 is the loop in line 7 over all intermediate positions in a substring, which does essentially the same search of a partition as in line 7 of Algorithm 1, and similarly leads to cubic complexity. When the parsing table T is constructed with the help of matrix multiplication in Algorithm 2, the search for those intermediate positions is implicitly carried out inside the matrix multiplication procedure: whenever a product of two Boolean matrices A, B ∈ Bn×n is calculated as a Boolean matrix C ∈ Bn×n with Ci,j =

n _

Ai,k ∧ Bk,j ,

k=1

every number k with Ai,k = Bk,j = 1 corresponds to a partition of some substring in the parsing algorithm. If, for some i and j, such a number k exists, then the matrix multiplication procedure sets Ci,j to 1, but the number k itself is not recorded anywhere. In order to construct a parse tree efficiently, the algorithm should compute extra data along with Boolean matrix multiplication: given Boolean matrices A, B ∈ Bn×n , 15

it should calculate a (not necessarily unique) integer matrix C 0 ∈ Zn×n , where ( −1, if Ci,j = 0, 0 Ci,j = any k with Ai,k ∧ Bk,j , if Ci,j = 1. These values are known as witnesses for Boolean matrix multiplication, and can be computed in time MM(n) logO(1) n by the algorithm of Alon and Naor [2], where MM(n) is the number of operations in a ring needed to multiply two n × n matrices over that ring. It remains to modify Algorithm 2 to calculate and store these values, and then reimplement Algorithm 3 to make use of them instead of recalculating the partitions. Let G = (Σ, N, R, S) be a grammar and let w = a1 . . . an be an input string. In addition to the parsing table Ti,j = { A ∈ N | ai+1 . . . aj ∈ LG (A) }, as in Algorithms 1–2, the new algorithm shall construct an (n + 1) × (n + 1) matrix P 0 of functions from N × N to {−1} ∪ {1, . . . , n − 1}, satisfying the following condition. ( −1, if ai+1 . . . aj ∈ / LG (BC), 0 Pi,j (B, C) = any k with ai+1 . . . ak ∈ LG (B) and ak+1 . . . aj ∈ LG (C), if ai+1 . . . aj ∈ LG (BC) This is essentially a collection of integer matrices for each relevant concatenation BC. Theorem 2. There exists an algorithm that, given a string w = a1 . . . an and a Boolean grammar G = 0 (Σ, N, R, S), constructs the tables Ti,j and Pi,j in time |G| · MM(n) logO(1) n, where MM(n) is the number of arithmetical operations needed to multiply two n × n integer matrices. Sketch of a proof. The new algorithm is generally the same as Algorithm 2, with the following differences. Each assignment statement involving a Boolean matrix product, such as PD = PD ∪ (TB × TC ) in line 14, 0 is now interpreted in the following sense. For every nonterminal symbols B and C and for every entry Pi,j 0 with (i, j) in the D-submatrix, where the current value of Pi,j (B, C) is −1, this statement should assign 0 Pi,j (B, C) := k, if there exists such a number k, that (i, k) lies in the B-submatrix, (k, j) lies in the Csubmatrix, and B ∈ Ti,k and C ∈ Tk,j . If multiple such values of k exist, then any of them can be written 0 into Pi,j (B, C). This can be achieved as follows. For every pair (B, C), the algorithm invokes an external procedure that determines witnesses for Boolean matrix multiplication, such as the algorithm of Alon and Naor [2]. Then, an appropriate offset is added to each non-negative value, and the functions in each entry of the result are merged with the appropriate entries of P 0 . Finally, the function f : 2N ×N → 2N is redefined to map a given function (from N × N to integers) to a set of nonterminals: f (P 0 ) = {A | ∃A → B1 C1 & . . . & Bm Cm & ¬D1 E1 & . . . & ¬Dm0 Em0 ∈ R : P 0 (Bt , Ct ) 6= −1 and P 0 (Dt , Et ) = −1 for all t}, 0 0 Consider that Pi,j (B, C) 6= −1 if and only if Pi,j (B, C) is true. If the values of Pi,j (B, C) computed by the resulting algorithm are accordingly considered as Boolean values, one can see that it computes all the same data as Algorithm 2, and uses them to calculate the same values of Ti,j . Therefore, the new algorithm is bound by the correctness statements in Lemmata 1–2, while its performance is described by Lemma 3. The extra data on the intermediate positions stored in P 0 are correct, because these are witnesses for the multiplication of the appropriate Boolean matrices.

Using these data structures, Algorithm 4 constructs a parse tree without any further search operations. Its running time is proportional to the number of nodes in the resulting condensed parse tree, and is therefore at most quadratic in the length of the string for any Boolean grammar. For an ordinary contextfree grammar, Algorithm 4 works in linear time. Together with the procedure described in Theorem 2, this yields a parsing algorithm with running time |G| · MM(n) logO(1) n. 16

Algorithm 4 (Constructing parse tree using the integer matrix P 0 ). Let G = (Σ, N, R, S) be a Boolean grammar in binary normal form, let w = a1 . . . an , with n > 1 and ai ∈ Σ, be an input string, and let 0 the sets Ti,j and the functions Pi,j : N → {−1} ∪ {1, . . . , n − 1} be available for all 0 6 i < j 6 n. Then, for every substring a`+1 . . . am and for every nonterminal A ∈ T`,m generating this substring, the following procedure constructs a condensed parse tree of a`+1 . . . am from A. Let M [A, `, m] be a global variable used to store the computed values of this procedure. Procedure parse(A, `, m): 1: if M [A, `, m] is defined then 2: return M [A, `, m] 3: else if m − ` = 1 then 4: M [A, `, m] = tree with root A → am connected to the leaf am 5: return M [A, `, m] 6: else 7: for all rules A → B1 C1 & . . . & Br Cr & ¬Br+1 Cr+1 & . . . & ¬Bs Cs & ¬ε ∈ R do 8: integer p[1..s] 9: for t = 1 to s do 0 (B, C) 10: p[t] = P`,m 11: if p[t] is defined for all t ∈ {1, . . . , r} and for none of t ∈ {r + 1, . . . , s} then 12: Create a node τ labelled A → B1 C1 & . . . & Br Cr & ¬Br+1 Cr+1 & . . . & ¬Bs Cs & ¬ε 13: for t = 1 to r do 14: Add descendant parse(Bt , `, p[t]) to τ 15: Add descendant parse(Ct , p[t], m) to τ 16: M [A, `, m] = τ 17: return τ

7. Generic algorithm The original Valiant’s algorithm was presented in a generic form, in which it computes a certain kind of closure of an upper triangular matrix. The elements of the matrix come from a set X, equipped with a product operation ⊗ : X × X → X and an associative and commutative sum operation t : X × X → X. The sum must be distributive over the product, which makes this structure a semiring without associativity of the product. Then, for any upper triangular matrix M ∈ X n×n , Valiant [26] defines its transitive closure T = T (M ) ∈ X n×n by j−1 G Ti,j = Mi,j t Ti,k ⊗ Tk,j . k=i+1

The application to formal grammars uses a transitive closure of a matrix of the following special form. For any sequence x = x1 . . . xn , with n > 1 and xi ∈ X, define the matrix M = M (x) ∈ X (n+1)×(n+1) , by Mi−1,i = xi and Mi,j = ∅ for i < j − 1. Its transitive closure is, by definition, the matrix T = T (x) ∈ X (n+1)×(n+1) satisfying the equations Ti−1,i = xi , Ti,j =

j−1 G

Ti,k ⊗ Tk,j .

k=i+1

The updated version of Valiant’s algorithm presented in this paper no longer relies on using a single data type X, as separating matrices of sets of nonterminals and matrices of sets of pairs becomes essential. Nevertheless, it can also be presented in a generic form, using operations over slightly different abstract structures, as in the following ad hoc definition. 17

Definition 4. In the context of this section, an algebra is a quintuple A = (X, Y, ⊗, t, f ), which consists of any two sets X and Y , a product operation ⊗ : X × X → Y , an associative and commutative sum operation t : Y × Y → Y and any function f : Y → X. For any sequence x = x1 . . . xn , with n > 1 and xi ∈ X, define the matrix T = T (x) ∈ X (n+1)×(n+1) by the equations Ti−1,i = xi , Ti,j = f

 j−1 G

 Ti,k ⊗ Tk,j .

k=i+1

Obviously, the table T can be calculated directly by the formula, as in Algorithm 1, which requires computing Θ(n3 ) products ⊗ and sums t, as well as evaluating the function f exactly n(n−1) times. In 2 order to carry out the same computation faster, the proposed algorithm shall rely on an external procedure for calculating matrix products of the following kind. Definition 5. Let A = (X, Y, ⊗, t, f ) be an algebra, as in Definition 4. For any two matrices A, B ∈ X n×n , define their product as a matrix C ∈ Y n×n , with Ci,j =

n G

Xi,k ⊗ Xk,j .

k=1

A matrix multiplication algorithm for a family of algebras A is given an algebra A ∈ A and a pair of matrices A, B, and calculates their product C. The proposed generalization of Algorithm 2 shall compute the table T for any given algebra A = (X, Y, ⊗, t, f ) and any given sequence x ∈ X + . The algebra is given to the algorithm as black-box procedures for calculating the basic operations and the matrix product, as well as black-box memory blocks for storing any elements of X and Y . It is assumed that the matrix product procedure runs in time s(m)·MM(n), where m is the size of the given algebra (under any measure of size) and the factor s(m) represents the dependence on this size, while MM(n) is the number of ring operations needed to multiply two n × n matrices. Theorem 3. Consider a family of algebras A , for which the product of two given n × n matrices over a given algebra of size m can be calculated in time s(m) · MM(n). Then there exists a generic algorithm, which is given the procedures for the sum t, the function f and the matrix product, as well as a string x ∈ X + of length n, and which computes the corresponding matrix
T (x) in time O(s(m)MM(n) log n). Assuming that MM(n) = n2+Ω(1) , the complexity is Θ s(m)MM(n) . The algorithm and its analysis are exactly the same as Algorithm 2 and Theorem 1, with symbols reinterpreted as follows. 1. All matrix products are taken with respect to Definition 5. 2. The elementwise union of matrices is replaced with elementwise abstract sum t. 3. The values of the base elements T`,`+1 are taken directly from the input string x ∈ X. Note that it is also possible to extend Valiant’s closure of a matrix over a semiring X to the case of matrices over an algebra A = (X, Y, ⊗, t, f ) as follows. For a matrix M ∈ Y n×n , define its transitive closure as the matrix T = T (M ) ∈ X n×n satisfying Ti,j = f Mi,j t

j−1 G


Ti,k ⊗ Tk,j .

k=i+1

Then Algorithm 2 can be easily modified to calculate the closure T (M ) for a given matrix M . However, since this paper does not explore any applications of this closure, this line of study is not pursued here. 18

8. Applications of the generic form In the generic form stated in Theorem 3, the algorithm can be applied to various families of grammars. For example, for an ordinary context-free grammar G = (Σ, N, R, S) in binary normal form and for an input string w = a1 . . . an , one can set X = 2N , Y = 2N ×N , ⊗ = ×, t = ∪, xi = { A ∈ N | A → ai ∈ R }, f (y) = { A ∈ N | ∃A → BC ∈ R : (B, C) ∈ y }.

(3)

For Boolean grammars, the only difference is in the function f , which has to handle more varied Boolean logic in the rules. For these product (⊗) and sum (t) operations, calculating the matrix product of Definition 5 amounts to multiplying several pairs of Boolean matrices, one for each pair (B, C) ∈ N × N that has a corresponding conjunct BC or ¬BC in any rule of the grammar. Therefore, the factor depending on the grammar is s(m) = O(m), where m = |G|, and the required generalized matrix multiplication takes time O(|G| · MM(n) log n), so that Theorem 3 implies the earlier Theorem 1. The generalized algorithm also directly applies to stochastic grammars, which define, for every string w ∈ Σ∗ and for every nonterminal symbol A ∈ N , a number between 0 and 1 indicating to which extent w has the property A, or, in other words, the probability of the string’s being in LG (A). Definition 6. A stochastic grammar is a quintuple G = (Σ, N, R, Φ, S), where R is a finite set of rules of ∗ the form A → Pα, with A ∈ N and α ∈ (Σ ∪ N ) , and Φ : R → [0, 1] sets the probability of each rule, which must satisfy α: A→α∈R Φ(A → α) = 1 for each A ∈ N . For every string w ∈ Σ∗ and for every nonterminal symbol A ∈ N , consider all parse trees of w according to A, with respect to the ordinary, non-probabilistic grammar G0 = (Σ, N, R, S). Then the probability of every such tree is the product of the probabilities of all its nodes (each determined as the probability of the rule it is labelled with), and the probability of a string w having the property A is the sum of probabilities of all its trees: X Y Φ(X → ξ). t is a tree v is a node in t, of w from A labelled with X → ξ

Consider that a stochastic grammar can be transformed to a normal form with all rules of the form A → BC, with B, C ∈ N , or A → a, with a ∈ Σ. Using this normal form, a fast algorithm for computing the probabilities for all substrings of a given string can be directly inferred from Theorem 3. Corollary 1. There exists an instance of the generic algorithm in Theorem 3, which, given a stochastic grammar G and an n-symbol string, determines the probability of this string, as defined by the grammar,
and does so in time O |G| · MM(n) log n , where MM(n) is the time needed to multiply two n × n numerical
matrices representing probabilities. Assuming that MM(n) = n2+Ω(1) , the complexity is Θ |G| · MM(n) . Sketch of a proof. Let G = (Σ, N, R, S) be the grammar and let w = a1 . . . an be the input string. Let X be the set of all functions from N to real or rational numbers in [0, 1] (or whatever numbers are used to represent probabilities); these are essentially |N |-tuples of numbers, with components corresponding to nonterminals. Similarly, let Y be the set of all functions from N × N to [0, 1]. Define x ⊗ x0 = y, 0

y t y = yb, f (y) = x,

with y(B, C) = x(B) · x0 (C), with yb(B, C) = y(B, C) + y 0 (B, C), X with x(A) = Φ(A → BC) · y(B, C). A→BC∈R

19

The desired probabilities of all substrings of w in the languages generated by all nonterminals are now contained in a matrix T = T (x) ∈ X (n+1)×(n+1) , defined by ( Ti−1,i = xi ,

Ti,j = f

with xi (A) =

 j−1 G

Φ(A → ai ), if A → ai ∈ R, 0, otherwise,

 Ti,k ⊗ Tk,j .

k=i+1

In order to calculate the matrix T according to Theorem 3, one should provide a matrix multiplication algorithm with respect to the sum t and the product ⊗. Given matrices M, M 0 ∈ X n×n , this algorithm separately considers each concatenation BC that occurs in any rules of the grammar, and extracts two real-valued n × n matrices of B-components of M and of C-components of M 0 . The product of these two matrices forms the (B, C)-component of the result. Thus, the product of M and M 0 is obtained using |G| · MM(n) operations. Using this matrix multiplication procedure, the generalized algorithm of Theorem 3 computes T in the stated time. Since the original Valiant’s [26] algorithm in its generic form is applicable to stochastic grammars as well, Corollary 1 is by no means a new result. On the other hand, Definition 6 was recently extended to stochastic conjunctive grammars by Zier-Vogel and Domaratzki [29]; these grammars are not covered by the generic form of Valiant’s algorithm, yet can be handled by the algorithm in Theorem 3. Definition 7 (Zier-Vogel and Domaratzki [29]). A stochastic conjunctive grammar is a quintuple G = (Σ, N, R, Φ, S), where G0 = (Σ, N, R, S) is a conjunctive grammar, whereas Φ : R → [0, 1] sets the probability of each rule. The probability of a string w ∈ Σ∗ having a property A ∈ N is defined as the following sum over all expanded parse trees: Y X Φ(X → ξ1 & . . . & ξm ). v is a node in t, t is an expanded tree labelled with X → ξ1 & . . . & ξm of w from A

Stochastic conjunctive grammars can be transformed to the binary normal form by the same procedure as for the non-stochastic conjunctive grammars, expanded to calculate the probabilities of the resulting rules. For grammars in binary normal form, a fast parsing algorithm is obtained by the same argument as in Corollary 1, with a small correction in the definition of f . Corollary 2. The generic algorithm in Theorem 3 has an instance, which, given a stochastic conjunctive grammar G in binary
normal form and an input string of length n, determines
its probability in time O |G| · MM(n) log n . If MM(n) = n2+Ω(1) , then the complexity is Θ |G| · MM(n) . Sketch of a proof. The sets X and Y are defined as in Corollary 1: functions in X map nonterminals to probabilities, whereas functions in Y map pairs of nonterminals to probabilities. The product of two Xfunctions and the sum of two Y -functions are also the same as in Corollary 1, while the mapping f : Y → X is defined by X f (y) = x, with x(A) = Φ(A → B1 C1 & . . . & Bm Cm ) · y(B1 , C1 ) · · · y(Bm , Cm ). A→B1 C1 &... & Bm Cm ∈R

Then the algorithm in Theorem 3 calculates the desired probabilities. The next Section 9 presents a further application of the generic algorithm to parsing Boolean grammars under the alternative definition of their semantics. 20

9. Application to the well-founded semantics The well-founded semantics of Boolean grammars were proposed by Kountouriotis, Nomikos and Rondogiannis [11]. These semantics are applicable to every syntactically valid Boolean grammar, and define a three-valued language generated by each nonterminal symbol. Three-valued languages, as defined by Kountouriotis et al. [11], are mappings from Σ∗ to {0, 21 , 1}, where 1 and 0 indicate that a string definitely is or definitely is not in the language, while 12 stands for “undefined”. This paper adopts the mathematically equivalent representation of three-valued languages as pairs (L, L0 ) with L ⊆ L0 ⊆ Σ∗ , where L and L0 represent a lower bound and an upper bound on a language that is not known precisely. A string in both L and L0 definitely is in the language, a string belonging to neither of them definitely is not, and if a string is in L0 but not in L, then its membership is not defined. In particular, if L = L0 , then the language is completely defined, and a pair (∅, Σ∗ ) means a language about which nothing ∗ is known. Denote the set of such pairs by 3Σ . Boolean operations and concatenation are generalized from two-valued to three-valued languages as follows: (K, K 0 ) ∪ (L, L0 ) = (K ∪ L, K 0 ∪ L0 ) (K, K 0 ) ∩ (L, L0 ) = (K ∩ L, K 0 ∩ L0 ) (L, L0 ) = (L0 , L) (K, K 0 )(L, L0 ) = (KL, K 0 L0 ) Two different partial orderings on three-valued languages are defined. First, they can be compared with respect to the degree of truth: (K, K 0 ) vT (L, L0 )

if K ⊆ L and K 0 ⊆ L0 .

This means that whenever a string belongs to the lesser language, it must be in the greater language as well, and if the membership of a string in the lesser language is uncertain, then it must be either uncertain or true for the greater language. The other ordering is with respect to the degree of information: (K, K 0 ) vI (L, L0 )

if

K ⊆ L and L0 ⊆ K 0 .

It represents the fact that (K, K 0 ) and (L, L0 ) are approximations of the same language, and that (L, L0 ) is more precise, in the sense of having fewer uncertain strings. If a string is definitely known to belong or not to belong to the lesser language, then it must have the same status in the greater language, and if a string is uncertain in the lesser language, then the greater language may have any value of this string, that is, keep it as uncertain or define it as a member or a non-member. Both orderings are extended to
vectors of three-valued languages. The truth-ordering has a bottom element ⊥T = (∅, ∅),
. . . , (∅, ∅) , that is, every language is completely defined as ∅; the top element
is (Σ∗ , Σ∗ ), . . . , (Σ∗ , Σ∗ ) . For the information-ordering, the bottom element is ⊥I = (∅, Σ∗ ), . . . , (∅, Σ∗ ) , in which all languages are fully undefined. There is no top element for vI , while all completely defined languages are maximal elements. As in the two-valued case, concatenation, union and intersection, as well as every combination thereof, are monotone and continuous with respect to the truth ordering; complementation is not monotone. With respect to the information ordering, concatenation and all Boolean operations, including complementation, are monotone and continuous, which extends to any combinations of these operations. These properties are crucial for the following definition, which uses a two-level fixpoint iteration, where the first level defines monotone sequences with respect to the truth ordering, while the second level is a single sequence, monotone with respect to the information ordering. Definition 8 (Kountouriotis et al. [11]). Let G = (Σ, N, R, S) be a Boolean grammar, let N = 0 Σ∗ |N | {A1 , . . . , A|N | }. For any vector K = ((K1 , K10 ), . . . , (K|N | , K|N , define a function ΘK : | )) ∈ (3 ) 21

∗

∗

(3Σ )|N | → (3Σ )|N | by substituting its argument into positive conjuncts and K into negative conjuncts: for each A ∈ N , the A-component of the vector [ΘK (L)]A is defined as \ m

[

[ΘK (L)]A =

A→α1 &... & αm & ¬β1 &... & ¬βm0 ∈R

Using this function, define Ω(K) =

0

αi (L) ∩

i=1

m \

 βj (K) .

j=1

G G T Θ`K (⊥T ) and let M = I Ωk (⊥I ). Then, according to the well`>0

k>0

founded semantics of Boolean grammars, the three-valued language generated by each A ∈ N is [M ]A ; it is denoted by LG (A) = (LA , L0A ), where LA ⊆ Σ∗ is its lower approximation and L0A ⊆ Σ∗ is its upper approximation. The main result justifying the validity of the well-founded semantics is that M is a solution of the following system of equations in three-valued languages: A=

[ A→α1 &... & αm & ¬β1 &... & ¬βm0 ∈R

\ m

0

αi (L) ∩

i=1

m \

 βj (L)

(for A ∈ N ).

j=1

There exists the following generalization of the binary normal form to the well-founded semantics: Theorem A (Kountouriotis et al. [11]). Every Boolean grammar, as in Definition 8, can be effectively transformed to a grammar in binary normal form, in which every rule is of the following form. A → B1 C1 & . . . & Bm Cm & ¬D1 E1 & . . . & ¬Dm0 Em0 & ¬ε

(m > 1, m0 > 0, Bi , Ci , Dj , Ej ∈ N )

A→a

(a ∈ Σ)

A → a & ¬A

(a ∈ Σ)

S→ε S → ε & ¬S (the latter two forms are only allowed if S never appears in the right-hand sides of any rules) The transformation maintains the three-valued language generated by the grammar. This normal form allows defining and algorithmically computing a parsing table similar to Ti,j in Section 3. The entries of this table are three-valued subsets of N , and such subsets are generally defined as follows: for every base set X0 , the set of its three-valued subsets is 3X0 = { (X, X 0 ) | X ⊆ X 0 ⊆ X0 }; the union and the Cartesian product of two three-valued sets are defined as (X, X 0 ) × (Y, Y 0 ) = (X × Y, X 0 × Y 0 ), (X, X 0 ) ∪ (Y, Y 0 ) = (X ∪ Y, X 0 ∪ Y 0 ). Now, for every input string w = a1 . . . an , the desired three-valued parsing table is T ∈ (3N )(n+1)×(n+1) , with
Ti,j = { A | ai+1 . . . aj ∈ LA }, { A | ai+1 . . . aj ∈ L0A } . Once the table is calculated, the three-valued membership of the string w in the language is given by whether S is in both components of T0,n , or only in the second component, or in neither. An extension of Algorithm 1 for computing the three-valued parsing table was defined and proved correct by Kountouriotis et al. [11, Sect. 6]. In effect, they have established the following theorem, stated here in the notation of this paper.

22

Theorem B (Kountouriotis et al. [11]). For every Boolean grammar G = (Σ, N, R, S) in binary normal form and for every input string w = a1 . . . an , the table T ∈ (3N )(n+1)×(n+1) can be computed in time O(|G| · n3 ) using the formulae
Ti−1,i = { A | A → ai ∈ R }, { A | A → ai ∈ R or A → ai & ¬A ∈ R } , Ti,j = f

 j−1 [

 Ti,k × Tk,j ,

k=i+1

where the three-valued logic in the rules of the grammar is expressed in the function f : 3N ×N → 3N , defined by f (P, P 0 ) = {A | ∃A → B1 C1 & . . . & Bm Cm & ¬D1 E1 & . . . & ¬Dm0 Em0 & ¬ε : (Bi , Ci ) ∈ P and (Dj , Ej ) ∈ / P 0 for all applicable i, j}, {A | ∃A → B1 C1 & . . . & Bm Cm & ¬D1 E1 & . . . & ¬Dm0 Em0 & ¬ε :
(Bi , Ci ) ∈ P 0 and (Dj , Ej ) ∈ / P for all applicable i, j} , for all P ⊆ P 0 ⊆ N × N. Note that the Cartesian product in the expression for Ti,j maps three-valued subsets of N to three-valued subsets of N × N , and the union is over three-valued subsets of N × N . The operations in Theorem B are composed exactly as in Algorithm 1, and directly computing them requires cubic time. However, the same data can again be computed more efficiently using matrix multiplication, which shall now be demonstrated by encoding all base sets and operations into the abstract form required by Theorem 3. Corollary 3. There exists an instance of the generic algorithm in Theorem 3, which, given a three-valued Boolean grammar G in binary normal form and an n-symbol
string, determines the membership status of this string in L(G), and does so in time O |G| · BMM(n) log n , where BMM(n) is the time needed to multiply
two n × n Boolean matrices. Assuming that BMM(n) = n2+Ω(1) , the complexity is Θ |G| · BMM(n) . Sketch of a proof. The proof adopts the notation of Definitions 4–5, into which the formalism of the wellfounded semantics has to be converted in order to apply Theorem 3. The algebra A = (X, Y, ⊗, t, f ) generalizes the one defined for ordinary two-valued grammars (3) by adopting three-valued sets. The sets X and Y encode three-valued membership of a single substring in all languages LG (A) and in all languages LG (BC), respectively. X = 3N N ×N

Y =3

(three-valued subsets of N ) (three-valued subsets of N × N )

The product ⊗ and the sum t are defined as the corresponding three-valued extensions of set-theoretic operations. ⊗=×

(three-valued Cartesian product)

t=∪

(three-valued union)

The function f : 3N ×N → 3N is the same as in Theorem B. It remains to present a matrix multiplication algorithm for these product and sum operations. Consider f ∈ (3N )n×n . For every nonterminal B ∈ N , consider the B-submatrix of M as a any two matrices M, M B pair of Boolean matrices M B , M 0B ∈ Bn×n , so that, for each element Mi,j = (X, X 0 ), the bit Mi,j is true if 0B 0 B ∈ X, and Mi,j is true if B ∈ X (these are the lower bound and the upper bound on the membership of B f is a pair of Boolean matrices in the three-valued set Mi,j ). Similarly, for every C ∈ N , the C-submatrix of M C 0C B B 0C 0C f ,M f . Then the Boolean matrix products M × M f and M × M f contain the same information on M 23

f. Thus, matrix multiplication with the membership of the pair (B, C) in the elements of the product M × M respect to the given operations ⊗, t can be done in time |G| · BMM(n), and Theorem 3 can be applied to obtain the desired algorithm. This establishes an analogue of Theorem 1 for the well-founded semantics. The algorithm can be further ´ extended to Boolean fuzzy sets of Esik and Kuich [9] by combining the elements in the proofs of Corollaries 1 and 3. 10. Conclusion Thus, one more key algorithm for the context-free grammars has been extended to the general case of Boolean grammars. Generalization entailed no complications whatsoever; on the contrary, the new algorithm came out simpler and clearer than the presentations of Valiant’s algorithm given in the literature [26, 10], for the reason that some unnecessary features, such as the intermediate algebraic layer, had to be removed to handle this case. Accordingly, the unintended second contribution of this paper is a pædagogic presentation of Valiant’s algorithm as it is. The observed smooth transition to Boolean grammars in this algorithm provides further evidence for the author’s long-time claim that Boolean grammars are the proper general case of the standard model of syntax: the context-free grammar. Although the disjunctive fragment of this model is certainly the most important one, it has been very well researched already, whereas the general case of all Boolean operations in essentially the same model is relatively unstudied and calls for future investigation. A key problem to be solved is to understand the limits of the expressive power of Boolean grammars. Acknowledgements The author is grateful to the anonymous referees for careful reading, for pointing out numerous inaccuracies, and for suggesting several valuable improvements to the presentation, such as the notation for submatrices in Algorithm 2. Research supported by the Academy of Finland under grant 257857. References [1] L. Adleman, K. S. Booth, F. P. Preparata, W. L. Ruzzo, “Improved time and space bounds for Boolean matrix multiplication”, Acta Informatica 11:1 (1978), 61–70. [2] N. Alon, M. Naor, “Derandomization, witnesses for Boolean matrix multiplication and construction of perfect hash functions”, Algorithmica, 16:4–5 (1996), 434–449. [3] V. L. Arlazarov, E. A. Dinic, M. A. Kronrod, I. A. Faradzhev, “On economical construction of the transitive closure of an oriented graph”, Soviet Mathematics Doklady, 11 (1970), 1209–1210. [4] J.-M. Bened´ı, J.-A. S´ anchez, “Fast stochastic context-free parsing: a stochastic version of the Valiant algorithm”, Pattern Recognition and Image Analysis (IbPRIA 2007, Girona, Spain, June 6–8, 2007) LNCS 4477, 80–88. [5] A. Bertoni, G. Mauri, N. Sabadini, “Membership problems for regular and context-free trace languages”, Information and Computation, 82:2 (1989), 135–150. [6] D. Coppersmith, S. Winograd, “Matrix multiplication via arithmetic progressions”, Journal of Symbolic Computation, 9:3 (1990), 251–280. [7] A. Je˙z, A. Okhotin, “Conjunctive grammars over a unary alphabet: undecidability and unbounded growth”, Theory of Computing Systems, 46:1 (2010), 27–58. [8] A. Je˙z, A. Okhotin, “Complexity of equations over sets of natural numbers”, Theory of Computing Systems, 48:2 (2011), 319–342. ´ [9] Z. Esik, W. Kuich, “Boolean fuzzy sets”, International Journal of Foundations of Computer Science, 18:6 (2007), 1197– 1207. [10] M. A. Harrison, Introduction to Formal Language Theory, Addison-Wesley, 1978. [11] V. Kountouriotis, Ch. Nomikos, P. Rondogiannis, “Well-founded semantics for Boolean grammars”, Information and Computation, 207:9 (2009), 945–967; [12] L. Lee, “Fast context-free grammar parsing requires fast Boolean matrix multiplication”, Journal of the ACM, 49:1 (2002), 1–15. [13] A. Okhotin, “Conjunctive grammars”, Journal of Automata, Languages and Combinatorics, 6:4 (2001), 519–535. [14] A. Okhotin, “Boolean grammars”, Information and Computation, 194:1 (2004), 19–48.

24

[15] A. Okhotin, “Generalized LR parsing algorithm for Boolean grammars”, International Journal of Foundations of Computer Science, 17:3 (2006), 629–664. [16] A. Okhotin, “Recursive descent parsing for Boolean grammars”, Acta Informatica, 44:3–4 (2007), 167–189. [17] A. Okhotin, “Unambiguous Boolean grammars”, Information and Computation, 206 (2008), 1234–1247. [18] A. Okhotin, “Fast parsing for Boolean grammars: a generalization of Valiant’s algorithm”, Developments in Language Theory (DLT 2010, London, Ontario, Canada, August 17–20, 2010), LNCS 6224, 340–351. [19] A. Okhotin, “Conjunctive and Boolean grammars: the true general case of the context-free grammars”, Computer Science Review, 2013, to appear. [20] A. Okhotin, C. Reitwießner, “Parsing by matrix multiplication implemented on a CPU–GPU system”, manuscript, 2010. [21] A. Okhotin, C. Reitwießner, “Parsing Boolean grammars over a one-letter alphabet using online convolution”, Theoretical Computer Science, 457 (2012), 149–157. [22] S. Rajasekaran, S. Yooseph, “TAL recognition in O(M (n2 )) time”, Journal of Computer and System Sciences, 56:1 (1998), 83–89. [23] W. Rytter, “Context-free recognition via shortest paths computation: a version of Valiant’s algorithm”, Theoretical Computer Science, 143:2 (1995), 343–352. [24] G. Satta, “Tree-adjoining grammar parsing and Boolean matrix multiplication”, Computational Linguistics, 20:2 (1994), 173–191. [25] V. Strassen, “Gaussian elimination is not optimal”, Numerische Mathematik, 13 (1969), 354–356. [26] L. G. Valiant, “General context-free recognition in less than cubic time”, Journal of Computer and System Sciences, 10:2 (1975), 308–314. [27] V. Vassilevska Williams, “Multiplying matrices faster than Coppersmith-Winograd”, 44th Symposium on Theory of Computing (STOC 2012, New York, USA, 19–22 May 2012), 887–898. [28] R. Williams, “Matrix-vector multiplication in sub-quadratic time (some preprocessing required)”, 18th Annual ACMSIAM Symposium on Discrete Algorithms (SODA 2007, New Orleans, USA, 7–9 January 2007), 995–1001. [29] R. Zier-Vogel, M. Domaratzki, “RNA pseudoknot prediction through stochastic conjunctive grammars”, CiE 2013, to appear.

25