Rough - Set - Like Approximations of Context and Regular

Rough - Set - Like Approximations of Context - Free and Regular Languages Gheorghe Paun

Institute of Mathematics of the Romanian Academy PO Box 1-764 Bucuresti 70 700 Romania

Lech Polkowski

Institute of Mathematics Warsaw University of Technology Pl. Politechniki 1 Warszawa 00 650 Poland

Abstract Using a grammar instead of an information system as usual in rough set theory, we de ne equivalence and tolerance classes of strings which lead to the notions of lower and upper approximations to the language generated by a given grammar. These approximations form an in nite hierarchy and they converge to the considered language. We investigate here the properties of the approximations for classes of context - free and regular languages. Our work brings together two very active branches of modern mathematics: formal language theory and rough set theory.

1 Introduction This work brings together two very active branches of modern mathematics: formal language theory and rough set theory. Surprisingly, a meager research was done at the intersection of these two elds in spite of the fact that most languages we deal with are quite imprecise, ambiguous , imprecisely formalized, intractable from the complexity point of view or even undecidable. Approximating languages is a natural problem, occurring in various contexts , and rough set theory is concerned with exactly this task of constructing and manipulating approximations (of concepts in the framework of information systems). We consider here the problem of approximating a language starting from a (context - free) grammar. In this problem, a grammar is given and the language generated by the grammar is not known. This is the case in many frameworks e.g. in programming languages where we have rigorously de ned language speci cation; our problem is in contrast to the basic problem of grammatical inference, where one looks for the grammar generating a language on the basis of the given samples of positive/negative examples. In case our language is in nite

Andrzej Skowron

Institute of Mathematics Warsaw University Banacha 2 Warszawa 02 097 Poland

(otherwise the problem can be trivial), the grammar has to work in nitely many steps in order to produce it. One can ask the natural questions (i) whether an approximation of the language with adequate properties can be produced in nitely many steps; (ii) using means simpler than those of the starting grammar; (iii) whether nite approximations to language converge to this language in a certain precise sense. We show here that the conditions (i) - (iii) can be satis ed satisfactorily by means of a rough set - like approach. The basic idea consists in considering (given a context - free grammar G) the set Ln of sentential forms generated in exactly n derivation steps ; the languages Ln approximate the language from above in a way related to pattern languages (Angluin 1980), (Jiang et al. 1994), (Kari et al. 1995). Speci cally, having a variable A occurring in a sentential form, say x1 Ax2 , we know that A would eventually derive a terminal string ( we may start from a reduced grammar in order to ensure that) but we do not know what this string will be. Hence, replacing A with an arbitrary terminal string leads to an approximation from above of our language. It is not obvious, but it turns out to be true, that these approximations converge to the language. To produce an approximation from below to our language, we consider the set Ln of terminal sentential forms obtained in at most n derivation steps ; languages Ln are in the language generated by G and their sequence also converges to the language. The approximations Ln; Ln turn out to be the lower, respectively, the upper, approximations in rough set theoretical sense generated from natural tolerance relations or even, in case of lower approximations, equivalence relations. Many questions pose themselves in connection with the above situation e.g. the rate of convergence of approximations, their relation to operations on language, classi cation of languages with respect to properties of their approximations etc. etc. We expect to address these problems in further work.

2 Rough set - theoretic preliminaries The basic assumption of rough set theory (Pawlak 1991) is that objects in a given universe U are perceived by means of certain attributes from a set A , each attribute a 2 A being a function on the universe U ; a fortiori, objects with identical descriptions should be regarded as indiscernible. The pair (U; A) is an information system. An information vector of x 2 U is the set InfB (x) = f(a; a(x)) : a 2 Ag;objects x; y 2 U are B - indiscernible, where B  A; whenewer InfB (x) = InfB (y); we denote the relation of B -indiscernibility by the symbol IND(B ). We denote by [x]B that equivalence class of IND(B ) which contains x: A concept (=a subset) X  U can be (and is) perceived throughout the knowledge represented by the set B of attributes : BX  X  BX where BX = fx 2 U : [x]B  X g is the B-lower approximation of X and BX = fx 2 U : [x]B \ X 6= ;g is the B - upper approximation of X. We say that X is a B-exact concept in case there exists a set Y  U such that BY = X = BY ; otherwise, X is B - rough. In many problems where rough set methods apply one is bound to relax these original notions by replacing the indiscernibility with a similarity (tolerance) of objects. This modi cation leads to tolerance rough sets and tolerance approximations (Polkowski et al. 1995), (Skowron et al. 1995). A relation  on a universe U is a tolerance relation in case it is re exive and symmetric. For x 2 U , the  - tolerance class of x is the set  (x) = fy : yxg ; tolerance classes form a covering of the universe, not necessarily a partition. The lower and upper approximations with respect to a tolerance  are de ned in analogy with classical notions: A X = fx 2 U :  (x)  X g is the  - lower approximation of X and A X = fx 2 U :  (x) \ X 6= ;g is the  - upper approximation of X; clearly, A X  X  A X: In case  is an equivalence IND(B ),  - approximations become B - approximations.

3 Formal language - theoretic preliminaries For an alphabet V , we denote by V  the set of words over V (the free monoid under the concatenation operation) including the empty word  ; we let V + = V  nfg: For x 2 V  , j x j denotes the length of x and j x jU is the number of occurrences of symbols from U  V  in x. A context - free grammar is a construct G = (N; T; S; P ) where N (non - terminals); T (terminals) are disjoint alphabets , S (the axiom) is in N , and P is a nite subset of N  (N [ T ): We write down elements of P in the form x ! w of production rules. If w 6=  for each rule x ! w in P , then G is  - free. If w 2 T [TN for each x ! w in P then G is regular. The

completion rule S !  is allowed in  - free and regular grammars in case the empty word belongs to language , but then S is not allowed to appear in the right hand side of any production. If x = x1 Xx2 ; y = x1 xx2 and X ! x 2 P then we write x =) y ; the symbol =) denotes the re exive and transitive closure of =). The language L(G) generated by the grammar G is de ned as L(G) = fx 2 T  : S =) xg: Symbols REG, resp. CF, will denote classes of regular, resp. context - free languages. We quote the known results (Salomaa 1973).

Proposition 1 (i) REG  CF ;

(ii) the emptiness and the membership questions are decidable for context - free grammars (i.e. given a grammar G and a word x, an algorithm exists which answers the question : L(G) = ;?; x 2 L(G)?); (iii) REG is closed under complement, whereas CF is closed under intersection with regular languages. Suppose we are given a context - free grammar G = (N; T; S; P ) and classes [x], somehow de ned for a tolerance or equivalence on V  , of words x 2 T  with [x] regular for any x.

Proposition 2 Under the above assumptions, both questions (i) is [x] a subset of L(G) ?; (ii) is the intersection [x]\L(G) empty ? are decidable. Proof. (i) Because [x]  L(G) i (T n[x])\L(G) = ; and T n[x] 2 REG hence (Proposition 1) (T n[x]) \ L(G) 2 CF , this problem is reduced to the emptiness problem forCF . (ii) Because [x] \ L(G) 2 CF , this

problem is again reduced to the emptiness problem for CF . An important for the sequel notion is that of a reduced grammar; A context - free grammar G = (N; T; S; P ) is said to be reduced if it has the following properties: 1. G is  - free; 2. there is no chain rule of the form X ! Y where X; Y 2 N in the set P ; 3. each non - terminal x is reachable in G: there is a derivation S =) x1 Xx2 in G; 4. each non - terminal X is productive: there is in G a derivation X =) x; where x 2 T :

Proposition 4. Given a context - free grammar G , an equivalent reduced grammar G0 can be algorithmically constructed; moreover, G0 is regular in case G is

regular. In what follows we always work with reduced grammars even when this is not mentioned explicitly; we also ignore the empty word and identify two languages diering only by the empty word.

4 Approximating a context - free language We work with a context - free grammar G as above. We let (i) SFo (G) = fS g; (ii) SFi (G) = (SFi?1 (G) \ T  ) [fw 2 (N [ T ) : z =) w , some z 2 SFi?1 (G)g: We consider also the regular substitution sT : (N [ T ) ! 2T  de ned by : sT (a) = fag for a 2 T ; sT (X ) = T + for X 2 N . Now, for i  0, we de ne the two approximations of L(G): Ai L(G) = SFi (G) \ T  , Ai L(G) = sT (SFi (G)). We include some examples.

Example 1 For G1 = (fsg; fa; bg; S; fs ! aSb; S ! abg) we have L(G1 ) = fan bn : n  1g 2 LIN nREG. We have SFj (G1 ) = fak bk : 1  k  j g [ faj Sbj g, Aj L(G1 ) = fak bk : 1  k  j g, Aj L(G1 ) = fak bk : 1  k  j g [ aj fa; bgbj : For G2 = (fS; X g; fa; bg; S; fS ! aX; X ! aX; X ! bX; X ! a; X ! bg) we have L(G2 ) = afa; bg+ 2 REG: In this case, SFj (G2 ) = fay : y 2 fa; bg+; j y j< j g [ fay : y 2 fa; bg+; j y j< j gX , Aj L(G2 ) = fay : y 2 fa; bg+; j y j< j g; Aj L(G2 ) = afa; bg+:

In the second case, the upper approximation gives the exact description of the language from step 1 on. Intuitively, this is also the rst case when both approximations converge to the language. This is the situation in general. We have

Proposition 4 For i  0, there are : an equivalence "i and a tolerance i such that Ai L(G) =A" L(G) and Ai L(G) = A L(G). i

i

Proof. For i  0, x; y 2 T ; consider Ci (x) = fw 2 SFi (G) : x 2 sT (w)g and let: x"i y i Ci (x) = Ci (y) and xi y i either x; y 2 sT (w) for some w 2 SFi (G) or x = y. For x 2 SFi (G) \ T , we have x 2 L(G) and sT (x) = fxg hence if y = 6 x then y 2= sT (x) i.e. "i (x) = fxg. This shows "i (x)  L(G) i.e. Ai L(G) A" L(G): Conversely, if x 2 T  and "i (x)  L(G) then x 2 L(G) and x 2 SFi (G): Thus, x 2 SFi (G) \ T  i.e. Ai L(G) A" L(G): Consider now x 2 sT (SFi (G)) along with the tolerance class i (x) = fy 2 T  : x; y 2 sT (w); some w 2 SFi (G)g: We have sT (w) \ L(G) = 6 ; for any w 2 SFi (G) hence i (x) \ L(G) = 6 ; and thus Ai L(G)  A L(G): Conversely, take x 2 T  with i (x) \ L(G) = 6 ;; if i (x) = i

i

i

fxg then x 2 SFi (G) \ T  hence x 2 sT (SFi (G)): If i (x) = 6 fxg then there is y 6= x with y; x 2 sT (w) for some w 2 SFi (G) hence x 2 sT (SFi (G)): It follows that Ai L(G)  A L(G): i

Our next proposition shows that the approximations are correctly related to the language.

Proposition 5. For i  0, Ai L(G)  L(G)  Ai L(G):

Proof is obvious by de nitions (when i = 0: A0 L(G) =

;, A0 L(G) = T +):

Approximations form monotone hierarchies.

Proposition 6. For i  0, Ai L(G)  Ai+1L(G)  Ai+1 L(G)  Ai L(G): Proof. The rst inclusion as well as the second follow from de nitions. For the third inclusion, let x 2 Ai+1 L(G); there is w 2 SFi (G) such that x 2 sT (w): If w 2 T  \ SFi (G) then x = w hence x 2 Ai L(G): In case w 2 T nSFi (G) there is z 2 SFi (G) such that z =) w hence w 2 sT (z ) and thus x 2 sT (SFi (G)) = Ai L(G): In case w 2= T , there is z 2 SFi (G) such that z =) w and z = u1 Xu2, w = u1vu2 for a rule X ! v in P . Clearly, sT (w) = sT (u1 )sT (v)sT (u2 )  sT (u1 )T +sT (u2 ) = sT (z ) and it follows that x 2 sT (SFi (G)) = Ai L(G): We de ne: A1 L(G) = [i0 Ai L(G) and A1 L(G) = \i0 Ai L(G). Proposition 7. A1L(G) = L(G) = A1 L(G): Proof. Clearly, both inclusions  hold. Observe that any x 2 L(G) has a derivation in G of length  2 j x j due to absence of chain rules, so x 2 SF2jxj(G) hence x 2 A2jxj L(G) A1 L(G): This proves the rst equality. Assume now that A1 L(G)nL(G) 6= ;; for x 2 A1 L(G)nL(G);take j > 2 j x j and observe that x 2 sT (SFj (G)) and j w j 2j for any w 2 SFj (G)nT  ; it follows that x 2 sT (SFj (G)) \ T  hence x 2 L(G): The proposition is proved. We have approximations converging to the language in set - theoretical sense. We de ne the new language BNi L(G) = Ai L(G)nAi L(G) for i  0 called the rough boundary language of rank i of L(G). Proposition 8. For i  0; languages Ai L(G); Ai L(G) and BNi L(G) are regular.

Proof. The assertion is trivial in case of Ai L(G) by

nitness of all these sets and follows in case of Ai L(G) by nitness of all sets SFi (G) and the fact that sT is a

regular substitution . As regular languages are closed under dierence, the assertion for BNi L(G) follows. By Proposition 6, the sequence fBNi (L(G))g is monotonically decreasing; by Proposition 7, \i0 BNi (L(G)) = ;.

5 The rate of convergence An important problem concerning the approximations is the quality of approximation of the language L. Obviously, in case of in nite languages one has to leave aside cardinality arguments; what remains is e.g. to look at the speed - up results. Such results can be obtained at the price of enlarging the initial grammar.

Proposition 9. For each context - free grammar G = (N; T; S; P ) where L(G) is in nite and for k  1 , we can nd a grammar Gk = (N; T; S; Pk ) such that L(G) = L(Gk ) and for each i  1 : Ai L(Gk ) =Ai+k L(G) and Ai L(Gk ) = Ai+k L(G):

Proof. Consider Pk = P [fS ! x : x 2 SFk+1 (G)g:

Then obviously, SFi (Gk ) = SFi+k (G) and the proposition follows. The construction above has also another signi cant consequence. We call a context - free language L k - reachable where k  1 if there is a grammar G = (N; T; S; P ) such that L = L(G) and Ak L(G) = L. Clearly, if a languageL is k - reachable then L is also k0 - reachable for any k0  k: However, it follows from Proposition 9 that a k - reachable language is also 1 reachable.

Proposition 10. Any k - reachable context - free language L is 1 - reachable.

Proof. Let G = (N; T; S; P ) be such that L = L(G) and Ak L(G) = L: For a grammar Gk?1 = (N; T; S; Pk?1 ) of Proposition 9, we have A1 L(Gk ) = A1+k?1 L(G) = L:

We give a characterization of reachable languages; we call a language L  T  marked in case there exist x1 ; x2 ; ::; xk+1 2 T  such that L = x1 T +x2 :::xk T +xk+1 ; any marked language is regular.

Proposition 11. A language L  T  is reachable i

it is a nite union of marked languages.

Proof. As L is 1 - reachable, A1L(G) = L

for some grammar G hence L = sT (SF1 (G)) . As SF1 (G) is nite and for each w = x1 X1 x2 :::xk Xk xk+1 2 SF1 (G) , the language sT (w) is marked, L is a nite union of marked languages. Conversely, if L = [jn xj;1 T +xj;2 T +:::xj;k T +xj;k +1 j

j

then L = A1 L(G) for G = (fS; X g; T; S; fS ! xj;1 Xxj;2 ::::xj;k Xxj;k +1 : j  ng) [ fX ! aX; X ! a : a 2 T g). j

j

In consequence, the class of reachable languages is very small. The problem remains open to nd other concepts of reachability, less restrictive, and to nd other speed - up results (maybe without increasing too much the starting grammar).

6 Conclusions and remarks 1. We have de ned approximations to a language generated by a context - free grammar. These approximations are partly conforming to the orthodox requirements of rough set theory in using partitions of the underlying universe of words and partly transcend this limitations by using coverings of the universe i.e. a tolerance relation, in agreement with the current research in rough set theory. Our approximations are based on sets SFi (G) of sentential forms generated by derivations of length i in G. The other approximation idea can be based on sets DTi (G) of all sentential forms having derivation trees of height i. In the case of linear (hence also for regular) grammars, the sets SFi (G) and DTi (G) coincide but for context - free grammars the approximations based on the sets DTi (G) deserve investagion; intuitively, these approximations are closer to the language L(G) than those based on the sets SFi (G): 2. Yet another improvement of approximations based on either of the above sequences of sets of words may consist in introducing hypothesized languages LX for each non - terminal X in G. Formally, for a context - free grammar G = (N; T; S; P ) and any non - terminal X , a regular language LX  T  can be found such that fy 2 T  : X =) yg  LX (in our considerations concerning Ai L(G); Ai L(G) we have taken LX = T + for each X 2 N ). The de nition of the regular substitution sT : (N [ T ) ! 2T  becomes now : sT (a) = fag for a 2 T ; sT (X ) = LX for X 2 N . This restriction can be too strong for regular languages (if LS = L(G) then L(G) = A1 L(G) ) but for context - free languages this approach may bring forth an increase in convergence rate. 3. Our de nitions of rough set - theoretic approximations, BX and BX , may seem self - contradictory at rst glance: to de ne these approximations to an unknown set X , we should know X itself. Obviously, this apparent contradiction can be easily removed by an adequate change of de nitions; however, in our case of context - free languages, when a grammar G is given , the contradiction is non - existent (at the cost of knowing the grammar of approximated language).

4. Concerning our approximations of language, the relation i is in general non - transitive; consider SFi (G) = faaXb; aY bbg where X =) a; Y =) b: We have L(G) = faaab; abbbg; aabb 2 sT (aaXb) \ sT (aY bb) hence aabb i aaab and aabb i abbb but not aaab i abbb: We can use neither the relation i nor the relation "i for both approximations Ai L(G); Ai L(G) . In our example, i (aaab) = faaab; aabbg is not included in L(G) and no element of "i (aabb) is in L(G) : if y 2 "i (aabb) then y is of the form y = aay0bb: It is an open problem whether the approximations Ai L(G); Ai L(G) can be introduced (with possibly their de nitions modi ed) by means of a single relation of tolerance or equivalence.

References [1] D. Angluin (1980). Finding patterns common to a set of strings. J. Comp. System. Sci. 21: 46-62. [2] T. Jiang, E. Kinber, A. Salomaa, K. Salomaa, S. Yu (1994). Pattern languages with and without erasing. Intern. J. Computer Math. 50 :147-157. [3] L. Kari, A. Mateescu, Gh. Paun, A. Salomaa (1995). Multi pattern languages. Theoretical Computer Sci. 141: 253 -268. [4] Z. Pawlak (1992). Rough Sets: Theoretical Aspects of Reasoning about Data. Dordrecht: Kluwer. [5] L. Polkowski, A. Skowron, J. Zytkow (1995). Tolerance based rough sets. In: T.Y. Lin and A.M. Wildberger (eds.), Soft Computing, 55-58. San Diego: Simulation Councils. [6] A. Salomaa (1973). Formal Languages. New York: Academic Press. [7] A. Skowron, J. Stepaniuk (1995). Generalized approximation spaces. In: T.Y. Lin and A.M. Wildberger (eds.), Soft Computing, 18-21. San Diego: Simulation Councils.