Learning Local and Recognizable !-Languages and ... - CiteSeerX

Learning Local and Recognizable

!-Languages

and Monadic Logic Programs

Ahmed Saoudi and Takashi Yokomori 1 L.I.P.N, Institut Galilee, Universite Paris XIII, 2 Department of Computer Science and Information Mathematics, University of Electro-Communications

Abstract !-languages called local ! -languages, and give two learning

We introduce two subclasses of regular

!-languages

and recognizable

algorithms for those subclasses. We also relate these results to the learning problem for the class of monadic logic programs.

1

Introduction

In the study of inductive inference of formal languages, Gold (1967) proved that the class of languages containing all nite sets and one in nite set is not identi able in the limit from positive data only. This implies that even the class of regular languages is not identi able in the limit from positive data. Angluin (1980) developed several conditions for identifying, a class of languages, in the limit from positive data, and presented some examples of these identi able classes. In this paper we will focus on languages of in nite words (i.e., !words), rather than nite words, and introduce two subclasses of regular ! -languages called local ! -languages and recognizable !-languages. For a motivation of introducing !-languages as well as a comprehensive review on !-langauges, refer to, e.g., Thomas (1990). By introducing a new class of ! -languages (i.e., local ! -languages), we rst prove that this class is identi able in the limit from positive data that are ultimately periodic in nite words. In fact, we present a linear time algorithm for learning the class of local ! -languages. Then, we de ne another new subclass of regular ! -languages, called recognizable !-languages, and give an algorithm for learning recognizable !languages from positive data and restricted subset queries(Angluin (1988)). Further, we introduce a class of logic programs called monadic logic programs and show that the set of in nite computations of a monadic logic program is exactly a recognizable ! -language. This leads us to a result that this class of programs is also learnable from positive data (i.e. sample of in nite computations) and restricted subset queries. 2

Preliminaries

2.1 Basic de nitions Let 6 be a xed nite alphabet and 63 be the set of all nite-length strings over 6. Further, let 6+ = 63 0 fg, where is the null string. By lg(u) we denote the length of string u. A language over 6 is a subset of 63 . The 1

Learning !-Languages

2

cardinality of a set S is denoted by jS j. Let Pk (w) (resp. Fk (w)) be the pre x (resp. the set of factors) of w of length k. These are de ned only when w has length k or more. If w has length k, then Pk (w) = w and Fk (w) = fwg. An in nite sequence u (in nite word, or ! -word ) over 6 is a mapping from [!] to 6, where [! ] denotes the set of all non-negative integers, and u is written in the form : u = u(0)u(1)u(2) 11 1, with u(i) 2 6. As a notation for segments of u, u(m; n) = u(m) 1 1 1 u(n 0 1). We denote the set of in nite sequences over 6 by 6! . An !-language, over 6, is a set of in nite sequences. Let L and K be two languages over 6, we de ne : (i) L! = fu 2 6! j u = u1 u2 1 1 1 and ui 2 Lg; (ii) KL! = fu 2 6! ju = u1 u2; u1 2 K; u2 2 L! g: The limit of L, denoted by lim L, is the set of in nite words u such that the set of initial segments of u meets in nitely many time the set L. (That is, lim L = fw 2 6! j9! n u(0; n) 2 Lg, where 9! n means that there exist in nitely many n.) An !-language L is called regular i there exist two sequences (Ai )1in and (Bi )1in of regular languages such that L = ni=1 Ai Bi! :

S

A ne (alphabetic) morphism is a mapping from a set 6 to a set 1, and is extended in the usual manner to be a mapping from the set of sequences over 6 to the set of sequences over 1. Let L be a class of (! -)languages to be identi ed over a xed alphabet 6. We consider a class of representations R with the property that for all L 2 L, there exists r 2 R such that r represents L(denoted by L(r) = L). For a given r 2 R, a positive presentation (or data) of L(r) is any in nite sequence of examples such that every w 2 L(r) occurs at least once in the sequence and no other examples not in L(r) appear in the sequence. Let r be a representation in R representing a given L(i.e., L = L(r)). An algorithm A is said to identify (or learn) a language L in the limit from positive data i for any positive presentation of L, the in nite sequence of representations ri in R produced by A satis es the property that there exists a representation r0 in R such that for all suciently large i, the i-th conjecture (representation) ri is identical to r0 and L(r0) = L(r ). A class of languages L is identi able in the limit from positive data i there exists an algorithm A that, given an L in L, identi es L in the limit from positive data.

2.2 Automata on in nite words The idea of using automata for recognizing in nite sequences is due to the late Buchi (1960). Buchi used ! -automata to prove the decidability of the monadic second order theory of natural numbers with the successor relation, which is called S1S. De nition 1. A Buchi automaton is a structure M =< Q; 6; q0 ; ; F >, where Q is a nite set of states, 6 is the input alphabet, q0 is the initial state, : Q 2 6 ! 2Q is the transition function, and F Q is the set of designated states. A computation of M over an in nite word u is a mapping Cu : [!] ! Q satisfying the following conditions :

Ahmed Saoudi and Takashi Yokomori 3 (i) Cu (0) = q0 , and (ii) for each i 2 [!], Cu (i + 1) 2 (Cu (i); u(i)). M accepts the in nite word u i there exists a designated state which occurs in nitly often in this computation, i.e., fq 2 Qj9! nCu (n) = q g\F 6= ;. Muller (1963) introduced a new condition for accepting in nite sequences and used it, with nite state automata, to analyze asynchronous circuits.

De nition 2. A Muller automaton is a structure M =< Q; 6; q0; ; F >, where Q; 6; q0; are de ned as before, and F is the family of designated sets of states, i.e., F 2Q .

M accepts an in nite word u in the sense of Muller i the set of states occurring in nitely often in the computation Cu belongs to F , i.e., fq 2 Qj9! nCu (n) = qg 2 F . An automaton M is said to be deterministic i for each state q and each input symbol a, we have j(q; a)j 1. Finally, L! (M ) denotes the set of in nite words accepted by M . It is known that an ! -language L is accepted by a deterministic Buchi automaton i L = limK , for some regular language K (Thomas (1990)). 3

Learning Local

! -Languages

3.1 Characterizing Local ! -Languages In this section, by introducing the notion of local systems, we will extend the notion of local language to ! -languages and compare the power of local systems with Muller and Buchi automata. A local system over 6 is an ordered pair S =< I; C >, where I 6, and C 62 . We denote by L! (S ) the set of in nite sequences u, starting with a symbol from I , such that u does not contains any segment (i.e. factor) from C : L! (S ) = I 6! 0 63 C 6! : An ! -language L over 6 is called local if there exists a local system S over 6 such that L! (S ) = L. We denote by LOC ! (6) the set of local ! -languages over 6.

LOC ! (6) = fLjL = L! (S ); for some local system S g: (When 6 is implicitly stated, LOC ! is used.)

Since regular ! -languages are closed under boolean operations, and L! (S ) belongs to the boolean closure of regular !-languages, the set of local ! -languages forms a subclass of regular !-languages. In order to obtain an automata characterization of local ! -languages, we de ne the following types of automata.

A B-automaton(or B-type machine) is a special type of a Buchi automaton M =< Q; 6; q0; ; F >; where F = Q.

An M-automaton(or M-type machine) is a special type of a Muller automaton M =< Q; 6; q0; ; F >; where F = 2Q .


4

Let us compare the power of these automata by associating them with the following families of ! -languages. 1. 2. 3. 4.

DB` = fLjL = L! (M ); for some deterministic B-type machine M g B` = fLjL = L! (M ); for some B-type machine M g DM` = fLjL = L! (M ); for some deterministic M-type machine M g M` = fLjL = L! (M ); for some M-type machine M g.

Since B-type and M-type machines accept an in nite word u i there exists a comutation on u, one can apply the well-known procedure, that is the subset construction, to these machines and get an equivalent deterministic one. Hence, one can prove the following:

Theorem 3. For each !-language L, the following conditions are equivalent : (i) L is accepted by a deterministic M-automaton, (ii) L is accepted by a nondeterministic M-automaton, (iii) L is accepted by a nondeterministic B-automaton, and (iv) L is accepted by a deterministic B-automaton. Thus, we have that DB` = B` = DM` = M` . Let us now compare the accepting power of local systems with deterministic B-automata. Actually, as shown below, we can show an interesting relationship between the two.

Theorem 4. L belongs to DB` i there exist a ne morphism ' and L1 in LOC ! such that '(L1) = L. Proof. (only if part) Let M =< Q; 6; q0 ; ; Q > be a deterministic B-type machine such that L = L! (M ). Then, construct SL =< IL ; CL > as follows: IL = fq0g262Q, and CL = 02 0f(q; a; q 0)(q 0; b; q 00 )j(q; a; q0 ); (q0 ; b; q 00 ) 2 Trans(M )g, where 0 = Q 2 6 2 Q and Trans(M ) = f(q; a; q0 )j(q; a) = q0 g. Further, de ne '((q; a; q 0)) = a for all (q; a; q 0) 2 0. Let L1 = L! (SL ). Then, it is clear that '(L1) = L! (M ) = L. (if part) Let L1 2 LOC ! , i.e., L1 = L! (S ) for some S =< I; C >. We show that L1 is in DB ` by presenting a construction procedure, Algorithm (1), for a deterministic B-type machine M =< Q; 6; q0; ; Q > such that L1 = L! (M ). Algorithm (1) Input: S =< I; C > Output: M =< Q; 6; q0 ; ; Q > such that L1 = L! (M ) Procedure Let Q = fq0 g [ f[a]ja 2 I g [ f[ab]jab 2= C g [ f$g ; forall a 2 I do (q0; a) = [a] ; forall q 2 Q 0 fq0g do forall a 2 6 do if q = [b] and ba 2= C then ([b]; a) = [ba] ; else if q = [bc] and ca 2= C ; then (q; a) = [ca] ; else (q; a) = $

5

Ahmed Saoudi and Takashi Yokomori

M is clearly deterministic and it holds that L! (M ) = L1. Since closed under ne morphisms, '(L1 ) is also in DB ` .

DB` is

We will prove that LOC ! is a proper subclass of DB ` . As a terminology, an ! -word u is ultimately periodic i there exist p 1; q 0 such that for any n q one has u(n + p) = u(n). In this case, we write u = u1 u!2 , where u1 = u(0; q 0 1) (if q 1), = (if q = 0), and u2 = u(q; q + p).

Lemma 5. The sets containing only one ultimately periodic word of the form u1 u2u1 u!2 , where u1; (6=)u2 2 6+, j6j 2, are not local !-languages. Proof. Let L = fu1 u2u1 u!2 g be any singleton set consisting only one ultimately periodic word u, where u = u1 u2u1 u!2 , u1; u2 2 6+. Assume that there exists S =< I; C > such that L! (S ) = fu1 u2u1 u!2 g = ! I 6 0 63 C 6! . Then, P1 (u1) 2 I , and (F2 (u1u2 u1 ) [ F2(u2 u2)) \ C = ;. This implies that u1u!2 must be in L! (S ) which is a contradiction. Proposition 6. The class of local ! -languages (i.e. included in DB` .

LOC ! ) is properly

Proof. It suces to prove that the inclusion is proper. From Lemma 5, L = fabab! g is not a local !-language. One can construct a deterministic B-type machine M =< Q; fa; bg; q0 ; ; Q > such that fabab! g = L! (M ), where Q = F = fq0; q1 ; q2; q3 g; (q0 ; a) = q1; (q1; b) = q2 ; (q2 ; a) = q3; and (q3 ; b) = q3 . 3.2 Learning Algorithm We now consider the problem of learning local ! -languages. In order to do this, we de ne the notion of a characteristic sample and show the existence of a characteristic sample for every local !-language. De nition 7. Let L be a local ! -language over 6 and L = L! (S) for some local system S =< I; C >. Then S is said to be minimal for L if for any S 0 =< I 0; C 0 >, L = L! (S 0 ) implies I I 0 and C C 0 . Since, for a xed 6, the number of distinct local systems over 6 is nite, we may assume that for any local !-language L = L! (S ), S is minimal.

De nition 8. Let T be a nite sample of ultimately periodic in nite words. Let ST =< IT ; 62 0 CT > be the local system such that P1 (T ) = IT and F2(T ) = CT , where P1 (T ) = fP1 (w)jw 2 T g and F2 (T ) = w2T F2(w). Then, the set L = L! (ST ) is called the local !-language associated with the sample T .

S

Using the last de nition, one can easily prove the following lemma.

Lemma 9. Let ST and ST be two local systems with two nite samples T and T 0. Then, we have: (i) T L! (ST ), (ii) If T T 0 then L! (ST ) L! (ST ). (iii) If L is an ! -language such that T L, then L! (ST ) L. 0

0

6


Let L be a local ! -language. A nite subset E is called a characteristic sample for L if L is the smallest local !-language containing E .

Lemma 10. If E is a characteristic sample for a local !-language L, then L = L! (SE ). Further, if E T L, then L = L! (ST ). Let T = fu1v1! ; :::; un vn! g be a sample of ultimately periodic in nite words. We denote by Tk the set fu1v1k ; :::; unvnk g. By assuming that lg(vi ) 6= 0 (for i = 1; :::; n), one can easily prove the following lemma.

Lemma 11. The !-language associated with T is equal to the one associated with T2 . This lemma tells us that for learning local !-languages, from a nite sample of ultimately periodic in nite words, only a nite representation of these in nite words is actually needed. Further, from the manner of constructing deterministic B-type machine M in Algorithm (1), it is easily seen that there eectively exists a nite sample set T of a local !-language L! (M ) from which a local system ST such that L! (ST ) = L! (M ) can be reconstructed. In other words, T is quali ed enough to be a characteristic sample for L! (M ).

Theorem 12. There eectively exists a characteristic sample for any local !-language. Let us present an algorithm that learns an unknown local !-language in the limit from positive data.

Algorithm (2) Input : A positive presentation of an unknown local !-language L. Output : An in nite sequence SEi (i = 1; 2; :::) such that L! (SEi ) are local !-languages. Procedure : let E0 = ; ; let SE0 =< ;; 62 > ; repeat let SEi =< Ii; Ci > be the current conjecture ; read the next positive example w = uv 2 ; scan w to compute P1(w) and F2(w) ; let SEi+1 =< Ii [ fP1 (w)g; Ci 0 F2 (w) > ; let Ei+1 = Ei [ fw g ; output SEi+1 as new conjecture ; forever Lemma 13. Let SE0 , SE1 ... SEi , ... be the sequence of conjectures by Algorithm (2). Then (i) for all i 0, L! (SEi+1 ) L and (ii) there exists r 0 such that for all i 0, L! (SEr ) = L! (SEr+i ) = L. Proof. Since for all i 0, Ei L, (i) immediately follows from Lemma 10. Now consider a suciently large number n0 > 0 such that EL En0 , where EL is a characteristic sample for L. Since EL is nite, this is always possible. Then, from Lemma 10, L = L! (SEL ) = L! (SEn0 ), noting that En0 L. Also, since for all i > n0 , EL Ei L, we again obtain that L! (SEi ) = L.


7

By summarizing all the lemmas, we obtain the following theorem.

Theorem 14. Given an unknown local !-language L, Algorithm (2) learns, in the limit from positive data, a local system SE such that L! (SE )= L. [Time Complexity Analysis of Algorithm (2)] The time complexity of Algorithm (2) depends on the size of the positive data provided. From the observation that an ultimately periodic in nite word w = uv! is completely speci ed when u and v are speci ed, we may take the sum of lengths of u and v as the size of w, denoted by lg (w). Here is a problem that there are possibly several pairs (u; v ) specifying the same ini nte word. It is, however, known that for an ultimately periodic in nite word w there uniquely exists a pair (u; v ) such that w = uv ! , where lg(u) and lg(u) + lg(v ) are the shortest (Johansen 1988). Such a pair is called the canonical representation of w. Hence, we may de ne the size of w(= uv ! ) be the sum of the lengths of u and v, that is, lg(w) be lg(u) + lg(v ), where (u; v ) is the canonical representation of w. We then have O(lg (uv! )) = O(lg (uv2)) = O(lg (uv)). Let N be the sum of the sizes of all positive data En = fw1 ; :::; wn g provided. Then, the time complexity of Algorithm (2) is bounded by O(N ), where N = ni=1 lg(wi ).

P

4

Learning Recognizable

!-Languages

In this section, we consider the learning problem for a subclass of regular ! -languages called recognizable ! -languages. The method for learning this class is based on learning from positive examples and uses restricted types of subset queries. An ! -language is said to be recognizable i it is accepted by a deterministic B-type machine. (In the standard terminology, \recognizable" is often used as an equivalent for \regular". Nevertheles, here we prefer to use this rather than any other.) Speaking of the language learnability in the nite string case, it is wellknown that the (full) class of regular languages is not learnable in the limit from only positive data. So, the problem arises, besides positive data, what is minimally needed to learn the class of regular languages. Concerning the learnability on the full class of regular languages, Angluin shows that membership queries, restricted equivalence queries and subset queries are insucient for eciently learning the class in the sense of exact identi cation (Angluin 1988). More exactly, it is shown that any algorithm that exactly learns any regular language using restricted equivalence, membership, and subset queries must make at least exponential number of queries in the worst case. Getting back to the learnability of !-languages, we will show, by using a weak version of subset queries called restricted subset queries, that the class of recognizable ! -languages is learnable from positive data. We proved Theorem 4 in the last section which states that an ! -language U is recognizable i it is a morphic image of a local !-language L (i.e. U = h(L) for some morphism h). We will show how to derive a learning algorithm, for recognizable !-langauges, from the one that learns local ! -languages.

8


We observe, from the proof of Theorem 4, the following : [Observations] (1) The alphabet 0(= Q 2 6 2 Q) contains mn2 elements, where n is the number of states of minimum deterministic B-type machine MU for U , and m = j6j. (2) For w = a1 1 1 1 at in U , let g (w) = (i1 ; a1 ; j1)(i2; a2; j2 ):::(it; at; it ) be a word over 0. A word g (w) is called a good word for w if for all k = 1; 2; :::; t 0 1, jk = ik+1 and i1 (it ) is the initial( nal) state of MU , respectively. Then, the number of good words in h01 (w) is no more than nlg(w) . (3) Since MU is a minimum deterministic machine, for each w = a1 1 1 1 at 2 U , there uniquely exists a good word g(w) 2 h01 (w) corresponding to a sequence of acceptance for w by MU . (4) Let L be a local !-language over 0 such that U = h(L), and let RL be a characteristic sample for L. Then, there exists a nite set of positive data SU of U such that RL h01 (SU ). (That is, RL is a member of the power set of h01 (SU ).) These observations lead us to an algorithm that learns recognizable !languages via homomorphic coding.

Algorithm (3) Input: A positive presentation of an unknown recognizable !-language U , n=the number of states of a minimum deterministic B-type machine for U . Output : A local system S such that U = h(L! (S )). Query: restricted subset query (Angluin 1988). Procedure Initialize E0 = ;, i = 0 ; Construct the initial local system SE0 =< ;; 02 > ; repeat let SEi =< Ii ; Ci > be the current conjecture ; read the next positive example w ; let G(w) = f1; :::; t g be the set of good words for w in h01 (w) ; let j = 0 ; while j < t do j := j + 1 ; scan j to compute P1 (j ) and F2 (j ) ; let SEi =< Ii [ fP1 (j )g; Ci 0 F2 (j ) > ; let answerj =Is-Subset-of(h(L! (SEi )); U ); if answerj =\yes", then Ei := Ei [ fj g ; Ii := Ii [ fP1(j )g ; Ci := Ci 0 F2 (j ) ; let Ei+1 = Ei ; let SEi+1 =< Ii; Ci > ; output SEi+1 ; forever where Is-Subset-of(Li ; U ) returns \yes" if Li U , and \no" otherwise. Further, a notation X := Y denotes that X is newly de ned by Y .

[Correctness of Algorithm (3)]


9

We note that the algorithm always produces a conjecture SEi such that h(L! (SEi )) U . Now, to see the correctness of Algorithm (3), we need the following lemma :

Lemma 15. Let n be the number of states for a minimum automaton accepting the unknown recognizable ! -language U . After at most t(n) number of queries, Algorithm (3) produces a conjecture SEi such that Ei includes a characteristic sample for a local ! -language L with the property that U = h(L), where t(n) is depending on U . Proof. Let w1; :::; wp be a sequence of positive data provided. From the observation (2), the cardinality of Ei is at most nlg(w1 ) + 1 1 1 + nlg(wp ) pn` , where ` is the maximum length of positive data provided. Now, remember the description on the eective existence of a characteristic sample, established by Theorem 12, and the de nition of a local ! -language associated with a sample set in De nition 8. Let m0 = j0j = mn2, then the maximum length of all strings which may be potentially elements of a characteristic sample for L such that U = h(L) is bounded by 3m02 = 3m2n4 = q (n). Since h is a coding, i.e., length-preserving mapping, after providing all strigns whose lengths are no more than q (n), there exists a characteristic sample for L(Observation (4)). Thus, letting r(n) be the number of positive data of U whose lengths are no more than q(n), then after at most t(n)(= r(n)n` ) number of queries, the algorithm nd a nite set Ei of positive data of L with the property that Ei inculudes a characteristic sample for L and U = h(L). By Lemma 15, the algorithm produces a conjecture SEi such that L L! (SEi ) h01(U ). Since h(L) = h(h01(U )) = U and from the property of SEi , we have that h(L! (SEi )) = U . Thus, we obtain the following theorem.

Theorem 16. Given an unknown recognizable !-language U , Algorithm (3) eciently learns in the limit, from positive data and restricted subset queries, a local system S such that U = h(L! (S )). 5

Learning Monadic Logic Programs

In this section, we will de ne a class of programs called monadic logic programs. We show that the set of computations of a monadic logic program is recognizable, that is accepted by a deterministic B-type automaton. A monadic logic program is a structure P =< 5; 6; X; F; C; G >, where 5 is the set of unary predicates, 6 is the set of unary functional symbols, X (= fxg) is the set of individual variables containing only one variable, F is the set of facts of the form A(a) with A 2 5 and a 2 6, C is the set of clauses of the form A(fx) B (x) with f 2 6, A; B 2 5, and the goal having the form G(x) with G 2 5. Example 1 Facts: A(a) ; Sequence(a) ; Sequence(b) Clauses : Sequence(ax) Sequence(x) Sequence(bx) Sequence(x) A(ax) Sequence(x) Goal: A ( x)

10


This example tells us that a monadic logic program computes sets of both nite and in nite sequences. More precisely, the set of nite computations of this program is equal to a(a + b)3 and its in nite computations are a ( a + b) ! .

Theorem 17. For each ! -language L, the following conditions are equivalent. (i) L is accepted by a deterministic B-automaton, and (ii) L is the set of in nite computations of a monadic logic program. This equivalence is due to the fact that a monadic logic program can be viewed a B-type automaton, where the states are the predicates. This correspondence is such that the number of states of the B-type machine is equel to the number of unary predicates of the program computing the set accepted by the B-type automaton. Similarly, a B-type automaton can be viewed as a monadic logic program such that the set of in nite sequences accepted by the automaton is exactly the set of in nite computations of this monadic logic program. The following theorem is a consequence of Theorem 4 and the last theorem.

Theorem 18. L is the set of in nite computations of a monadic logic program i there exist a ne morphism ' and L1 in LOC ! such that '(L1 ) = L. By using Algorithm (3) and the fact that there exists a minimal monadic logic program, where \minimal" means that it has the minimum number of predicates, one can derive an algorithm for learning monadic logic programs with the same complexity as Algorithm (3). This leads to the following result.

Theorem 19. Given an unknown set L of in nite computations of a monadic logic program, Algorithm (3) learns in the limit, from positive data and restricted subset queries, a local system S such that L = h(L! (S )). Acknowledgements

The authors are grateful to the referees for their useful comments on an early draft of this paper. The rst author was sponsored by the Department of Research and Technology under contract MRT 92 C 0038, while the second author was supported by Grants-in-Aid for Scienti c Research No.04229105 from the Ministry of Education, Science and Culture, Japan. Bibliography

1. Angluin, D. (1980). Inductive inference of formal languages from positive data. Information and Control, 45, 117{135. 2. Angluin, D. (1988). Queries and concept learning. Machine Learning, 2, 319{342. 3. Buchi, J.R. (1960). On a decision method in restricted second order arithmetic. Proc. Cong. Logic Method and Philos. of Sci., Stanford Univ. Press, California.


4. Gold, E. M. (1967). Language identi cation in the limit, and Control, 10, 447{474.

11 Information

5. Maler, O. and Pnueli, A. (1991). On the learnability of In nitary Regular Sets, Proc. 4th Annual Workshop on Computational Learning Theory, 128{136, Morgan Kaufman, San Mateo. 6. Muller, D. E. (1963). In nite sequences and nite machines, Proc. 4th IEEE on Switching Circuit Theory and Logical Design, 3{16. 7. Nivat, M. and Saoudi, A. (1989). Automata on in nite objects and their applications to logic and programming, Information and Computation, 83, 41{64. 8. Johansen, P. (1988). Inductive inference of ultimately periodic sequences, BIT, 28, 573{580. 9. Thomas, W. (1990). Automata on in nite objects, Handbook of Theoretical Computer Science(Van Leeuwen ed.), 133{191, North-Holland, Amsterdam. 10. Wilke, T. (1993). Locally threshold testable languages of in nite words, Proc. 10th Annual Symposium on Theoretical Aspects of Computer

(STACS'93), Lecture Notes in Computer Science, 665, 607{616, Springer-Verlag.

Science

11. Yokomori, T. (1989). Learning Local Languages from Positive Data, Proc. Fujitsu Workshop on Computational Learning Theory, 1{16, Numazu, Japan. 12. Yokomori, T., Ishida, N. and Kobayashi, S. (1994). Learning Local Languages and Its Applications to Protein -chain Identi cation, Proc. 27th Annual Hawaii International Conference on System Sciences Vol.V, 113-122, January, Hawaii.