On the Decidability of the Freeness of Matrix

On the Decidability of the Freeness of Matrix Semigroups Julien Cassaigne

Institut de Mathematiques de Luminy, Luminy Case 930 F-13288 Marseille Cedex 9, France [email protected]

This research was done while this author visited TUCS.

Juhani Karhumaki

Department of Mathematics, University of Turku FIN-20014 Turku, Finland

[email protected]

Supported by the Academy of Finland under the grant 14047.

Tero Harju

Department of Mathematics, University of Turku FIN-20014 Turku, Finland

[email protected]

Supported by the Academy of Finland under the grant 14047.

Turku Centre for Computer Science TUCS Technical Report No 56 November 1996 ISBN 951-650-875-8 ISSN 1239-1891

1 Introduction

It is well known that free semigroups +k can be embedded into the semigroup of 2  2 matrices over nonnegative integers, N22. Moreover, such embeddings can be done in many dierent ways. For example, the morphisms de ned by

and

8 2 0 > a 7 ? ! <  20 11  > : b ?7 ! 0 1

(1)

8 1 1 > < a ?7 ! 0 1 1 0 > : b ?7 ! 1 1

(2)

yield such embeddings of the 2-generator free semigroup +2 . The former one extends immediately  k i to an embedding of +k = fa0; : : : ; ak?1g+ into N22 by setting ai 7! 0 1 . The above connection is very fruitful, and in particular, allows to deduce results on matrices from those of words, and vice versa. Typical examples of the former ones are several undecidability results of matrices established by translating the Post Correspondance Problem into a problem of matrices. The undecidability of the existence of the zero matrix in a nitely generated semigroup of 3  3 matrices over integers, due to Paterson [7], is one of the earliest results of this type. Another example is the inclusion problem for Nrational series, i.e. languages with multiplicities accepted by nite automata, as presented in [3]. On the other hand, a fundamental compactness result of free semigroups, often referred to as the Ehrenfeucht Compactness Property, cf. e.g. [4], is an elegant example of the reverse deduction. Embeddings as in (1) and the well known decidability of whether a given nite set of words is a free generating set, cf. [1], motivate a natural question: Is it decidable whether a nitely generated semigroup of 2  2 matrices over nonnegative integer is free? Two natural extensions of this problem are to replace the semiring N by the rings Zor Q, respectively, or to consider matrices of higher dimensions. It was shown in [5] that the above problem for 3  3 nonnegative integer matrices is undecidable. One of our goals is to extend this result slightly by showing that the problem remains undecidable even in the case when the matrices are upper-triangular. In doing this we establish a variant of the Post 1

Correspondence Problem, which implies the result rather easily, and which, we believe, might turn useful in other applications as well. As another goal we tackle the freeness problem for 2  2 matrices. Actually, we restrict to the case when the semigroup is generated by just two matrices, which moreover are upper-triangular. Even this simple-looking problem appears to be hard. We introduce two sucient conditions for freeness, but we are not able to conclude whether they are necessary. We also give some examples demonstrating the non-triviality of our restricted problem, which is | due to the embeddings of type (1) | quite natural.

2 A variant of PCP In this section we introduce an undecidable variant of the Post Correspondence Problem, which suits particularly well to show that the freeness of nitely generated matrix semigroups is undecidable. Another variant is presented in [2]. We need some terminology. For a nite set  let  be the free monoid generated by  and + =  n fg, where  denotes the unit element. By an equality set of a pair h; g :  !
 of morphisms we mean the set E (h; g) = fw 2 + jh(w) = g(w)g : Now, the Post Correspondence Problem, PCP for short, asks to decide for a given pair (h; g) whether or not E (h; g) = ;. Elements in E (h; g) are called solutions of the instance (h; g) of PCP. Our variant of PCP is as follows. The mixed modi cation of PCP, MMPCP for short, asks to decide for two given morphisms h; g :  !
 whether there exists a word w = a1 : : :ak , with ai 2  and k  1, such that h1(a1 )h2 (a2) : : :hk (ak ) = g1 (a1)g2 (a2 ) : : :gk (ak ) ; (3) where, for each i, hi; gi 2 fg; hg and, for some j , hj 6= gj . Again a word w satisfying (3) is called a solution of the instance (h; g) of MMPCP. We shall prove

Theorem 1. MMPCP is undecidable.

Proof. As usual we reduce PCP to MMPCP. So let (h; g) be an instance of

PCP. Assume that h; g :  !
 and that c, d and e are new letters not in  [
. Further let `; r :
 ! (
[ fdg) be morphisms de ned as `(a) = da and r(a) = ad for a 2
. Finally, for each a 2
we de ne two morphisms ha ; ga : ( [ fd; eg) ! (
[ fc; d; eg) by setting ha (x) = `(h(x)); ga (x) = r(g(x)) for x 2 ; ha (d) = c`(h(a)); ga (d) = cdr(g(a)); ha (e) = de; ga (e) = e: 2

It follows directly that the instance (h; g) of PCP has a solution aw if and only if the instance (ha ; ga) of PCP has a solution dwe. Since the pair (ha ; ga ) cannot have solutions of other forms, we conclude that it is undecidable whether for a given a 2  and morphisms h; g :  !
 the instance (ha ; ga) of PCP has a solution. Now the theorem follows when we show that the pair (ha ; ga), as an instance of PCP, has a solution if and only if the same pair, as an instance of MMPCP, has a solution. To simplify notations we denote h = ha and g = ga . The implication in one direction is clear. So assume that the pair (h; g), as an instance of MMPCP, has a solution, and let w = a1 : : :ak be such a solution of minimal length. We claim that then h(w) = g(w), i.e. w is a solution of the instance (h; g) of PCP. In notations of (3), the minimality of w implies that h1 6= g1 and hk 6= gk , so that necessarily a1 = d and ak = e, and ai 62 fd; eg for i = 2; : : : ; k ? 1. Assuming, by symmetry, that h1 = h and g1 = g we show that hi = h and gi = g for all i = 1; : : : ; k. If this is not the case, then there exists the smallest t such that gt = h or ht = g. Consider the rst alternative. Then we have g(a1 : : :at?1)h(at ) 2 cd(d)+ (d)+ ; so that the shortest pre x of the right hand side of (3), which is not a pre x in c(d)! ends with dd. But no choice of the left hand side of (3) matches with this pre x: if hi 6= g for all i, then h1(a1 ) : : :hk (ak ) 2 c(d)+ , and if hi = g for some i, then the shortest pre x of the required form is in c(d)+ . In the second alternative a similar argumentation can be used to derive a contradiction, starting now from the relation h(a1 : : :at?1)g(at ) 2 c(d)+ (d)+ .

3 The undecidability of the freeness In this section we apply Theorem 1 in order to re-prove a result of [5], or in fact its sharper form. Let  = fa1; : : : ; ang and, for each w 2  , let b(w) be the number that w represents as a reverse n-adic number, i.e. b() = 0 and if w = ai1 : : :aik then

b(w) =

k X j =1

ij nj ?1 :

We de ne, following [7], a mapping :    ! N33 by setting 0 njuj 0 b(u) 1

(u; v) = @ 0 njvj b(v) A : 0 0 1 3

It is straightforward to compute that is a morphism, and moreover it is trivially injective. Now, we are ready for

Theorem 2. It is undecidable whether a semigroup generated by a nite set of upper-triangular 3  3 matrices of nonnegative integers is free. Proof. Let (h; g) be an instance of MMPCP. Obviously we can assume that h and g are endomorphisms of  . Then de ne

M = f (a; h(a)); (a; g(a))ja 2 g and let S be the semigroup generated by M . Now, for any i1 ; : : : ; ip ; j1 ; : : : ; jq 2 M , say it = (ait ; hit (ait )) and js = (bjs ; gjs (bjs )), with hit ; gjs 2 fh; gg and ait ; bjs 2 , for t = 1; : : : ; p and s = 1; : : : ; q, we have i1 : : :ip = j1 : : : jq in S ; if and only if (by the de nition of ) (i1 : : :ip )`;3 = ( j1 : : : jq )`;3 for ` = 1; 2 ; if and only if (by the uniqueness of n-adic representations) ai1 : : :aip = bj1 : : :bjq and hi1 (ai1 ) : : :hip (aip ) = gj1 (bj1 ) : : :gjq (bjq ) : Consequently, S is nonfree if and only if the instance (h; g) of MMPCP has a solution. This completes the proof of Theorem 2. Our proof of Theorem 2 deserves a few comments. It is not more complicated than the one presented in [5]. Moreover, the result is stronger in the following two respects. The number of matrices needed to make the problem undecidable is in our case 18, while in [5] it is 29, assuming that in the PCP we consider instances over 7 letter domain alphabet, as can be done cf. [6]. Second, as we already mentioned, our result is for upper-triangular matrices, which are quite dierent from the general ones in many respects, as is also indicated in our next section.

4 The freeness problem for 2  2 matrices

We have seen that the freeness problem is undecidable for 3  3 integer matrices, even when it is restricted to triangular matrices. We now concentrate on the 4

case of dimension 2, for which the decidability is open (the freeness is obviously decidable in dimension 1). First, we give some evidence that if this problem is undecidable, then it cannot be proved using a construction similar to the one used for dimension 3. The rest of this section is then devoted to one of the simplest nontrivial cases, with only two generators, and upper-triangular matrices. We studied it in the hope of nding a decidable subproblem, but even for this apparently simple problem, we were only able to nd sucient conditions for the freeness.

4.1 Coding two independent words is impossible

To prove the undecidability of the freeness for 3  3 matrices, we used matrices to represent a pair of words in a free monoid, such that the product of the matrices representing (u; v) and (u0 ; v0 ) represents (uu0 ; vv0 ). In other terms, we used an injective morphism of    into N33. We prove here that there is no such embedding into N22.

Theorem 3. There is no injective semigroup morphism ' :    ?! C 22 for any alphabet  with at least two elements. Proof. It is clearly sucient to prove the proposition in the case where  =

f0; 1g. Then S =    has the presentation

ha; b; c; d; ejac = ca; ad = da; bc = cb; bd = db; ae = ea = a; be = eb = b; ce = ec = c; de = ed = d; ee = ei ; as is easy to see using the isomorphism a 7! (0; ), b 7! (1; ), c 7! (; 0), d 7! (; 1), e 7! (; ). Suppose that an injective morphism ' exists, and let A = '(a), B = '(b), C = '(c), D = '(d), E = '(e). Since the conjugation by an invertible matrix does not in uence the injectivity of ', we can suppose, allowing complex values, that A is in Jordan normal form.   Suppose rst that A has two dierent eigenvalues. Then A = 0 0 , and the matrices commuting with A are exactly the diagonal matrices. But then C and D are diagonal and commute, which contradicts injectivity since c and d do not commute in S .  0  1 If A has only one eigenvalue, then either A = 0  or A = 0  . The rst case is excluded, because A would then commute with B . In the 5

x y second case, the matrices commuting with A have the form 0 x , and they commute together, which yields a contradiction as above.

This result says nothing directly about the decidability of the freeness for 2  2 matrices, but it means that if the problem is undecidable, then the proof must use a completely dierent argument than the current proofs for 3  3 matrices.

4.2 A restricted problem: two 2  2 upper-triangular matrices We now consider the following problem (P), a further restriction of the general freeness problem: given two upper-triangular 2  2 matrices

A=

  a a  1 2 and B = b1 b2 0 b3 0 a3

with entries in Q, determine whether these matrices freely generate the semigroup fA; B g+ . Note that in the case where B is a power of A, or conversely, the semigroup is free but with only one generator. In the sequel, by \the semigroup is free" we shall mean \the semigroup is free with two generators". 6

Standard form. The rst step is to eliminate trivial cases, and to reduce the matrices into a standard form.

Lemma 1. If one of the matrices A or B is singular (i.e. has determinant 0), then the semigroup fA; B g+ is not free.

Proof. Suppose that det A = 0, and let t be the trace of A, so that A2 = tA.

Then A2 BA = ABA2 , and so fA; B g+ is not free. We now suppose that A and B are nonsingular (i.e. invertible). Lemma 2. If the matrices A and B are nonsingular and fA; Bg+ is not free, then A and B satisfy an equation U = V where U and V are products of the same number of A's and B 's (i.e. commutatively equivalent), and U begins with A and V with B . Proof. Let U = V be the shortest equation satis ed by the semigroup. The two members must begin with dierent matrices, because if they both began with A there would be a shorter equation A?1U = A?1 V . If they are not commutatively equivalent, consider instead the equation UV = V U .

Lemma 3. Let ;  2 fA; B g+ is free.

Q = Q n f0g. Then

fA; B g+ is free if and only if

Proof. If the semigroup is not free, let U (A; B ) = V (A; B ) be an equation

satisfying the hypotheses of Lemma 2. Then U (A; B ) = i j U (A; B ) = V (A; B ), where (i; j ) is the common Parikh vector of U and V . A consequence of Lemma 3 is that it does not make the problem easier to restrict to matrices having integer entries.

Proposition 1. Theproblem(P) is decidable  b if1 and  only if the restricted proa 0 blem (P') where A = 0 1 and B = 0 1 , with a; b 2 Q nf?1; 0; 1g, is decidable. Proof. The necessary part is trivial. Suppose that we have an instance (A; B )

of (P). If A or B is singular, then we can decide the problem by Lemma 1. Otherwise, we apply Lemma 3 with  = a13 and  = b13 to get an equivalent instance with the lower entries  ?1 equal to 1. a 2 If a1 = ?1, i.e. A = 0 1 , then A2 = I and the semigroup is not 1 a   1 na  2 n free. If a1 = 1, i.e. A = 0 1 , then A = 0 1 2 . Let b1 = qp with p 2 Zand q 2 N. Then p  Ap B = 0q b2 +1 pa2 = BAq 7

and we have an equation Ap B = BAq or B = A?p BAq depending on the sign of p. Therefore, the semigroup is not free if ja1j = 1 or jb1j = 1. If (a1 ? 1)b2 = (b1 ? 1)a2, then the two matrices A and B commute and so the semigroup is not free. Otherwise, let r2 = 1?a2a1 and r1 = r2(b ? 1) + b2 6= 0.  The matrix R = r01 r12 is nonsingular, and therefore fA; B g+ is free if and only if fR?1AR; R?1BRg+ is free since they are always isomorphic semigroups. It can be checked that the matrices R?1AR and R?1BR have the desired form. From now on, we consider only problem (P'). An instance of the problem is completely described by the pair (a; b). Note that the argument for ja1j = 1 can also be applied to any other element in the semigroup: if we nd C in fA; B g+ with an upper left entry equal to 1 or ?1, then there is an equation of the form C 2 = I or C p B = BCq or B = 1 C ?p BC q . For example, the semigroup generated by 20 01 and 02 11 is not free. Lemma 4. The instances (a; b), (b; a), ( a1 ; 1b ), ( 1b ; a1 ) of problem (P') are equivalent.

Proof. Since the order is not important in problem (P), we can exchange A and

B in the construction, and then get instance (b; a) of problem (P'), which is therefore equivalent to (a; b). The instances (A; B ) and (A?1 ; B ?1 ) are equivalent: if one semigroup satis es an equation, then the other satis es the mirror equation. The construction of Proposition 1 applied to (A?1 ; B ?1 ) yields ( a1 ; 1b ), which is therefore equivalent to (a; b).

Sucient conditions. We can now establish two dierent sucient conditions for freeness. The rst one uses p-adic arguments. If p is a prime
? number and x 2 Q, we denote by vp (x) the p-adic valuation of x, de ned by vp pn zy = n if n; y; z 2 Zand p does not divide y and z . By convention, vp (0) = +1. Proposition 2. If there is a number p such that vp (a) > 0 and vp(b) > 0, then fA; B g+ is free. Proof. Suppose that there is a relation. By Lemma 2, we can suppose that it has the form AX = BY . It is clear that the elements of fA; B g+ , in particular X and Y , all have entries of non-negative p-adic valuation (in other words, A 0  x x and B can be viewed as matrices with entries in Zp). Let X = 0 1 and  y y0  Y = 0 1 . Then the upper right entry of AX is ax0 , with positive p-adic 8

valuation vp (ax0 ) = vp (a) + vp (x0)  vp (a) > 0, whereas the upper right entry of BY is by0 + 1, and vp (by0 + 1) = 0 since vp (by0 ) > 0 and vp (1) = 0, hence the relation AX = BY cannot hold. For example, Proposition careof the classical example mentioned in  2 0 2 takes 2 1 the introduction, 0 1 ; 0 1 , i.e. (a; b) = (2; 2). Using Lemma 4, we can also conclude that fA; B g+ is free when vp (a) < 0 and vp (b) < 0, for example if (a; b) = ( 34 ; ? 91 ). Our second condition is apparently very dierent, since it uses inequalities on real numbers.

Lemma5. Let (a; b) be an instance of problem (P') with jaj < 1 and jbj < 1. 0 If X = x0 x1 2 fA; B g+ , then x0 2 I where I is the interval de ned as follows:

h

h

 if a > 0 and b > 0, then I = 0; 1?1 b ;

i

h

 if a < 0 and b > 0, then I = 1?a b ; 1?1 b ;  if a > 0 and b < 0, then I = [0; 1];

i

h

 if a < 0 and b < 0, then I = 1?aab ; 1?1ab . Proof. In each case, it is easily checked that 0; 1 2 I , aI  I and bI + 1  I .

Proposition 3. If jaj + jbj  1, then fA; Bg+ is free. Proof. Suppose that there is a relation AX = BY with X 2 fA; B g+ . The

upper right entry of AX = BY must be in aI \ (bI +1). But when jaj + jbj  1, this intersection is empty except in one case: a > 0, b < 0, I = [0; 1], jaj + jbj = 1, aI \ (bI + 1) = fag = f1 + bg. In this case, according to Lemma 2, we assume that there is the same number of A's in each side of the equation. But it is easy to see that the only matrices in fA; B g+ with an upper right entry equal to 1 are of the form BAk , and X and Y cannot be both of this form. Using Lemma 4, we can also that fA; B g+ is free when ja1j + j1bj  1. conclude   2 0 ; 3 1 For example, the semigroup is free. 0 1 0 1 9

A partial algorithm. Our two sucient conditions (Propositions 2 and 3) are certainly not necessary, see the example below. We can do a little better by combining them. Let us assume here that jaj < 1 and jbj < 1. Let U and V be two xed elements of fA; B g+ . Arguments similar to Propositions 2 and 3 give sucient conditions that ensure that there is no relation of the form UX = V Y . If we are able to nd such a sucient condition for every pair (U; V ) 2 AfA; B gk  B fA; B gk for a xed k, then we have proved that fA; B g+ is free. The conditions can be expressed as follows: let U=

 u u0   v v0   x x0   y y0  ; V = ; X = ; Y = 0 1 0 1 0 1 0 1 :

Then UX = V Y if and only if ux = vy and ux0 + u0 = vy0 + v0 . If p is a prime number such that vp (a)  0 and vp (b)  0, one of the two being positive, and if min(vp (u); vp (v)) > vp (u0 ? v0 ), then the equation UX = V Y does not hold. If (uI + u0) \ (vI + v0 ), where I is the interval de ned in Lemma 5, then the equation does not hold either. For example, take (a; b) = ( 72 ; 34 ). Propositions 2 and 3 do not apply, since a + b > 1 and the numerators have no common factor, and they do not apply to ( a1 ; 1b ) either. However, we nd that two dierent sucient conditions are satis ed for two values of (U; V ):



4 49 0









 for (U; V ) = (AA; B ) we have U = 0 1 , V = 0 uI + u0 = [0; 16 49 [ and vI + v = [1; 4[ are disjoint;



3 14

2 7

 for (U; V ) = (AB; B ) we have U = 0 1 , V = ? 57 , v3 (u) = v3 (v) = 1 > v3(u0 ? v0 ) = 0.

3 4 1



0 1 , I = [0; 4[, 3 4 1



0 0 0 1 , u ?v =

Together, these two conditions imply that fA; B g+ is free, since any equation (with identical initial letters removed) is of one of the forms AAX = BY or ABX = BY . In practice, we can use a breadth- rst walk of the tree of possible equations, cutting branches when one of the conditions is satis ed, and at the same time testing if an equation is actually found. When the semigroup is not free, this algorithm always terminates with the correct answer. When the semigroup is free, it may answer so, but it may also never terminate. This does not seem to happen very often in practice. However, it does not terminate in a reasonable time for the instance ( 32 ; 35 ). Indeed, we do not know whether this instance is free. 10

4.3 Conclusions We considered the problem of deciding whether a given nitely generated semigroup of 2  2 matrices (with rational entries) is free. This problem is natural since all nitely generated free semigroups can be embedded into a 2-generator matrix semigroup (over nonnegative integers), and freeness is a well-known decidable property for nitely generated subsemigroups of free semigroups. The problem seems to be also very challenging. The corresponding problem for 3  3 matrices is undecidable, cf. [5], even in the case when the matrices are upper-triangular as was proved here. We noted that the methods of proving these results cannot work for 2  2 matrices. Moreover, as an evidence of the nontriviality of the problem, we studied a restricted case of the problem where there are only two upper-triangular matrices. Even this very simple-looking problem is quite subtle: we were able to obtain only necessary conditions for freeness, and of the concrete example

A=



2 3 0



0 1 ;B =



3 1 5



0 1

we do not know whether it is a free generating set, although it does not satisfy any equation where the lengths of both sides are at most 20. The fact that our sucient conditions apply in many cases is illustrated in Figure 1. In this gure, a point with rational coordinates (a; b) represents the corresponding instance of the problem. A few areas for which the result is known are represented: the grey areas indicate points for which Proposition 3 apply. On any straight line passing through the origin, there are only nitely many points (white circles) for which the semigroup is not free, since Proposition 2 applies most of the time; however, there are also nitely many points to which this proposition dos not apply and that are either free (black circles, on the line with slope ?6=5) or unsettled (grey circles, on the line with slope ?9=10). The symmetry of the gure is quite striking. Part of this symmetry, namely with respect to the rst diagonal, or by inversion (a; b) 7! ( a1 ; 1b ), follows from previous simple observations. But there seems also to be symmetry with respect to both axes (the set of chosen lines is not symmetric, but this is just not to overload the gure : in fact, the lines with slopes 6=5, ?9=4, etc. look exactly like their symmetric counterpart). In all the examples we have computed, (a; b), (a; ?b), (?a; b) and (?a; ?b) yield the same answer. We have no explanation for this, and when the semigroups are not free there is no apparent similarity between the relations that they satisfy. 11

4 4/3

-6/5

-9/10

1

3

not free

free

free

2

not free

9/4

1/2

1 1/4 free -2

not free

-1

0

1

2

3

4

-1

-2 free

not free

-3

-4

-3

-4

Figure 1: The restricted problem for two 2  2 matrices

12

free

References [1] J. Berstel and D. Perrin. Theory of codes. Academic Press, 1986. [2] C. Chorut, T. Harju, and J. Karhumaki. A note on decidability questions in presentations of word semigroups. Manuscript. [3] S. Eilenberg. Automata, Languages, and Machines. Academic Press, 1974. [4] T. Harju and J. Karhumaki. Morphisms. In G. Rozenberg and A. Salomaa, editors, Handbook of Formal Languages. Springer Verlag, to appear. [5] D. A. Klarner, J.-C. Birget, and W. Satter eld. On the undecidability of the freeness of integer matrix semigroups. Int. J. Algebra Comp., 1:223{226, 1991. [6] Y. Matiyasevich and G. Senizergues. Decision problems for semi Thue systems with a few rules. Manuscript. [7] M. S. Paterson. Unsolvability in 3  3 matrices. Studies in Appl. Math., 49:105{107, 1970.

13

Turku Centre for Computer Science Lemminkaisenkatu 14 FIN-20520 Turku Finland http://www.tucs.abo.

University of Turku  Department of Mathematical Sciences

 Abo Akademi University  Department of Computer Science  Institute for Advanced Management Systems Research

Turku School of Economics and Business Administration  Institute of Information Systems Science