Nil-automorphisms of groups with residual properties

Nil-automorphisms of groups with residual properties Carlo Casolo Dipartimento di Matematica “U. Dini”, Universit`a di Firenze Viale Morgagni 67A, I-50134 Firenze, Italy casolo @ math.unifi.it

Orazio Puglisi

arXiv:1203.3645v3 [math.GR] 22 May 2012

Dipartimento di Matematica “U. Dini”, Universit`a di Firenze Viale Morgagni 67A, I-50134 Firenze, Italy puglisi @ math.unifi.it

May 23, 2012

1

Introduction

Following Plotkin [P] we say that the automorphism x of the group G is a nil-automorphism if, for every g ∈ G, there exists n = n(g) such that [g,n x] = 1 (the commutator is taken in the holomorph of G and, following the usual notation, the element [g,n x] is defined inductively by [g,0 , x] = g and, when n > 0, [g,n x] = [[g,n−1 x], x]). If the integer n can be chosen independently of g, then x is said to be unipotent. Nil and unipotent automorphisms can be regarded as a natural extension of the concept of Engel element, since a nil-automorphism x is just a left Engel element in Ghxi. Another way to look at nil-automorphisms, is to consider them as a generalization of unipotent automorphisms of vector spaces. For these reasons there are several natural questions that can be asked about nil-automorphisms, which are suggested by known facts about Engel groups or unipotent linear groups. The Engel condition is, however, rather weak and it is often necessary to add some additional hypothesis in order to investigate this property. Similar difficulties arise when we study groups of nil-automorphisms. In this paper we shall focus on groups satisfying some finiteness conditions. The first result we prove is a consequence of a classical theorem of Baer [B]. Recall that a group G satisfies Max if every subgroup of G is finitely generated. Theorem A If G satisfies Max, and H is any group of nil-automorphisms of G, then [G, H] is contained in the Fitting radical of G and H stabilizes a finite subnormal series in G. In particular H is nilpotent. An interesting consequence of Theorem A is that groups of of nil-automorphisms of finite groups are nilpotent. On the other hand every finite nilpotent group has a faithful representation as a group of nil-automorphisms of a suitable finite group, so that the following question seems natural: Question 1 Is it true that every finite group of nil-automorphisms is nilpotent? Although the question is still far from a general solution, we provide a partial answer by restricting ourselves to a particular class of groups. Namely the following theorem can be proved. 1

2

Carlo Casolo and Orazio Puglisi

Theorem B Let G be a locally graded group, and H a finite group of nil-automorphisms of G. Then H is nilpotent. The definition of locally graded groups will be recalled in the next section, but it is worth remarking that many interesting types of groups (e.g. locally nilpotent groups, locally finite groups, residually finite groups) are locally graded. A natural hypothesis, suggested by the “linear” setting, is to require that for some n, the identity [g,n x] = 1 holds for all g ∈ G and x ∈ H ≤ Aut(G). In this case we say that H is a group of n-unipotent automorphisms. It is then reasonable to ask to what extent the linear case can be generalized. Question 2 Let G be a group and H ≤ Aut(G). Assume that, for some n, the identity [g,n x] = 1 holds for all g ∈ G and x ∈ H. Under which hypothesis on G is it true that H is (locally) nilpotent? A well known result of Wilson [W1] says that finitely generated residually finite n-Engel groups are nilpotent, suggesting that Question 2 may have an affirmative answer for groups G in this class. It is easy to see that, rather than looking at residually finite groups, it is better to work with profinite groups, in order to take advantage of their well developed theory. We get the following theorem Theorem C Let G be a profinite group and H an abstract finitely generated group of nunipotent automorphisms. Assume that G has a basis of open normal subgroups whose members are normalized by H. Then F = clG ([G, H]) is pro-nilpotent and H is nilpotent. In particular theorem C applies when the profinite group G is finitely generated because, in that case, G has a basis whose elements are characteristic subgroups. Since any residually finite group can be embedded into its profinite completion, it is not hard, using theorem C, to derive a similar result for groups in this class. Another immediate corollary of Theorem C is the result of Wilson stated before. Finally we wish to point to the reader’s attention the recent work of Crosby and Traustason on Engel groups. In particular the results contained in [CT], will be needed at some stage of our proofs.

2

Groups with residual properties

In order to investigate groups with residual properties, we need information about nil-automorphisms of finite groups. This information will be obtained as a special case of a more general fact which, in turn, is a consequence of the following classical result of Baer [B].

Theorem 2.1 Let G be a group satisfying Max. Then the set of left Engel elements of G coincides with the Fitting radical of G. Consequently an Engel group with Max is a finitely generated nilpotent group.

Nil automorphisms

3

Theorem A can now be proved easily. Theorem A If G satisfies Max, and H is any group of nil-automorphisms of G, then [G, H] is contained in the Fitting radical of G and H stabilizes a finite subnormal series in G. In particular H is nilpotent. Proof Assume, first of all, that G is abelian. Let k be the rank of G and set T for the torsion subgroup of G. The subgroup T is finite and G/T is free abelian. For every prime p, the group Ap = G/Gp is elementary abelian and can be regarded as an Fp -vector space of dimension at most k. Thus Hp = H/CH (Ap ) is a unipotent group of automorhisms of Ap , hence [Ap ,k Hp ] = 1. T From this it follows that [G,k H] ≤ p∈P Gp = T . On the other hand T is finite and each element of H acts as a nil-automorphism, so that there exists m such that [T,m H] = 1. Thus [G,k+m H] = 1 and H stabilizes a finite series in G. When G is nilpotent, H acts as a group of nil-automorphisms on each factor of the descending central series hence the above argument can be used to see that H stabilizes a finite central series of G. Let now G be any group satisfying Max, and choose h ∈ H. It is clear that h is a left Engel element of the group Ghhi which, in turn, satisfies Max. By theorem 2.1 h belongs to the Fitting radical of Ghhi. It follows that the subgroup [G, h] is contained in the Fitting radical of G. Therefore [G, H] is contained in the Fitting radical of G and property Max implies that [G, H] is finitely generated, hence nilpotent. Now, by the first part of the proof, H stabilizes a finite normal series in [G, H], showing that H stabilizes a finite subnormal series in G. By a well known result of Hall (see [H1]), H is nilpotent. ✷ As a corollary we obtain

Corollary 2.1 Let G be a finite group and H a group of nil-automorphisms of G. Then H is nilpotent and [G, H] ≤ Fit(G).

We concentrate now on the main objects of our investigation, namely groups with residual properties. We start by recalling the definition of locally graded groups.

Definition 2.2 A group G is said to be locally graded if every non trivial finitely generated subgroup of G has a proper normal subgroup of finite index. Of course any residually finite group is locally graded but the converse is not true. E.g. the groups PSL(2, F), where F is an infinite locally finite field, are locally finite (hence locally graded) but, being simple, they do not have proper subgroups of finite index. Question 1 has a positive answer for locally graded groups. Theorem B Let G be a locally graded group, and H a finite group of nil-automorphisms of G. Then H is nilpotent. Proof By way of contradiction we assume the claim false, so that there exist pairs (G, H) where G is locally graded, H ≤ Aut(G) is finite and consists of nil-automorphisms, and H is

4

Carlo Casolo and Orazio Puglisi

not nilpotent. Among them we select a pair (G, H) in such a way that H has minimal order. Therefore every proper subgroup of H is nilpotent and the structure of H can be easily described. There exist two different primes p, q, such that H = Qhxi where Q is an elementary abelian q-group, x has order pm for some m, and Q is an irreducible hxi-module. In order to achive a contradiction, we shall prove that, for each i ∈ N, if [g,i x] = 1 then [g, Q] = 1. When i = 0 this is true, because [g,0 x] = g. Assume we know the claim holds for all j < i, and choose g ∈ G such that [g,i x] = 1. The element a = [g, x] satysfies [a,i−1 x] = 1, therefore [a, Q] = 1. The subgroup A = [g−1 , Q] is H-invariant. To see this we need only to show that A is normalized by x. In fact for any y ∈ Q, we have [g−1 , y]x = [[x, g]g −1 , y x ] = [a−1 g −1 , y x ] = [g−1 , y x ] ∈ A Assume A 6= 1. The group A is finitely generated, so that its finite residual R is a proper subgroup. Since unipotent groups of automorphisms of finite groups are nilpotent, the subgroup Q must act trivially on every finite image given by H-invariant normal subgroups of A. Therefore Q acts trivially on A/R. Consider the map η : Q −→ A/R defined by (y)η = [g−1 , y]R. If u, v are in Q, then [g −1 , v, u] ∈ R, hence (uv)η = [g−1 , uv]R = [g −1 , vu]R = [g −1 , u][g −1 , v]u R = [g −1 , u][g−1 , v][g−1 , v, u]R = (u)η (v)η The map η is therefore a surjective homomorphism, hence A/R is finite and R is trivial. Thus Q centralizes A. Moreover A is isomorphic to a quotient of Q and x acts on A as a unipotent automorphism. Since x has order pm the only possibility is that 1 = [A, x] = [g−1 , Q, x]. Pick y ∈ Q. Witt’s identity gives 1 = [g −1 , y −1 , x]y [y, x−1 , g−1 ]x [x, g, y]g

−1

On the other hand [g −1 , y −1 , x] = 1 because [g −1 , Q, x] = 1, and [x, g, y] = 1 because Q centralizes a = [g, x]. Hence the above identity reduces to [y, x−1 , g −1 ] = 1. This holds for every y ∈ Q, thus [Q, x, g −1 ] = 1. But Q is an irreducible hxi-module, so that [Q, x] = Q, proving that Q centralizes g −1 , hence g. The inductive step is then complete. Choose g ∈ G. There exists n = n(g) such that [g,n x] = 1. Thus [g, Q] = 1 and Q turns out to centralize G. Therefore Q must then be trivial and this contradiction proves that the claim holds. ✷ We focus now on Question 2. Since this question has a positive answer for finite groups, it seems natural to study the same problem for residually finite groups. To start off let us consider a finitely generated residually finite group G. It is easily seen that there exists a residual system N , whose members are characteristic subgroups. The completion of G with respect to N gives rise to a profinite group G on which Aut(G) acts as a group of continuous automorphisms. For this reason it is convenient to consider Question 2 in the class of profinite groups and take advantage of the well-developed theory of profinite groups. For each N ∈ N indicate by N0 the closure of N in G. Each N0 is normalized by Aut(G) so that there is an induced action of Aut(G) on G/N0 = GN0 /N0 ≃ G/N0 ∩ G = G/N . If H ≤ Aut(G) is a group of nil-automorphisms, then, for each N ∈ N , its elements are nilautomorphims in their action on the finite group G/N0 . Therefore [G, H]N0 /N0 is nilpotent,

Nil automorphisms

5

by corollary 2.1. It readily follows that the closure of [G, H] in G is pro-nilpotent and it is isomorphic to the cartesian product of its Sylow p-subgroups. This argument suggests that it might be useful, as a first step, to consider Question 2 for pro-p-groups. The first lemma we need is actually a fact about finite p-groups.

Lemma 2.3 Let G be a solvable p-group of exponent pl and derived length d. If H ≤ Aut(G) and [g,n h] = 1 for all g ∈ G and h ∈ H, then H has exponent bounded in terms of pl , n and d. Proof The proof is by induction on the derived length of G. When G is abelian we can invoke Lemma 16 of [C]. Assume now the claim holds for groups l of derived length at most d−1 > 1, and let G have derived length d. The group K = H f (p ,n,d−1) l l acts trivially on both G/G(d−1) and G(d−1) . Thus [G, K, K] = 1 so that [g, xp ] = [g, x]p = 1 for l all g ∈ G, x ∈ K. Therefore, setting f (pl , n, d) = f (pl , n, d − 1)pl , we get [g, hf (p ,n,d) ] = 1 for all g ∈ G, h ∈ H proving that exp(H) divides f (pl , n, d), as claimed. ✷

Lemma 2.4 Let H be a group of n-unipotent automorphisms of the abelian group A. If [A, H] has finite exponent, then H has finite exponent. Proof It is enough to prove the lemma when the exponent of [A, H] is power of a prime p. If [A, H] has exponent p, define k by pk−1 ≤ n < pk . Then, given a ∈ A and h ∈ H, one has k k k 1 = [a,n h] = a(h − 1)n = a(h − 1)p = a(hp − 1). Thus hp centralizes A and H has exponent dividing pk . Arguing by induction assume that the exponent of [A, H] is pr . Fix h ∈ H and r−1 r−1 has exponent pr−1 , there consider its action on the group A/[A, H]p . Since [A, H]/[A, H]p r−1 r−1 which has exponent exists m such that H m centralizes A/[A, H]p . Thus [A, H m ] ≤ [A, H]p m k k p. Thus H has exponent dividing p , and H has exponent dividing mp . ✷ The next lemma shows that, when a group of automorphisms of a p-group G is n-unipotent, then it is possible to construct several powerful subgroups of G.

Lemma 2.5 Let G be a finite p-group, H ≤ Aut(G) a group of n-unipotent automorphisms and L any H-invariant subgroup of G. Then there exists k such that R = [L, H k ] is powerful. Proof We treat the case p 6= 2 first. Let k = f (p, n, 2) as defined in lemma 2.3, set K = H k and R = [L, K]. The group H acts on R as a group of n-unipotent automorphisms, hence [R, K] ≤ R(2) Rp . Clearly [R, K g ] ≤ R(2) Rp for every element g ∈ L. Therefore R′ = [R, [K, L]] ≤ R(2) Rp . Dedekind’s modular law gives R′ = R′ ∩ R(2) Rp = R(2) (Rp ∩ R′ ) but, since R(2) is contained in the Frattini subgroup of R′ , we get R′ = Rp ∩ R′ . Therefore R′ ≤ Rp , proving the claim. When p = 2 choose k = f (4, n, 2) and apply the same argument. ✷ This lemma can be easily adapted to the case of pro-p-groups. Our main interest is the case of finitely generated pro-p-groups but it is better to consider a slightly more general setting, that we describe here below.

6

Carlo Casolo and Orazio Puglisi

Definition 2.6 We say that (G, H) is a (n, p)-couple if 1. G is a pro-p-group and H ≤ Aut(G) is finitely generated; 2. there exists n such that each h ∈ H acts as an n-unipotent automorphism on G; 3. G has a basis of open subgroups whose members are normalized by H. If G is a finitely generated pro-p-group and H is a finitely generated group of n-unipotent automorphisms of G, then (G, H) is a (n, p)-couple. In fact, for any fixed natural number k, G contains only finitely many subgroups of index k, so that, given N E G open, the normal T subgroup NH = h∈H N h is H-invariant and of finite index. By a well known result of Serre, NH is open (see [W2], theorem 4.3.5) hence, if B is a basis of open subgroups for G, then BH = {NH | N ∈ B} is a basis of H-invariant open subgroups. On the other hand the set AH = {CH (G/N ) | N ∈ BH } is a basis of open subgroups for H, with respect to which H is an Hausdorff topological group. When considering a topological group U , for every subgroup A, we shall indicate by clU (A) its topological closure in U . As usual a closed subgroup A is said to be finitely generated when A is the closure of a finitely generated discrete subgroup A0 .

Lemma 2.7 Let (G, H) be a (n, p)-couple. Then the following facts hold 1. There exists k such that, for every H-invariant subgroup L of G, the group R = clG ([L, clH (H k )]) is powerful. 2. If d = d(H) is the minimal number of generators for H, there exists r = r(p, n, d) such that, for every x ∈ G, the group L = clG (xH ) can be generated by r elements. Proof Let k = f (p, n, 2) or f (4, n, 2) when p = 2, as defined in lemma 2.5, and set K = clH (H k ). Let B = {Nλ | λ ∈ Λ} be a basis of normal H-invariant open subgroups of G and choose L an Hinvariant subgroup of G. For each λ ∈ Λ define Lλ = LNλ /Nλ and Rλ = [L, K]Nλ /Nλ = [Lλ , K]. T Since K = λ∈Λ H k CH (G/Nλ ), we have [L, K] ≤ [L, H k ]Nλ so that Rλ = [L, H k ]Nλ /Nλ = [Lλ , H k ]. Each Lλ is a finite p-group and, by lemma 2.5, Rλ is powerful. Therefore the group R = clG ([L, K]), which is the inverse limit of the Rλ , is powerful. The group H/K is d-generated, residually finite and of exponent bounded by k, which is a function of p and n. By Zelmanov’s solution of the restricted Burnside problem, H/K is finite of order bounded by a a suitable function r = r(p, n, d). Choose x ∈ G and set L = clG (xH ). Arguing as in the first paragraph, we see that [L, K] ≤ L(2) Lp and, in particular [L, K] ≤ L′ Lp ≤ Φ(L). Choose X = {hi | i = 1, . . . , r} a left tranversal for K in H. Given any y ∈ K and h ∈ X, we have xhy = xh [xh , y] ≡ xh

(mod Φ(L))

Thus, for each λ ∈ Λ, the set Xλ = {xh Nλ | h ∈ X} generates Lλ = LNλ /Nλ , modulo its Frattini subgroup. Thus hXλ i = Lλ . Since each Lλ can be generated by r elements, the same holds for their inverse limit L. ✷

7

Nil automorphisms

The next fact concerns the action of nil automorphisms on uniformly powerful group.

Proposition 2.8 Let G be a finitely generated uniformly powerful pro-p-group and consider the usual additive structure defined on it. If α is a r-unipotent automorphism of G, then the automorphism induced by α on (G, +) is r-unipotent. Proof Given k ≥ 1 and x ∈ G, define k

C(x, k) = xα x−(1)α k

k−1

k−1

· · · x(−1)

k (k−1 )α x(−1)k =

k Y k−i i k x(−1) ( i )α i=0

We stick to the notation of chapter 4 of [DDMS]. Fix n ≥ 1 and any x ∈ G. The group Gn+1 /G2n+2 is abelian, whence n

n

[xp ,k α] ≡ C(xp , k)

(mod G2n+2 )

and the relation k−i i k k−i i k n n = (x(−1) ( i )α )p (xp )(−1) ( i )α

holds for every i = 0, 1, . . . , k. Thus n

(C(xp , k))p

−n

k

= xα +n x−(1)α k

k−1

k

+ n x( 2 ) α

k−2

k

+n . . . +n x(−1) = an (x, k)

Lemma 4.10 of [DDMS] shows that, for each n, n

an (x, k) ≡ bn (x, k) = [xp ,k α]p

−n

(mod Gn )

so that the sequences {an (x, k) | n ∈ ω} and {bn (x, k) | n ∈ ω} have the same limit. The limit of the first sequence is, by definition, the element k X i=1

x

(−1)i (ki)αk−i

  k X k−i i k (−1) = [x,k α] xα = i i=1

of (G, +). Since α is r-unipotent, [g,r α] = 1 for each g ∈ G. Thus each bn (x, r) = 1, so that [x,r α] = 0 in (G, +), showing that α acts unipotently on (G, +). ✷ At this stage we can prove some facts concerning unipotent automorphisms of nilpotent groups. These results will be used in the proof of the general case, but have some interest in their own. We begin by considering abelian groups.

Proposition 2.9 Let (A, H) be a (n, p)-couple and assume that A is a torsion-free abelian group. There exists m = m(n) such that [A,m H] = 1. In particular H stabilizes a series of length at most m in A and it is nilpotent of class at most m − 1.

8

Carlo Casolo and Orazio Puglisi

Proof We refer to the terminology defined in [CT]. Let G = AH be the semidirect product of A and H. Thus A is a residually hypercentral n-Engel normal subgroup of G. Hence, by theorem 3 of [CT], Af (n) ≤ ζm(n) (G) for suitable functions f, m. Being A torsion-free, it follows that A ≤ ζm(n) (G), and H stabilizes a series of length at most m(n) in A. ✷ A straightforward consequence is the following.

Corollary 2.2 Let (G, H) be a (n, p)-couple and assume that G is a finitely generated uniformly powerful pro-p-group. Then H is torsion-free and there exists m = m(n) such that H is nilpotent of class at most m. Proof Consider H as a subgroup of Aut((G, +)) = GL(r, Zp ). By proposition 2.9 H is a unipotent subgroup of GL(r, Zp ) so that it is nilpotent and torsion-free. Clearly ((G, +), H) is a (n, p)-couple so that, again by 2.9, the nilpotency class H can be bounded in terms of n. ✷ The following is an easy fact that we shall need later. Its proof can be easily derived from the results about isolators collected in [H2].

Lemma 2.10 Let V be a torsion-free nilpotent group, U a normal subgroup of finite index and α an automorphism of V . 1. The groups U and V have the same nilpotency class. 2. If α centralizes U , then α = 1.

Proposition 2.11 Let G be a finitely generated nilpotent pro-p-group, and H a d-generated group of n-unipotent automorphisms of G. Then the following hold 1. H stabilizes a central series in G and is therefore nilpotent; 2. there exists κ = κ(p, d, n) such that γκ (H) is finite. Proof Under these hypotheses G has an H-invariant basis of open subgroups, so that (G, H) is a (n, p)-couple. Assume first that G is torsion-free. It is easily seen, arguing by induction on the nilpotency class of G, that H is torsion-free. The key observation is that a nil automorphism of a torsionfree abelian group, has finite order if and only if it is the identity. Let K be the subgroup of H described in lemma 2.7 and set R = clG ([G, K]). The group R is uniformly powerful because it is powerful and torsion-free (see theorem 4.8 in [DDMS]). The group G acts on R by conjugation, and this action gives rise to an action on (R, +). Proposition 2.8 shows that each element of G acts unipotently on (R, +), whence G/CG (R) is isomorphic to a closed group of unitriangular matrices over Zp . Each non-trivial closed subgroup of GL(r, Zp ) has rank bounded in terms of r whence, by theorem 3.13 of [DDMS], it possesses a characteristic powerful

Nil automorphisms

9

subgroup of finite index. Such subgroup is uniformly powerful, because the unipotent subgroups of GL(r, Zp ) are torsion-free. Let M = M/CG (R) be such a subgroup. The group H acts on both R and M/CG (R) and we can use corollary 2.2 to see that there exists m = m(p, d, n) such that γm (H) acts trivially on both R and M/CG (R). Set S = γm (H) ∩ K. Since S ≤ CGH (R), then [M, S] ≤ [G, K] ∩ CG (R) = R ∩ CG (R) = ζ(R) . Notice that, by proposition 2.9, H stabilizes a finite series of length at most m in ζ(R). Fix x ∈ M and consider the map θx : S −→ ζ(R) defined by (s)θx = [s, x]. It is readily seen that θx is an homomorphism. For any k ∈ K, s ∈ S we have [sk , x] = [s, x[x, k −1 ]]k = [s, x]k because [s, k−1 ] belongs to R and S centralizes R. Thus θx is an homomorphisms of K-modules, showing that S/ ker(θx ) has a K-central series of length at most m. The subgroup M has finite index in G, so that it is (topologically) finitely generated. Let x1 , . . . , xr be a set of generators for M . Then r \

ker(θxi ) = {s ∈ S | [s, xi ] = 1 ∀ i = 1, . . . , r} = {s ∈ S | [s, x] = 1∀ x ∈ M } = CS (M )

i=1

By lemma 2.10 CS (M ) = CS (H) = 1. For this reason S embeds, as a K-module, into the direct Q product ri=1 S/ ker(θxi ), so that S has a finite K-central series of length at most m. This shows that γ2m (K) = 1 and, by lemma 2.10, the same holds for H. Setting κ(p, d, n) = 2m gives the claim. For the general case set T for the torsion subgroup of G and put G = G/T . If G is abelian, then T is pure in G, so that T p = T ∩ Gp . Thus T Gp /Gp ≃ T /T ∩ Gp = T /T p and this shows that T is finitely generated, hence finite. By induction on the nilpotency class of G, we see that T is always finite and, in particular, closed in G. Thus G is a torsion-free pro-group and , if κ = κ(n, p, d) is the integer defined in the previous paragraph, the group γκ (H) acts trivially on G. Any h ∈ γκ (H) is then completely determined, once we know the r elements th,i = [xi , h] so that |γκ (H)| ≤ |T |r and the claim is established. ✷ This result has a rather strong consequence.

Proposition 2.12 Let (G, H) be a (n, p)-couple with G a torsion-free nilpotent pro-p-group. If H can be generated by d elements, there exists κ = κ(p, d, n) such that γκ (H) = 1. Proof Let κ be the integer defined in proposition 2.11. Given any x ∈ G, the group L = clG (xH ) is, by lemma 2.7, finitely generated. By proposition 2.11 we have [L, γκ (H)] = 1, because G is torsion-free so that a non-trivial n-unipotent automorphism of G can not have finite order. In particular, γκ (H) ≤ CH (x). Being x a generic element of G, this shows that γκ (H) = 1. ✷ When the prime p is large enough, some useful bounds can be obtained.

Proposition 2.13 Let G be a finite p-group of class m, and H a group of n-unipotent automorphisms of G. There exists integers c = c(n, m), l = l(n) such that, if p > l, then H is nilpotent of class at most c.

10

Carlo Casolo and Orazio Puglisi

Proof First of all assume G abelian. Set A = GH and let k = k(n), l = l(n) be the integers defined in lemma 3 of [CT]. For each g ∈ G and a ∈ A, we have [g,n a] = 1 and [G,r H] = 1 for some r. Lemma 3 of [CT] can be used to get that, for every g ∈ G and h1 , h2 , . . . , hk ∈ H, one has [g, h1 , h2 , . . . , hk ]l = 1. Since p > l the element [g, h1 , h2 , . . . , hk ] must be trivial, showing that [G,k H] = 1. Thus [G, γk (H)] = 1 and the class of H is bounded by c(n, 1) = k(n)−1. When G has class m > 1 we consider the action of H on the factors of the ascending (or descending) central series and apply the above argument. It is readily seen that [G,mk H] = 1 so that H has class at most c(n, m) = mk(n) − 1. ✷ This proposition can be immediately extended to pro-p-groups.

Corollary 2.3 Let (G, H) be a (n, p)-couple, with G nilpotent of class m, and let l(n), c(n, m) be the integers defined in proposition 2.13. If p > l(n), then H is nilpotent of class at most c(n, m) Proof If N is a basis of H-invariant open normal subgroups of G, then G/N is a finite p-group of class at most m, for all N ∈ N . The group H acts on each G/N as a group of n-unipotent automorphism so that, by proposition 2.13, γc(n,m)+1 (H) centralizes all these quotients of G. T ✷ Therefore [G, γc(n,m)+1 (H)] ≤ N ∈N N = 1 and H has class at most c(n, m). Before embarking in the proof of our main result, three technical lemmata are needed. The first of them is due to Hartley and its proof can be found in [Ha].

Lemma 2.14 Let G be a group with an ascending series with abelian factors. If the members of the series are characteristic, then G contains a characteristic subgroup U such that U is nilpotent of class at most two, and CG (U ) = ζ(U ).

Lemma 2.15 Let G be a group and H a group of n-unipotent automorphisms of G. Suppose that H has a normal abelian subgroup A such that H = Ahhi, and assume that [G, A, A] = 1. Then H is nilpotent. Proof The subgroup M = [G, A] is a normal abelian subgroup of G, and will be regarded as an hhi-module. For each k = 1, 2, . . . , n let Xk = {g ∈ G | [g,k h] = 1} and set A0 = A, Ak = CA (Xk ) when k = 1, . . . , n. The subgroups Ak form a descending chain in A and An = 1. Choose g ∈ X1 and consider the map

θg : A −→ [G, A] a 7−→

[a, g].

11

Nil automorphisms

An easy calculation shows that θg is an hhi-module homomorphism, so that A/ ker(θg ) is hhi-isomorphic to a submodule of [G, A]. In particular h acts as an n-unipotent automorphism on A/ ker(θg ). Thus \ ker(θg ) A = A0 / g∈X1

embeds, as an hhi-module, into a cartesian product of copies of M , hence it is acted upon by h as an n-unipotent automorphim. Therefore the element h stabilizes a series of length at most n in A0 /A1 . Suppose we have already shown that hhi stabilizes a series of length at most n, in each factor Ai /Ai+1 for i = 1, . . . , k < n. Let S be the semigroup generated by x = h−1 , and ZS its semigroup algebra. Pick g ∈ Xk+1 and define the map θg : Ak −→ M = [G, A] a 7−→

[a, g].

Using the fact that [g, h] ∈ Xk , we see that, for every a ∈ Ak , one has (ax )θg = ((a)θg )x . Therefore Ak / ker(θg ) is isomorphic, as a ZS-module, to a submodule of M . In particular, being (x − 1)n ∈ ZS, for every u ∈ Ak / ker(θg ), the identity [u,n x] = 1 holds. Then the same identity holds for the action on Ak /Ak+1 , because this group embeds, as a ZS-submodule, into a cartesian power of copies of M . Therefore hhi stabilizes a finite series of length at most n in Ak /Ak+1 . Hence hhi stabilizes a series of length at most n2 in A, showing that H is nilpotent. ✷

Lemma 2.16 Let (G, H) be a (n, p)-couple and assume that H = Rhhi with R contained in the Hirsch-Plotkin radical of H. Let N E H be such that H/N is nilpotent and N hhi is locally nilpotent. If R is nilpotent then H is nilpotent. Proof The group H is finitely generated and metanilpotent hence, by 15.5.1 of [R], the HirschPlotkin radical of H coincides with the Fitting subgroup and is nilpotent. The subgroup N hhi is subnormal in H because H/N is nilpotent. Thus N hhi is contained in R, the Hirsch-Plotkin radical of H. Hence hhi ≤ R and from this it follows that H = R. ✷

Proposition 2.17 Let (G, H) be a (n, p)-couple, and let l(n), c(n, m) be the integers defined in proposition 2.13. (1) If p > l(n), then [G, γc(n,2)+1 (H), γc(n,2)+1 (H)] = 1 In particular γc(n,2)+1 (H) is abelian. (2) H is nilpotent. Proof Let N = {Ni | i ∈ I} be a base of H-invariant open subgroups of G and set Gi = G/Ni . Each Gi is a finite p-group so that, by lemma 2.14, it has a characteristic subgroup Ui = Mi /Ni Q of nilpotency class at most two, such that CGi (Ui ) = ζ(Ui ). The group G = i∈I Gi can Q be topologized by assigning the set B = {N J = j∈J Gj | |I \ J| finite } as a base of open neighborhoods for the identity. In this way G is a compact pro-p-group, and H acts on it as a

12

Carlo Casolo and Orazio Puglisi

Q group of n-unipotent automorphisms. The subgroup U = i∈I Ui is closed in G, is acted upon continuously by G and H, and CG (U ) = ζ(U ). Choose any cofinite subset J of I and define Q U J = U ∩ N J = j∈J Uj . The subgroups U J form a basis of open neighborhoods of the identity in U . If A = CH (U J ), we have that [G, A] ≤ CG (U J ). Thus [Uj , [G, A]Nj /Nj ] = 1 for all j ∈ J, hence [G, A]Nj /Nj ≤ CGj (Uj ) = ζ(Uj ) when j ∈ J. Hence [[G, A]Nj /Nj , A] ≤ [ζ(Uj ), A] ≤ [Uj , A] = 1 for all j ∈ J, showing that [G, A, A] ≤ ∩j∈J Nj = 1 which, in turn, proves that A is abelian. In particular, if C = CH (U ), then [G, C, C] = 1 and C is abelian. If p > l(n) we can apply corollary 2.3 to the action of H/CH (U ) on U , showing that γc(n,2)+1 (H) ≤ CH (U ) so that (1) holds. S Let X = {h ∈ H | hH is abelian }. Assume that U = h∈H\X CU (h). Since H consists of continuous automorphisms of U , each CU (h) is closed and, being H \ X finite or countable, we may invoke Baire’s theorem to prove that, for some h ∈ H \ X, the subgroup CU (h) contains an open subset. Therefore CU (h) must contain an open subgroup of the form U J . The subgroup U J is normalized by H, so that U J ≤ CU (hx ) for all x ∈ H showing that U J centralizes hH . Thus hH ≤ CH (U J ) and, by the previous paragraph, hH is abelian, a contradiction. It is therefore S possible to find u ∈ U \ h∈H\X CU (h), so that CH (u) ≤ X. The group V = clU (uH ) is nilpotent of class at most 2, it is finitely generated ( see lemma 2.7) and CH (V ) is contained in CH (u). Since CH (V ) is contained in X, for every element c ∈ CH (V ) we have that cH is abelian and it follows that CH (V ) is a 2-Engel group. In particular CH (V ) is nilpotent of class at most 3. Moreover proposition 2.11 shows that H/CH (V ) is nilpotent. Thus H is metanilpotent and, in particular, it is solvable. To complete the proof we argue by contradiction, selecting H a counterexample of minimal derived length. Under this assumption each finitely generated subgroup of H ′ is nilpotent, so that H ′ is contained in R, the Hirsch-Plotkin radical of H. However H is metanilpotent hence, by 15.5.1 of [R], R is nilpotent and coincides with the Fitting subgroup of H. Each counterexample of minimal derived length, is therefore nilpotent-by-abelian. We may choose H in such a way that d(H/R), the minimal number of generators of H/R, is minimal. If H/R = hh1 R, . . . , hl Ri and l ≥ 2, then the two subgroups H1 = hh1 , Ri and H2 = hh2 , . . . hl , Ri, are both normal and nilpotent. To see this let F be a finite subset of Hi and consider W = hF i. Clearly Fit(W ) ≥ W ∩ R, hence W/Fit(W ) is an homomorphic image of W R/R ≤ Hi /R, showing that W/Fit(W ) has minimal number of generators strictly smaller than d(H/R). By assumption W is nilpotent. The group Hi is then locally nilpotent and normal in H, thus it is contained in R, the Hirsch-Plotkin radical of H, proving that H = H1 H2 ≤ R, a contradiction. Therefore d(H/R) = 1 and it is possible to write H = Rhhi for a suitable h ∈ H. Moreover we may choose H in such a way that the nilpotency class of R is minimal. Consider S = CH (clU (uH )) and the subgroup Shhi. Since H/S is nilpotent, then Shhi is subnormal in H and, by lemma 2.16, it can not be locally nilpotent. If, for every s ∈ S, the subgroup hs, hi is nilpotent, it is easy to show that Shhi is hypercentral, hence locally nilpotent. For this reason there must exist a ∈ S such that ha, hi is not nilpotent. Since a ∈ S ⊆ X, then aha,hi is abelian. For this reason there is no loss of generality in assuming that H = ha, hi = Ahhi where A = aH . Notice that A ≤ R.

13

Nil automorphisms

The group H is then metabelian, so that it satisfies Max-n, the maximal condition on normal subgroups. If W is a pro-p-group, and σ : H −→ Aut(W ) is an homomorphism, we say that σ is an n-unipotent representation if (W, H σ ) is a (n, p)-couple. The set N = {N | N ≤ H is the kernel of an n-unipotent representation and H/N is not nilpotent} is not empty, so that it has a maximal element M . By looking at the representation related to M , we may assume that • if 1 6= N E H and H/N acts faithfully on a pro-p-group W in such a way that (W, H/N ) is a (n, p)-couple, then H/N is nilpotent. Denote by C the centralizer of U in H. Two cases should be considered. Case 1. C is not trivial. Thus H/C is nilpotent and, as we have seen before, C is abelian and contained in R. Since [G, C, C] = 1, we may apply lemma 2.15 to see that the subgroup Chhi is nilpotent. Lemma 2.16 can now be used to see that H is nilpotent, a contradiction. Case 2 C = 1. The group H acts faithfully on U . If U is torsion, then it must be of finite exponent, because it is nilpotent and compact. If this is the case, for each u ∈ U , the subgroup cl(uH ) has, by lemma 2.7, a uniformly bounded number of generators. By Zelmanov’s solution of the restricted Burnside problem, cl(uH ) has uniformly bounded order. In particular cl(uH ) = uH and, for some m, H m ≤ CH (uH ). Hence \ Hm ≤ CH (uH ) = CH (U ) = 1 u∈U

and H is finite, hence nilpotent. Thus U is not a torsion group. When W is a nilpotent group, we shall indicate by T (W ) its torsion sugbroup. Let S = CH (U/T (U )) and, for each s ∈ S, consider the homomorphism

ψs : U −→

U/ζ(U )

u 7−→ [u, s]ζ(U ). This homomorphism is continuous and for this reason [U, s]ζ(U )/ζ(U ) is compact. On the other hand [U, s] is contained in the torsion subgroup of U , hence it has finite exponent, say e1 . For this reason [cl(U e1 ), s] ≤ ζ(U ). The same argument applies to the continuous homomorphism

ϕs : clU (U e1 ) −→ ζ(U ) u 7−→ [u, s]

14

Carlo Casolo and Orazio Puglisi

proving that clU (U e1 )e2 ≤ ker(ϕs ). It is then straightforward to see that, for each s ∈ S, there exists e = e(s) such that [clU (U e ), s] = 1. The subgroup S can be generated, as a normal subgroup, by finitely many elements s1 , . . . , sl . Choosing e = max{e(si ) | i = 1, . . . , l}, we get [clU (U e ), S] = 1. If S 6= 1 then P = CH (clU (U e )) ≥ S is not trivial and therefore H/P is nilpotent, by our choice of H. Thus γa (H) ≤ P for some a. The group H acts on W = U/clU (U e ), (W, H/CH (W )) is a (n, p)-couple and W is nilpotent of finite exponent. As we have seen before, H/CH (W ) is a finite p-group. There exists b such that γb (H) ≤ CH (U/clU (U e )) and, if c = max{a, b}, then D = γc (H) ≤ CH (U/clU (U e )) ∩ CH (clU (U e )). In particular [U, D, D] = 1 and D is abelian. By lemma 2.15 the subgroup Dhhi is nilpotent and, by lemma 2.16, H would be nilpotent. For this reason the subgroup S must be trivial. For every u ∈ U set L(u) for the closure in U of uH , and C(u) = CH (L(u)). Recall that each L(u) can be generated by a uniformly bounded number of elements. If C(u) = 1 for some u, then H is nilpotent, by proposition 2.11, a contradiction. Thus C(u) is always non-trivial, and H/C(u) is nilpotent, because L(u) is a finitely generated pro-p-group, hence countably based. By proposition 2.11, there exists κ such that γκ (H)C(u)/C(u) is finite. Therefore [L(u), γκ (H)] is contained in T (L(u)) ≤ T (U ). It then follows γκ (H) ≤ S = CH (U/T (U )) = 1, so that H is nilpotent. This final contradiction proves that the claim holds so that H must be nilpotent. ✷ We are now in a positon to prove Theorem C. Theorem C Let G be a profinite group and H a finitely generated group of n-unipotent automorphisms. Assume that G has a basis of open normal subgroups whose members are normalized by H. Then F = clG ([G, H]) is pro-nilpotent and H is nilpotent. Proof The fact that F is pro-nilpotent has been already noticed in the paragraph before lemma Q 2.3. Write F = p∈I Fp where each Fp is the p-Sylow subgroup of F . Assume first that H acts faithfully on F . By theorem 2.17 each H/CH (Fp ) is nilpotent, say of class cp and, when p > l(n), [Fp , γc(n,2)+1 (H), γc(n,2)+1 (H)] = 1. Therefore, if c = max{c(n, 2), cp | p ≤ l(n)} + 1}, we get [F, γc (H), γc (H)] = 1 showing that H is abelian-by-nilpotent. If, by way of contradiction, we assume the claim false, the same argument used in the proof of theorem 2.17 can be applied, to see that there must exist a counterexample H that is nilpotentby cyclic. Write H = Rhhi with R the Fitting subgroup of H. Lemma 2.15 says that γc (H)hhi is nilpotent, hence H is nilpotent, by lemma 2.16, a contradiction. Therefore H can not act faithfully on F and C = CH (F ) is not trivial. We point out that the previous argument shows that H/C is nilpotent. The group C satysfies [G, C, C] = 1, whence H is abelian-by-nilpotent. Assuming the claim false we apply once again the argument of theorem 2.17, to show that H may be assumed to be nilpotent-by-cyclic. At this stage the same considerations of the previous paragraph provide a contradiction, thus proving that H must be nilpotent. ✷ As anticipated in the introduction, our results imply a well known theorem due to Wilson [W1]. Theorem 2.18 Let H be a finitely generated residually finite n-Engel group. Then H is nilpotent

Nil automorphisms

15

Proof Let G be the profinite completion of H. Since H/ζ(H) is a finitely generated group of n-unipotent automorphisms of G, we apply theorem C to show that H/ζ(H) is nilpotent. Thus H is nilpotent and the theorem is proved. ✷

References [B]

R. Baer, Engelsche Elemente Noetherscher Gruppen, Math. Ann. 133 (1957), 256–270.

[C]

C. Casolo, Groups with all subgroups subnormal, Note Mat. 28 (2008), suppl. 2, 1-153.

[CT]

G. Crosby, G. Traustason, On right n-Engel subgroups, J. Algebra 324 (2010), no. 4, 875–883.

[DDMS] J. D. Dixon, M. P. F. du Sautoy, A. Mann and D. Segal , Analytic pro-p groups, LMS Lecture Notes Series, n.157, Cambridge University Press, 1991. [H1]

P. Hall, Some sufficient conditions for a group to be nilpotent, Illinois J. Math. 2 (1958), 787–801.

[H2]

P. Hall, The Edmonton notes on nilpotent groups, Queen Mary College Mathematics Notes, Queen Mary College Mathematics Department, London 1969.

[Ha]

B. Hartley, Finite groups of automorphisms of locally finite soluble groups, J. Algebra 57 (1979), no. 1, 241–257.

[Hu]

T. W. Hungerford, Algebra, GTM 73, Springer-Verlag, New York - Berlin - Heidelberg 1974.

[P]

B. I. Plotkin, Groups of automorphisms of algebraic systems, Wolters- Noordhoff Publishing, 1972.

[R]

D. J. S. Robinson, A course in the theory of groups, GTM 80, Springer-Verlag, New York - Berlin - Heidelberg 1982.

[W1]

J. S. Wilson, Two-generator conditions for residually finite groups, Bull. London Math. Soc. 23 (1991), no. 3, 239–248.

[W2]

J. S. Wilson, Profinite groups, LMSM New Series 19, Clarendon Press, Oxford University Press, New York, 1998.