permutation groups with multiply transitive suborbits - CiteSeerX

PERMUTATION GROUPS WITH MULTIPLY TRANSITIVE SUBORBITS By P. J. CAMERON [Received 8 August 1971]

In 1929, Manning ([5]) proved that if G is a primitive permutation group on Cl which is not 2-transitive, and if the stabilizer Ga of a point a e f i i s 2-transitive on an orbit F(a) with | F(a) | = v > 2, then Ga has an orbit A(a) with |A(a)| = w, where w > v and w\v(v— 1). In [1], I gave a shorter proof of this theorem, using combinatorial techniques developed by Sims, D. G. Higman, and others; I showed that, in the notation of that paper, we can take A = F*°F. Put w = v(v— l)/k, where k is a positive integer. Although Manning's theorem tells us only that k 2. The Mathieu group M22 (or its automorphism group) has a primitive rank 3 representation of degree 77 on the blocks of the associated Steiner system. The subdegrees are 1, 16, 60; the constituent of degree 16 is doubly transitive, and k = 4. The Higman-Sims group HS (or its automorphism group) has a primitive rank 3 representation of degree 100. The subdegrees are 1, 22, 77; the constituent of degree 22 is triply transitive, and k = 6. The purpose of this paper is to obtain a stronger result than that of Manning, and to obtain still stronger results under the assumption that 6?a is more than doubly transitive on F(a). The precise result (which is probably still less than the whole truth) is as follows. Suppose that G is a primitive, not 2-transitive permutation group on Q, and that G^ is ^-transitive on F(a) (t ^ 2), with |F(a)| = v > 2 and | (F*o F)(a) | = v{v-l)/k. Then the following results hold, (i) jfe 1, and I have been unable to prove their non-existence in general, but I have several sufficient conditions (including the condition 2 ^ A ^ 103) for their non-existence. Section 1 contains a result on paired suborbits in a transitive permutation group. A corollary of this result is the fact that if 0 is transitive on Q and Ga is 2-transitive on r(a), then Ga is 2-transitive on the paired suborbit F*(a).

0. Notation All permutation groups and sets considered in this paper are finite. A sketch of the graph-theoretic notation developed by Sims, T>. G. Higman, and others will be given. For the general theory and notation of finite permutation groups, see Wielandt ([7]). If Q is a transitive permutation group on Q, and A is a subset of the Cartesian product ClxCl which is fixed by G (acting in the natural way on Q x Q), then A(a) = {/3| (a,j8) e A} is a subset of Q.fixedby Ga; A thus defines an 'orbital'. This procedure sets up a one-to-one correspondence between (r-orbits in D x Q and (ra-orbits in £1. (The number of such orbits is the rank of G). A* = {(/?, a) | (a, j8) e A} is the orbital (or subset of Q x Q) paired with A; A is self-paired if A = A*. Note that |A(a)| = |A*(a)| = | A|/| Q |. If T and A are fixed sets of G in QxQ, let F o A denote the set {(oc,P)\ there exists y e Q with (a, y ) e F, (y5J8) e A ; a ^ j 3 } ; this is also a fixed set of G. The diagonal {{a, a) | a e £1} is a single 'trivial' #-orbit. Suborbits of G are (2a-orbits in Q; where there is no confusion, I shall use the term for (r-orbits in Q x Q also. If F is a non-trivial 6r-orbit in Q, x Cl, the F-graph is the regular directed graph whose point set is Q and whose edges are precisely the ordered pairs in P. If F is self-paired, the F-graph can be regarded as an undirected graph. It is helpful to think of a colour associated with each non-trivial suborbit, and to suppose that in the complete directed graph on Q, all edges belonging to the F-graph are coloured with the colour associated with F. A connected component of any such graph is a block of imprimitivity for G\ G is primitive if and only if each such graph is connected.

GROUPS WITH MULTIPLY TRANSITIVE SUBORBITS

429

1. Paired suborbits

Let G be a transitive permutation group on Cl, and let F(a) and F*(a) be paired suborbits. Crar(a) and GJ"( 1). Let G be a transitive permutation group on Cl, having paired suborbits F and F*, with | F(a) | = v. If # a r ( a > has 0>, then GaT*{Ci) also has 2P.

Proof. Suppose that the theorem is false, and let the permutation group G on the set Q be a counter-example, with F and F* the paired suborbits and 0> the suitable property. A path of length k (for k ^ 0) is an ordered (& + l)-tuple (a0, ...,ak) of points of Q. with the property that ajt_x is transitive on F(aft_1), where (a0, ...ja^.^) is a path of length k—1.) A± and B x are obvious. So suppose that k ^ 1 and that Ak and Bk hold. Let (al5 ...,ock) be a path of length k—1, and let H = Gai ak. By hypotheses Ak and B^, Hr(0lk) is transitive and has 0>. Also, /F* ( a i ) is transitive, but since (rair*(ai) is assumed not to have 8P and H ^ Gw condition 2 of the definition shows that Hr*{GA. Then L