Automorphisms of a family of cubic graphs

Transcription

1 Automorphisms of a family of cubic graphs Jin-Xin Zhou Mohsen Ghasemi ICM 2012, March, Al Ain Abstract A Cayley graph Cay(G, S) on a group G with respect to a Cayley subset S is said to be normal if the right regular representation R(G) of G is normal in the full automorphism group of Cay(G, S). For a positive integer n, let Γ n be a graph having vertex set {x i, y i i Z 2n } and edge set {{x i, x i+1 }, {y i, y i+1 }, {x 2i, y 2i+1 }, {y 2i, x 2i+1 } i Z 2n }. In this paper, it is shown that Γ n is a Cayley graph and its full automorphism group is isomorphic to Z 3 2 S 3 for n = 2, or to Z n 2 D 2n for n > 2. Furthermore, we determine all pairs of group G and Cayley subset S satisfying Γ n = Cay(G, S) is non-normal for G. Using this, all connected cubic non-normal Cayley graphs of order 8p are constructed explicitly for each prime p. Key Words: Cayley graphs, vertex-transitive graphs, automorphism groups 2000 Mathematics Subject Classification: 05C25, 20B25. 1 Introduction For a finite, simple, undirected and connected graph X, we use V (X), E(X), A(X) and Aut(X) to denote its vertex set, edge set, arc set and full automorphism group, respectively. For u, v V (X), denote by {u, v} the edge incident to u and v in X. A graph X is said to be vertex-transitive, edge-transitive and arc-transitive (or symmetric) if Aut(X) acts transitively on V (X), E(X) and A(X), respectively. In particular, if Aut(X) acts regularly on A(X), then X is said to be 1-regular. Let G be a permutation group on a set Ω and a Ω. Denote by Γ a the stabilizer of a in G, that is, the subgroup of G fixing the point a. We say that G is semiregular on Ω if G a = 1 for every a Ω and regular if G is transitive and semiregular. Given a finite group G, an inverse closed subset S G \ {1} is called a Cayley subset of G. The Cayley graph Cay(G, S) on G with respect to a Cayley subset S is defined to have vertex set G and edge set {{g, sg} g G, s S}. A Cayley graph Cay(G, S) is connected if and only if S generates G. Given a g G, define the permutation R(g) on G by x xg, x G. Then R(G) = {R(g) g G}, called the right regular representation of G, is a permutation group isomorphic to G. In general, a graph X is isomorphic to a Cayley graph on a group G if and only if its automorphism group has a subgroup isomorphic to G, acting regularly on the vertex set of X (see [?, Lemma 4]). It is well-known that R(G) Aut(Cay(G, S)). So, Cay(G, S) is vertex-transitive. A Cayley graph Cay(G, S) is said to be normal if R(G) is normal in Aut(Cay(G, S)). Determining automorphism groups or, equivalently, studying normality of Cayley graphs, plays an important role in the investigation of various symmetry properties of graphs, and have been becoming a very active topic in the algebraic graph theory. The concept of normal Cayley Department Of Mathematics, Beijing Jiaotong University, Beijing , P.R. China Department of Mathematics, Urmia University, Urmia 57135, Iran 1

2 2 graph was first introduced by Xu [?], and following this article, the normality of Cayley graphs have been extensively studied from different perspectives by many authors. Note that Wang et al. [?] obtained all disconnected normal Cayley graphs. For this reason, it suffices to consider the connected ones when one investigates normality of Cayley graphs. It can be easily obtained from elementary permutation group theory that a Cayley graph of prime order is normal if the graph is neither empty nor complete. The normality of Cayley graphs of order a product of two primes was determined by Dobson et al. [?,?,?]. Recently, Dobson et al. [?] determined the normality of Cayley graphs on Z 3 p. There also has been a lot of interest in the studying of normality of small valent Cayley graphs. For the 4-valent case, Baik et al. [?] determined all non-normal Cayley graphs on abelian groups with valency at most 4. Wang and Xu [?] determined all tetravalent non-normal 1-regular Cayley graphs on dihedral groups. Feng and Xu [?] proved that every connected tetravalent Cayley graph on a regular p-group is normal when p 2, 5. Li et al. [?,?] investigated the normality of tetravalent edge-transitive Cayley graphs on G, where G is either a group of odd order or a finite non-abelian simple group. Kovács [?] classified all connected tetravalent nonnormal arc-transitive Cayley graphs on dihedral groups satisfying one additional restriction: the graphs are bipartite, with the two bipartition sets being the two orbits of the cyclic subgroup within the dihedral group. Recently, Zhou [?] determined all tetravalent non-normal Cayley graphs of order 4p for each prime p. For the 3-valent case, Fang et al. [?] proved that the vast majority of connected cubic Cayley graphs on non-abelian simple groups are normal, and Xu et al. [?,?] classified all connected cubic non-normal Cayley graphs on non-abelian simple groups. Let p and q be two primes. In [?,?,?], all connected cubic non-normal Cayley graphs of order 2pq are determined. For more results on the normality of Cayley graphs, we refer the reader to [?,?,?]. Let n 2 be an integer. The graph Γ n (see Fig.??) is defined to have vertex set V = {x i, y i i Z 2n } and edge set E = {{x i, x i+1 }, {y i, y i+1 }, {x 2i, y 2i+1 }, {y 2i, x 2i+1 } i Z 2n } x 0 x 1 x 2 x 3 x 4 x 2n 3 x 2n 2 x 2n y 0 y 1 y 2 y 3 y 4 y n 3 y 2n 2 y 2n 1 Figure 1: The graph Γ n This family of graphs have been investigated in several recent research papers, see, for example, [?,?,?]. In this paper, it is shown that Γ n is a Cayley graph, and its full automorphism group is isomorphic to Z 3 2 S 3 for n = 2, or to Z n 2 D 2n for n > 2. Furthermore, we determine all possible pairs of group G and Cayley subset S such that Γ n = Cay(G, S) is non-normal for G. Using this, all connected cubic non-normal Cayley graphs of order 8p are constructed explicitly for each prime p. It appears that there are seven sporadic and four infinite families of cubic non-normal Cayley graphs of order 8p. To end this section, we introduce some notations and definitions as well as some preliminary results which will be used later in the paper. For a regular graph X, use d(x) to represent the valency of X, and for any subset B of V (X), the subgraph of X induced by B will be denoted by X[B]. Let X be a connected vertex-transitive graph, and let G Aut(X) be vertex-transitive on X. For a G-invariant partition Σ of V (X), the quotient graph X Σ is defined as the graph with vertex set Σ such that, for any two vertices B, C Σ, B is adjacent to C if and only if there exist u B and v C which are adjacent in

3 3 X. Let N be a normal subgroup of G. Then the set Σ of orbits of N in V (X) is a G-invariant partition of V (X). In this case, the symbol X Σ will be replaced by X N. For a positive integer n, denote by Z n the cyclic group of order n as well as the ring of integers modulo n, by Z n the multiplicative group of Z n consisting of numbers coprime to n, by D 2n the dihedral group of order 2n, and by C n and K n the cycle and the complete graph of order n, respectively. We call C n an n-cycle. For two Cayley subsets S and T of a group G, if there is an α Aut(G) such that S α = T then S and T are said to be equivalent, denoted by S T. One may easily show that if S and T are equivalent then Cay(G, S) = Cay(G, T ) and then Cay(G, S) is normal if and only if Cay(G, T ) is normal. For two groups M and N, N M denotes a semidirect product of N by M. For a subgroup H of a group G, denote by C G (H) the centralizer of H in G and by N G (H) the normalizer of H in G. Then C G (H) is normal in N G (H). 2 Automorphism group of Γ n Let n 2 be a positive integer. The main purpose of this section is to determine the automorphism groups of the graphs Γ n given in last section. To do this, we first introduce several groups of order 4n which will be used throughout the paper. G 0 4n = a, b a2n = 1, a n = b 2, b 1 ab = a 1 ; G 1 4n = a, b a2n = 1, a n = b 2, b 1 ab = a n 1 (n is even); G 2 4n = a, b an = b 4 = 1, b 1 ab = a 1 (n is even); G 3 4n = a, b an = b 4 = 1, b 2 a = ab 2, b 1 ab = a 1 b 2 (n is even); G 4 4n = a, b a2n = b 2 = 1, b 1 ab = a 1 ; G 5 4n = a, b, c an = b 2 = c 2 = 1, b 1 ab = a 1, ac = ca, bc = cb (n is even); G 6 4n = a, b a8 = b 2 = 1, bab = a 5 (n = 4); G 7 4n = a, b, c a3 = b 3 = (ab) 2 = c 2 = 1, ac = ca, bc = cb (n = 6); G 8 4n = a, b, c a4 = b 2 = c 3 = 1, bab = a 1, ac = ca, bc = cb (n = 6). (1) Lemma 1 Let G be a finite group. If G has a normal subgroup N of order 2 such that G/N = D 2n, then G is isomorphic to one of the groups G i 4n (0 i 5). Proof Let N = c. As G/N = D 2n, let G/N = an, bn a n N = b 2 N = N, b 1 abn = a 1 N. Then G = a, b, c. As b 2 N = N, one has b 2 = c or 1. Similarly, a n = c or 1. So, we have the following four cases: Case 1: b 2 = c, a n = c. In this case, b 2 = a n and a 2n = 1. Since b 1 abn = a 1 N, one has b 1 ab = a 1 or a 1 c. For the former, we have G = G 0 4n. Let b 1 ab = a 1 c. Since c = a n, one has b 1 ab = a n 1, implying that (n 1, 2n) = 1. Thus, n is even, and hence G = G 1 4n. Case 2: b 2 = c, a n = 1. In this case, we also have b 1 ab = a 1 or a 1 c. Let b 1 ab = a 1. If n is odd, then let x = ac and y = b. Then G = x, y x 2n = 1, y 2 = x n, x 1 yx = y 1 = G 0 4n. If n is even, then G = G 2 4n. Let b 1 ab = a 1 c. Then a and a 1 c have the same order. It follows that n is even, and hence G = G 3 4n. Case 3: b 2 = 1, a n = c. In this case, a 2n = 1. If b 1 ab = a 1 then G = G 4 4n. If b 1 ab = a 1 c = a n 1, then n is even. Set x = a, y = ba. Then, y 2 = a n = x n and y 1 xy = a n 1 = x n 1. So, G = x, y x 2n = 1, x n = y 2, y 1 xy = x n 1 = G 1 4n.

4 4 Case 4: b 2 = 1, a n = 1. If b 1 ab = a 1, then G = G 5 4n. Let b 1 ab = a 1 c. Set x = a, y = ba. Then G = x, y x n = y 4 = 1, y 2 x = xy 2, y 1 xy = x 1 y 2 = G 3 4n. The following is the main result of this section. Theorem 2 If n = 2 then Γ n = Q3, the three-dimensional hypercube of order 8 and Aut(Γ n ) = Z 3 2 D 6. If n > 2, then Aut(Γ n ) = Z n 2 D 2n. Furthermore, Γ n is isomorphic to the non-normal Cayley graph Cay(G, S), where G and S are as the following: (1) G = Z 4 Z 2 = a b, S {a, a 1, b} and X = Q 3 ; (2) G = G 4 8, S {b, a, a 1 } or {b, ba, ba 4 } and X = Q 3 ; (3) G = G 1 4n and S {b, b 1, ba} (n is even); (4) G = G 3 4n and S {b, b 1, ab} (n is even); (5) G = G 4 4n and S {b, ba, ban } (n > 2); (6) G = G 5 4n and S {b, bc, ba} (n is even). Proof If n = 2, then by MAGMA [?] Γ n is symmetric. By [?], up to isomorphism there is a unique cubic symmetric graph of order 8. It follows that Γ 2 = Q3, and Aut(Γ 2 ) = Z 3 2 D 6. By [?, Theorem 2.3], we have the Cases (1) and (2). Let n > 2 and A = Aut(Γ n ). Let α : x i x i+2, y i y i+2 (i Z 2n ), β : x i y i, y i x i (i Z 2n ), and δ : x i x 2n+1 i, y i y 2n+1 i (i Z 2n ) be three permutations on V (X). It is easy to check that a, b and g are automorphisms of X. Furthermore, α, β, δ is regular on V (X). It follows that Γ n is a Cayley graph. Set D i = {x i, y i } for i Z 2n. For any g A, assume D g i D i. Take v D g i D i. Without loss of generality, let v = x i with i even. Since n > 2, (x i, x i+1, y i, y i+1 ) is the unique 4-cycle in Γ n passing through x i. Set D g i = {x i, a}. Clearly, there is a 4-cycle in Γ n passing through x i and a. It follows that a {x i+1, y i, y i+1 }. Since Γ n [D i ] = Γ n [D g i ] = 2K 1, one has a = y i and hence D i = D g i. Let B = {D i i Z 2n }. Then B is an A-invariant partition of V (Γ n ). Consider the quotient graph (Γ n ) B and let K be the kernel of A acting on B. Then (Γ n ) B = C2n and A/K D 4n. It is easy to see that A/K is vertex-transitive but not edge-transitive on (Γ n ) B. So, A/K = D 2n. Furthermore, K = (x 1 y 1 )(x 2 y 2 )... (x 2n 1 y 2n 1 )(x 0 y 0 ) = Z n 2. Thus, A = K α, δ = Z n 2 D 2n. Let G be an arbitrary subgroup of A acting regularly on V (Γ n ). Then G = 4n. Since G is transitive on V (Γ n ), one has GK/K = A/K = D 2n. It follows that G K = Z 2 and G/(G K) = D 2n. Set N = G K = c. By Lemma??, G is isomorphic to one of G i 4n (i Z 6). Since G is regular on V (Γ n ), Γ n is a Cayley graph on G. Without loss of generality, assume that Γ n = Cay(G, S), where S = S 1 and 1 / S. Then V (Γ n ) = G and G acts on V (Γ n ) by right multiplication. Let D 0 = {1, c}(= N). Then each D i (i Z 2n ) is a right coset of D 0 in G. Let D 1 = {x, cx} and D 2n 1 = {y, cy}. Then S = {x, cx, y}. In what follows, we consider the following two cases. Case 1: x has order greater than 2. Then xc also has order greater than 2. Since S = S 1, x 1 = xc, and y is an involution. Then c = x 2. It follows that G = x, y with x an element of order 4. Note that G 0 4n has a unique involution a n. Since this involution and any other element of G 0 4n can not generate G0 4n, one has G G 0 4n. If G = G4 4n = a, b a2n = b 2 = 1, b 1 ab = a 1, then x a and y 1 xy = x 1. It follows that G = D 8 and hence n = 2, a contradiction. Similarly, it can be shown that G G 5 4n.

5 5 Let G = G 2 4n = a, b an = b 4 = 1, b 1 ab = a 1. Clearly, G 2 4n = {ai, a i b 2, a i b, a i b 1 i Z n }. Furthermore, for each i Z n, a i b and a i b 1 have order 4. So, y a b 2. Since b 1 ab = a 1, y is in the center of G 2 4n, forcing G2 4n = x, y is abelian, a contradiction. Thus, G = G1 4n or G3 4n. Let G = G 1 4n = a, b a2n = 1, a n = b 2, b 1 ab = a n 1 (n is even). Clearly, G 1 4n = {ai, a i b 0 i 2n 1}. It is easy to check that the center of G 1 4n is an. Furthermore, if i is odd then a i b has order 2, and if i is even then a i b has order 4. Since x, y = G 1 4n and n > 2, one has x = a i b and y = a j b where i Z 2n is even and j Z 2n is odd. It is easy to see that the map a i b b, a a can induce an automorphism of G 1 4n. So, we may let x = b. Again, since x, y = G 1 4n, one has j Z 2n. Let β be the automorphism of G1 4n induced by aj a, b b. Then S b = {b, b 1, ba j } β = {b, b 1, ba}. Thus, S {b, b 1, ba}, that is, Case (3) happens. Let G = G 3 4n = a, b an = b 4 = 1, b 2 a = ab 2, b 1 ab = a 1 b 2. Clearly, G 3 4n = {ai, a i b 2, a i b, a i b 1 i Z n }. It is easy to check that the center of G 3 4n is b2. Furthermore, if i is odd then a i b and a i b 1 have order 2, and otherwise they have order 4. Since x 2 = c, one has {x, xc} = {a i b, a i b 1 }, where i Z n is even. It is easy to see that the map a a, a i b b can induce an automorphism δ of G 3 4n. So we may let S = {b, b 1, y}. Since n > 2 and G 3 4n = x, y, one has y = a j b or a j b 1 for some odd j Z n. Let θ and σ be the automorphisms of G 3 4n induced by a j a, b b, c and a a, b b 1, c c, respectively. Then S θ = {b, b 1, ab} or {b, b 1, ab 1 }. Since {b, b 1, ab} σ = {b, b 1, ab 1 }, one has S {b, b 1, ab}, that is, Case (4) happens. Case 2: x has order 2. Since c is in the center of G, cx also has order 2. So, y must be an involution, and hence x, y is a dihedral group. Let c x, y. Then G = x, y. The only possible case is G = G 4 4n = a, b a2n = b 2 = 1, b 1 ab = a 1. It is easy to check that the center of G 4 4n is an and that the set of involutions of G 4 4n is {an, a i b 0 i 2n 1}. Then c = a n. Since the automorphism group Aut(G 4 4n ) of G4 4n is transitive on {a i b 0 i 2n 1}, one may let x = b and y = ba j for some j Z 2n 1. Since G 4 4n = x, y, one has j Z 2n. Let α be the automorphism of G4 4n induced by aj a, b b. Then {b, ba n, ba j } α = {b, ba n, ba}. Therefore, S {b, ba n, ba}, and hence we have Case (5). Let c / x, y. Then G = x, y c = D 2n Z 2. If n is odd, then G = G 4 4n = a, b a2n = b 2 = 1, bab = a 1, implying c = a n. Furthermore, we may let x = b. Since x, y = D 2n, one may let y = ba 2j for some j mathbbzn. Then (2j + n, 2n) = 1, and hence the maps β : a 2j+n a, b b and γ : a a, b ba can induce two automorphisms of the group G 4 4n. Since Sβγ = {b, ba n, ba}, one has S {b, ba n, ba}. We again have Case (5). If n is even, then G = G 5 4n = a, b, c an = b 2 = c 2 = 1, bab = a 1, ac = ca, bc = cb. We may let x, y = a, b = D 2n. Then x = ba i, y = ba j for some i, j Z n. It is easy to see that the group of automorphisms of G 0 4n fixing c is transitive on {ba k k Z n }. So, we may let x = b. Since x, y = a, b, one has j Z n. Let θ be the automorphism of G 5 4n induced by aj a, b b, c c. Then S θ = {b, bc, ba j } θ = {b, bc, ba}, and hence S {b, bc, ba}. The Case (6) happens. 3 Proof of the main theorem Let p be an odd prime. In this section, we shall classify all connected cubic non-normal Cayley graphs of order 8p. In fact Zhou and Feng [?], classified the normality of cubic cayley graph of order 4p as following: Theorem 3 Let p be a prime and let X = Cay(G, S) be a connected cubic Cayley graph on a

6 6 group G of order 4p. Then either Aut(X) = R(G).Aut(G, S) or one of the following happens: (1) G = Z 4 Z 2 = a b, S = {a, a 1, b}, Aut(X) = (Z 2 Z 2 Z 2 ) S 3, and X = Q 3, the three-dimensional hypercube of order 8; (2) G = D 8, S = {b, a, a 1 } or {b, ab, a 2 b}, and X = Q 3 ; (3) G = a, b a 2p = b 2 = 1, b 1 ab = a 1 (p 3), S = {b, ab, a p b}, Aut(X) = Z p 2 D 2p, and X = Cay(D 4p, {b, ab, a p b}). The lexicographic product C p [2K 1 ] is defined as the graph with vertex set {x i, y i i Z p } and edge set {{x i, x i+1 }, {y i, y i+1 }, {x i, y i+1 }, {y i, x i+1 } i Z p }. Lemma 4 Let p be an odd prime. Let A = Aut(C p [2K 1 ]) and let P be a Sylow p-subgroup of A. Then the centralizer of P in A has order 2p. Proof It is easily known that A = Z p 2 D 2p. Let K be the maximal normal 2-subgroup of A. Then A/K = D 2p and K = k 0 k 1... k p 1, where k i = (x i y i ) for i Z p. Let α = (x 0 x 1... x p 1 )(y 0 y 1... y p 1 ). It is easy to see that α is an automorphism of C p [2K 1 ] of order p. Without loss of generality, let P = α. It is easy to see that k 0 k 1... k p 1 = C A (P ) K. Therefore, C A (P ) 2p. Suppose that C A (P ) > 2p. Then C A (P )K = A, implying A/K = C A (P )/(C A (P ) K) = Z 2p, a contradiction. Thus, C A (P ) = 2p. Lemma 5 Let X = Cay(G, S) be a connected cubic symmetric Cayley graph on a group G of order 16. Then X is non-normal if and only if one of the following holds. (1) G = G 0 16 and S {b, ba, ba3 }; (2) G = G 1 16 and S {ba, a, a 1 }; (3) G = G 6 16 and S {b, a, a 1 } or {b, ab, (ab) 1 }; Proof It can be obtained by MAGMA [?] that the Cayley graphs Cay(G 0 16, {b, ba, ba3 }), Cay(G 1 16, {ba, a, a 1 }), Cay(G 6 16, {b, a, a 1 }) and Cay(G 6 16, {b, ab, (ab) 1 }) are non-normal and symmetric. We only need to prove the necessity. Since X is symmetric, by [?] there is a unique cubic symmetric graph of order 16. It follows that X is isomorphic to the Möbius-Kantor graph. With the help of MAGMA [?], G is isomorphic to one of G 0 16, G1 16 or G6 16. Let G = G 0 16 = a, b a8 = b 2 = 1, bab = a 1. By [?, Theorem 2.1], S {b, ba, ba 3 }. Let G = G 1 16 = a, b a8 = 1, a 4 = b 2, b 1 ab = a 3. By MAGMA [?], G cannot be generated by any three involutions. So, S consists of an involution, say x, and an element, say y, of order greater than 2 and its inverse. Then, x {a 4, ba, ba 3, ba 5, ba 7 }, and y = a i for some i Z 8 because the Möbius-Kantor graph has girth 6. Since ba, ba 3, ba 5 and ba 7 are conjugate and S generates G, one may assume that x = ba. It is easy to see that for each i Z 8, the map a a i, b ba 1 i can induce an automorphism of G. Thus, one may assume that y = a, and hence S {ba, a, a 1 }. Let G = G 6 16 = a, b a8 = b 2 = 1, bab = a 5. Then G has three involutions a 4, b and a 4 b, and furthermore, a 4 is in the center of G. Since S generates G, one has a 4 / S. Note that the map a a, b a 4 b can induce an automorphism of G. So, we may assume S = {b, x, x 1 } with x G. Since b together with an element of order 4 can not generate G, one may assume x {a, a 3, ab, a 5 b}. Note that the maps a a l, b b (l Z 8 ) can induce automorphisms of G. So, we may take x = a or ab, and hence S {b, a, a 1 } or {b, ab, (ab) 1 }.

7 7 Lemma 6 Let X = Cay(G, S) be a connected cubic symmetric Cayley graph on a group G of order 24. Then X is non-normal if and only if one of the following holds. (1) G = G 1 24 and S {ba, a, a 1 }; (2) G = G 3 24 and S {a3, ba, b 1 a 3 }, {a 3, ba, b 1 a 5 }, or {ba 3, b 2 a 3, b 1 a}; (3) G = G 7 24 and S {abc, ac, a 1 c}; (4) G = G 8 24 and S {b, ac, a 1 c 1 }. Proof By MAGMA [?], we can obtain the sufficiency. We only need to prove the necessity. With the help of MAGMA [?] we can obtain that G = G 1 24, G3 24, G7 24 or G8 24. Let G = G 1 24 = a, b a12 = 1, a 6 = b 2, b 1 ab = a 5. By MAGMA [?], G can not be generated by any three involutions. So, S consists of an involution, say x, and an element, say y, of order greater than 2 and its inverse. Then x = a 6 or ba i for some odd i Z 12. Further, for any odd i Z 12, there exists an automorphism of G mapping bai to ba. Then we may assume x = ba because S generates G. By [?], the cubic symmetric graph of order 24 has girth 6. It follows that y has order at least 6. Then y = a i for some i Z 12 because ba, a2 = G. Note that the map a a i, b ba 1 i (i Z 12 ) can induce an automorphism of G. Thus, one has S {ba, a, a 1 }. Let G = G 3 24 = a, b a6 = b 4 = 1, b 2 a = ab 2, b 1 ab = a 1 b 2. Clearly, S contains an involution, say x. Since all Sylow 2-subgroups of G are conjugate, one may assume x a 3, b and x b 2. Since b 1 a 3 b = b 2 a 3 and b 1 (ba 3 )b = b 1 a 3, one may assume x = b 2, a 3 or ba 3. If b 2 S, then by MAGMA [?], one has S G, a contradiction. Let x = a 3. In this case, if S also contains an element of order greater than 2, then by MAGMA [?] X is either disconnected or non-symmetric, a contradiction. It follows that S consists of three involutions, say S = {a 3, y, z}. If y {b 2, b 2 a 3, } then by MAGMA [?], X is disconnected, a contradiction. Thus, y, z I = {ba, ba 3, ba 5, b 1 a, b 1 a 3, b 1 a 5 }. Since all elements in I are conjugate under the centralizer of a 3 in G, one may take y = ba. By MAGMA [?], one can obtain that S = {a 3, ba, b 1 a 3 } or {a 3, ba, b 1 a 5 }. Let x = ba 3. If S also contains an element, say y, of order greater than 2, then y has order 6 because X has girth 6. In this case, by MAGMA [?], it can be obtained that X is non-symmetric, a contradiction. Thus, S consists of three involutions. By MAGMA [?], one can obtain that S = {ba 3, b 2 a 3, b 1 a} or {ba 3, b 2 a 3, b 1 a 5 }. It is easy to see that the map a a 1, b b can induce an automorphism of G. Thus, S {ba 3, b 2 a 3, b 1 a}. Let G = G 7 24 = a, b, c a3 = b 3 = (ab) 2 = c 2 = 1, ac = ca, bc = cb. Set P = ab, ba, c. Then P is a normal Sylow 2-subgroup of G. So, S consists of an involution, say x, and an element, say y, of order greater than 2 and its inverse. Since X has girth 6, y has order at least 6. It is easy to check that every element of G of order 6 is conjugate to ac or a 1 c. So, we may assume that y = ac. Since S generates G and X is symmetric, by MAGMA [?], x = abc, bac or ab 2 ac. Since a 1 (ab)a = ba and a 1 baa = ab 2 a, one has S {abc, ac, a 1 c}. Let G = G 8 24 = a, b, c a4 = b 2 = c 3 = 1, bab = a 1, ac = ca, bc = cb. Clearly, every Sylow 2-subgroup of G is normal. So, S consists of an involution, say x, and an element, say y, of order greater than 2 and its inverse. Clearly, x a 2. It is easy to check that there is an automorphism of G mapping x to b. So we may take x = b. Since X is connected and symmetric, by MAGMA [?], y has order 12. Also, one can check that there is an automorphism of G fixing b and mapping y to ac. Thus, S {b, ac, a 1 c 1 }. The following is the main result of this section. Theorem 7 Let G be a group of order 8p with p a prime. A connected cubic Cayley graph Cay(G, S) on G is non-normal if and only if one of the following happens:

8 8 (1) G = G 0 16 and S {b, ba, ba3 } (p = 2); (2) G = G 1 8p and S {ba, a, a 1 } (p = 2, 3); (3) G = G 1 8p and S {b, b 1, ba}; (4) G = G 3 24 and S {a3, ba, b 1 a 3 }, {a 3, ba, b 1 a 5 }, or {ba 3, b 2 a 3, b 1 a} (p = 3); (5) G = G 3 8p and S {b, b 1, ab}; (6) G = G 4 8p and S {b, ba, ban }; (7) G = G 5 8p and S {b, bc, ba}; (8) G = G 6 16 and S {b, a, a 1 } or {b, ab, (ab) 1 } (p = 2); (9) G = G 7 24 and S {abc, ac, a 1 c} (p = 3); (10) G = G 8 24 and S {b, ac, a 1 c 1 } (p = 3). Proof The sufficiency can be obtained from Theorem??, Lemmas?? and??. We only need to prove the necessity. Let X = Cay(G, S) be a connected cubic non-normal Cayley graph of order 8p. Set A = Aut(X). We consider two cases: p = 2 and p > 2. Case 1: p = 2. In this case, X = 16. If X is symmetric then by Lemma??, we can obtain the Cases (1),(2) and (8) in the theorem. In what follows, assume that X is non-symmetric. Then the vertex-stabilizer A v of v V (X) in A is 2-group. This implies that A is also a 2-group and hence the center of A is non-trivial. Take a subgroup, say N, of order 2 in the center of A. Consider the quotient graph X N of X relative to the orbit set of N, and let K be the kernel of A acting on V (X N ). Then X N has order 8 and valency 2 or 3, and A/K is a vertex-transitive group of automorphisms of X N. Let X N have valency 3. Then the stabilizer K v of v V (X) in K fixes the neighborhood of v in X pointwise because K fixes each orbit of N setwise. By the connectivity of X, K v fixes each vertex in V (X), forcing K v = 1. Hence, K = N. By [?], X N is a Cayley graph, and furthermore, either X N = Q3, the three dimensional hypercube, or Aut(X N ) 16. Note that if X N = Q3 then Aut(X N ) = S 4 Z 2. So, we always have A/N 16. It follows that A 32, and hence R(G) is normal in A, a contradiction. Let X N have valency 2. Let V (X N ) = {B i i Z 8 } with B i B i+1. Since d(x) = 3 and X is connected, d(x[b 0 ]) = 0 or 1. Assume d(x[b 0 ]) = 1. Then the stabilizer K u of u V (X) in K fixes the neighborhood of u in X pointwise because K fixes each orbit of N. By the connectivity of X, one has K u = 1 and hence K = N = Z 2. Since X N = C8, one has A/K Aut(C 8 ) = D 16 and hence A 32. It follows that R(G) A, a contradiction. Assume d(x[b 0 ]) = 0. Since d(x) = 3, one may let X[B 0 B 1 ] = 2K 2 and X[B 0 B 7 ] = C 4. Let B i = {x i, y i } for each i Z 8. By the transitivity of A on V (X), we may assume that x i x i+1, y i y i+1, x 2i y 2i+1 and y 2i x 2i+1 for each i Z 8. Then X = Γ 4. From Theorem?? we obtain the Cases (3) and (5)-(7) in the theorem. Case 2: p > 2. Assume that X is symmetric. By [?, Theorem 5.1], either 3 p 1 and X is 1-regular, or X = F24, F40, F56B or F56C (see [?] for the definition of such notations). By MAMGA [?], F40, F56B and F56C are not Cayley graphs. Let X = F24. From Lemma?? we obtain the Cases (2), (4), (9) and (10) in the theorem. Let X be 1-regular. Then 3 p 1 and A = 24p. Let P be a

9 9 Sylow p-subgroup of A. It follows from [?, Theorem 5.1] that P A and hence P R(G). By [?, Lemmas 3.2,3.4], A/P = A 4 Z 2. It follows that R(G)/P is the unique Sylow 2-subgroup of A/P and hence it is normal. Thus, R(G) A, a contradiction. In what follows, assume that X is non-symmetric. Then the stabilizer A 1 of the vertex 1 V (X) = G in A has order a power of 2. It follows that A = G A 1 = 2 3+n p for some integer n. Since X is non-normal, one has n > 1. Let P be a Sylow p-subgroup of A such that P R(G). Clearly, P = Z p. We first prove the following claim. Claim: P is non-normal in A. Suppose to the contrary that P A. Consider the quotient graph X P of X relative to the orbit set of P, and let K be the kernel of A acting on V (X P ). Then X P has order 8 and valency 2 or 3, and A/K is a vertex-transitive group of automorphisms of X P. Let X P have valency 3. Then the stabilizer K v of v V (X) in K fixes the neighborhood of v in X pointwise because K fixes each orbit of P setwise. By the connectivity of X, K v fixes each vertex in V (X), forcing K v = 1. Hence, K = P. By [?], X P is a Cayley graph, and furthermore, either X P = Q3, the three dimensional hypercube, or Aut(X P ) 16. Note that if X P = Q3 then Aut(X P ) = S 4 Z 2. So, we always have A/P = 2 3+n 2 4, contrary to the fact that n > 1. Now let X P have valency 2, that is, X P = C8. Then A/K Aut(X P ) = D 16. Let V (X P ) = {B i i Z 8 } with B i B i+1 for each i Z 8. If some B i contains an edge of X, then the connectivity of X P implies that X[B i ] has valency 1. This forces that B i = p is even, a contradiction. Thus, X[B i ] = pk 1 for all i Z 8. Since X is cubic, assume that X[B 0 B 7 ] = pk 2 and X[B 0 B 1 ] = C 2p. Then A/K is not edge-transitive on X P, and hence A/K = D 8. Since p > 2, the subgroup K of K fixing B 0 pointwise also fixes B 1 and B 7 pointwise. The connectivity of X gives K = 1, and consequently, K Aut(B 0 B 1 ) = D 4p. Since K fixes B 0, one has K = Z p or D 2p. It follows that A = 2 3+n p K A/K 16p, contrary to the fact that n > 1. Now we know that the claim is true, that is, P A. By Burnside p a q b -theorem, A is solvable because A = 2 3+n p. Let N be the maximal normal 2-subgroup of A. Since A is solvable and P A, one has N > 1. Then P N/N A/N, namely, P N A. If P P N, then P is characteristic in P N, and hence it is normal in A, a contradiction. Thus, P is not normal in P N. Consider the quotient graph X N of X relative to the orbit set of N, and let K be the kernel of A acting on V (X N ). Then X N has valency 2 or 3. Let B be an orbit of N. Since p > 2, one has B = 8, 4 or 2, and hence p X N. Since A/K is transitive on V (X N ), K is a 2-group. The maximality of N gives K = N. Let B = 8. Then X N = p. Since p > 2, X N has valency 2 and hence A/N Aut(X N ) = D 2p. If X[B] = 8K 1, then the subgraph induced by any two adjacent orbits is of valency 1 or 2. This forces that X N = p is even, a contradiction. Thus, X[B] has valency 1. In this case, it is easy to see that N is semiregular, and hence N = 8. It follows that A = 2 3+n p 16p, contrary to the fact that n > 1. Let B = 2. Then X N = 4p. If X N has valency 3, then we also have N v = 1 for each v V (X). It follows that N = Z 2 and hence P P N, a contradiction. If X N has valency 2, then X N = C4p and A/N Aut(X N ) = D 8p. Since A/N is transitive on V (X N ), A/N = D 4p, Z 4p or D 8p. This implies that A/N always has a normal subgroup, M/N, of order 2. This is contrary to the maximality of N. Let B = 4. Assume that either X N has valency 3, or X N has valency 2 and X[B] = 2K 2. Then it is easily seen that N is semiregular, and hence N = 4. If p > 3, Sylow Theorem implies that P P N, a contradiction. If p = 3 then X N = C6, L 3 (the Ladder graph of order 6) or K 3,3. When X N = C6 or L 3, A/N = 3 2 n+1 Aut(X N ) = 12, forcing n 1, a contradiction. When X N = K3,3, we have A/N = It is easy to see that A/N always has a normal 2-subgroup, contrary to the fact that N is a maximal normal 2-subgroup of A. Thus, X N has valency 2 and X[B] = 4K 1. Let V (X N ) = {B i i Z 2p } with B i B i+1. Since X

10 10 is cubic, one may assume that X[B 0 B 1 ] = C 8 or 2C 4 and X[B 0 B 2p 1 ] = 4K 2. Suppose X[B 0 B 1 ] = C 8. The subgroup N of N fixing B 0 pointwise also fixes B 1 and B 2p 1 pointwise. The connectivity of X and the transitivity of A/N on V (X N ) imply N = 1, and consequently, N Aut(X[B 0 B 1 ]) = D 16. Since N fixes B 0 setwise, one has N 8. Clearly, A/N is not edgetransitive on X N. So, A/N = D 2p. It follows that A = 2 3+n p = N A/N 16p, forcing n 1, a contradiction. Thus, X[B 0 B 1 ] is a union of two 4-cycles, say (x 0 0, x1 0, x0 1, x1 1 ) and (x0 2, x1 2, x0 3, x1 3 ), where B i = {x i 0, xi 1, xi 2, xi 3 } with i = 0 or 1. Remember that X N = (B 0, B 1,..., B 2p 1 ) is a 2pcycle. Hence, A has an element, say σ, of order p such that Bi σ = B i+2 for each i Z 2p. With no loss of generality, assume σ = 3 i=0 (x0 i x 2 i... x 2p 2 i )(x 1 i x 3 i... x 2p 1 i ). Then for each i Z 2p, B i = {x i 0, xi 1, xi 2, xi 3 }, and (x2j 0, x2j+1 0, 1, x2j+1 1 ) and ( 2, x2j+1 2, 3, x2j+1 3 ) are the two 4-cycles of X[B 2j B 2j+1 ] for each j Z p. Since X is connected, one may assume that X[B 2j B 2j+1 B 2j+2 B 2j+3 ] is one of the following cases: Case I x 2j Case II For each j Z p, Cj 0 = ( 0, x2j+1 0, 1, x2j+1 1 ) and Cj 1 = ( 2, x2j+1 2, 3, x2j+1 3 ) are two 4- cycles. Set Ϝ = {Cj i i Z 2, j Z p }. Since p > 2, it is easy to see that passing through each vertex in V (X) there is one and only one 4-cycle in X, and this 4-cycle belongs to Ϝ. Clearly, any two distinct 4-cycles in Ϝ are vertex-disjoint. This implies that Ω = {V (Cj i) i Z 2, j Z p } is an A-invariant partition of V (X). Consider the quotient graph X Ω. For Case I, X Ω = C2p, and it is easy to see that X = G 8p. By Theorem??, we obtain the Cases (3) and (5)-(7). For Case II, X Ω = Cp [2K 1 ]. Let K be the kernel of A acting on Ω. Then A/K Aut(C p [2K 1 ]) = Z p 2 D 2p. Noting that between any two adjacent vertices of X Ω there is only one edge of X, K fixes each vertex in V (X) and hence K = 1. If p = 3 then A = 2 3+n p = 2 4 3, contrary to the fact that n > 1. Thus, p > 3. Recall that X N = C2p and A/N is not edgetransitive on X N. So, A/N = D 2p. Since R(G) is regular on V (X), one has R(G)/(R(G) N) = R(G)N/N = A/N = D 2p. Let M = R(G) N. Note that P R(G) is a Sylow p-subgroup A. Since p > 3, Sylow Theorem implies that P MP. It follows that MP = M P, and hence M C A (P ). However, C A (P ) = 2p by Lemma??, a contradiction. References [1] Y.G. Baik, Y.-Q. Feng, H.S. Sim, M.Y. Xu, On the normality of Cayley graph of abelian groups, Algebra Colloq. 5 (1998) [2] N. Biggs, Algebraic Graph Theory, Cambridge University Press, Cambridge, [3] W. Bosma, C. Cannon, C. Playoust, The MAGMA algebra system I: The user language, J. Symbolic Comput. 24 (1997)

11 11 [4] I. Z. Bouwer (ed.), The Foster Census, Charles Babbage Research Centre, Winnipeg, [5] P. Camerona, J. Sheehanb, P. Spiga, Semiregular automorphisms of vertex-transitive cubic graphs, European J. Combin. 27 (2006) [6] J. L. Chen, Y. Q. Feng and D. J. Wang, A Family of non-normal Cayley digraphs, Acta Mathematica Sinica, English series, 17 (2001), [7] M.D.E. Conder, P. Dobcsányi, Trivalent symmetric graphs on up to 768 vertices, J. Combin. Math. Combin. Comput. 40 (2002) [8] J.D. Dixon, B. Mortimer, Permutation Groups, Springer-Verlag, New York, [9] E. Dobson, I. Kovács, Automorphism groups of Cayley digraphs of Z 3 p, Electronic J. Combin. 16 (2009) #R149. [10] E. Dobson, D. Witte, Transitive permutation groups of prime-squared degree, J. Algebraic Combin. 16 (2002) [11] S.F. Du, Y.-Q. Feng, J.H. Kwak, M.Y. Xu, Cubic Cayley graphs on dihedral groups, in Mathematical Analysis and Applications, edited by S. Nanda, G.P. Raja Sekhar, Narosa Publishing House, New Delhi, 2004; pp [12] S.F. Du, R.J. Wang, M.Y. Xu, On the normality of Cayley digraphs of order twice a prime, Austral. J. Combin. 18 (1998) [13] X.G. Fang, C.H. Li, D.J. Wang, M.Y. Xu, On cubic Cayley graphs of finite simple groups, Discrete Math. 244 (2002) [14] X.G. Fang, C.H. Li, M.Y. Xu, On edge-transitive Cayley graphs of valency four, European J. Combin. 25 (2004) [15] Y. Q. Feng, T. P. Gao, Automorphism groups and isomorphisms of Cayley digraphs of abelian groups, Australasian Journal of Combinatorics, 16 (1997), [16] Y.-Q. Feng, J.H. Kwak, R.J. Wang Automorphism groups of 4-valent connected Cayley graphs of p-group, Discrete Math. 248 (2002) [17] Y.-Q. Feng, J.H. Kwak, Classifying cubic symmetric graphs of order 8p or 8p 2, European J. Combin. 26 (2005) [18] Y.-Q. Feng, Z.P. Lu, M.Y. Xu, Automorphism groups of Cayley digraphs, in Application of Group Theory to Combinatorics, edited by J. Koolen, J.H. Kwak, M.Y. Xu, Taylor & Francis Group, London, 2008; pp [19] Y.-Q. Feng, K. Wang, s-regular cyclic coverings of the three-dimensional hypercube Q 3, European J. Combin. 24 (2003) [20] Y.Q. Feng, R.J. Wang, M.Y. Xu, Automorphism groups of 2-valent connected Cayley digraphs on regular p-group. 18 (2002) [21] Y.-Q. Feng, M.Y. Xu, Automorphism groups of tetravalent Cayley graphs on regular p- groups, Discrete Math. 305 (2005) [22] A. Hassani, M.A. Iranmanesh, C.E. Praeger, On vertex-imprimitive graphs of order a product of three distinct odd primes, J. Combin. Math. Combin. Comput. 28 (1998)

12 12 [23] Q. X. Huang, J. X. Meng, On the isomorphisms and automorphism groups of circulants, Graphs Combin., 12 (1996), [24] B. Huppert, Eudliche Gruppen I, Springer-Verlag, Berlin, [25] I. Kovács, B. Kuzman, A. Malnič, On non-normal arc-transitive 4-valent dihedrants, Acta Math. Sinica (Engl. ser.), in press. [26] C. H. Li, Isomorphisms of connected Cayley digraphs, Graphs Combin., 14 (1998), [27] C.H. Li, Z.P. Lu, H. Zhang, Tetravalent edge-transitive Cayley graphs with odd number of vertices, J. Combin. Theory B 96 (2006) [28] Z.P. Lu, M.Y. Xu, On the normality of Cayley graphs of order pq, Austral. J. Combin. 27 (2003) [29] D. Marušič, T. Pisanski, Symmetries of hexagonal graphs on the torus, Croat. Chemica Acta 73 (2000) [30] B.D. McKay, G. Royal, Cubic transitive graphs, gordon/remote/cubtrans/index.html. [31] G.A. Miller, H.C. Moreno, Non-abelian groups in which every subgroup is abelian, Trans. Amer. Math. Soc. 4 (1903), [32] C.E. Praeger, Finite normal edge-transitive graphs, Bull. Austral. Math. Soc. 60 (1999) [33] G. Sabidussi, On a class of fix-point-free graphs, Proc. Amer. Math. Soc. 9 (1958) [34] C.Q. Wang, D.J. Wang, M.Y. Xu, On normal Cayley graphs of finite groups, Science in China A 28 (1998) [35] C.Q. Wang, M.Y. Xu, Non-normal one-regular and 4-valent Cayley graphs of dihedral groups D 2n, European J. Combin. 27 (2006) [36] H. Wielandt, Finite Permutation Groups, Academic Press, New York, [37] M. Y. Xu, Automorphism groups and isomorphisms of Cayley digraphs, Discrete Math., 182 (1998), [38] X.H. Yu, J.X. Zhou, L.L. Wang, Cubic Cayley graphs of order 2p 2, Advances In Mathematics. 35 (2006) [39] M.Y. Xu, Automorphism groups and isomorphisms of Cayley digraphs, Discrete Math. 182 (1998) [40] S.J. Xu, X.G. Fang, J. Wang, M.Y. Xu, On cubic s-arc transitive Cayley graphs of finite simple groups, European J. Combin. 26 (2005) [41] S.J. Xu, X.G. Fang, J. Wang, M.Y. Xu, 5-Arc transitive cubic Cayley graphs on finite simple groups, European J. Combin. 28 (2007) [42] H.X. Yu, J.-X. Zhou, L. Wang, Cubic Cayley graphs of order 2p 2, Adv. in Math. (China) 35 (2006)

13 13 [43] C. Zhang, J.-X. Zhou, Y.-Q. Feng, Automorphisms of cubic Cayley graphs of order 2pq, Discrete Math. 309 (2009) [44] C.-X. Zhou, Y.-Q. Feng, Automorphism groups of connected cubic Cayley graphs of order 4p, Algebra Colloq. 14 (2007) [45] J.-X. Zhou, Cubic vertex-transitive graphs of order 4p (Chinese), J. Sys. Sci. & Math. Sci. 28 (2008) [46] J.-X. Zhou, Tetravalent non-normal Cayley graphs of order 4p, Electronic J. Combin. 16 (2009) #R118.