On the restriction of Deligne-Lusztig characters

Transcription

1 On the restriction of Deligne-Lusztig characters Mark Reeder Department of Mathematics, Boston College Chestnut Hill, MA May 5, Introduction This paper was motivated by the following restriction problem for representations of finite orthogonal groups. Let F be an algebraic closure of a finite field f of cardinality q, a power of a prime p > 2. Let G = SO(V ) be the special orthogonal group of a 2n + 1- dimensional F-space V with nondegenerate quadratic form Q. Assume V and Q are defined over f, and let F denote the corresponding Frobenius endomorphisms of V and G. Fix v V F with Q(v) 0 and let H be the stabilizer of v in G. Let π Irr(G F ), σ Irr(H F ) be complex irreducible cuspidal representations of the respective groups G F and H F of f-rational points. The problem is to compute the multiplicity π, σ H F = dim Hom H F (π, σ) of σ in the restriction of π to H F. Using unpublished work of Bernstein and Rallis (independently) on p-adic orthogonal groups, it can be shown that Supported by NSF grant DMS π, σ H F = 0 or 1. 1

2 In this paper, we compute π, σ H F exactly, when π and σ are irreducible cuspidal Deligne-Lusztig representations [8]. We do not rely on the above-mentioned work of Bernstein and Rallis. Our calculation follows from a qualitative study of restrictions of Deligne-Lusztig characters for general simple algebraic groups, to be described later in this introduction. To state our multiplicity result for orthogonal groups, we first recall the inducing data. Let T G, S H be F -stable anisotropic tori in G and H. There are unique partitions λ = (j λj ), µ = (j µ j ) of n (here λ j, µ j are the number of parts equal to j) such that T F j (f 1 2j) λ j, S F j (f 1 2j) µ j, where, for any d > 1, f d = F F d is the extension of f in F of degree d, and f 1 2j is the kernel of the norm mapping f 2j f j. The number of parts j µ j is even if H is split, and odd if H is nonsplit. Let χ Irr(T F ) and η Irr(S F ) be irreducible characters of T F and S F which are regular in the sense that χ and η have trivial stabilizers in the respective Weyl groups W G (T ) F and W H (S) F. We may write where χ = j χ j, η = j η j, χ j = χ j1 χ jλj Irr ( (f 1 2j) λ j ), each χ jk is a character of f 1 2j, and likewise for η. Let Γ 2j Z/2jZ be the Galois group of f 2j /f. Definition 1.1 We say that χ and η intertwine if η jk is a Γ 2j -conjugate of χ jk for some 1 j n, 1 k λ j, 1 k µ j. Note that χ and η can intertwine even if T S. However, if λ and η have no common parts, that is, if λ j µ j = 0 for all j, then χ and η do not intertwine. By Deligne-Lusztig induction, we have virtual representations RT,χ G of GF and RS,η H of HF, respectively. By the regularity assumptions on χ and η, these are actually irreducible characters, up to sign. In fact, we have ( 1) rk G R G T,χ Irr(G F ), ( 1) rk H R H S,η Irr(H F ). These two irreducible characters are cuspidal, since T and S are anisotropic. We prove: 2

3 Theorem 1.2 Let T and S be anisotropic F -stable maximal tori in G and H, respectively, and let χ Irr(T F ), η Irr(S F ) be regular characters. Then { ( 1) rk G+rk H RT,χ, G RS,η H 0 if η, χ intertwine H F = 1 if η, χ do not intertwine. If T and S are arbitrary F -stable maximal tori, but χ and η are still regular, then the multiplicity is either zero or a power of two; see (60) below. The multiplicity result 1.2 is used in [12] to verify some cases of the conjectures of [11] describing restrictions from p-adic SO 2n+1 to SO 2n, in terms of symplectic local root numbers and the parametrization of depth-zero supercuspidal L-packets given in [7]. As already mentioned, Theorem 1.2 follows from a qualitative result, in a general setting, on multiplicities of Deligne-Lusztig representations. Let G be a a connected simple algebraic group defined over f, and let H be a connected reductive f-subgroup of G. Fix F -stable maximal tori T G and S H, along with arbitrary characters χ Irr(T F ) and η Irr(S F ). From this data Deligne and Lusztig [8] construct virtual characters RT,χ G and RS,η H on GF and H F, respectively. Let, H F be the canonical pairing on virtual characters of H F. We are interested in the multiplicity R G T,χ, R H S,η H F, where R G T,χ is viewed as a virtual character of HF, by restriction. Let B and B H be Borel subgroups of G and H, respectively, and let δ be the minimum codimension of a B H -orbit in G/B. The invariant δ is called the complexity of the H-variety G/B. The theory of complexity was first studied for reductive groups over fields of characteristic zero (cf. [1] and references therein). In that setting, it is proved in [1] that δ governs the growth of multiplicities in restrictions of algebraic representations. We will show that δ also governs the growth of multiplicities in restrictions of Deligne-Lusztig representations. Because we are in nonzero characteristic, we need to make an assumption. Let g, h be the Lie algebras of G and H. Assumption 1.3 There is an Ad(H)-stable decomposition g = h m, and a non-degenerate symmetric bilinear form B on m, invariant under Ad(H). This assumption holds if p is a good prime for g and the Killing form of g is nondegenerate on h [24, I.5.3]. For G = SO N+1, H = SO N, our assumption holds for p > 2. 3

4 For an integer ν 1, let N T ν : T F ν T F be the norm map, and let χ (ν) = χ N T ν, η (ν) = η N S ν. Under Assumption 1.3, we prove the following. Theorem 1.4 There is a polynomial of degree at most δ: M(t) = At δ + Q[t], whose coefficients depend on χ and η, and an integer m 1 such that R G T,χ (ν), R H S,η (ν) H F ν = M(q ν ) for all positive integers ν 1 mod m. The degree δ is optimal: if q is sufficiently large, there exist χ, η such that the leading coefficient A is nonzero. We also give an explicit formula for the leading term A in Theorem 1.4 (see Proposition 7.4). For G = SO N+1, H = SO N, we have δ = 0, and our explicit formula for A leads to Theorem 1.2 (see Section 9). Even if δ > 0 one can sometimes use Theorem 1.4 to compute exact multiplicities, by exploiting the polynomial nature of M(t). In Section 10 we illustrate this for G = SO 7, H = G 2, where δ = 1. Our formula for A also allows us to show, for general G and H, and very regular χ (see section 8), that the multiplicity R G T,χ, St H H F of the Steinberg representation St H is a monic polynomial in q of degree δ, while the multiplicity of the trivial representation R G T,χ, 1 H H F is a polynomial in q of degree strictly less than δ. In particular, for G = SO 2n+1 and H = SO 2n, we have for very regular χ. R G T,χ, St H H F = 1, R G T,χ, 1 H H F = 0, To prove Theorem 1.4 we use a method introduced by Thoma [27] for the study of the restriction of irreducible representations from GL n (f) to GL n 1 (f) 4

5 (where again δ = 0). In that situation, the Green s functions giving the character on unipotent elements were explicitly known. Hagedorn [13], in his 1994 PhD thesis, showed how some of Thoma s methods could be generalized to Deligne- Lusztig characters for other pairs of classical groups, where the Green s functions are less explicit. The abstract results of Hagedorn gave me the courage to attempt such calculations for general groups, and to obtain closed multiplicity formulas for orthogonal groups. It is a pleasure to thank Dick Gross for initiating the work in [12] which led to this paper, for helpful remarks on an earlier version, and for aquainting me with Hagedorn s thesis. The referee read the original version of this paper with care and insight, made valuable comments and simplified some of the arguments. In particular, the proof of Lemma 3.1 given below is due to the referee, and is much shorter than the original one. Some general notation: The cardinality of a finite set X is denoted by X. Equivalence classes are generally denoted by [ ], sometimes with ornamentation. If g is an element of a group G, we write Ad(g) for the conjugation map Ad(g) : x gxg 1, and also write g T := gt g 1 for a subgroup T G. The center of G is denoted Z(G) and the centralizer of g G is denoted C G (g). We write, H for the pairing on the space of class functions on a finite group H, for which the irreducible characters of H are an orthonormal basis. If G, G H are finite overgroups of H and ψ, ψ are class functions on G, G respectively, then ψ, ψ H is understood to mean ψ H, ψ H H, where H denotes restriction to H. 5

6 Contents 1 Introduction 1 2 Remarks on Maximal Tori 7 3 On the centralizer of a semisimple element Deligne-Lusztig characters 12 5 Multiplicity as a polynomial Summation on H F A partition of S Restriction of Deligne-Lusztig characters Green functions A progression of powers of Frobenius Character sums Multiplicity as a polynomial Complexity and the degree of M(t) A formula for the complexity Degree of Ψ α (t) A remark on the multiplicity formula The leading term of M(t) 27 8 Optimality 29 9 Restriction from SO 2n+1 to SO 2n Restriction from SO 7 to G Cuspidal Multiplicities

7 2 Remarks on Maximal Tori Let G be a connected reductive algebraic F-group. We assume G is defined over f and has Frobenius F. If T is a maximal torus in G we denote its normalizer in G by N G (T ) and write W G (T ) = N G (T )/T for the Weyl group of T in G. If T is F -stable, we have W (T ) F = N G (T ) F /T F, by the Lang-Steinberg theorem. The reduction formula for Deligne-Lusztig characters (recalled in section 4 below) involves a sum over the following kind of subset of G F. Fix an F -stable maximal torus T G, and let s be a semisimple element in G F. We must sum over the set N G (s, T ) F := {γ G F : s γ T }. Note that N G (s, T ) F, if non-empty, is a union of G F s N G (T ) F -double cosets, where G s := C G (s) is the identity component of the centralizer C G (s) of s in G. To say that s γ T is to say that γ T G s, so determining the G F s N G (T ) F - double cosets in N G (s, T ) F amounts to determining the G F s -conjugacy classes of F -stable maximal tori in G s which are contained in a given G F -conjugacy class. Such classes of tori are parameterized by twisted conjugacy classes in Weyl groups of G s and G. The aim of this section is to parameterize the G F s N G (T ) F -double cosets in N G (s, T ) F in terms of the fiber of a natural map between twisted conjugacy classes in the Weyl groups of G s and G. This parameterization will be fundamental to our later calculations with Deligne-Lusztig characters. We begin by recalling the classification of F -stable maximal tori in G. See [5, chap. 3] for more details in what follows. Fix an F -stable maximal torus T 0 in G contained in an F -stable Borel subgroup of G, and abbreviate N G = N G (T 0 ), W G = W G (T 0 ). Let T (G) denote the set of all F -stable maximal tori in G. Then T (G) is a finite union of G F -orbits. For any T T (G), let [T ] G := { γ T : γ G F } denote the G F -orbit of T. There is g G such that T = g T 0. Since T is F -stable, we have g 1 F (g) N G. This gives an element w := g 1 F (g)t 0 W G. 7

8 The map Ad(g)t = gtg 1 is a f-isomorphism Ad(g) : (T 0, wf ) (T, F ), where the second component denotes the action of Frobenius under an f-structure. For any finite group A with F -action, we let H 1 (F, A) denote the set of F - conjugacy classes in A. These are the orbits of the action of A on itself via (a, b) abf (a) 1. Let [b] H 1 (F, A) denote the F -conjugacy class of an element b A. For g, T, w as above, the F -conjugacy class of w is independent of the choice of g. Hence we have a well-defined class For each ω H 1 (F, W G ), the set cl(t, G) := [w] H 1 (F, W G ). T ω (G) := {T T (G) : cl(t, G) = ω} is a single G F -orbit in T (G), and all G F -orbits are of this form. Thus, the partition of the set of F -stable maximal tori into G F -orbits is given by T (G) = T ω (G). ω H 1 (F,W G ) Let s G F be semisimple, and let T s be an F -stable maximal torus of G s contained in an F -stable Borel subgroup of G s, and let W Gs be the Weyl group of T s in G s. The partition of T (G s ) into G F s -orbits is given, as above, by T (G s ) = T υ (G s ). υ H 1 (F,W Gs ) If T T (G), the set of F -stable maximal tori in G s which are G F -conjugate to T is a finite union (possibly empty) of G F s -orbits. We want to describe this union in terms of F -conjugacy classes in W Gs. That is, given ω H 1 (F, W G ), we have T ω (G) T (G s ) = υ M ω T υ (G s ) (1) for some subset M ω H 1 (F, W Gs ), and our task is to find M ω. 8

9 The first point is that T s is generally not contained in an F -stable Borel subgroup of G. Let g G be such that g T s = T 0, and let ẏ s := gf (g) 1 have image y s W G. Then cl(t s, G) = [y s ] H 1 (F, W G ), and Ad(g) is an f-isomorphism Ad(g) : (T s, F ) (T 0, y s F ). Now T 0 is also a maximal torus in Ad(g)G s, whose Weyl group W G s := Ad(g)W Gs is a subgroup of W G, stable under Ad(y s ) F. Define j Gs : H 1 (F, W Gs ) H 1 (F, W G ) to be the composition of maps j Gs : H 1 (F, W Gs ) Ad(g) H 1 (y s F, W G s ) incl H 1 (y s F, W G ) τys H 1 (F, W G ), (2) where the middle map is induced by the inclusion W G s W G and τ ys is the twisting bijection given by τ ys [x] = [xy s ]. Now let T be an arbitrary F -stable maximal torus in G s. Write T = h T s, with h G s, so that h 1 F (h) cl(t, G s ). For g G as above, we have T = hg 1 T 0. Since gh 1 F (hg 1 ) = g(h 1 F (h))g 1 gf (g) 1, it follows that This proves: cl(t, G) = j Gs (cl(t, G s )). (3) Lemma 2.1 For each ω H 1 (F, W G ) and T T ω (G s ), we have T ω (G) T (G s ) = T υ (G s ). υ j 1 Gs (ω) We can also parameterize the G F s -orbits in [T ] G T (G s ) via the mapping N G (s, T ) F := {γ G F : s γ T } [T ] G T (G s ), γ γ T. (4) 9

10 Note that G F s acts on N G (s, T ) F by left multiplication, and that (4) factors through the quotient N G (s, T ) F := G F s \N G (s, T ) F. (5) The action of N G (T ) F on N G (s, T ) F by right multiplication commutes with the G F s -action, hence factors through an action on N G (s, T ) F, where T F acts trivially. This gives an action of W G (T ) F on N G (s, T ) F, whose orbits are the G F s N(T ) F -double cosets in N(s, T ) F. Lemma 2.2 The mapping (4), sending γ γ T, induces a bijection N G (s, T ) F /W G (T ) F G F s \ ( [T ] G T (G s ) ) with the property that the stabilizer in W G (T ) F of the class γ N(s, T ) F is isomorphic, via Ad(γ), to W Gs ( γ T ) F. Proof: The bijectivity is straightforward and left to the reader. Let w W G (T ) F, and let ẇ N G (T ) F be a representative of w. Then γ w = γ G F s γẇ = G F s γ Ad(γ)ẇ N Gs ( γ T ). This implies the assertion about the stabilizer. Combining Lemmas 2.1 and 2.2, we get an explicit formula for N G (s, T ) F. Corollary 2.3 Let ω H 1 (F, W G ) and T T ω (G). Then the set N G (s, T ) F is non-empty if and only if the fiber j 1 G s (ω) is non-empty, in which case, we have N G (s, T ) F = υ j 1 Gs (ω) W G (T ) F W Gs (T υ ) F, where, for each υ j 1 G s (ω), the torus T υ is chosen arbitrarily in T υ (G s ). 3 On the centralizer of a semisimple element. Let s G F be semisimple. In the previous section we parameterized the set of G F s -conjugacy-classes maximal tori in G s which are contained in a given G F - conjugacy class, in terms of fibers of the map j Gs : H 1 (F, W Gs ) H 1 (F, W G ). To compute this map j Gs concretely, we must find an element y s W G such that cl(t s, G) = [y s ], where T s T (G s ) is contained in an F -stable Borel subgroup of 10

11 G s. This amounts to finding the f-isomorphism class of the connected centralizer G s. An elegant formula for y s was given by Carter [6], using the Brauer complex. Here we explain a different method that is suited to our later computations; namely we show how the class [y s ] can be determined from the effect of F on a diagonalized G-conjugate of s. Unfortunately, both the present method, as well as that of [6] require that C G (s) be connected. That is, we must assume that G s = C G (s). This holds for any semisimple s G if G has simply-connected derived group. Our method generalizes that of Gross [10], who determined C G (s) when this group is a torus (over an arbitrary field). Let Φ denote the set of roots of T 0 in G. Let ϑ denote the automorphisms of Φ and W G induced by F. For α Φ with corresponding reflection s α W G, we have α F = qϑ 1 α, ϑ(s α ) = s ϑ α. let Here is our recipe for finding cl(t s, G). Let t T 0 be a G-conjugate of s, and Φ t = {α Φ : α(t) = 1}. Since t has a conjugate in G F, there is w W (not necessarily unique) such that Choose such a w arbitrarily. From (6) it follows that F (t) = t w. (6) wϑ Φ t = Φ t. (7) Now choose any positive system Φ + t Φ t. Then (7) implies that wϑ Φ + t is another positive system in Φ t. Being the Weyl group of Φ t, the group W Gt acts simply transitively on positive systems in Φ t, so there is a unique x W Gt such that wϑ Φ + t = x Φ + t. (8) Setting y = x 1 w, we see that w can be factored uniquely as w = xy, (9) where x W Gt and yϑ Φ + t = Φ + t. Since C G (t) is connected, the group W Gt is the full stabilizer of t in W G. This means that a different choice of w satisfying (6) will change x, but not y. 11

12 Lemma 3.1 With y constructed as above, we have cl(t s, G) = [y] H 1 (F, W G ). Proof: The following proof was provided by the referee; it is shorter than the original proof. Choose g G such that ẏ = g 1 F (g) N G is a representative of y. Then yf (t) = t x = t, which implies that g t G F. Since C G (s) is connected, any element of G F which is G-conjugate of s is in fact G F -conjugate to s. Hence, by multiplying g on the left by an element of G F, we may assume that s = g t. By definition of y, there is an Ad(ẏ)F -stable Borel subgroup B t G t containing T 0. Hence g B t is an F -stable Borel subgroup of G s, containing the F -stable maximal torus T s := g T 0. Since T s is G F s -conjugate to T s, it follows that as claimed. cl(t s, G) = [g 1 F (g)] = [y], 4 Deligne-Lusztig characters Let T T (G) be an F -stable maximal torus in G, and let χ Irr(T F ). The Deligne-Lusztig character RT,χ G has the following reduction formula [8]: For u unipotent in G F s, we have R G T,χ(su) = γ N G (s,t ) F χ(γ 1 sγ)q Gs γt γ 1 (u). (10) The summation is over the set NG (s, T ) F defined in (5), and for any reductive f-group H, and S T (H), the Green function Q H S on the unipotent set of HF is defined by Q H S (u) = R H S,1(u). In this section we describe the summation over N G (s, T ) F in (10) in terms of fibers of the map j Gs studied in the previous two sections. Breaking the sum (10) into W G (T ) F -orbits, we have RT,χ(su) G = Q Gs T υ (u) γ O χ(γ 1 sγ), (11) υ j 1 Gs (ω) υ 12

13 where ω = cl(t, G), T υ is any torus in T υ (G s ), and O υ is the W G (T ) F -orbit in N G (s, T ) F corresponding to υ j 1 G s (ω) as in Lemma 2.2. By the stabilizer assertion in Lemma 2.2, the inner sum in (11) can be written as follows. For any γ N G (s, T ) F and χ Irr(T F ), the value at s of the transported character γ χ := χ Ad(γ 1 ) Irr( γ T F ) depends only on the image γ N G (s, T ) F. We have γ O υ χ(γ 1 sγ) = 1 W Gs (T υ ) F x W G (T ) F γx χ(s), (12) where γ on the right side of (12) is an arbitrary element of N G (s, T ) F such that γ O υ. In our later computations with R G T,χ it will be useful to let s vary in GF in such a way that G s is unchanged. Let Z(G s ) denote the center of G s. For υ H 1 (F, W Gs ), the function χ υ := γ O υ γ χ (13) is well-defined on Z(G s ) F, and we have RT,χ(zu) G = Q Gs T υ (u)χ υ (z), if G z = G s. (14) υ j 1 Gs (ω) 5 Multiplicity as a polynomial In this section we begin the proof of Theorem 1.4, and will show that the multiplicity is given by a polynomial function. Let G be a connected reductive algebraic group over f. Let H G be a connected reductive f-subgroup of G, and let S be an F -stable maximal torus of H. 5.1 Summation on H F. Suppose we are given a function f : H F C, invariant under conjugation by H F, with the property that if h H F has Jordan decomposition h = su, then f(h) = 0 unless the conjugacy class Ad(H F ) s meets S. Our first aim is to 13

14 express the sum of f over H F as a sum of rational functions in q over an index set which does not depend on q. Let H ss and H upt be the sets of semisimple and unipotent elements of H. Let S(H F ) and U(H F ) be the sets of Ad(H F )-orbits in (H ss ) F and (H upt ) F, respectively. By the vanishing assumption on f, we have 1 H F f(h) = 1 H F h H F = 1 H F s (H ss ) F The map γ s γ induces a bijection f(su) u (Hs upt ) F Ad(HF ) s Ad(H F ) s S s S F C H (s) F \N H (s, S) F [u] U(H F s ) Ad(H F ) s S, Ad(H F s ) u f(su). (15) so that Recalling that Ad(H F ) s S = N H(s, S) F. C H (s) F N H (s, S) F = H F s \N H (s, S) F, we get 1 H F h H F f(h) = 1 1 N f(su). (16) H (s, S) F C s S F [u] U(Hs F ) Hs (u) F 5.2 A partition of S To this point, the overgroup G has not played a role. Now G is used to partition the sum over S F in (16), as follows. Let I(S) be an index set for the set of subgroups {G s : s S}. Note that each element of I(S) is determined by a subset of the roots of S in G, hence I(S) is finite. For ι I(S) let G ι be the corresponding connected centralizer, and let S ι := {s S : G s = G ι }. 14

15 Thus, S is finitely partitioned as S = ι I(S) S ι. The F -action on S induces a permutation of I(S), and we let I(S) F be the F -fixed points in I(S). Note that if S F ι is nonempty, then ι I(S) F. For ι I(S), we set H ι := (H G ι ), which is none other than H s for any s S ι. Note that if s S ι, then s S C H (s), which implies that s H ι G ι. (17) Returning to our sum (16), we now have 1 H F h H F f(h) = ι I(S) F [u] U(H F ι ) s S F ι 1 N H (s, S) F f(su) C Hι (u) F. (18) 5.3 Restriction of Deligne-Lusztig characters We now consider the function f arising in our multiplicity formula. Let H, S be as above, let T be an F -stable maximal torus of G, and let χ Irr(T F ), η Irr(S F ) be arbitrary characters. Using the function f : H F C given by we have The map f(h) = R G T,χ (h) RH S,η(h), (19) R G T,χ, R H S,η H F = 1 H F j Gs : H 1 (F, W Gs ) H 1 (F, W G ) defined in (2) depends only on G s, so we set We have an analogous map j Gι := j Gs, for any s S ι. j Hι : H 1 (F, W Hι ) H 1 (F, W H ). 15 h H F f(h). (20)

16 Likewise, the sets N G (s, T ) F and N H (s, S) F depend only on ι, so we now write N G (ι, T ) F := N G (s, T ) F, NH (ι, S) F := N H (s, S) F, for s S F ι. Using (14) for G and H, along with (18), our multiplicity formula becomes R G T,χ, R H S,η H F = ι I(S) F [u] U(H F ι ) υ, ς Q Gι T υ (u)q Hι S ς (u) N H (ι, S) F C Hι (u) F χ υ (s)η ς (s), (21) where the middle sum runs over υ j 1 G ι (cl(t, G)) and ς j 1 H ι (cl(s, H)). The character sums χ υ and η ς are as defined in (13). 5.4 Green functions We digress from our multiplicity formula (21), to recall the polynomial nature of Green functions Q G T, defined on the unipotent set of GF, for a connected reductive f-group G with Frobenius F and F -stable maximal torus T in G. For u = 1, we have s S F ι Q G T (1) = ɛ G (w)[g F : T F ] p, (22) where [G F : T F ] p is the maximal divisor of the index [G F : T F ] which is prime to p, w cl(t, G) and ɛ G : W G {±1} is the sign character of W G. Note that ɛ G (w) = ( 1) rk G+rk T [5, 7.5.2]. For u 1, the Green functions Q G T (u) can be expressed as polynomials which are known explicitly by tables for exceptional groups [3], [18] and for classical groups by recursive formulas [19] which can be implemented on a computer [9]. It will suffice for us to know the leading terms of these Green polynomials, which can be expressed in a uniform way. Let B G be the variety of Borel subgroups of G, and let BG u be the variety of u-fixed points in B G. The irreducible components of BG u all have the same dimension, and we set d G (u) := dim BG. u Steinberg proved that 2d G (u) = dim C G (u) rk G, (23) 16

17 where rk G is the absolute rank of G. Assume that p is a good prime for G. For each unipotent class [u] U(G F ) and twisted conjugacy class [w] = ς H 1 (F, W G ), there is a polynomial of degree at most d G (u), such that Q w,u (t) = Q ς,u (t) Z[t], Q G T (u) = Q w,u (q) if cl(t, G) = [w] (see [20] and references therein). The coefficient of t d G(u) in Q w,u (t) is tr[w, H 2d G(u) (B u G)], where w acts on the l-adic cohomology of BG u via the Springer construction (see [21], [14], [16]). If we take u = 1 then d G (1) = N is the number of positive roots of G and Q G w,1(t) = ɛ G (w)t N + lower powers of t, (24) which is easily seen to be consistent with (22). Suppose now that we replace F by F ν for some ν 1. The G F ν -class of T is then represented by (wϑ) ν ϑ ν W G, where ϑ is the automorphism of W G induced by F. Suppose ν 1 mod m, where m is a positive integer divisible by the exponent of the finite group W G ϑ. This implies that F ν = F on W G and that (wϑ) ν ϑ ν = w for all w W G. It follows that H 1 (F, W G ) = H 1 (F ν, W G ) and that the class cl(t, G) is the same with respect to F or F ν. Likewise, the class of u in G F or G F ν is determined by the G-conjugacy class C G containing u, together with a class in H 1 (F, A G (C)) or H 1 (F ν, A G (C)), where A G (C) is the component group the centralizer of some F -fixed element in C. As in the preceding paragraph, we may take m sufficiently divisible so that F ν = F on A G (C) and that the class of u in G F or G F ν corresponds to the same class in H 1 (F, A G (C)). We may choose m so that this holds for every C, since there are finitely many unipotent classes. 17

18 Let Q G T,ν be the Green function for T on GF ν. For m sufficiently divisible as in the previous two paragraphs and ν 1 mod m we have Q G T,ν(u) = Q w,u (q ν ). (Note the difficulty with the exceptional class in E 8 is avoided since our conditions on m imply that ν is odd, see [20, Remark 6.2].) 5.5 A progression of powers of Frobenius The indices of and terms of the summations in (21) depend on F, and we wish to remove this dependence for infinitely many powers of F, in order to represent the sum in (21) as the value of a rational function. There is a positive integer m such that F m acts trivially on the finite set I(S) and the divisibility conditions on m from the previous section hold when G is replaced by G ι or H ι for every ι I(S). In particular, m is divisible by the orders of the component groups A ι (u) of the centralizers in H ι of all unipotent elements u Hι F for every ι I(S) F and that F m is the identity automorphism on A ι (u) for all such ι and u. This implies that for each ι I(S) F and [u] U(Hι F ), there is a polynomial P ι,u (t) Z[t], of degree equal to dim C Hι (u), such that C Hι (u) F ν = P ι,u (q ν ) (25) for all ν 1 mod m. Moreover, each polynomial P ι,u (t) is of the form A ι (u) times a monic polynomial in Z[t]. The above conditions on m also ensure that the indices in the outer two summations in (21), as well as the quantity N H (ι, S) F are unchanged if F is replaced by F ν for ν 1 mod m. To handle the inner sum, we add more conditions: in the next section we will define certain subgroups Z J of S, indexed by subsets J I(S) F. We also insist that m be divisible by Z J /ZJ and that F m acts trivially on Z J /ZJ for each J I(S) F. 5.6 Character sums In order to interpret the inner sum of (21) as a rational function, we shall replace each summand Sι F by the group Zι F, where Z ι := Z(G ι ) S. (26) 18

19 It is easy to check that Z ι Z(H ι ), S ι Z ι S, G ι = C G (Z ι ). (27) Let χ υ and η ς be the character sums appearing in (21). Our aim is to express the sum 1 χ Zι F υ (s)η ς (s) (28) s S F ι as the value of a rational function. Define a partial ordering on I(S) by Equivalently, we have ι ι G ι G ι. ι ι Z ι Z ι. Let be the complement of S ι in Z ι. Y ι := Z ι S ι Lemma 5.1 For every ι I(S) we have Y ι = ι <ι Z ι. Proof: Let s Y ι. Then s S ι for some ι I(S), with ι ι, so s Z ι. Since Y ι Z ι, we have G ι = C G(Z ι ) C G(Y ι ) G s = G ι, so ι < ι. Conversely, let s Z ι, with ι < ι. Note that s Z ι. If s / Y ι, then s S ι. This implies that G ι = C G (Z ι ) G s = G ι, contradicting ι < ι. This proves the lemma. 19

20 For a subset J I(S), let Z J = Z ι. ι J There is a polynomial f J C[t] of degree dim Z J, such that For ν 1, let f J (q ν ) = Z F ν J, for all ν 1 mod m. N Tυ ν : T F ν υ Tυ F, Nν Sς be the norm mappings. These are surjective. Set χ (ν) υ : S F ν ς S F ς := χ υ Nν Tυ, η ς (ν) := η ς Nν Sς. Assume that ι J I(S) F. Then Z J is F -stable and Z F J Z F ι Z(G ι ) F Z(H ι ) F. Both χ υ and η ς are defined on the latter group (see (13)), so we may restrict them to ZJ F. Our conditions on m at the end of section 5.5 ensure that the restricted norm mapping Nν Sς : Z F ν J ZJ F is also surjective. This implies, for all integers ν 1 mod m, that χ (ν) υ, η (ν) ς Z F ν J = χ υ, η ς Z F J. Hence for each J, we have χ (ν) υ (z)η ς (ν) (z) = χ υ, η ς Z F J f J (q ν ). (29) z Z F ν J Let I(ι, S) := {ι I(S) : ι < ι}. It now follows from Möbius inversion that the rational function Θ ι,υ,ς (t) := χ υ, η ς Z F ι + ( 1) J f J (t) χ υ, η ς Z F J (30) f ι (t) J I(ι,S) F has the property that Θ ι,υ,ς (q ν ) = 1 Z F ν ι s S F ν ι χ (ν) υ (s)η (ν) (s), for all ν 1 mod m. Since dim Z J dim Z ι for all J I(ι, S) F, we have ς deg Θ ι,υ,ς 0. (31) 20

21 5.7 Multiplicity as a polynomial We return to our multiplicity formula (21). We have shown that R G T,χ, R H S,η H F = α Ψ α (q)θ α (q), (32) where α runs over quadruples α = (ι, u, υ, ς), with ι I(S) F, [u] U(H F ι ), υ j 1 G ι (cl(t, G)), ς j 1 H ι (cl(s, H)), (33) Θ α (t) = Θ ι,υ,ς (t) is the rational function defined in (30) and Ψ α (t) is the rational function defined by Q Gι υ,u(t)q Hι ς,u(t) Ψ α (t) = f ι (t) N H (ι, S) F P ι,u (t). (34) Here Q Gι υ,u(t) and Q Hι ς,u(t) are the Green polynomials from section 5.4 and P ι,u (t) is the polynomial from (25). If F is replaced by F ν with ν 1 mod m, where m is as in section 5.5, the summation indices α are unchanged, so that the rational function M(t) := α Ψ α (t)θ α (t) (35) has the property that R G T,χ (ν), R H S,η (ν) H F ν = M(q ν ), (36) for all ν 1 mod m. In particular, M(q ν ) is an integer for all ν 1 mod m. We next observe that the numerator of each term in M(t) belongs to Z[t], and the denominator of each term in M(t) is an integer times a monic polynomial in Z[t]. Hence there is a Z such that am(t) = f(t) g(t), where f(t) and g(t) are in Z[t] and g(t) is monic. We can therefore write am(t) = p(t) + r(t), 21

22 where p(t) Z[t] and r(t) is a rational function of negative degree. On the other hand, r(q ν ) = am(q ν ) p(q ν ) = a R G T,χ (ν), R H S,η (ν) H F ν p(q ν ) Z for all ν 1 mod m. Since r(q ν ) 0 as ν, we must have r(t) 0, so M(t) = 1 a p(t). This shows that M(t) is a polynomial, as claimed. 6 Complexity and the degree of M(t) From now on, the algebraic group G is simple. That is, the center Z(G) is finite and contains every normal subgroup of G. Recall that the complexity δ is the minimum codimension of a B H -orbit in G/B. In this section we will complete the proof of the first assertion of Theorem 1.4 by showing that δ is an upper bound on the degree of the multiplicity polynomial M(t) defined in (35). 6.1 A formula for the complexity In this section we show that δ has the simplest conceivable formula. Let g and h be the Lie algebras of G and H. We are assuming that g is simple. We also invoke 1.3. That is, we assume that g = h m, stable under Ad(H), and that there is a nondegenerate Ad(H)-invariant symmetric form B on m. Hence Ad restricts to a homomorphism Ad : H SO(m). Lemma 6.1 Assume that H G. Then ker[ad : H SO(m)] = Z(G) H. Proof: Containment is clear. We prove containment. Set N := ker[ad : H SO(m)] and let n be the Lie algebra of N. We have n = ker[ad : h so(m)], 22

23 so n is an ideal in h. But [n, m] = 0, so n is in fact an ideal in g. Since g is simple and not equal to h, we have n = 0. Hence N is a finite normal algebraic subgroup of H. By [4, 22.1], N is central in H, hence Ad(N) acts trivially on h, as well as on m. It follows that N is central in G. This completes the proof. Let B and B H be Borel subgroups of G and H, respectively. Let U and V be their respective unipotent radicals. After conjugating, we may assume that B H = SV, B = T U with S T, V U. Proposition 6.2 The complexity δ is given by { dim G/B dim B H if H G δ = 0 if H = G. Proof: If H = G, the fact that δ = 0 is clear from the Bruhat decomposition. Assume from now on that H G. We must show that B H has an orbit in G/B with finite stabilizers. Let w be the element of W G (T ) such that w B B = T. Then every element of UwB/B can be uniquely expressed as uwb for u U. For v V, s S, we have vs uwb = v(sus 1 )wb. By uniqueness of expression, vs fixes uwb if and only if v = usu 1 s 1. It follows that the projection B H S gives an isomorphism from the B H -stabilizer of uwb to the S-stabilizer of u 1 V in the quotient variety U/V. We will show there exists u U such that the latter stabilizer is finite. Denote the Lie algebras of U, V, T, S by u, v, t, s. The tangent space to U/V at ev is u/v. We have g/h = t/s u/v ū/ v, where ū = Ad(w)u is the opposite nilradical of u and v is the opposite nilradical of v. Since ker[ad : S GL(g/h)] is finite by Lemma 6.1, it follows that ker[ad : S GL(u/v)] is finite. This latter kernel is the set of common zeros of the roots Φ(S, U/V ) of S in u/v (see [4, 8.17]). We have u/v = (u/v) S (u/v) α. α Φ(S,U/V ) 23

24 A vector in u/v whose α-component is nonzero for every α Φ(S, U/V ) will therefore have finite stabilizer in S. Proposition 6.2 now follows from a basic result: Lemma 6.3 Let k be an algebraically closed field. Suppose a k-torus S acts on a smooth irreducible affine k-variety X, fixing a point x X, so that S acts on the tangent space T x X at x. If there exists v T x X having finite stabilizer S v S, then there exists y X having finite stabilizer S y S. This lemma can be proved as follows. Since the torus S acts completely reducibly on the coordinate ring k[x], the argument of Lemme 1 in [15] shows that there is an S-equivariant morphism ϕ : X T x X such that ϕ(x) = 0, and whose differential dϕ x : T x X T x X is bijective. The set U of points in T x X with finite stabilizers is open, and non-empty by hypothesis. Since ϕ is dominant, the preimage ϕ 1 (U) is nonempty. If y ϕ 1 (U), then S y S ϕ(y), and the latter stabilizer is finite. Lemma 6.3 can also be proved using a T -equivariant embedding of X in a linear representation of T. 6.2 Degree of Ψ α (t) We return now to our rational function We have Q Gι υ,u(t)q Hι ς,u(t) Ψ α (t) = f ι (t) N H (ι, S) F P ι,u (t). deg P ι,u (t) = dim C Hι (u), deg f ι (t) = dim Z ι. (37) From section 5.4, and equation (23) we find that deg Ψ α (t) dim Z ι + d Gι (u) + d Hι (u) dim C Hι (u) [ = dim Z ι dim CGι (u) dim C Hι (u) rk G rk H ]. (38) The fixed point spaces g s, h s, m s are the same for any s S ι ; we denote them by g ι, h ι, m ι. Thus we have an Ad(H)-stable decomposition g ι = h ι m ι 24

25 and dim C Gι (u) dim C Hι (u) = dim m u ι dim m ι = dim C Gι (1) dim C Hι (1). (39) Define δ ι := dim Z ι + dim B Gι dim B Hι dim S = dim Z ι [dim m ι rk G rk H]. (40) For example, if ι 0 is the minimal element of I(S), then G ι0 = G, Z ι0 = Z(G) S and m ι0 = m. (41) Since Z(G) is finite, Proposition 6.2 implies that δ ι0 = 1 [dim m rk G rk H] = δ, if H G. (42) 2 Lemma 6.4 We have deg Ψ α (t) δ ι, with equality only if u = 1. Proof: The inequality follows from (38) and (39), and the last assertion follows from section 6.1. We now seek a bound on deg Ψ α which is independent of ι. We will show that δ ι δ, and that equality holds only in rather special circumstances. Let m ι be the sum of the eigenspaces of Ad(s) in m with eigenvalues 1, for any s S ι. Since det Ad(H) = 1 on m, the dimension dim m ι is even. We have m = m ι m ι, the form B is nondegenerate on m ι, and Ad : H SO(m) restricts to a homomorphism Ad ι : H ι SO(m ι). Lemma 6.5 For every ι I(S) we have δ ι δ. Moreover, if H G then the following are equivalent. 1. δ ι = δ 2. dim(z ι ) = 1 2 dim m ι 3. Ad ι (Z ι ) is a maximal torus in SO(m ι). When these hold, the derived group of H ι acts trivially on m ι. 25

26 Proof: If H = G then δ = 0 and δ ι = dim Z ι dim S 0. From now on assume H G. From (40) and (42), we have Now, the group δ δ ι = 1 2 dim m ι dim Z ι. (43) N ι := ker[ad ι : Z ι SO(m ι)] is finite. Indeed, since Z ι Z(G ι ), it follows that N ι centralizes m ι, as well as m ι. Hence we have N ι ker[ad : H SO(m)] = Z(G) H, the latter equality from Lemma 6.1. Hence N ι Z(G), and the latter is finite since G is simple. Since Ad(Z ι ) is a torus in SO(m ι) and 1 2 dim m ι is the dimension of a maximal torus in SO(m ι), this proves that both sides of (43) are 0 and that (1-3) are equivalent. For the last assertion, recall that Z ι Z(H ι ). If (1-3) hold then Ad ι (H ι ) centralizes a maximal torus in SO(m ι), hence is contained in that torus. With this lemma, the first assertion of Theorem 1.4 has been proved. 6.3 A remark on the multiplicity formula The formula (35), as written, contains more terms than are necessary. For, if we write Ψ α (t)θ α (t) = P α (t) + R α (t), where P α (t) is a polynomial and deg R α (t) < 0, then M(t) = P α (t) and R α (t) = 0, α α since M(t) is a polynomial. From (31) and Lemma 6.4 we have deg P α δ ι, where α = (ι, u, υ, ς). It follows that R G T,χ, R H S,η H F = M(q) = α P α (q), (44) where the sum is over just those α = (ι, u, υ, ς) such that δ ι 0. 26

27 7 The leading term of M(t) We have shown that the multiplicity polynomial M(t) has the form M(t) = At δ + (lower powers of t). In this section we find an explicit and effective formula for the leading term A of M(t). Recall from (35) that M(t) = α Ψ α (t)θ α (t), where α runs over quadruples (ι, u, υ, ς) as in (33), and Q Gι υ,u(t)q Hι ς,u(t) Ψ α (t) = f ι (t) N H (ι, S) F P ι,u (t) Θ α (t) = χ υ, η ς Z F ι + ( 1) J f J (t) χ υ, η ς Z F J f ι (t) J I(ι,S) F By Lemmas 6.4 and 6.5, only quadruples α with u = 1 and δ ι = δ contribute to the leading term; henceforth we assume α is of this form. As a power series in t, we then have Ψ α (t) = A α t δ + (lower degree terms), where A α = [Z F ι : Zι F ] ( 1)rk(Gι)+rk(T )+rk(hι)+rk(s) N H (ι, S) F. (45) At first glance, each function Θ α (t) could contribute many terms to A, coming from various ι < ι with dim Z ι = dim Z ι, since Z ι may be disconnected. We now show that in fact Θ α (t) contributes only one term. Lemma 7.1 If δ ι = δ and ι < ι then dim Z ι < dim Z ι. Proof: If H = G, we have δ ι = dim Z ι dim S 0 = δ with equality iff Z ι = S. The lemma holds since S is connected. Now assume H G. Suppose δ ι = δ and ι < ι, yet dim Z ι = dim Z ι. Then Z ι Z ι Z ι. (46) 27

28 From Lemma 6.5, the image Ad ι (Z ι ) is a maximal torus in SO(m ι). It follows that Ad ι (Z ι ) = Ad(Z ι ). Thus, for each z Z ι there is z 0 Z ι such that z 1 := zz 1 0 ker[ad ι : Z ι SO(m ι)]. By Lemma 6.1, we have z 1 Z(G) H. Hence We have shown that z = z 0 z 1 Z ι (Z(G) H). (47) Z ι = Z ι (Z(G) H). (48) Now S ι is stable under multiplication by Z(G) H. Moreover, S ι is open in Z ι, so S ι meets some connected component of Z ι in an open dense set. But then (48) implies that S ι meets every connected component of Z ι in an open dense set. Likewise, S ι meets some component of Z ι in an open dense set. By (46), every such component of Z ι is also a component of Z ι. Therefore S ι and S ι meet a common component of Z ι in a dense open set. This implies that S ι S ι is nonempty, hence ι = ι, contradicting ι < ι. As an aside, we mention the following consequence of (48) which simplifies our eventual formula for A when G is adjoint. Lemma 7.2 Suppose G is simple adjoint. If δ ι = δ then Z ι is connected. Return now to Θ α (t). For each J I(ι, S), the subgroup Z J is contained in some Z ι with ι < ι. Lemma 7.1 implies that deg f J (t) < deg f ι (t), which shows that the leading term of Θ α (t) has the following simple form. Corollary 7.3 Let α = (ι, 1, υ, ς) be a quadruple appearing in M(t) with δ ι = δ. Then Θ α ( ) = χ υ, η ς Z F ι. From (45) and 7.3 we get the following expression for the leading term A. 28

29 Proposition 7.4 The leading term A of M(t) in 1.4 is given by A = ι A ι, where ι runs over those ι I(S) F with δ ι = δ, and A ι = ( 1) rk(gι)+rk(t )+rk(hι)+rk(s) ZF ι /Zι F N H (ι, S) χ υ, η ς Z F ι. In the last summation, υ and ς run over j 1 G ι (cl(t, G)) and j 1 H ι (cl(s, H)), respectively. As a simple illustration of 7.4, we show how it reduces to the Deligne-Lusztig inner-product formula [8, thm. 6.8], when G = H. For ι I(S), we have then δ ι = dim Z(G ι ) dim S 0 = δ with equality iff G ι = S = H ι. This means ι is the maximal element of I(S) F, and M(t) = A = A ι is the inner product RT,χ G, RG S,η G F. By 7.4, if T is not G F conjugate to S then j 1 G ι (cl(t, G)) =, so A = 0. Otherwise we may take S = T, and the fiber of j Gι over cl(s, G) is the singleton {υ} corresponding to the class of S in itself. We have χ υ = w χ, η υ = w η, NG (ι, S) F = W G (S) F, and the result: w W G (S) F w W G (S) F A = χ υ, η υ S F W G (S) F is the original Deligne-Lusztig formula for R G S,χ, RG S,η G F. 8 Optimality Recall that G is simple. In this section we show that the degree δ is optimal. We may assume that H G. Let T G and S H be arbitrary F -stable maximal tori. We will show that for sufficiently large q, there are characters χ Irr(T F ) and η Irr(S F ) such that the leading coefficient A is nonzero. In fact, we can take η to be the trivial character. For each ι I(S) F with A ι 0, the fiber j 1 G ι (cl(t, G)) is non-empty. This means that Z ι is G F -conjugate to a subgroup Z ι T. There are only finitely many of these subgroups Z ι. Recall from (41) that ι 0 I(S) F is the minimal 29 υ,ς

30 element, for which Z ι0 = Z(G) S. If dim Z ι = 0 and δ ι = δ then Lemma 7.1 implies that ι = ι 0. Hence, ( if ι ι 0, the ) torus T/ Z ι has strictly smaller dimension than that of T, so that Irr T F / is a polynomial in q of degree strictly less Z F ι than dim T. Hence for sufficiently large q there are characters χ Irr(T F ) which are trivial on Zι F 0 and non-trivial on every Z ι F for ι ι 0. We call these χ very regular. For very regular χ and ι such that A ι 0, we have { 1 if ι = ι 0 χ υ, 1 Z F ι = (49) 0 if ι ι 0. It follows that for χ very regular, and η = 1, the coefficient A of t δ in M(t) is given by A = ɛ G (x)ɛ H (y), (50) where x cl(t, G) and y cl(s, H). Let ϑ be the automorphism of W H induced by F and let ψ be the character of an irreducible representation of ϑ W H. For each y W H, choose an F -stable torus S y in H such that y cl(s y, H). We have then a class function Rψ H of HF defined by R H ψ = 1 W H y W H ψ(ϑy)r H S y,1. For example, the trivial (1) and sign (ɛ H ) characters of W H extend to ϑ W H (trivially on ϑ). It is known (cf. [5, 7.6]) that R H 1 = 1 H and R H ɛ H = St H are the trivial and Steinberg characters of H F, respectively. For very regular χ, (50) implies that R G T,χ, R H ψ H F = ɛ G (x) ɛ H, ψ WH q δ + (lower powers of q). (51) In particular, we have ɛ G (x)r G T,χ, St H H F = q δ + (lower powers of q), while ɛ G (x)rt,χ G, 1 H H F has degree < δ. This last result is to be expected, in view of the results in [2] and [17]. 9 Restriction from SO 2n+1 to SO 2n. We return to the situation at the beginning of the introduction. So p > 2 and (V, Q) is a 2n + 1-dimensional quadratic F-space, defined over f, with Frobenius 30

31 F. Fix v V F with Q(v) 0, and let U be the orthogonal space of v in V. We take G = SO(V ), H = G v = SO(U), with f-structure on both groups induced from that on V. Assumption 1.3 holds: we may identify the quadratic spaces (m, B) = (U, Q). Let T, S be F -stable maximal tori in G and H respectively, and let χ Irr(T F ), η Irr(S F ) be characters, which for the moment are arbitrary. We have δ = dim B G dim B H dim rk H = n 2 (n 2 n) n = 0. From now on, we only consider ι I(S) F with δ ι = 0. Since G is adjoint, each such Z ι is connected, by 7.2. Proposition 7.4 gives the multiplicity formula ( 1) rk T +rk S R G T,χ, R H S,η H F = ι I(S) F δ ι=0 ( 1) rk(gι)+rk(hι) N H (ι, S) χ υ, η ς Z F ι, (52) υ,ς where υ and ς run over j 1 G ι (cl(t, G)) and j 1 H ι (cl(s, H)), respectively. The connectedness of Z ι implies that 1 is not an eigenvalue of any s S ι. The last assertion of 6.5 implies that s S ι has distinct eigenvalues on V/V s. It follows that G ι = SO(V s ) Z ι, H ι = SO(U s ) Z ι. (53) Note that dim V s is odd, say dim V s = 2a + 1. The decompositions (53) imply that if two F -stable maximal tori in G F ι are G F -conjugate, then they are G F ι -conjugate, and likewise for H. In other words, we have j 1 G ι (cl(t, G)) j 1 H ι (cl(s, H)) 1. Hence the inner sum of (52) has at most one term. To make this precise, we recall that tori in orthogonal groups are described by pairs (λ, λ ) of partitions. We write partitions as λ = (j λ j ), meaning that λ has λ j parts equal to j, and set λ = j jλ j. We have pairs of partitions 31

32 (ν, ν ), (λ, λ ), (µ, µ ) such that Z F ι j (f j )ν j (f 1 2j) ν j, ν + ν = n a T F j S F j (f j )λ j (f 1 2j) λ j, λ + λ = n, (f j )µ j (f 1 2j) µ j, µ + µ = n. (54) We have j 1 G ι (cl(t, G)) j 1 H ι (cl(s, H)) = 1 precisely when ν j λ j, µ j and ν j λ j, µ j (55) for all j. We assume (55) holds from now on. Note that if T and S are anisotropic then λ j = µ j = ν j = 0 for all j. We count the number of ι in the sum (52) giving rise to a fixed pair of partitions (ν, ν ). For s S F ι, consider the components (s j1,..., s jµj ; s j1,..., s jµ j) of s in the j th block (f j )µ j (f 1 2j) µ j S F. Then ι is determined by the pair of subsets {k [1, µ j ] : s jk = 1}, {k [1, µ j] : s jk = 1}. It follows that there are ( )( ) µ µ ν ν elements ι I(S) F giving rise to (ν, ν ), where ( ) µ := ( ) ( ) µj µ, := ν ν j j ν j ( µ j ν j ). From equations (2.3) and j 1 H ι (cl(s, H)) = 1 we have N H (ι, S) F = W ( )( ) H(S) F µ µ W Hι (S) F = (ν ν ν j!)(ν j!)(2j) ν j (2j) ν j. (56) Using 3.1 for G ι and H ι, we find that j ( 1) rk Gι+rk Hι = ( 1) rk G+rk H+P ν j. (57) 32

33 (One can also arrive at (57) by decomposing U F into irreducible fz F ι -modules, and calculating discriminants.) Finally, we must calculate the pairing χ υ, η ς Z F ι. We may conjugate T and S to arrange that Z ι T S. Then χ υ = 1 W Gι (T ) F x W G (T ) F ( x χ) Zι, η ς = 1 W Hι (S) F y W H (S) F ( y η) Zι. We now assume that χ and η are regular, in the sense that they have trivial stabilizers in W G (T ) F and W H (S) F, respectively. On the j th block (f j )µ j (f 1 2j) µ j of S F, we have η = η j1 η jµk η j1 η jµ k. Likewise, on the j th block (f j )λ j (f 1 2j) λ j of T F, we have Define χ = χ j1 χ jλj χ j1 χ jλ j. I j = {k [1, µ j ] : η jk Γ j {χ jl, χ jl 1 }, for some l [1, λ j ]}, I j = {k [1, µ j] : η jk = Γ 2j χ jl, for some l [1, λ j]}. (58) For every pair of subsets {k 1,, k νj } I j, {k 1,, k ν j } I j, each of the conjugates of the character (ν j )!(ν j)!(2j) ν j (2j) ν j η jk1 η jkνj η jk 1 η jk ν j contributes exactly once to the pairing χ υ, η ς Z F ι, by the regularity assumption (1.3). It follows that χ υ, η ς Z F ι = ( )( ) Ij I j (ν ν j j ν j j )!(ν j)!(2j) ν j (2j) ν j. (59) 33

34 Set e := ( 1) rk G+rk T +rk H+rk S. Inserting (56), (57) and (59) into (52), and summing over all (ν, ν ) satisfying (55), we get e RT,χ, G RS,η H H F = ( 1) P ( )( ) ν j Ij I j ν,ν j ν j = I j ( ) I j Ij ( ) ( 1) ν j I j ν j ν j =0 j ν ν j =0 j { 2 r if I j is empty for all j = 0 otherwise, ν j (60) where r = j I j. If either T or S is anisotropic, then r = 0. This proves Theorem Restriction from SO 7 to G 2 The previous situation had δ = 0. We now consider a case where δ = 1. The simplest such case is G = G 2, H = SL 3, which we leave to the reader. Here we take G = SO 7, H = G 2, embedded in G via the irreducible 7- dimensional representation V of G 2. We have δ = 9 8 = 1. We assume p 5. We will calculate the multiplicities RT,χ G, RH S,η HF, using the formula of section 6.3. We do not need any detailed knowledge of Green functions, beyond the general facts about their degrees and leading terms that we have already used. Let α, β be simple roots of a maximal f-split torus S 0 in H, with α short. The nonzero weights of H in V are the short roots of S 0. We view the maximal f-split tori T 0 and S 0 as T 0 = {(x, y, z) F 3 : xyz 0}, S 0 = {(x, y, z) F 3 : xyz = 1}, 34

35 in such a way that the coordinate functions e 1, e 2, e 3 on T 0 restrict to the roots 2α + β, α, α β on S 0. In this realization, the simple co-roots of S 0 in H are ˇα(t) = (t, t 2, t), ˇβ(t) = (1, t, t 1 ), and the corresponding simple reflections r α, r β in the Weyl group W H act by r α (t 1, t 2, t 3 ) = (t 1 3, t 1 2, t 1 1 ), r β (t 1, t 2, t 3 ) = (t 1, t 3, t 2 ). Since S 0 contains regular elements in T 0, it follows that W H is a subgroup of W G. If W G is realized as the group of the cube, then W H is the subgroup preserving a diagonal of the cube; as coset representatives for W G /W H we may take the identity and each coordinate sign change. Let T, S be F -stable maximal tori in G and H, corresponding to the conjugacy classes of x W G, and y W H, respectively. Let χ Irr(T F ), η Irr(S F ). We will use the refined multiplicity formula of section 6.3. We first tabulate the pairs (ι, u) in H, with ι I(S 0 ), and u H ι, for which We find four types as shown: dim Z ι + d Gι (u) + d Hι (u) dim C Hι (u) 0. (61) type ι u a (1, 1, 1) 1, u 0 b (1, t, t 1 ), t q = t ±1 1 c (1, t, t 1 ), t q = t 1 ±1 1 d regular 1 The middle column shows a typical element in S 0 for each type of ι. There can be more than one ι of the same type. Here u 0 H F is a long root element, which has Jordan partition on V. From equation (32), we have M(q) = M a (q) + M b (q) + M c (q) + M d (q), where each term on the right is the sum Ψα Θ α 35