REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS. Marko Tadić. Introduction

Transcription

1 Compositio Mathematica vol. 90, 1994, pages REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS Marko Tadić Introduction A non-archimedean field ofa characteristic different from two is denoted by F. In this paper we consider the representation theory ofgroups Sp(n, F ) and GSp(n, F ). The inner geometry ofthese groups motivates us to consider the representations ofthese groups as modules over representations ofgeneral linear groups. Such an idea goes back to D. K. Faddeev in the finite field case ([F]). D. Barbasch had also such point ofview in [Ba]. Besides the module structure, we also have a comodule structure. Our motivation for such approach is to make symplectic case more close to the well understood theory ofgroups GL(n), as it was developed by J. Bernstein and A.V. Zelevinsky ([BnZ1], [BnZ2], [Z1]), and to ideas developed in [T1]. The basic idea was to realize some ofthe properties ofthe representation theory ofsymplectic groups as a part ofthe structure theory ofcertain modules. This point ofview is helpful in searching ofnew square integrable representations, and in examining reducibility ofparabolically induced representations (see [T4]). In this paper, we are developing this approach in the symplectic case. Other classical groups can be treated in a similar way. We describe now the content ofthe paper according to sections. Let us point out first that the parameter n in Sp(n, F )orgsp(n, F ) denotes the semi simple rank ofthese groups. In the first section we collect some general facts about representations of finite length ofreductive groups over F. Besides the group R(G) ofvirtual characters ofg, we introduce a group R(G) which is constructed from the representations of finite length of G. Two algebras ofrepresentations ofgl(n, F ) are considered in the second section. The first algebra R was introduced by J. Bernstein and A.V. Zelevinsky ([BnZ2]). It is realized as the direct sum of R(GL(n, F )) s. The other algebra R is realized as the direct sum of R(GL(n, F )) s. The multiplication in both cases is defined with the help ofthe parabolic induction from the maximal parabolic subgroups. It is well known that R is a Hopfalgebra. The comultiplication is defined using Jacquet modules for maximal parabolic subgroups. The direct sum ofall R(Sp(n, F )) (resp. R(Sp(n, F ))) is denoted by R[S] (resp. R[S]). We introduce also corresponding groups for GSp(n, F ) s. They are denoted by R[G] and R[G]. In the fourth section, the groups R[S] and R[S] are considered as modules over R and R respectively. Analogously, we consider R[G] and R[G] as modules. We list then some important properties ofthese modules. A structure ofa comodule on R[S] and R[G] over R, is introduced in the fifth section. The comultiplication is defined again using Typeset by AMS-TEX 1

2 2 MARKO TADIĆ Jacquet modules ofmaximal parabolic subgroups. We describe the Langlands classification for groups Sp(n, F ) and GSp(n, F ) more explicitly in the sixth section. N. Winarsky obtained in [Wi] a necessary and sufficient conditions for reducibility of the unitary principal series representations of Sp(n, F ). D. Keys obtained exact number of irreducible pieces and he showed that these representations are multiplicity free ([Ke]). In the seventh section, using the above results we obtain corresponding results for GSp(n, F ). A necessary and sufficient conditions for reducibility of the unitary principal series and the length ofthese representations is obtained here. Then we derive a necessary and sufficient condition for the reducibility of the non-unitary principal series representations of Sp(n, F ) and GSp(n, F ) using the Langlands classification and above results. Regular characters ofa maximal torus in a split reductive group, for which corresponding non-unitary principal series representations have square integrable subquotients were characterized by F. Rodier in [R1]. In the eighth section we give an explicit characterization ofregular characters for groups Sp(n, F ) and GSp(n, F ). All square integrable representations of GSp(n, F ) which may be obtained as subquotients ofnon-unitary principal series representations induced by regular characters, are described explicitly in this section. There is a considerable number ofsquare integrable representations obtainable in this way. Let us describe the parameters ofthat representations in the case when the residual characteristic is odd. Such square integrable representations, up to a twist by a character of GSp(n, F ), are parameterized by all pairs (k, lψ 1 + mψ 2 ) where k, l, m are non-negative integers which satisfy l, m 1, k + l + m = n and ψ 1,ψ 2 are different characters of F oforder two. Restriction ofthe square integrable representations ofthe eighth section to the group Sp(n, F ), is studied in the ninth section. These representations split without multiplicities and we give a parameterization ofthe irreducible pieces. We get a considerable number ofsquare integrable representations ofsp(n, F ) in this way. Let us mention that the only square integrable representation of Sp(n, F ) which corresponds to a regular character, is the Steinberg representation. The Steinberg representation of Sp(n, F ) is a subquotient ofa non-unitary principal series representation π = Ind Sp(n,F ) P φ ( 1/2 P φ ). It is well known that π is a multiplicity free representation of length 2 n. It has exactly one square integrable subquotient, the Steinberg representation. The Steinberg representation and the trivial representation are the only unitarizable subquotients of π. There exist very different examples ofnon-unitary principal series representations of Sp(n, F ) which possess square integrable subquotients. For suitable choice of F, there exists a non-unitary principal series representation π 1 of Sp(2n, F ) which has exactly 2 n irreducible square integrable subquotients. They are all of multiplicity one. There exists a subquotient of π 1 whose multiplicity is 2 n.forn =1we have proved in [SaT] that all irreducible subquotients are unitarizable. In the last section, one such representation is analyzed in detail. Let us denote this representation of Sp(4,F) by π 2 (there is no additional assumptions on the field F ). The length of π 2 is 36. It has exactly 25 different irreducible subquotients. We find all multiplicities. The last example is an illustration ofapplication ofsome ofthe methods which were considered in the previous sections. Further development along these lines, and more advanced applications ofthese techniques and ideas, are announced in [T4]. The present

3 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 3 paper should be considered as an introduction to this point ofview on representations of p-adic symplectic groups. We apply this approach systematically to the representations of GSp(2,F) (and also Sp(2,F)) in the joint paper [SaT] with P.J. Sally. The first version ofthis paper was written when the author was visiting the Mathematical Department ofthe University ofutah. The last revision ofthe paper took place during the author was a guest ofthe Sonderforschungsbereich 170 in Göttingen. We are thankful to both institutions for the hospitality and excellent working conditions. The referee s comments and corrections helped a lot in bringing this paper to the present form. I. Mirković helped a lot in improving the style ofthe paper. The field ofreal numbers is denoted by R in the paper. The subring ofintegers is denoted by Z, the non-negative integers are denoted by Z +, while the strictly positive integers ara denoted by N. One technical remark at the end. We have already mentioned algebra R and modules R[S] and R[G]. Let me note that in this paper we could work simply with representations, instead ofdoing calculations in R, R[S] orr[g] (this is equivalent). Regardless of this, we introduced this algebra and these modules because they arose naturally in our considerations. 1. Groups of representations In this section we shall recall some well known facts from the representation theory of reductive p-adic groups. We fix a local non-archimedean field F. The group ofrational points ofa reductive group defined over F is denoted by G. The category ofall smooth representations ofg and G-intertwinings among them, is denoted by Alg(G). The full subcategory of representations offinite length in Alg(G) is denoted by Alg f.l. (G). For each isomorphism class of representations in Alg f.l. (G), we fix a representative. The set ofall such representatives is denoted by R + (G). Note that R + (G) is a set. The full subcategory of Alg f.l. (G) whose objects consist of R + (G), is denoted by R + (G) again. We denote by In(G) the set ofall equivalence classes ofall smooth indecomposable non-zero representations offinite length. The subset ofall irreducible classes is denoted by G. The subset ofall unitarizable classes in G is denoted by Ĝ. We shall consider Ĝ G In(G) R + (G). Let π 1,π 2 R + (G). We denote by π 2 + π 2 a unique representation in R + (G) which is equivalent to π 1 π 2. It is clear that the addition is associative and commutative. The representation on 0-dimensional space is the zero ofthe additive semigroup R + (G). It is obvious that In(G) generates R + (G) as a semigroup with zero. In other words, each π R + (G) may be written as π = π π m with π i In(G). It is easy to see that π 1...π m In(G) are determined uniquely, up to a permutation, by π R + (G) ([Bu], ch. 8, n o 2, Theorem 1).

4 4MARKO TADIĆ We denote by R(G) the free Abelian group over the basis In(G). According to the above observation, we may identify R + (G) with a subset of R(G) in a natural way. Also, R + (G) is an additive subsemigroup of R(G). The Grothendieck group ofthe category Alg f.l. (G) (or equivalently, of R + (G)) will be denoted by R(G). The canonical mapping will be denoted by s.s. : R + (G) R(G). Recall that R(G) may be identified with the free Abelian group over the basis G. We shall do so. Denote s.s.(r + (G)) by R + (G). There is a unique extension ofs.s. to an additive homomorphism of R(G) intor(g). This extension will be denoted again by s.s.. For x 1,x 2 R(G) we shall write x 1 x 2 if x 2 x 1 R + (G). Let χ be a character of G. Then π χπ induces automorphisms of R(G) and R(G). They are positive, i.e. χ(r + (G)) R + (G) and χ(r + (G)) R + (G). If π is a smooth representation of G, then π denotes the smooth contragredient of π and π denotes the complex conjugate of π. One extends and to R(G) and R(G) additively. These are involutive automorphisms of R(G) and R(G) and they commute. Let Z(G) be the Bernstein center of G. It is the algebra ofall invariant distributions T on G such that the convolution T f is compactly supported for any locally constant compactly supported function f on G. For each smooth representation (π,v ) ofg, Z(G) acts naturally on V. Ifπ is irreducible, then Z(G) acts by scalars. The corresponding character of Z(G) is denoted by θ π. It is called the infinitesimal character of π (here we follow mainly notation of [BnDKa]). The set ofall infinitesimal characters ofrepresentations in G is denoted by Θ(G). The set Θ(G) is in a natural one-to-one-correspondence with the set ofall cuspidal pairs modulo conjugation (a cuspidal pair (M,ρ) consists of a Levi factor M ofa parabolic subgroup P = MN in G and ofan irreducible cuspidal representation ρ of M). We identify these two sets. For θ Θ(G), Gθ denotes the set of all π G such that θ π = θ. The set G θ is finite. If θ =(M,ρ), then G θ is just the set ofall irreducible subquotients ofind G P (ρ), i.e. π G θ ifand only ifπ G and π s.s.(ind G P (ρ)). Set R θ (G) = Zπ π G θ Note that R(G) = R θ (G),θ Θ(G), is a gradation ofthe group R(G). This gradation is compatible with the order on R(G), i.e. if π i = πθ i R(G) = R θ(g), i =1, 2, then π 1 π 2 ifand only ifπθ 1 π2 θ for any θ Θ(G). Let θ Θ(G). Denote by R + θ (G) the set ofall π R+ (G) such that each irreducible subquotient of π is in G θ. The additive subgroup of R(G) generated by R + θ (G) is denoted by R θ (G). Set In θ (G) =In(G) R + θ (G). Now In θ (G) is a basis of R θ (G). Considering the action ofthe commutative algebra Z(G), one obtains in a standard way that we have the following disjoint union In(G) = In θ (G). Θ(G)

5 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 5 This was already proved without use of Z(G) by W. Casselman ([Cs], Theorem and 7.3.2). Thus R(G) = Θ(G) R θ (G). is a gradation on R(G). Clearly s.s.(r θ (G)) = R θ (G). For a smooth representation π of G and for an automorphism σ of G, σπ denotes the representation (σπ)(g) =π(σ 1 (g)). In this way one obtains automorphisms σ : R(G) R(G) and σ : R(G) R(G). Let P be a parabolic subgroup of G. Denote by N the unipotent radical of P. Let P = MN be a Levi decomposition. The modular character of P is denoted by P. Using the normalized induction functor we define in a natural way homomorphisms Ind G P : R + (M) R + (G), Ind G P : R(M) R(G) Ind G P : R(M) R(G). Recall now the notion ofa Jacquet module. For a smooth representation (π, V ) ofg, we denote by rp G (π) the representation of M on N-coinvariants twisted by ( 1/2 P ) M. We have the Frobenius reciprocity for τ R + (M) and π R + (G): Hom R+ (G)(π, Ind G P (τ)) = Hom R+ (M)(r G P (π),τ). In a natural way one obtains homomorphisms r G P : R(G) R(M), rp G : R(G) R(M). Take the opposite parabolic subgroups P = M N of P. By Corollary of[cs] we have [ r G P (π) ] = r Ḡ P ( π). for π R + (G). We can reformulate the Frobenius reciprocity: ( Hom R + (G) Ind G P (τ),π ) ( = HomR + (M) τ,r Ḡ P (π)), π R + (G),τ R + (M) ([Si2], Theorem 2.4.3). The set ofall irreducible essentially tempered representations ofg (i.e. representations of G which become tempered after twisting by a suitable character of G) is denoted by T (G). The essentially square integrable (resp. cuspidal) representations in G are denoted by D(G) (resp. C(G)). Let T u (G) =T (G) Ĝ, Du (G) =D(G) Ĝ and Cu (G) =C(G) Ĝ. For τ T (G) there exists a unique positive valued character χ of G such that χ 1 τ T u (G). Define ν(τ) =χ and τ u = χ 1 τ u.thusτ = ν(τ)τ u.

6 6 MARKO TADIĆ 2. Algebras of representations for GL(n) Let R + n = R + (GL(n, F )), R n = R(GL(n, F )), R n + = R + (GL(n, F )), R n = R(GL(n, F )). We denote R = R n, n 0 R = R n. n 0 The maps s.s.: R n R n naturally extend to s.s. : R R. Let R + = n 0 R + n, R + = n 0 R + n. For π i R + n i,i=1, 2, we denote by π 1 π 2 (resp. π 1 π 2 ) the unique representation in R + n 1 +n 2 which is isomorphic to the parabolically induced representation from the standard parabolic subgroup P (resp. P ) with respect to the upper (resp. lower) triangular matrices, whose Levi factor is naturally isomorphic to GL(n 1,F) GL(n 2,F)byπ 1 π 2 (see [BnZ]). The induction that we consider is normalized in such a way that it carries unitarizable representations to unitarizable ones. Conjugation by a suitable element ofthe Weyl group gives the following equality in R π 1 π 2 = π 2 π 1. We have also (π 1 π 2 ) π 3 = π 1 (π 2 π 3 ), where π 3 R + n 3. We extend and to Z-bilinear mappings on R R. In this way (R, +, ) becomes a graded associative ring with identity. We extend π π, π π to R. These are involutive automorphisms ofthe ring. We shall define now a binary operation on R which will be denoted again by. Let π 1,π 2 R. We may consider π 1,π 2 Rsince GL(n, F ) In(GL(n, F )). Therefore, we have defined π 1 π 2 R.Nowπ 1 π 2 R is defined to be s.s. (π 1 π 2 ). In this way R becomes a graded associative commutative ring with identity. In a natural way one defines automorphisms π π and π π on R. A character χ of F = GL(1,F) is identified with a character of GL(n, F ) using the determinant homomorphism. We consider the map χ : π χπ, π R + n, and extend it

7 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 7 Z-linearly to R. In this way, χ is an automorphism of R. One defines χ : R R in a natural way. We shall denote by t g the transposed matrix of g GL(n, F ). The matrix transposed with respect to the second diagonal is denoted by τ g. The representations t π 1 : g π( t g 1 ) and τ π 1 : g π( τ g 1 ) are equivalent for π R + n, i.e. τ π 1 = t π 1 in R +.We extend π t π 1 Z-linearly to R. One has directly t (π 1 π 2 ) 1 = t π 1 1 t π 1 2 = t π 1 2 t π 1 1. Thus, π t π 1 is an involutive antiautomorphism of R. Observe that t π 1 = π for an irreducible representation π ([GfKa]). Thus s.s. ( π) = s.s. (t π 1) for any π R. Let α =(n 1,...,n k ) be an ordered partition of n and let π R + n. We denote by r α,(n) (π) the Jacquet module introduced in 1.1 of[z1]. It is a representation ofgl(n 1,F)... GL(n k,f). Working with standard parabolics with respect to the lower triangular matrices, instead ofthe upper triangular ones, we introduce r α,(n) (π) in an analogous way. Now [ r α,(n) ( π) = r α,(n) (π)]. We consider s.s. ( r (k,n k),(n) (π) ) R k R n k. For π GL(n, F ) set m (π) = n s.s. ( r (k,n k)(n) (π) ) R R, k=0 m (π) = n k=0 ( ) s.s. r (k,n k),(n) (π) R R. With comultiplication m,(r, +, ) is a Hopfalgebra (the similar statement holds for m and (R, +, ).We shall denote Irr = GL(n, F ), n 0 Irr u = GL(n, F ), n 0 In = D = C = In(GL(n, F )), n 0 D(GL(n, F )), n 1 C(GL(n, F )), n 1

8 8 MARKO TADIĆ The modulus of F will be denoted by F. We denote by ν n, or simply by ν, the positive valued character g det g F of GL(n, F ). For each δ D there exists a unique α R such that ν α δ is unitarizable. This α will be denoted by e(δ). If X is a set,then we shall denote by M(X) the set ofall finite multisets in X. By definition, M(X) is the set ofall possible n-tuples ofelements ofx, with all possible n Z +. The set M(X) is an additive semigroup for the operation (x 1,...,x n )+(y 1,...,y m )=(x 1,...,x n,y 1,...,y m ). We shall now describe the gradation obtained on R and R from Θ(GL(n, F )) (see the first section). For π Irr we shall say that ω =(ρ 1,...,ρ n ) M(C) is the support of π if π ρ 1... ρ n in R. The support of π is denoted by supp π. Let Irr ω = {π Irr; supp π = ω}. We denote by R ω the subgroup of R generated by Irr ω.now R = R ω. M(C) This is a gradation ofthe Hopfalgebra R. Put In ω = {π In; s.s.(π) R ω }. The subgroup of R generated by In ω is denoted by R ω. We have R = R ω. M(C) This is a gradation ofthe ring R. We shall introduce a new gradation on R which may be be useful in the study of representations of p-adic symplectic groups. We shall write ρ 1 ρ 2 if ρ 1 = ρ2 or ρ 1 = ρ2 for ρ 1,ρ 2 Irr. The set ofall equivalence classes in C for this relation, is denoted by C. The canonical projection is denoted by κ : C C. Now we define κ : M(C) M(C ) (ρ 1,...,ρ n ) (κ(ρ 1 ),...κ(ρ n )). Note that κ(ω 1 )+κ(ω 2 )=κ(ω 1 + ω 2 ). For Ω M(C ) set R Ω = R ω (ω M(C),κ(ω) =Ω). Again R = M(C ) R Ω, and this is another gradation ofthe Hopfalgebra R.

9 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 9 3. Symplectic groups In the rest ofthis paper we shall assume that char F 2. The vector space ofall n m matrices over F is denoted by M (n,m) (F ). We denote M (n,n) (F )bym n (F ). Let J n denotes the matrix : in M n (F ). The identity n n matrix is denoted by I n. For S M 2n (F ), according to [F ], set S = [ 0 Jn J n 0 ] t S [ 0 Jn J n 0 Clearly (S 1 S 2 )= S 2 S 1.If [ ] A B S =, A,B,C,D M C D n (F ), then By definition, [ τ D S = τ B τ C τ A ]. Sp(n, F )={S M 2n (F ); SS= I 2n }. We may say also that Sp(n, F ) is the set ofall matrices S M 2n (F ) which satisfy t S [ 0 Jn J n 0 ] S = [ 0 Jn J n 0 The [ third ] way to describe Sp(n, F ) is to say that Sp(n, F ) is the set ofall matrices A B, A, B, C, D M C D n (F ) which satisfy τ DA τ BC = I n, τ DB = τ BD and τ AC = τ CA. Then [ ] 1 [ A B τ ] D = τ B C D τ C τ. A Now GSp(n, F )={S M 2n (F ); SS (F )I 2n }. We may describe GSp(n, F ) also as the set ofall S M 2n (F ) which satisfy [ ] [ ] t 0 Jn S S (F 0 Jn ). J n 0 J n 0 ] ]

10 10 MARKO TADIĆ For S GSp(n, F ) there exists a unique ψ(s) F such that SS = ψ(s)i 2n. It is easy to see that {[ ] } In 0 Sp(n, F ) ; λ F = GSp(n, F ). 0 λi n ( [ ]) In 0 Note that ψ g = λ for g Sp(n, F ). Clearly, ψ is multiplicative. Also, 0 λi n Sp(n, F ) is the derived subgroup of GSp(n, F ). Take Sp(0,F) to be the trivial group and take GSp(0,F)tobeF. We consider Sp(0,F) and GSp(0,F) as0 0 matrices formally. The diagonal subgroup in Sp(n, F ) (resp. GSp(n, F )) will be taken for a maximal split torus. These maximal tori are denoted by A 0. We fix the Borel subgroup in Sp(n, F ) (resp. GSp(n, F )) which consists ofall upper triangular matrices in Sp(n, F ) (resp. GSp(n, F )). These Borel subgroups are denoted by P φ. We parametrize A 0 in Sp(n, F ) in the following way (x 1,...,x n ) In GSp(n, F ) we do it as follows: a :(F ) n A 0, [ diag(x1,...,x n ) 0 a :(F ) n F A 0 [ diag(x1,...,x (x 1,...,x n,x) n ) 0 diag(x 1 n 0,...,x 1 1 ) 0 x diag(x 1 n ].,...,x 1 1 ) The Weyl groups defined by the above maximal tori in Sp(n, F ) and GSp(n, F ) are naturally isomorphic. These groups are denoted by W. The simple roots determined by the Borel subgroup in Sp(n, F ) are and In GSp(n, F ) the simple roots are and α i (a(x 1,...,x n )) = x i x 1 i+1, 1 i n 1, α n (a(x 1,...,x n )) = x 2 n. α i (a(x 1,...,x n,x)) = x i x 1 i+1, 1 i n 1, α n (a(x 1,...,x n,x)) = x 2 nx 1. The Weyl group W has 2 n n! elements. The action of W by conjugation on A 0 Sp(n, F ) is generated by transformations a(x 1,...,x i,x i+1,...,x n ) a(x 1,...,x i+1,x i,...,x n ), 1 i n 1 ].

11 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 11 and a(x 1,...,x n 1,x n ) a(x 1,...,x n 1,x 1 n ). In the case of A 0 GSp(n, F ) generating transformations are a(x 1,...,x i,x i+1,...,x n,x) a(x 1,...,x i+1,x i,...,x n,x), 1 i n 1, and a(x 1,...,x n 1,x n,x) a(x 1,...,x n 1,xx 1 n,x). The standard parabolic subgroups of Sp(n, F ), and also of GSp(n, F ), are parametrized by subsets of {α 1,...,α n }. We shall use the following parametrization. First we consider the case of Sp(n, F ). Fix n Z +. Take an ordered partition α =(n 1,...,n k )ofm, where m n.if m = 0, then the only partition is denoted by (0). Set M α = g gk h τ g 1 k... 0 τ g 1 1 ; g i GL(n i,f),h Sp(n m, F ) Further, P α = M α P φ is a parabolic subgroup of Sp(n, F ). correspond to the subset These parabolic subgroups {α 1,...,α n }\{α n1,α n1 +n 2,...,α n n k }. The unipotent radical of P α is denoted by N α. One obtains standard parobolic subgroups (resp. Levi factors) in GSp(n, F ) from the standard parabolic subgroups P α (resp. Levi factors M α )insp(n, F ) by multiplying them with the subgroup {[ ] } In 0 ; λ F. 0 λi n These subgroups in GSp(n, F ) are denoted again by P α and M α. Then M α = where g i GL(n i,f),h GSp(n m, F ).... g k g 1 0 h ψ(h) τ g 1 k... 0 ψ(h) τ g 1 1

12 12 MARKO TADIĆ Suppose that we have two standard parabolic subgroups P α and P α. Then they are associate ifand only ifα and α are partitions ofthe same number, and ifthey are equal as unordered partitions. The characters of F will be identified with characters of GSp(n, F ) in the following way χ (g χ(ψ(g))). 4. Modules of representations Let R + n [S] =R + (Sp(n, F )), R n [S] =R(Sp(n, F )), R n + [S] =R + (Sp(n, F )), R n [S] =R(Sp(n, F )), R[S] = R n [S], n 0 R[S] = R n [S], n 0 R + [S] = R + n [S], n 0 R + [S] = n 0 R + n [S], Irr[S] = Sp(n, F ), n 0 In[S] = In(Sp(n, F )), n 0 C[S] = C(Sp(n, F )), n 0 T [S] = T (Sp(n, F )). n 0 One introduces analogous notation R + n [G], R n [G],... for the groups GSp(n, F ). Take π R + n and σ R + m[s]. We take the maximal parabolic subgroup P (n) in Sp(n + m, F ). Using the identification (g, h) g h 0 ; g GL(n, F ),h Sp(m, F ), 0 0 τ g 1 we identify GL(n, F ) Sp(m, F ) with M (n). In this way we consider π σ as a representation of M (n). Let π σ = Ind Sp(n+m,F ) P (n) (π σ),

13 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 13 We extend and Z-bilinearly to Nowwehave π σ = Ind Sp(n+m,F ) t P (n) (π σ). R R[S] R[S]. Proposition 4.1. (i) With the action : R R[S] R[S], R[S] is a Z + -graded module over (R, +, ) (ii) With the action : R R[S] R[S], R[S] is a Z + -graded module over (R, +, ). (iii) For π R + n and σ R + m[s] we have π σ = τ π 1 σ, (π σ) = π σ, (π σ) = π σ. Proof. Only the associativity π 1 (π 2 σ) =(π 1 π 2 ) σ is not evident in (i). This follows from the transitivity of the induction in stages ([BnZ2]). The same situation is with (ii). One obtains (iii) using conjugation with [ 0 J n+m J n+m 0 ]. Now we define : R R[S] R[S] by s.s.(π) s.s.(σ) = s.s.(π σ), π R, σ R[S] (in the above formula we use the definition of π σ R[S] which we introduced before). One has Proposition 4.2. The additive group R[S] is a Z + -graded module over R. Wehave for π R and σ R[S]. π σ = π σ Proof. The relation π σ = π σ follows from part (iii) of the previous proposition since P (n) and t P (n) are associate parabolics ([BnDKa]). We shall describe now the gradations of R[S] and R[S] by infinitesimal characters. The disjoint union ofthe sets ofall cuspidal pairs modulo conjugation, which correspond to Sp(n, F ), n 0, may be identified with the set M(C ) C[S].

14 14MARKO TADIĆ Let ω =(x, σ) M(C ) C[S]. Take (ρ 1,...,ρ n ) M(C) such that κ((ρ 1,...,ρ n )) = x. Let Irr ω [S] ={τ Irr[S]; τ ρ 1 ρ 2... ρ n σ}. The subset ofall π In[S] all ofwhose irreducible subquotients are in Irr ω [S] is denoted by In ω [S]. Let R ω [S] be the subgroup of R[S] generated by Irr ω [S] and let R ω [S] be the subgroup of R[S] generated by In ω [S]. For (x 1,...,x n ) M(C ) and ((y 1,...y m ),σ) M(C ) C[S] set Now (x 1,...,x n )+((y 1,...,y m ),σ)=((x 1,...,x n,y 1,...,y m ),σ). R[S] = R ω [S] M(C ) C[S] is a graded module over R = R ω for both structures. We have an analogous situation M(C ) for R[S]. We consider now the case of GSp(n, F ). Let π R + n and σ R + m[g]. Using the identification (g, h) g h 0,g GL(n, F ),h GSp(m, F ), 0 0 ψ(h) τ g 1 we identify GL(n, F ) GSp(m, F ) with M (n). Let We extend again and Z-bilinearly to We factor also to a Z-bilinear map π σ = Ind GSp(n+m,F ) P (n) (π σ), π σ = Ind GSp(n+m,F ) t P (n) (π σ). R R[G] R[G]. R R[G] R[G]. Now we have an analogue ofpropositions 4.1 and 4.2. Proposition 4.3. (i) For the mapping (resp. ): R R[G] R[G], the additive group R[G] is a Z + -graded module over (R, +, ) (resp. (R, +, )). (ii) Let π R + n and σ R + m[g]. For a character χ of F we have χ(π σ) =π χσ.

15 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 15 Suppose that π has a central character, say ω π. Then π σ = t π 1 ω π σ, (iii) The additive group R[G] is a Z + -graded module over R. Wehave π σ = π ω π σ for π Irr and σ R[G]. (iv) If π R + n and σ R + m[g], then for the restrictions to symplectic groups we have the following equalityin R[S]: (π σ) Sp(n + m, F )=π (σ Sp(m, F )). (v) We identify F with the center of GL(n, F ) using the homomorphism λ λi n. Also, using the homomorphism λ λi 2n, we identify F with the center of GSp(n, F ). Let π i R + n i, n =1,...,k, be representations which have central characters ω πi,i =1,...,k. Let σ R + m[g] be a representation having a central character ω σ.ifm>0, then the central character of π 1 π 2 π k σ is ω π1 ω π2...ω πk ω σ. If m =0, then the central character is ω π1...ω πk ω 2 σ. (vi) Let σ R + m[g] be a representation with a central character, say ω σ. Let χ be a character of F.Ifm 1, then the central character of χσ is χ 2 ω σ. (vii) We have for π R +, σ R + [G] (π σ) = π σ, (π σ) = π σ. 5. Comodules of representations Take an ordered partition α =(n 1,...,n k )ofm and take n m. Identifying (g 1,g 2,...,g k,h),g i GL(n i,f),h Sp(n m, F ),

16 16 MARKO TADIĆ with the matrix... g k g 1 0 h τ g 1 k... 0 τ g 1 1 we identify GL(n 1,F)... GL(n k,f) Sp(n m, F ) with M α Sp(n, F ). Let σ R + n [S]. The Jacquet module for N α is denoted by s α,(0) (σ) =r Sp(n,F ) P α (σ). The Z-linear extension to R n [S] is denoted by s α,(0) again. Let β =(n 1,...,n k ) be an ordered partition of m n. We shall write β α if m m and ifthere exists a subsequence p(1) <p(2) <...<p(k) of{1, 2,...,k } such that n n p(1) = n 1 n p(1) n p(2) = n 2. n p(k 1) n p(k) = n k (for α = (0) we assume that k = 0). The relation is transitive. If β α and σ R(M α ), then we set s β,α (σ) =r M α M β (σ). Now, s β,α are transitive. This means that α 1 α 2 α 3 implies Note that we may identify s α1,α 3 = s α1,α 2 s α2,α 3., M α GL(n 1,F)... GL(n k,f) Sp(n m, F ), where α =(n 1,...,n k ) is a partition of m n. Thus, we have a natural identification We can lift also s β,α to These mappings are transitive again. R(M α ) R n1 R nk R n m [S]. s.s.(s β,α ):R(M α ) R(M β ).

17 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 17 We define now a Z-linear mapping For σ R n [S], the formula is µ (σ) = µ : R[S] R R[S]. n s.s.(s (k),(0) (σ)) k=0 n R k R n k [S]. Note that µ is additive and it is Z + -graded. From the transitivity of s α,β s one obtains that µ is coassociative. This means that the following diagram commutes R[S] k=0 µ R R[S] µ id µ R R[S] m id R R R[S]. The mapping µ is graded with respect to M(C )-gradation on R and M(C ) C[S] gradation on R[S]. In other words µ (R Ω [S]) R ω R Ω [S] ω+ω =Ω where Ω, Ω M(C ) C[S],ω M(C ). We make now necessary modifications for the case of GSp(n, F ). We identify first M α with GL(n 1,F)... GL(n k,f) GSp(n m, F ), when α =(n 1,...,n k ) is a partition of m n. The identification mapping is (g 1,...,g k,h) g gk h ψ(h) τ g 1 k... 0 ψ(h) τ g 1 1 In the same way as above, we introduce s α,(0) and s β,α for GSp(n, F ). Again, s β,α s are transitive. The map µ : R[G] R R[G] is defined sothat acts on σ R n [G] µ (σ) = n s.s.(s (k),(0) )(σ). k=0 This map is additive, Z + -graded and coassociative.

18 18 MARKO TADIĆ 6. Langlands classification Let t =((δ 1,...,δ n ), τ) M(D) T [S]. We shall write t simply as (δ 1,...,δ n,τ). Denote D + = {δ D; e(δ) > 0}. If t =(δ 1,...,δ n,τ) M(D + ) T [S], then we say that t is written in a standard order if e(δ 1 ) e(δ 2 )... e(δ n ) Let t =(δ 1,...,δ n,τ) M(D + ) T [S]. Suppose that it is written in a standard order. The representation δ 1 δ 2... δ n τ R + [S] is uniquely determined by t. This is a consequence ofirreducibility oftempered induction for GL(n)-groups ([Jc] or [Z1]). This representation will be denoted by λ(t). Similarly, the representation δ 1 δ 2... δ n τ is uniquely determined by t. It will denoted by λ(t). Observe that the fourth section implies s.s.(λ(t)) = s.s.(λ(t)). We also have λ(t) = δ 1 δ 2... δ n τ. We shall now describe the Langlands classification in the case of Sp(n)-groups. Let t M(D + ) T [S]. The representation λ(t) has a unique irreducible quotient which we denote by L(t). The mapping t L(t) is a one-to one mapping of M(D + ) T [S] ontoirr[s]. One can describe L(t) in a few different ways. We recall now two such descriptions. The representation λ(t) has a unique irreducible subrepresentation. This subrepresentation is isomorphic to L(t). There exists an integral intertwining operator from λ(t) into λ(t) whose image is L(t) (for the explicit formula one may consult [BlWh]). The multiplicity of L(t) in λ(t) is one. Thus, the multiplicity of L(t) in λ(t) is one. This implies that the intertwining space between λ(t) and λ(t) is one-dimensional. Therefore, ifwe have an intertwining between λ(t) and λ(t) which is injective or surjective, then λ(t) is irreducible. Let t =(δ 1,...,δ n,τ) M(D + ) T [S]. Suppose that it is written in a standard order and suppose that δ i GL(k i,f). Set e (t) =(e(δ 1 ),...,e(δ 1 ),...,e(δ n ),...,e(δ n ), 0, 0,...,0), }{{}}{{}}{{} k 1 times k n times m times

19 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 19 where τ Sp(m, F ). We consider a partial order on R k defined by (x 1,...,x k ) (y 1,...,y k ) x 1 y 1, x 1 + x 2 y 1 + y 2,. x x k y y k. Let t M(D + ) T [S] and let σ be a subquotient of λ(t) different from L(t). Then σ = L(t 1 ) for some t 1 M(D + ) T [S], t 1 t. Moreover then it must hold e (t 1 ) <e (t). We could introduce the order on R k in a different way. Let β 1,...,β n be the basis of R k biorthogonal to the basis (1, 1, 0, 0,...,0), (0, 1, 1, 0,...,0),. (0, 0,...,0, 1, 1, 0), (0, 0,...,0, 0, 1, 1), (0, 0,...,0, 0, 0, 2), where we consider the usual inner product on R k. Simple computation gives β i =(1, 1,...,1, 0, 0,...,0), 1 i n 1, }{{} i times and β n = 1 (1, 1,...,1, 1). 2 It is easy to check that for x, y R k x y (x, β i ) (y, β i ), i =1,...,k. We shall write now the Langlands parameter ofthe contragredient representation. Let t =(δ 1,...,δ n,τ) M(D + ) T [S]. Suppose that it is written in a standard order. Then we have an injective intertwining operator L(t) δ 1 δ 2... δ n τ.

20 20 MARKO TADIĆ Applying Proposition 4.1, one obtains δ 1 δ 2... δ n τ = δ 1 δ 2... δ n τ Passing to contragredients, one obtains a surjective intertwining operator δ 1 δ 2... δ n τ L(t). Since (δ 1,...,δ n, τ) M(D + ) T [S] and since it is written in a standard order, we obtain L((δ 1,...,δ n,τ)) = L((δ 1,...,δ n, τ)). We set t =( δ 1..., δ n, τ) fort =(δ 1,...,δ n,τ) M(D + ) T [S]. Then L(t) = L( t). We shall recall now ofa criterion for square integrability ofrepresentations, according to [Cs]. Let π Sp(n, F ). Let P α be any standard parabolic subgroup being minimal with the property that s α,(0) (π) 0 (all such P αs are associate parabolic subgroups). Write α =(n 1,...n k ), where α is a partition of m n. Let σ be any irreducible subquotient of s α,(0) (π). Then we write. σ = ρ 1 ρ k ρ. All representations ρ 1,...,ρ k and ρ must be cuspidal. Let e (σ) =(e(ρ 1 ),...,e(ρ 1 ),...,e(ρ k ),...,e(ρ k ), 0,...,0 ). }{{}}{{}}{{} n 1 times n k times (n m) times We are able to present now the criteria of[cs]: (i) Suppose that the following conditions hold (e (σ),β n1 ) > 0, (e (σ),β n1 +n 2 ) > 0,. (e (σ),β m ) > 0, for any α and σ as above. Then π is a square integrable representation. (ii) If π is a square integrable representation, then all inequalities of(i) hold for any α and σ as above.

21 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 21 Note that the conditions in (i) are equivalent to the following conditions (e (σ),β 1 ) > 0, (e (σ),β 2 ) > 0,. (e (σ),β n ) > 0, if α (0). Now we describe the case of GSp(n, F )-groups. We write t =((δ 1,...,δ n ),τ) M(D + ) T [G] again simply as t =(δ 1,...,δ n,τ). We say that t =(δ 1,...,δ n,τ) M(D + ) T [G] is written in a standard order if e(δ 1 ) e(δ 2 )... e(δ n ). If t =(δ 1,...,δ n,τ) is written in a standard order, then the representation λ(t) =δ 1 δ n τ R + [G] is uniquely determined by t. The representation δ 1... δ n τ is denoted by λ(t). The representation λ(t) has a unique irreducible quotient. Denote it by L(t). The mapping t L(t) is a one-to-one mapping of M(D + ) T [G] ontoirr[g]. The representation L(t) maybe characterized as the unique irreducible subrepresentation of λ(t). The multiplicity of L(t) in λ(t) is one. There exists an integral intertwining operator from λ(t) into λ(t) whose image is L(t). The intertwining space between λ(t) and λ(t) is one-dimensional. We have L((δ 1,...,δ n,τ)) = L(( δ 1,..., δ n, τ)). We compute the Langlands parameters ofthe contragredient representations in the same way as before. One gets L((δ 1,...,δ n,τ)) = L((δ 1,...,δ n,ω 1 δ 1 If χ is a character of F, then we have χλ((δ 1,...,δ n,τ)) = λ((δ 1,...,δ n,χτ))...ω 1 δ n τ)). and χl((δ 1,...,δ n,τ)) = L((δ 1,...,δ n,χτ)).

22 22 MARKO TADIĆ Remark 6.1. Note that χl((δ 1,...,δ n,τ)) = L((δ 1,...,δ n,τ)) if and onlyif χτ = τ. We consider now the following inner product on R n+1 ((x i ), (y i )) = n n n x i y i +(2x n+1 + x i )(2y n+1 + y i ) i=1 i=1 i=1 Let β 1,...,β n be the basis dual to the basis (1, 1, 0,...,0), (0, 1, 1, 0,...,0), ofthe subspace Then. (0, 0,...,0, 1, 1, 0), (0, 0,...,0, 0, 2, 1), {(x i ) R n+1 ;2x n+1 + β 1 = n x i =0}. i=1 ( 1, 0,...,0, 1 ), 2 β 2 =(1, 1, 0,..., 1), β n 1 = β n =. ( 1, 1,...,1, 0, n 1 2 ( 1, 1,...,1, 1, n 2 For τ T [G], there exists a unique γ R such that the representation unitarizable. Set e 0 (τ) =γ. Let α = (n 1,...,n k ) be a partition of m n. Take δ i D + GL(n i,f) and τ T (GSp(n m, F )). Suppose that t =(δ 1,...,δ k,τ) is in a standard order. Set e (t) =(e(δ 1 ),...,e(δ 1 ),...,e(δ k ),...,e(δ k ), 0,...,0,e }{{}}{{}}{{} 0 (τ)) n 1 times n k times (n m) times ). ), e (t) =(e(δ 1 ),...,e(δ 1 ),...,e(δ k ),...,e(δ k ), 0,...,0 ) }{{}}{{}}{{} n 1 times n k times (n m) times γ F τ is

23 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 23 Suppose that for t, t 1 M(D + ) T [G], L(t 1 ) is a subquotient of λ(t) and suppose that t t 1. Then (e (t 1 ),β 1) (e (t),β 1),. (e (t 1 ),β n) (e (t),β n), and the strict inequality holds for at least one index 1 i n. Ife (t) =(x i ), e (t 1 )=(y i ), then the above condition becomes y 1 x 1, y 1 + y 2 x 1 + x 2,. y y n x x n. The strict inequality holds again for at least one case. We are going to repeat the criterion for square integrability, in the case of GSp(n, F ). Fix π GSp(n, F ). Take a standard parabolic subgroup P α such that it is minimal among standard parabolic subgroups which satisfy s α,(0) (π) 0. Recall that all such P α s are associate. Assume that α =(n 1,...,n k ) is a partition of m n. Take any irreducible subquotient σ of s α,(0) (π). Write it as σ = ρ 1... ρ k ρ. Set e (σ) =(e(ρ 1 ),...,e(ρ 1 ),...,e(ρ k ),...,e(ρ k ), 0,...,0,e }{{}}{{}}{{} 0 (ρ)), n 1 times n k times (n m) times e (σ) =(e(ρ 1 ),...,e(ρ 1 ),...,e(ρ k ),...,e(ρ k ), 0,...,0 ) }{{}}{{}}{{} n 1 times n k times (n m) times Suppose that we have for any α and σ as above (e (σ),β n 1 ) > 0, (e (σ),β n 1 +n 2 ) > 0,. (e (σ),β m) > 0. Ifthe central character ofπ is unitary, then π is a square integrable representation. For an irreducible representation with a unitary central character to be square integrable the

24 24MARKO TADIĆ above conditions are also necessary.the above inequalities are equivalent to the following inequalities (e (σ),β n1 ) > 0, (e (σ),β n1 +n 2 ) > 0,. (e (σ),β m ) > 0 (we work in R n with the standard inner product). At the end ofthis section we relate Langlands classifications for GSp and Sp groups. Lemma 6.2. Let t =(δ 1,...,δ n,τ) M(D + ) T [G]. Suppose that L(t) is a representation of GSp(p, F ) and suppose that τ is a representation of GSp(q, F). We decompose τ Sp(q, F) =τ τ k into a direct sum of irreducible representations of Sp(q, F). Then τ 1,...,τ k are tempered representations of Sp(q, F). Denote t i =(δ 1,...,δ n,τ i ) M(D + ) T [S] Suppose that τ i τ j for i j. Then we have a direct sum decomposition L(t) Sp(p, F )=L(t 1 )+ + L(t k ). In particular, L(t) Sp(p, F ) is a multiplicityone representation. Proof. We may assume that t =(δ 1,...,δ n,τ) is written in a standard order. Now L(t) is the unique irreducible subrepresentation of λ(t). We have a direct sum λ(t) Sp(p, F )=λ(t 1 )+ + λ(t k ). Let U be an irreducible subrepresentation of λ(t) Sp(p, F ). Then for each i we consider the composition U λ(t 1 )+ + λ(t k ) λ(t i ). The image of U is contained in L(t i ). Thus U L(t 1 )+L(t 2 )+ + L(t k ). In particular, since L(t) Sp(p, F ) is a sum ofirreducible representations (see [Si1]), we have L(t) Sp(p, F ) L(t 1 )+ + L(t k ). Note that L(t i ) are not equivalent for different i. Since U is irreducible, we see that U = L(t i0 ) for a unique 1 i 0 k. Thus L(t) Sp(p, F )= L(t i ) i X for a unique X {1, 2,...,k}. Note that GSp(p, F ) acts by conjugation on L(t i ) s and {L(t i ),i X} is invariant for this action. Now L(t i ) is again invariant. Thus L(t i ) i X i X is a GSp(p, F )-subrepresentation. Therefore if X {1,..., k}, then L(t i ) has an i X irreducible GSp(p, F )-subrepresentation V L(t). Both representations are irreducible subrepresentations of λ(t). We obtained a contradiction. Thus X = {1, 2,..., k} and this completes the proof.

25 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 25 Lemma 6.3. Let t =(δ 1,...,δ n,τ) M(D + ) T [G]. Suppose that L(t) is a representation of GSp(p, F ) and that τ is a representation of GSp(q, F). We always have a following decomposition k τ Sp(q, F) =m in R[S], where each τ i is irreducible, and, for different i s, theyare inequivalent. Let i=1 t i =(δ 1,...,δ n,τ i ). Then L(t i ) are not equivalent for different i s and we have L(t) Sp(p, F )=m k i=1 τ i L(t i ) in R[S]. Proof. Let τ Sp(q, F) =τ τ mk be a decomposition into a direct sum ofirreducible representations. Suppose that U is an irreducible subrepresentation of λ(t) Sp(p, F ). Let t i =(δ 1,...,δ n,τ i). As in the proofofthe preceding lemma, we find that U L(t 1)+ + L(t mk). Thus U = L(t i ) for some 1 i k. Denote by π the action of GSp(p, F )onλ(t). Note that GSp(p, F ) acts transitively on {π(g)u; g GSp(p, F )}. Thus V = span {π(g)u; g GSp(p, F )} is GSp(p, F )-invariant. It is offinite length. Also, as an Sp(p, F )-representation, it is completely reducible. There exists an irreducible GSp(p, F )-subrepresentation V 1 of V.ThusV 1 = L(t). Now U = L(t i ) is isomorphic to a subquotient of V 1 as a representation of Sp(p, F ). Since U was arbitrary (we could take any L(t i )foru, since λ(t) Sp(p, F )=λ(t 1)+ + λ(t mk )), we see that each L(t i) appears as a subrepresentation of L(t) Sp(p, F ). We have proved that in R[S] wehave L(t) Sp(p, F ) = m k i=1 L(t i ). Nowwehave m 2 k = card {χ (F ) ; χτ = τ} = card {χ (F ) ; χl(t) =L(t)} =(m ) 2 k. ([GeKn], Lemma 2.1.). Thus m = m. This finishes the proofofthe lemma.

26 26 MARKO TADIĆ 7. Non-unitary principal series representations The trivial one-dimensional representation ofa group G will be denoted by 1 G.IfGis the trivial group, then we shall denote 1 G simply by 1. Let χ 1,...,χ n (F ) (resp. (F ) ). The representations χ 1... χ n 1 will be called the unitary principal series representations (resp. the non-unitary principal series representations) of Sp(n, F ). By Theorem 1 ofn. Winarsky s paper [Wi], a unitary principal series representation χ 1... χ n 1 is reducible ifand only ifthere exists a character χ i,1 i n, whose order is two (ifsuch character exists then it is clear from SL(2)-case that χ 1... χ n 1 is reducible). One can obtain a more precise information about the reducible unitary principal series representations from paper [Ke] of D.Keys. First, a unitary principal series representation χ 1... χ n 1 is a multiplicity one representation, by Theorem C n of[ke]. The computation ofr-group in the proofof Theorem C n of[ke] gives that the length ofχ 1...χ n 1 equals 2 to the number of different characters oforder 2 among χ 1,...,χ n. We can easily determine the reducibility points for the non-unitary principal series representations of Sp(n, F ). Theorem 7.1. Let χ 1,...,χ n (F ). Consider the following three conditions: (i) For any 1 i n, χ i is not of order 2. (ii) For any 1 i n, χ i ν ±1. (iii) For any 1 i<j n, χ i ν ±1 χ ±1 j (all possible combinations of two signs are allowed). The non-unitaryprincipal series representation χ 1... χ n 1 of Sp(n, F ) is irreducible if and onlyif conditions (i), (ii) and (iii) hold. Proof. Suppose that condition (i) or (ii) is not satisfied for some χ i. Then we know from the representation theory of SL(2,F) that χ i 1 is reducible. Write χ i 1=π 1 + π 2 in R[S], where π 1,π 2 R[S], π 1 > 0, π 2 > 0. By the fourth section we have the following equalities in R[S]: χ 1... χ i... χ n 1= χ 1... χ i 1 χ i+1... χ n χ i 1= χ 1... χ i 1 χ i+1... χ n π 1 + χ 1... χ i 1 χ i+1... χ n π 2. Thus, χ 1... χ n 1 is reducible. Suppose that (iii) does not hold for some 1 i<j n. Write χ i = ν ɛ 1 χ ɛ 2 j where ɛ 1,ɛ 2 {±1}. Note that in this situation we know that in R we have χ i χ ɛ 2 j = π 1 + π 2 where π 1,π 2 R, π 1 > 0, π 2 > 0. We have the following equalities in R[S]: χ 1... χ i... χ j... χ n 1=

27 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 27 χ 1... χ i... χ ɛ 2 j... χ n 1= χ 1... χ i χ ɛ 2 j χ i+1... χ j 1 χ j+1... χ n 1= χ 1... χ i 1 π 1 χ i+1... χ j 1 χ j+1... χ n 1+ χ 1... χ i 1 π 2 χ i+1... χ j 1 χ j+1... χ n 1. The final result shows that χ 1... χ n 1 is reducible. We have proved that conditions (i), (ii) and (iii) are necessary for irreducibility. Now we shall suppose that conditions (i), (ii) and (iii) hold. Note that condition (iii) implies the following condition which will be denoted (iii) : for any 1 i j n, χ ±1 i ν ±1 χ ±1 j. We want to prove irreducibility of χ 1... χ n 1. For any ɛ 1,...,ɛ n {±1} we have by the fourth section that χ 1 χ 2...χ n 1=χ ɛ χ ɛ n n 1 in R[S]. It is enough now to prove irreducibility of χ ɛ 1... χ ɛ m 1 for some ɛ i s as above. Note that χ ɛ 1 1,...,χ ɛ n n again satisfy conditions (i), (ii) and (iii). With a suitable choice of ɛ is, we can get that e(χ i ) 0, 1 i n. We shall assume this in the rest ofthe proof. For any permutation p of {1, 2,...,n} we have equality χ 1... χ n 1=χ p(1)... χ p(n) 1 in R[S]. Therefore, it is enough to prove irreducibility of χ p(1)... χ p(n) 1 for some p. With a suitable choice of p, we can assume that e(χ 1 ) e(χ 2 )... e(χ n ) 0. We introduce j {0, 1, 2,..., n 1,n} in the following way. If all e(χ i ) > 0 (resp. all e(χ i ) = 0), then we set j = n (resp. j = 0). Suppose that there exist χ k and χ l such that e(χ k ) > 0 and e(χ l ) = 0. Then, one denotes by j the index which satisfies e(χ j ) > 0 and e(χ j+1 ) = 0 (such an index must exist in this situation). Set τ = χ j+1... χ n 1 (if j = n, then we take τ = 1). Since we do not have characters oforder 2 and since τ is a unitary principal series representations, the representation τ is irreducible. Clearly, τ is a tempered representation. Set t =(χ 1,...,χ j,τ). Then t M(D + ) T [S]. Also, t =(χ 1,...,χ j,τ) is written in a standard order. In R[S] we have χ 1... χ n 1=χ 1... χ j (χ j+1... χ n 1) = χ 1... χ j τ = λ(t).

28 28 MARKO TADIĆ Recall that µ 1 µ 2 is irreducible ifand only ifµ 1 µ 1 2 ν ±1,µ 1,µ 2 (F ).Ifµ 1 µ 2 is irreducible, then we have in R Also, if µ 1 1 is irreducible, then we have µ 1 µ 2 = µ 2 µ 1. µ 1 1=µ 1 1 1=µ 1 1. in R[S]. The condition (iii) implies that χ i χ k is irreducible for all i, k. Therefore we have the following sequence of equalities in R[S]: λ(t) =(χ 1 χ 2 )... χ n 1=(χ 2 χ 1 ) χ 3... χ n 1=... = χ 2 χ 3 χ 4... χ n χ 1 1= = χ 2... χ n (χ 1 1 1) by (i) and (ii). Furthermore, we have in R[S] λ(t) =χ 2... χ n 1 (χ 1 1 χ n ) 1=... =(χ 1 1 χ 2 )... χ n 1. Continuing the procedure with χ 2 and then with χ 3,...,χ j, we obtain that in R[S] we have λ(t) =χ χ 1 j χ j+1... χ n 1=χ 1 1 χ χ 1 j τ = λ(t) By the sixth section λ(t) is irreducible. This finishes the proofofthe theorem. We study now the case of GSp(n, F ). Let χ 1,...χ n,χ (F ) (resp. (F ) ). The representation χ 1... χ n χ is called the unitary principal series representation (resp. the non-unitary principal series representation) of GSp(n, F ). Recall that (χ 1 χ 2... χ n χ) Sp(n, F )=χ 1 χ 2... χ n 1 We shall consider now unitary principal series representations of GSp(n, F ). Lemma 7.2. Let χ 1,...,χ n,χ (F ). Decompose χ 1 χ 2... χ n χ = σ 1... σ m into a direct sum of irreducible representations of GSp(n, F ). Then σ i ψσ j for any ψ (F ) and any 1 i j n. In particular, χ 1... χ n χ is a multiplicityone representation. Proof. Suppose that σ i = ψσ j for some ψ and i j. Then σ i Sp(n, F )=σ j Sp(n, F ) since ψ is trivial on Sp(n, F ). This implies that χ 1... χ n 1 is not a multiplicity one representation of Sp(n, F ) which is a contradiction. For σ R + [G] set X Sp(n,F ) (σ) ={χ (F ) ; χσ = σ} This is clearly a subgroup of(f ). We shall study this group in more detail.

29 REPRESENTATIONS OF p-adic SYMPLECTIC GROUPS 29 Lemma 7.3. Let χ 1,...,χ n,χ (F ). For an irreducible subquotient σ of χ 1... χ n χ we have X Sp(n,F ) (σ) X Sp(n,F ) (χ 1 χ 2... χ n χ). If χ 1 χ 2... χ n 1 is a multiplicityone representation of Sp(n, F ), then the equality holds. In particular, if χ 1,...,χ n,χ are all unitarycharacters, then the equalityholds. Proof. Suppose that ψσ = σ. Then σ = ψσ is a composition factor of ψ(χ 1... χ n χ) = χ 1... χ n ψχ which is again a non-unitary principal series representation. Since χ 1... χ n χ and ψ(χ 1... χ n χ) have a common composition factor, they have the same Jorden-Hölder sequences ([BnZ2], Theorem 2.9.). This implies that they are equal in R[G]. Thus ψ X Sp(n,F ) (χ 1... χ n χ). Suppose now that ψ X Sp(n,F ) (χ 1... χ n χ). Decompose χ 1... χ n χ = σ 1 σ m into a direct sum ofirreducible representations. Suppose that χ 1 χ 2... χ n 1isa multiplicity one representation. Consider σ 1... σ m = ψ(σ 1 σ m )=ψσ 1 ψσ n. Now the multiplicity one condition on χ 1... χ n 1 and the simple argument from the proofofthe preceding lemma, imply σ i = ψσ i, for all 1 i m. Thus, ψ X Sp(n,F ) (σ i ) for any 1 i m. Lemma 7.4. If χ 1,...,χ n,χ (F ), then the group X Sp(n,F ) (χ 1... χ n χ) is equal to the subgroup of (F ) generated byall characters of order two in the set {χ 1,...,χ n }. Proof. Since χ(χ 1... χ n 1 F )=χ 1... χ n χ, wehave X Sp(n,F ) (χ 1... χ n χ) =X Sp(n,F ) (χ 1... χ n 1 F ). For any permutation p of {1,...,n} and for ɛ i {±1} the following equalities hold in R[G]: χ 1... χ n χ = χ p(1)... χ p(n) χ and χ 1... χ n χ = χ ɛ 1 1 χ ɛ 2... χ ɛ n n ( n i=1 χ (1 ɛ i)/2 i ) χ. By the above remarks, it is enough to prove the lemma in the following situation: the sequence χ 1,...,χ n,χ equals ϕ 1,...,ϕ }{{} 1,ϕ 2,...,ϕ 2,...,ϕ }{{} k,...,ϕ k, 1 }{{} F n 1 times n 2 times n k times

30 30 MARKO TADIĆ where ϕ i ϕ j and ϕ i ϕ 1 j for any i j, 1 i, j k. Note that the Weyl group of GSp(n, F ) (and of Sp(n, F )) is isomorphic to the semidirect product ofthe group ofpermutations ofn elements and {±1} n. The second factor is normal. If p is a permutation of {1,...,n}, then p acts as χ 1... χ n χ χ p 1 (1)... χ p 1 (n) χ. If ɛ =(ɛ i ) 1 i n is a sequence in {±1}, then ɛ acts in the following way ( χ 1... χ n χ χ ɛ χ ɛ n n χ χ (1 ɛ i)/2 i Let X be the subgroup of(f ) generated by all characters oforder two among ϕ 1,...,ϕ k. To prove the lemma we need to prove that X = X Sp(n,F ) (χ 1... χ n χ). Suppose that ϕ X. Then ϕ(χ 1... χ n χ) =χ 1... χ n ϕχ. By the Frobenius reciprocity the last representation has χ 1... χ n ϕχ for a subquotient (in fact, for a quotient) ofthe Jacquet module for the standard minimal parabolic subgroup. Note that χ 1... χ n ϕχ is in the same orbit ofthe Weyl group by the above remark about the action ofthe Weyl group. This implies that we have in R[G] χ 1... χ n χ = ϕ(χ 1... χ n χ). Thus ϕ X Sp(n,F ) (χ 1... χ n χ). This proves X X Sp(n,F ) (χ 1... χ n χ). Suppose that ϕ X Sp(n,F ) (χ 1... χ n χ). Then i.e. ϕ(χ 1... χ n χ) =χ 1... χ n χ, χ 1... χ n ϕχ = χ 1... χ n χ. Note again that χ 1... χ n ϕχ is a subquotient ofthe Jacquet module ofχ 1... χ n ϕχ for the standard minimal parabolic, while χ,... χ n χ is a subquotient ofthe Jacquet module of χ 1... χ n χ for the standard minimal parabolic also. The equality of representations in R[G] implies that χ 1... χ n ϕχ is in the Weyl group orbit of χ 1... χ n χ. Take an element w ofthe Weyl group such that χ 1... χ n ϕχ = w(χ 1... χ n χ). Write w = ɛp 1 where ɛ =(ɛ i ) 1 i n is a sequence in {±1}, and p is a permutation of {1, 2,...,n}. Thus χ 1... χ n ϕχ = χ ɛ 1 p(1)... χ ɛ n p(n) χ n i=1 ). χ (1 ɛ i)/2 p(i). The condition that ϕ i ϕ 1 j and ϕ i ϕ j for any i j, and χ 1... χ n = χ ɛ 1 p(1)... χ ɛ n p(n),