Unramified computation of tensor L-functions on symplectic groups

Transcription

1 Unramified computation of tensor L-functions on symplectic groups A DISSERTATION SUBMITTED TO THE FACULTY OF THE GRADUATE SCHOOL OF THE UNIVERSITY OF MINNESOTA BY Xin Shen IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Prof. Dihua Jiang June, 2013

2 c Xin Shen 2013 ALL RIGHTS RESERVED

3 Acknowledgements First of all, I would like to thank my adviser, Prof. Dihua Jiang. For the past six years he has supported me consistently, and has taught me a lot through our meetings almost every week. He shared his knowledge and professional experience without any hesitation, and patiently led me to the world of automorphic forms. He always believes that I will succeed in mathematics, which strengthens my confidence whenever I have difficulties in my career. He suggested this thesis project, and spent much time directing my research. I would also like to thank my academic brother Lei Zhang. He taught me much of what I know about automorphic forms and representation theory. He encouraged me when I was frustrated during my third year. He never says no when I have questions to discuss with him. Thanks to Prof. David Ginzburg for giving me advice on the unramified calculation of the tensor L-function in this thesis. Prof. James Cogdell gave me advice and support when I most needed it. Prof. Solomon Friedberg, Prof. Wee Teck Gan and Prof. Yiannis Sakellaridis made helpful comments on this paper. Thanks to Prof. Benjamin Brubaker, Prof. Kai-wen Lan and Prof. Richard McGehee for reviewing my thesis and serving on my thesis defense committee. Last but not least, it is a great pleasure to study mathematics with my other academic brother Baiying Liu. i

4 Abstract Tensor L-function is one of the important cases in the Langlands conjecture on the analytic properties of L-functions. Using the method of Rankin-Selberg convolution, Ginzburg, Jiang, Rallis and Soudry in [15] found an integral representation of the tensor L-functions for symplectic groups with non-generic representations. In this thesis we calculated the local integrals at the unramified places. First we gave a formula for the Whittaker-Shintani functions for symplectic groups, which is a generalization of the Casselman-Shalika formula for the Whittaker function in the generic case. Then we applied our formula and carried out the unramified calculation. We also investigated the local integrals at the non-archimedean, possibly ramified places and obtain some basic properties, such as convergence, rationalities, and non-vanishing of the local integrals for any given complex number s. ii

5 Contents Acknowledgements Abstract i ii 1 Introduction Introduction Automorphic forms and automorphic representations Langlands L-function L-function machine Tensor L-function Metaplectic groups Local metaplectic groups and Weil representation Global metaplectic groups Local L-function on the metaplectic groups Integral representation using Fourier-Jacobi models Whittaker-Shintani functions Introduction Notation Integral expression for the pairing Uniqueness of the pairing for generic (χ, ξ) Rationality(I) Double coset decomposition iii

6 2.7 Vectors invariant under certain open compact subgroups The I M -Invariant vectors in Ind MJ B (ξ, ψ) M J The I G -invariant vectors in Ind G B G (χ) γ-factor The calculation of γ(χ, ξ, w α, 1) Calculation of γ(χ, ξ, 1, w β ) Formula for generic (χ, ξ) Normalization Uniqueness of the Whittaker Shintani function The formula for the normalized Whittaker-Shintani function Application Local theory Introduction Notation Two transformation of the local integral First transformation Second transformation Unramified Computation Estimation Non-vanishing theorem References 112 iv

7 Chapter 1 Introduction 1.1 Introduction Given an irreducible cuspidal representation, it is a restricted tensor product of local representations. Almost all of them are unramified. Following Langlands, one can define the local L-function at all unramified places by their Satake parameters. The product of these L-functions is called a partial L-function. Around 1970 Langlands conjectured that one can define local L-functions at all other places, so that we can define a complete L-function as the product of local L-functions at all places, and each complete L-function should satisfy nice analytic properties. There are two main approaches toward this conjecture. One is via the integral representation, usually referred to as the Rankin-Selberg method. One is via the Fourier coefficient of Eisenstein series, usually referred to as Langlands-Shahidi method. Roughly speaking one first constructs an integral over a group over the Adele ring of a given global field, which we call the global integral. Assume it is a meromorphic function on C. Then we unfold the global integral to an Euler product of local integrals, that is, integrals over the groups defined over local fields. At almost every place, the local integral is unramified, i.e., all the vectors in the integral are fixed by some maximal compact subgroups. If one can equate the unramified integrals to the local L-functions at the unramified places, 1

8 2 (we call this step unramified calculation ), then it is expected that one could use the local integrals at other places to define local L-functions too. Then the analytic properties of complete L-function can be obtained from those of the global integral. So the unramified calculation is a criterion for finding the right global integral for the desired partial L-functions. One of the important cases of the Langlands conjecture is the tensor L- functions for G GL n where G is a classical group. Assume that we know enough analytic properties of them, then combining with the Converse theorem by Cogdell and Piatetski-Shapiro [8], the lifting from cuspidal representations of G to those of general linear groups is implied, in accordance with the Langlands functoriality [20]. In [15], Ginzburg, Jiang, Rallis and Soudry used the Fourier-Jacobi models to construct integrals for the tensor L-functions for Sp 2n GL k. The result of the unramified calculation is stated in [15, Theorem 4.3]. The purpose of this thesis is to give an explicit proof of this theorem with certain restriction on k and n. The paper is organized as follows. In Chapter 1, we will recall the statement of the Langlands conjecture, some definitions of related concepts, and the integral constructed in [15]. In Chapter 2 and 3 we will cover the two main steps in the unramified calculation. In Chapter 2 we will find an explicit formula for the unique pairing occurring at the unramified local integrals, which are called the Whittaker-Shintani functions. In Chapter 3 we will use this formula to compute the local integrals at the unramified places. In Chapter 3 we will also discuss some basic properties of the local integrals, including the convergence, meromorphic continuation, and non-vanishing of the local integrals at non-archimedean places. These properties are early steps of defining the local L-functions at ramified non-archimedean places.

9 3 1.2 Automorphic forms and automorphic representations In this section we recall some basic facts about automorphic forms and automorphic representations following [5] and lecture 2 and 3 of [9]. Let G be a split reductive algebraic group over a number field F. Let A be the Adele ring of F. We let G = v G(F v ) and g be the Lie algebra of G (viewed as a real group). For finite places v, let O v be the maximal compact subring of F v, and let K v = G(O v ). Let G(A f ) = v finiteg(f v ), where v finiteg(f v ) = { v< g v g v K v for almost every v}. Let G(A) = G G(A f ). A function f : G(A) C is smooth if f(x; y), where x G and y G(A f ), is a C function in x and locally constant function in y. Definition Fix a maximal compact subgroup K of G. A smooth function f on G(A) is an automorphic form on G(A) if 1. f is left G(F )-invariant. 2. f is right K-finite, where K = K v finite K v. 3. f is Z-finite, where Z is the center of the enveloping algebra of g. 4. f is slowly increasing, i.e., there exists a positive integer n and a constant C such that f(x) C x n for all x G(A). An automorphic form f is called an cuspidal automorphic form, or simply a cusp form, if for any proper F -parabolic subgroup P of G, with N being its unipotent radical, we have N(F )\N(A) dnf(ng) = 0 (1.1) for any g G(A). Now we define the Hecke algebra of G(A). Let U(g ) be the enveloping algebra of g, and A K the algebra of finite measure on K. Let k be the Lie

10 4 algebra of K. We define H = U(g ) U(k) A K. For each finite place v, we let H v be the convolution algebra of smooth and compactly supported functions on G(F v ). We let H f = v finiteh v, where the restricted tensor product means that v< h v v finiteh v if and only if h v is the characteristic function of K v for almost every v. The Hecke algebra of G(A) is defined as H = H H f. Definition An irreducible representation of H is automorphic (resp. cuspidal) if it is isomorphic to a sub-quotient of a representation of H in the space of automorphic (resp. cusp) forms on G(A). Remark Since the right translation of G does not preserve K -finiteness, automorphic representations are not G(A)-modules, but always H G(A f )- modules. Now we define another set of automorphic forms, called the smooth automorphic forms. It contains all the automorphic forms we define above, and they form a G(A)-module. Definition A smooth function ϕ on G(A) is called a smooth automorphic form if 1. it is left G(F )-invariant, 2. it is right K f -finite, 3. it is Z-finite, 4. there exists a positive integer r such that for all differential operators X U(g), Xϕ(g) C X g r If moreover ϕ satisfies (1.1), then it is called a smooth cusp form. The space of smooth automorphic forms is denoted by A (G(F )\G(A)). Similarly we can define smooth automorphic representations.

11 5 Definition A smooth representation of G(A) is automorphic if it is a closed irreducible sub-quotient of A (G(F )\G(A)). Remark Since the action of H or G(A) preserves cuspidality, we can define cuspidal representations as closed sub-quotients of cusp forms in a similar way. However all cusp forms are rapidly decreasing, from this one can deduce that any sub-quotients of the space of cusp forms are actually sub-representations. 1.3 Langlands L-function From now on we only consider smooth automorphic forms and smooth automorphic representations, which are G(A)-modules. Let π be an irreducible cuspidal representation of G(A). Then we have the following decomposition theorem. Theorem (Flath [10]). If (π, V ) is a smooth automorphic representation of G(A), then there exist irreducible admissible smooth representations (π v, V v ) of G(F v ), which are smooth Frechet representations of moderate growth if v, such that π = π π f where π = v π v is the topological tensor product of smooth Frechet representations and π f = v< π v is the restricted tensor product of smooth representations of G(F v ). Here the restricted product is defined as follows. Let V v be the representation space of π v. For almost every finite place v, we fix an element x 0 v V v which is nonzero and fixed by K v. Then π v is spanned by x v, where x v V v for all v, and x v = x 0 v for almost every v. By this theorem, for almost every finite v, π v contains a nonzero K v -invariant element. We call them unramified (or spherical) representations, and the K v - invariant elements are called unramified (or spherical) vectors. One can associate

12 6 a complex analytic group L G to G, which is called the L-group of G. Then by the Satake isomorphism each unramified representation π v can be associated with a conjugacy class z v in L G. Take r be a representation of L G of dimension n. Then the local Langlands L-function at an unramified place v is defined as L v (s, π, r) = det(i r(z v )q s v ) 1. Here q v is the residue cardinality of F v. Let S be a finite set of places such that all v / S are finite and unramified. Then we define the partial Langlands L-function as L S (s, π, r) = v / S L v (s, π, r). Theorem (Langlands). The partial Langlands L-function L S (s, π, r) is absolutely convergent for Re(s) >> 0. Conjecture (Langlands). One can define L v (s, π, r) at all other places, so the complete L-function L(s, π, r) = v L v (s, π, r) has good analytic properties, including meromorphic continuation to the whole plane, and a functional equation relating its value at s to 1 s. 1.4 L-function machine Following [13] we use the term L-function machine for the five main steps toward the analytic properties of Langlands L-functions. Step 1 establishing a global zeta integral, plus an Euler product of local integrals. Step 2 analyzing the meromorphic behavior of the global integral, and its function equation. Step 3 equating the unramified local integrals to L-functions desired. Step 4 analyzing the meromorphic behavior of the local integrals, and their function equations.

13 7 Step 5 establishing basic properties of the local zeta-integrals, namely the definition, existence, and properties of the greatest common divisors L v (s, π, r) of the local integrals. (Step 4 and 5 are called local theory ) Remark For the tensor L-functions for Sp 2n GL k, step 1 and 2 are completed in [15] where the formula for step 3 is given in theorem Tensor L-function A Langlands L-function L(s, π, r) is called a tensor L-function when π = π 1 π 2 is the cuspidal representation of H GL n, where H is a split classical group, and when r is the standard representation of L H L GL n. We simply denote it by L(s, π 1 π 2 ). When H = GL m, the L-function L(s, π 1 π 2 ) is well studied by Jacquet, Piatetski-Shapiro and Shalika in [17]. One of the important ingredients in the construction of the global integral for the L-function of GL n GL m is the Whittaker model. For a representation (τ, V ) of GL n (over local fields or Adele ring), the Whittaker functional is defined as follows. Let N be the unipotent of the Borel subgroup B of GL n, and let ψ N be a generic character on N. A Whittaker functional on (τ, V ) is a functional Λ on V satisfying Λ(τ(n)ξ) = ψ N (n)λ(ξ) for any n N. Whittaker function of ξ. For any ξ V and g GL n, let W ξ (g) = Λ(τ(g)ξ) be the (τ, V ) to Ind GLn N (ψ N ). By Frobenius reciprocity law The map ξ W ξ (g) is a GL n -homomorphism from Hom GLn (τ, Ind GLn N (ψ)) = Hom N (τ, ψ), the existence of non-trivial Whittaker function is equivalent to that of a nontrivial Whittaker functional. In this case we call τ generic. We have

14 8 Theorem (Piatetski-Shapiro [25], Shalika [27]). Every cuspidal automorphic representation of GL m (A) is generic. The space of functions W ξ (g) is called the Whittaker model of τ. On the other hand, we have Theorem (Gelfand and Kazhdan [14], Shalika [27]). If π = vπ v is an irreducible cuspidal representation of GL(A), then the dimension of Whittaker model on π is at most 1. The existence and uniqueness of the Whittaker model leads to the decomposition of the global integral into an Euler product. At almost every place one needs to calculate the unramified local integrals, where the Casselman-Shalika formula for unramified Whittaker functions is crucial. For a representation of a split algebraic group one can define the Whittaker model in a similar way. Now take H = Sp 2r, and consider the L-function L(s, π 1 π 2 ), where π 1 (resp. π 2 ) is the cuspidal automorphic representation of H (resp. GL n ). When π 1 is generic, the construction of L(s, π 1 π 2 ) is first given by Gelbart and Piatetski- Shapiro in [12] when r = n, and later by Ginzburg, Rallis and Soudry in [16] for general r. The case when π 1 is non-generic is more complicated. Ginzburg, Jiang, Rallis and Soudry gives a global zeta integral in [15]. The Whittaker models used in the generic case is replaced by the Fourier-Jacobi model, which is a pairing between two representations. By the work of Sun [30] and Liu and Sun [21], such pairing is unique locally up to a scalar, which implies that one can decompose the global integral into an Euler product. For the unramified calculation of local integrals, it is expected in [15] that they are equal to the tensor L-function L(s, π 1 π 2 ) divided by certain normalizers of the Eisenstein series. An explicit proof will be given in chapter 2 and 3 in this thesis.

15 1.6 Metaplectic groups Local metaplectic groups and Weil representation Let F be a local field with characteristic 0. Let W be a vector space over F with a non-degenerate anti-symmetric form,. Take X = {e 1,..., e n } and Y = {e 1,..., e n} be a polarization of W such that e i, e j = δ j i. Let H 2n+1 be a Heisenberg group over F of 2n+1 variables. It is the group consisting of elements in X Y F with the group structure (x 1, y 1, z 1 ) (x 2, y 2, z 2 ) = (x 1 + x 2, y 1 + y 2, z 1 + z 2 + x 1, y 2 x 2, y 1 ) 1 x y z 1 y It can also be realized as (x, y, z) = 1 x. Here x = (x 1,..., x n ) F n 1 corresponds to the element x i e i X and y = (y 1,..., y n ) F n corresponds to yi e n+1 i Y, and z F and x = t (x n..., x 1 ), y = t (y n,..., y 1 ). The action of Sp(W ) on H 2n+1 (defined by trivial extension of the action of Sp(W ) on W to W F ) is compatible with the conjugation of the matrix expression of Sp(W ) with respect to the basis {e 1,..., e n, e n,..., e 1} on the matrix expression of H 2n+1. Theorem (Stone-von Neumann theorem). For each non-trivial additive character ψ on F, there is a unique representation (up to isomorphism) of H 2n+1 such that the center acts as ψ. The group Sp(W ) normalizes H 2n+1 and stabilizes the center. So by the Stone- Von Neumann theorem, for each g Sp(W ) there is an automorphism A(g) on the space of the Weil representation, such that A(g) 1 w ψ (h)a(g) = w ψ (h g ) for any h H 2n+1. This gives a projective representation of Sp(W ). It can be lifted to a representation of the double cover Sp(W ), which is also called a Weil representation of Sp(W ).

16 10 When F C, Sp(W ) (note that W is defined over F ) has a unique nontrivial double cover, which we denote by Sp(W ). One can realize Sp(W ) explicitly by using the Rao normalized cocycle c W (, ) : Sp(W ) Sp(W ) {±1} ([26, Lemma 5.1]). Precisely, Sp(W ) = {(g, ɛ) Sp(W ) {±1}} with the group law (g 1, ɛ 1 ) (g 2, ɛ 2 ) = (g 1 g 2, c W (g 1, g 2 )ɛ 1 ɛ 2 ). For F being non-archimedean with characteristic 0, let w ψ be the Weil representation of Sp(W )(F ) H 2n+1 (F ) with respect to the non-trivial additive character ψ. It is realized on the space of Schwartz functions on X. Below are the formulas we will need. w ψ (x, 0, 0)φ(x 0 ) = φ(x 0 + x), w ψ (0, y, z)φ(x 0 ) = ψ(z + x 0 y )φ(x 0 ), w ψ a (φ)(x) = γ ψ (det a) det a 1 a 2 φ(ax), I n b (φ)(x) = ψ( x, bx )φ(x). w ψ I n Here γ ψ is the Weil index on F /(F ) 2 associated to the second degree character ψ(x 2 ). It satisfies γ ψ (det a)γ ψ (det b) = γ ψ (det ab)c W ( a Global metaplectic groups, a 2 a 1 a 2 ) (1.2) In this subsection let F be a number field, and F v its completion. Suppose W is a vector space over F with a anti-symmetric form, an let W v be its completion

17 11 at v. We know that the definition of Sp(W ) A is the restricted tensor product of Sp(W v ) with respect to the maximal compact subgroups at each place. To define the double covering Sp(A), one needs the following fact. Lemma (page 43, [22]). For each non-archimedean F v with odd residual characteristic, the maximal compact subgroup K v of Sp(W v ) is split in Sp(F v ), i.e., there exists a group homomorphism K v Sp(W v ), k v (k v, ɛ v (k v )). So we can define Ŝp(W ) A as the restricted tensor product of Sp(F v ) with respect to {K v v is finite and odd }. Let C be the subgroup { v(i, ɛ v ) v ɛ v = 1}. Then Sp(W ) A := C \Ŝp(W ) A is a double cover of Sp(W ) A. Let j : Sp(W ) A Sp(W ) A be the projection. Now we consider the preimage of Sp(W )(F ), the rational points. Let g Sp(W )(F ), and g v its image in Sp(W )(F v ), then g v K v for almost every v. By the properties of ɛ v in [26], we have Lemma For almost every v that is finite and odd, ɛ v (g v ) = 1. We also have Lemma ([26]). For g 1, g 2 Sp(W )(F ), the product v c Wv (g 1, g 2 ) = 1. With these two lemmas, we can easily deduce that the map g C v(g, 1) is a homomorphism from Sp(W )(F ) to Sp(W ) A. Let N be the maximal unipotent radical of Sp(W ). From the definition of c Wv it is not hard to see that c Wv (n 1, n 2 ) = 1 for every n 1, n 2 Sp(W v ). So N(A) is split in Sp(W ) A too. With these fact about K v, Sp(W )(F ) and N(A) one can define the automorphic form and cusp form on Sp(W ) A in a similar way. Let ψ be an additive character on F \A. One can define the Weil representation of H 2n+1 (A) Sp(W ) A with respect to ψ on S(A n ), the space of Schwartz functions on A n, such that for φ = v φ v, w ψ (g)(φ) = v w ψv (g v )(φ v ). Then we let θ φ (g) = ξ F w ψ (g)φ(ξ). It is an automorphic form on Sp(W ) A H 2n+1 (A).

18 1.6.3 Local L-function on the metaplectic groups 12 Let v be finite and odd. Let B v be the preimage of the Borel subgroup B in Sp(W v ). Note that K v is split in Sp(W v ), so we have the Iwasawa decomposition Sp(W v ) = B v K v. A representation of Sp(W v ) is called genuine if (1, ɛ) acts by multiplying ɛ. For a genuine spherical representation of Sp(W v ), one can assign the local L-factor with respect to the additive character ψ v on F v as follows. Suppose π is a spherical representation which can be embedded to Ind Sp(W )(F v) B v ( χ) where χ(diag(t 1,..., t n, t 1 n,..., t 1 1 )n, ɛ) ɛ i χ i (t i )γ 1 ψ v (t 1... t n ) where χ is an unramified character on the torus. By (2.39) χ is a character. Then if we let z π = (χ 1 (ϖ),..., χ n (ϖ), χ n (ϖ) 1,..., χ 1 (ϖ) 1 ), then the local L-factor with respect to ψ v is defined as L ψv (s, π, r) = det(i q s T r(r(z π ))) 1 where ϖ is the uniformizer in F v. Note that when π is genuine, the representation π w ψv is trivial on ɛ, so it factors through Sp(W v ) J = Sp(W v ) H 2n+1,v, which we call the Jacobi group. By the discussion in Chapter 2 in [2], π π w ψv is a bijection between genuine representation of Sp(W v ) and representation of Sp(W v ) J with central character ψ v. In particular when π = Ind Sp(W v) B v, π w ψv is isomorphic to Ind Sp(Wv)J B (χ, ψ v J v ), where B J = B Y Z (Y Z is the group of the elements (0, y, z) in the Heisenberg group), and (χ, ψ v )(b(0, y, z)) = χ(b)ψ v (z).

19 Integral representation using Fourier-Jacobi models In this section we recall the integral representation for the tensor L-function of symplectic groups using Fourier-Jacobi models as introduced in [15]. Let F be a number field, and A its ring of adeles. Let G = Sp 2n, and M = Sp 2m. Let π be an irreducible, cuspidal, automorphic representations of G(A). For any r < n, let Pr n be the parabolic subgroup of G with Levi decomposition Pr n = GL r 1 Sp 2(n r) Vr n. Then one can define the r-th Fourier Jacobi coefficient of ϕ π π as FJ φ ψ V n(ϕ π )(h) = dv ϕ(vh) θ φ r [Vr n] ψ,n r (l n r(v)h)ψ V n r (v) (1.3) with h Sp 2(n r). Here φ S(A n r ), the space of Schwartz function on A n r, and ψ is an additive character on F \A. θ φ ψ,n r θ φ ψ,n r (h) = ξ F n r w ψ (h)φ(ξ), is the theta series defined as and [Vr n ] is the quotient Vr n (F )\Vr n (A). The group H 2(n r)+1 is the Heisenberg group of 2(n r) + 1 variables, and map l n r : Vr n H 2(n r)+1 is defined as l n r (v) = (v r,r+1, v r,r+2,..., v r,2k r ; v r,2k r+1 ). (1.4) It is known that (a) the function FJ φ ψ V n(ϕ π )(h) is an genuine automorphic function on Sp 2(n r) (A), r and (b) when r is the largest number such that FJ φ ψ V n(ϕ π )(h) is not zero for some r ϕ π, the representation generated by FJ φ ψ V n(ϕ π )(h) is cuspidal. r When (b) is satisfied, we call FJ φ ψ V n(ϕ π )(h) the top Fourier-Jacobi coefficient r

20 of π. For any genuine automorphic representation π of Sp 2(n r) (A) and ϕ π π, we define the pairing 14 FJ P ψv n r (ϕ π, φ, ϕ π ) = [Sp 2(n r) ] dh FJ φ ψ V n r (ϕ π )(h)ϕ π (h) (1.5) assuming it is convergent. Assume now that when r = n m, the Fourier Jacobi coefficient is the top one, and σ is a summand in the complex conjugate of the cuspidal automorphic representation generated by FJ φ ψ V n r (ϕ π )(h), so the pairing FJ P ψv n r (ϕ π, φ, ϕ σ ) is not identically zero. Then we can consider integral as follows. Let k be a positive integer with k + m < n, and let P k,m be the parabolic subgroup of Sp 2(m+k) with Levi decomposition P k,m = GL k M V k,m. Let τ be an irreducible cuspidal automorphic representation of GL k (A), and let Ẽτ, σ (h, s) be an Eisenstein Series on Sp 2(m+k) (A) associated to a holomorphic section f τ, σ,s in the induced representation Ind Sp 2(m+k) (A) P k,m (A) (γ ψ τ det s 1 2 σ). Then with the same notation as above, we consider the integral I(ϕ π, φ, f τ, σ,s ) :=FJ P ψv n (ϕ π, φ, Ẽτ, σ (h, s)) n m k = dh FJ φ ψ V n (ϕ π )(h) Ẽτ, σ (h, s). n m k [Sp 2(m+k) ] By [15, Lemma 3.1], this integral is convergent absolutely and uniformly on vertical strips in C for s being away from poles of Ẽτ, σ (h, s). Then by the calculation in [15, Theorem 3.2], the integral is equal to M(A)V m+k (A)\Sp k 2(m+k) (A) R(A) dh dr FJ P ψv n (π(wrh)ϕ π, [ω ψ (l m+k (r)h)φ] m, f Wτ n m, σ,s(h)). (1.6) Here w is a Weyl element in G and R = {R(r) r Mat (n m k),k } is a unipotent

21 subgroup of G such that 15 w = I n m k I k I 2m I n m k, R(r) = I n m k r I k I 2m I k r, I k I n m k and ω ψ is the Weil representation on S(A m+k ), and [φ] m S(A m ) is defined as [φ] m (y 1,..., y m ) = φ(0,..., 0, y }{{} 1,..., y m ). k The function f Wτ, σ,s is defined on Sp 2(m+k)(A) taking the value in V σ. The trilinear form FJ P ψv n n m on V π S(A m ) V σ satisfies FJ P ψv n (π(vh)ϕ π, ω ψ (l m (v)h)φ, n m σ(h)ϕ σ ) = ψ V n n m (v)fj P ψv n (ϕ π, φ, n m ϕ σ ) (1.7) for all v Vn m(a) n and h M(A). For each v, the restriction of FJ P ψv n on n m local representation spaces satisfies an equation similar to (1.7), and by [30] and [21] such trilinear form is unique up to a scalar. Take T v be such trilinear forms for each v, then by suitable normalization, and assume that all the vectors in the integral are factorizable, we obtain an Euler product I(ϕ π, φ, f τ, σ,s ) = v I v (ϕ πv, φ v, f Wτ, σ,s,v), (1.8) where I v (ϕ πv, φ v, f Wτ, σ,s,v ) = M(F v)v m+k k (F v)\sp 2(m+k) (F v) dh R(F v) T v (π(wrh)ϕ π,v, [ω ψ (l m+k (r)h)φ v ] m, f Wτ, σ,s,v(h)). (1.9) dr

22 16 In the following two chapters, we are going to compute I v (ϕ πv, φ v, f Wτ, σ,s,v ) at unramified places, that is, where v is non-archimedean and all the vectors are spherical. First we calculate in Chapter 2 the value of T v (π(g)ϕ π,v, φ v, f ) explicitly. This function is called a Whittaker-Shintani function, and we denote it σ by Wχ,ξ,ψ(g) 0 (after normalization). Theorem (Theorem ). For every (χ, ξ) C n C m, the normalized Whittaker-Shintani function is given by X 0 m dxwχ,ξ,ψ(p 0 d xp f ) = ζ(1) m ζ(2i) i=1 b(wχ, w ξ)d(wχ)d (w ξ)((wχ) 1 δ 1 2 )(p f )((w ξ) 1 δ 1 2 )(p d ) w W G,w W M for d Λ + m and f Λ + n. Here ζ(s) = (1 q s ) 1 is the local zeta function. For definitions of notations, see theorem and section 2.2. If we let L(d, f ) = X 0 dxw0 χ,ξ,ψ(p d xp f ), and let S(d, f) = {d d Λ + m, f + d d Λ + n, d d}, then for each d S(d, f), there exists a d,f (d ) R independent of (χ, ξ, ψ) such that Wχ,ξ,ψ(p 0 d λp f ) = a d,f (d )L(d, f + d d ) d S(d,f) and that a d,f (d) > 0. In particular, we have W 0 χ,ξ,ψ(p f ) = L(0, f) Note that by the Iwasawa decomposition, we only need the last formula to compute (1.9) assuming all the data are unramified. This is done in Chapter 3. We also consider some basic properties of the local integrals at non-archimedean places. Theorem (Theorem 3.1.1). The local integrals I v (ϕ πv, φ v, f Wτ, σ,s,v) at the

23 non-archimedean places satisfy 17 (A) At the unramified places, I v (ϕ πv, φ v, f Wτ, σ,s,v ) = L(π v τ v, s) L ψv ( σ v τ v, s )L(τ v, sym 2, 2s) (1.10) as expected in [15, Theorem 4.3] (B) The local integral converges absolutely for Re(s) sufficiently large. (C) As a function of s, the local integral has a meromorphic continuation to the whole complex plain. (D) For any given s, there is a choice of data such that the local integral is nonzero.

24 Chapter 2 Whittaker-Shintani functions 2.1 Introduction In this chapter we give an explicit formula for the Whittaker-Shintani function. This chapter is separately submitted in [29]. Let G and M be symplectic groups, defined over a non-archimedean local field F of rank n and m respectively with n m + 1. Let Ind G B G (χ) be an unramified principal series of G. Let M J be the Jacobi group and B M J its Borel subgroup as defined in A in Section 2.2, and let Ind MJ B (ξ, ψ) be an unramified principal series M J of M J as defined in C in Section 2.2. Let U be the unipotent radical of a parabolic subgroup P n m 1 1 of G and ψ U be a character on U which is stabilized by M J (see 2.5 and 2.6). Then one can define an M J -invariant, (U, ψ U )-equivariant pairing l χ,ξ,ψ between Ind G B G (χ) and Ind MJ B (ξ, ψ). Let F0 M J χ and F 0 ξ,ψ be the normalized spherical vectors in Ind G B G (χ) and Ind MJ B (ξ, ψ), and we define M J W χ,ξ,ψ (g) = l χ,ξ,ψ (R(g)F 0 χ, F 0 ξ,ψ). This function is a Whittaker-Shintani function attached to (χ, ξ, ψ) (see Definition 2.2.4). We will show that for given (χ, ξ, ψ) (unramified) such function is 18

25 19 unique up to scalar. Denote by Wχ,ξ,ψ 0 the normalized Whittaker-Shintani function which is equal to 1 at the identity. In this paper we prove the following two theorems (for the definition of X 0, Z, K M J, p d, λ, p f, K G, see Section 2.2). Theorem Let (χ, ξ) C n C m, and let d Λ + m, f Λ + n. Let Wχ,ξ,ψ 0 be the normalized Whittaker-Shintani function attached to (χ, ξ, ψ). Then we have X 0 m dxwχ,ξ,ψ(p 0 d xp f ) = ζ(1) m ζ(2i) i=1 b(wχ, w ξ)d(wχ)d (w ξ)((wχ) 1 δ 1 2 )(p f )((w ξ) 1 δ 1 2 )(p d ). w W G,w W M (2.1) where and d(χ) = n ζ(χ a ± χ b ) ζ(χ i ), d (ξ) = m ζ(ξ a ± ξ b ) ζ(2ξ j ), 1 a<b n i=1 1 a<b m j=1 b(χ, ξ) = i<j+n m 1 j m ζ 1 (χ i ξ j ) ζ 1 (ξ j ) 1 i n 1 j m i>j+n m ζ 1 (χ i + ξ j ) ζ 1 ( χ i + ξ j ) Theorem Under the same notation and assumptions as in the previous theorem, the support of W 0 χ,ξ,ψ is on ZUK M J(p d λp f )K G. (2.2) d Λ + m,f Λ + n If we let L(d, f ) = X 0 dxw0 χ,ξ,ψ(p d xp f ), and let S(d, f) = {d d Λ + m, f + d d Λ + n, d d}, then for each d S(d, f), there exists a d,f (d ) R independent of (χ, ξ, ψ) such

26 that 20 Wχ,ξ,ψ(p 0 d λp f ) = a d,f (d )L(d, f + d d ) (2.3) d S(d,f) and that a d,f (d) > 0. In particular, we have W 0 χ,ξ,ψ(p f ) = L(0, f) (2.4) The paper is organized as follows. In Section 2.2 we give the notation we use in this paper. In Section 2.3 we apply the method of Rankin-Selberg convolution to find an integral expression for the pairing l χ,ξ,ψ when (χ, ξ) belongs to Z c C n C m which contains a Hausdorff open set. In Section 2.4 we show that the pairing l χ,ξ,ψ between Ind G B G (χ) and Ind MJ B (ξ, ψ) satisfying Condition A M J (see Definition 2.2.2) is unique up to a scalar. In Section 2.5 we apply Bernstein s theorem to extend this pairing defined by the integral to generic (χ, ξ). In Section 2.6 we discuss the double cosets of G on which the Whittaker-Shintani function is supported. By considering the vectors invariant under certain open compact subgroups (in Section 2.7) and applying the intertwining operators (in Section 2.8) we give an explicit formula in Section 2.9 for the Whittaker- Shintani function attached to generic (χ, ξ), and we obtain its value at the identity by an combinatorial argument in Section After showing the uniqueness of the normalized Whittaker-Shintani function in Section 2.11, we summarize our results in Section In Section 2.13 we use the formula we obtained to give an alternative proof of [24, Theorem 6.1], the unramified calculation of L-functions for Sp 2n GL Notation In this paper, we let F be a non-archimedean local field of characteristic 0. Let O be its maximal compact subring and p the uniformizer. Suppose the order of the residue field is q which is not a power of 2. All the groups are defined over F.

27 21 Through out the paper we fix ψ to be an additive character on F with conductor 0. (A) Groups. Let G = Sp 2n, H = Sp 2m+2 and M = Sp 2m, where m, n are two positive integers with n m + 1. Let M 2n 2n (F ) be the set of all 2n by 2n matrices over F. Identify G as a subset of M 2n 2n (F ), and identify g M (or g H) with diag(i n m, g, I n m ) G (or diag(i n m 1, g, I n m 1 ) G). For any subgroup P of G, and any i 0, we define P i as P i = P ( I 2n + M 2n 2n (p i O) ), and we always normalize the measure on P so that µ P (P 0 ) = 1. Let K G = G 0, and K M = M K G. For x, y F m and z F let 1 x y z I t J(x, y, z) = m y I m t H. x 1 J = {J(x, y, z) x, y F m, z F } is a Heisenberg group of dimension 2m+1. Let M J = M J, and K M J = K M J 0. Let X(x) = J(x, 0, 0), Y(y) = J(0, y, 0), Z(z) = J(0, 0, z) and denote the respective images of the functions X, Y and Z by X, Y and Z as well. Let B G, B H and B M be the standard Borel subgroups of G, H, M, and B M J = B M (Y Z), and let N G, N H, N M, N M J be their unipotent radicals respectively. Let T G be the toral part of of B G, and let Λ + k = {(d 1,..., d k ) Z k d 1 d 2... d k 0} T + G = {diag(t 1,..., t n, t 1 n,..., t 1 1 ) t 1... t n 1} T G = {t 1 t T + G }.

28 Define T + M and T M analogously. Let P n m 1 1 be the standard parabolic subgroup of G with Levi decomposition 22 Let ψ U be the character on U given by P n m 1 1 = GL n m 1 1 H U. (2.5) n m 1 ψ U (u) = ψ( u i,i+1 ). (2.6) Then M J stabilizes ψ U. We denote by I G, I M the Iwahori subgroups of G and M respectively. Let I M = I M J 0, and let I M J = I M (X 1 Y 0 Z 0 ). Let W G, W M be the Weyl groups of G and M with respect to T G and T M. i=1 (B) Elements. Let w0 G be the longest Weyl element in G. By abuse of notation we also use w0 G to denote one of its representatives in G. For k n, and for given t 1,..., t k F, we let d k (t 1,..., t k ) = diag(i n k, t 1,..., t k, t 1 k,..., t 1 1, I n k ) T G. Let Z = Z { }, and let v be the normalized valuation from F to Z. We define an order on Z k such that for a = (a 1,..., a k ) and b = (b 1,..., b k ) Z k, a b if and only if a i b i for 1 i k. We define min(a, b) = (min(a 1, b 1 ),..., min(a k, b k )). When a Z m, we let λ(a) = X(p a 1,..., p am ).

29 Here p = 0. Let λ = λ(0) = X(1, 1,..., 1). For a = (a 1,..., a k ) Z k, we let p a = d k (p a 1,..., p a k ). For α being a root in G, we define n α, N α as follows. For 1 i, j 2n, let E i,j M 2n 2n with 1 at the (i, j)-entry and 0 at others. Then for x F and i j, we let x(e i,j E 2n+1 i,2n+1 j ) if α = e i e j, x(e i,2n+1 j + E j,2n+1 i ) if α = e i + e j, n α (x) = xe n,n+1 if α = 2e n, x(e 2n+1 i,j + E 2n+1 j,i ) if α = e i e j, xe n+1,n if α = 2e n. Let N α be the group consisting of n α (x) for x F. When α is a simple root, we define T α and w α as follows. For 1 i n, let D i (t) = d n (1,..., 1, i th t, 1,..., 1). Then we let and D i (t)d i+1 (t 1 ) if α = e i e i+1, T α (t) = D n (t) if α = 2e n, E i,i+1 E i+1,i E 2n+1 j,2n+1 i + E 2n+1 i,2n+1 j if α = e i e i+1, w α = E n,n+1 E n+1,n if α = 2e n. By abuse of notation we also denote by w α its image in the Weyl group. For β being a root in M, we define n β, N β, w β and T β as elements in or subgroups of M in a similar way. 23 (C) Representations. Let χ and ξ be unramified characters on T G and T M. We parametrize them as χ = (χ 1,..., χ n ) C n and ξ = (ξ 1,..., ξ m ) C m

30 such that χ(d n (t 1,..., t n )) = n i=1 t i χ i 24 and ξ(d m (t 1,..., t m )) = m j=1 t j ξ j. By abuse of notation we also let χ i, ξ j be unramified characters on F so that χ i (p) = q χ i and ξ j (p) = q ξ j. Let e i Hom(T G, GL 1 ), for 1 i n, and e j Hom(T M, GL 1 ), for 1 j m such that e i (d n (t 1,..., t n )) = t i and e j(d m (t 1,..., t m )) = t j. Let Ind MJ B M J (ξ, ψ) be a representation of MJ consisting of smooth functions on M J such that f(b M (0, y, z)m J ) = ξδ 1 2 BM J (b m)ψ(z)f(m J ), with M J acting by right translation. Sometimes we write ξψ as a character on B M J such that ξψ(b M J(0, y, z)) = ξ(b M )ψ(z). Remark Although B M J\M J is not compact, functions in Ind MJ B (ξ, ψ) are M J compactly supported on B M J\M J by smoothness. In fact by Iwasawa decomposition on M, we have M J = B M JXK M. Suppose f Ind MJ B (ξ, ψ) which is right K M J f invariant for some open compact subgroup K f. Note that K f \K M J is finite. Let k be a representative in one of the cosets and suppose f(xk) 0. Then by the smoothness of f, f(xk) = f(xyk) when y is in a neighborhood of 0 F m. But then f(xyk) = ψ(2 x, y )f(xk), so ψ(2 x, y ) = 1 for all such y, which implies that x belongs to a compact subset. In particular, if f is K M J-invariant, then the restriction of f on X is up to a scalar equal to the characteristic function of X 0. So the spherical vector in Ind MJ B (ξ, ψ) is supported on B M J M JK MJ, and hence is unique up to a scalar. (D) Functions and Functionals. Let ζ(s) = (1 q s ) 1 be the local zeta function. For any set X we denote by Ch X the characteristic function of X. For ϕ 1 C c (G), we let F χ (ϕ 1 )(g) = BG χ 1 δ 1 2 B G (b G )ϕ 1 (b G g) d l b G. (2.7)

31 Then the map ϕ 1 F χ (ϕ 1 ) is surjective from C c for ϕ 2 C c (M J ), we let 25 (G) to Ind G B G (χ). Similarly F ξ,ψ (ϕ 2 )(m J ) = B M J (ξψ) 1 δ 1 2 BM J (b M J)ϕ 2(b M Jm J ) d l b M J. (2.8) The map ϕ 2 F ξ,ψ (ϕ 2 ) is surjective from C c (M J ) to Ind MJ B (ξ, ψ). Let M J K χ,ξ,ψ (b G w G 0 λj(0, y, z)b M u) = χ 1 δ 1 2 BG (b G )ξδ 1 2 B M J (b M)ψ(z)ψ 1 U (u). (2.9) We will prove in section 3 that the formula (2.9) gives a well-defined function on B G w G 0 λb M JU, which can be extended to a function on G by setting K χ,ξ,ψ (g) = 0 for all g outside of this set. For ϕ 1 C c (G) and ϕ 2 C c (M J ), let I χ,ξ,ψ (ϕ 1, ϕ 2 )(g) = G dg M J dm J ϕ 1 (g )K χ,ξ,ψ (g g 1 (m J ) 1 )ϕ 2 (m J ), (2.10) and let I 0 χ,ξ,ψ(g) = I χ,ξ,ψ (Ch KG, Ch KM J )(g). Let F0 χ = F χ (Ch KG ) and F 0 ξ,ψ = F ξ,ψ (Ch ), which are spherical in KM J IndG B G (χ) and Ind MJ B (ξ, ψ) respectively. M J Let H G be the spherical Hecke algebra of G, and H M J,ψ be the spherical Hecke algebra of M J with respect to ψ as defined in section 4 of [23], and let them act on Ind G B G (χ) K G and Ind M J B M J (ξ, ψ)k M J by characters ω χ and ω ξ respectively. For any function f on G, let (L(g 0 )f)(g) = f(g 1 0 g), and (R(g 0 )f)(g) = f(gg 0 ). Definition A pairing l χ,ξ,ψ between Ind G B G (χ) and Ind MJ B (ξ, ψ) will be M J said to satisfy Condition A if (i) l χ,ξ,ψ (F χ, F ξ,ψ ) = l χ,ξ,ψ (R(m J )F χ, R(m J )F ξ,ψ ) for any m J M J. (ii) l χ,ξ,ψ (R(u)F χ, F ξ,ψ ) = ψ U (u)l χ,ξ,ψ (F χ, F ξ,ψ ) for any u U. Remark By the definition of K χ,ξ,ψ, (2.10) actually gives a pairing between Ind G B G (χ) and Ind MJ B (ξ, ψ) satisfying Condition A, for those (χ, ξ) M J such that the integral is convergent for all choices of (ϕ 1, ϕ 2 ), (see proposition 2.3.2).

32 26 Definition For (χ, ξ) C n C m, a function W χ,ξ,ψ called a Whittaker-Shintani Function attached to (χ, ξ) if C (G) is (i) W χ,ξ,ψ (zuk M Jgk G ) = ψ 1 (z)ψ U (u)w χ,ξ,ψ (g). (ii) L(ϕ M J)R(ϕ G )W χ,ξ,ψ = ω ξ (ϕ M J)ω χ (ϕ G ) W χ,ξ,ψ for any ϕ M J H M J,ψ and ϕ G H G. The space of Whittaker-Shintani functions attached to (χ, ξ, ψ) is denoted by WS χ,ξ,ψ. Sometimes we omit ψ because it is fixed in this paper. A Whittaker- Shintani function is called a Normalized Whittaker-Shintani function if it equals 1 at the identity. We denote it by Wχ,ξ,ψ Integral expression for the pairing We use the function K χ,ξ,ψ, as defined in (2.9) to construct a pairing between Ind G B G (χ) and Ind MJ B (ξ, ψ) satisfying Condition A. First we show that M J Lemma The function K χ,ξ,ψ (g) as defined in (2.9) is well-defined. Proof. To show that K χ,ξ,ψ (g) is well-defined we only need to show that the way to write an element g B G w G 0 λyzb M U as g = b G w G 0 λyzb M u is unique. Take g = b G w G 0 λyzb M u, and let b M = tn be the Levi decomposition, then g = (b G t 1 )w G 0 (t 1 λt)(t 1 yt)(t 1 zt)nu. Note that (1) (b G t 1 ) B G and (t 1 λt)(t 1 yt)(t 1 zt)nu N G. (2) The way to write an element g B G w G 0 N G as g = bw G 0 n is unique. (3) λ t 1, y t 1, z t 1, n, u belongs to different root subgroups of N G. (4) For t, t T M, λ t = λ t if and only if t = t. Then our lemma is implied.

33 Observe that the function I χ,ξ,ψ defined on C c (G) C c (M J ) G by equation (2.10) factors through the mappings F χ : C c (G) Ind G B G (χ) and F ξ,ψ (ϕ 2 ) : Cc (M J ) Ind MJ B (ξ, ψ). Thus, we may define a pairing l M J χ,ξ,ψ on Ind G B G (χ) Ind MJ B (ξ, ψ) by setting l M J χ,ξ,ψ(f χ (ϕ 1 ), F ξ,ψ (ϕ 2 )) = I χ,ξ,ψ (ϕ 1, ϕ 2 )(e). Explicitly: l χ,ξ,ψ (F χ (ϕ 1 ), F ξ,ψ (ϕ 2 )) = G 27 M J ϕ 1 (g )K χ,ξ,ψ (g (m J ) 1 )ϕ 2 (m J ) dm J dg. (2.11) It is easy to see that the pairing l χ,ξ,ψ (F χ (ϕ 1 ), F ξ,ψ (ϕ 2 )) satisfies Condition A if the integral is convergent, and the integral is convergent if K χ,ξ,ψ is continuous on G. In the rest of this section we will prove the following proposition. Propostion Let Z c be the set of unramified characters (χ, ξ) satisfying Re(χ i χ i+1 ) > 1 for 1 i n m 1 Re(χ n m 1+j ξ j ) > 1 for 1 j m 2 Re( χ n m+j + ξ j ) > 1 for 1 j m 2 Re(χ n ) > 1 (2.12) then when (χ, ξ) Z c, the function K χ,ξ,ψ is continuous on G, and as a consequence, the integral (2.11) is convergent. Since K χ,ξ,ψ is defined continuously on B G w0 G λb M JU, which is a Zariski open subset of G (we will see this soon) and extended by 0 to G, we only need to show the continuity outside the Zariski open set, for the function K χ,ξ,ψ. The method we use here is similar to that in [19]. First by the Bruhat decomposition we have G = B G wn G. w W G And we know that B G w0 G N G is Zariski open in G. In fact we have

34 Lemma There exists α k o[g] for 1 k n such that 28 B G w G 0 N G = {g α k (g) 0 for all k }. Proof. For g G, let its matrix be g = (g ij ) 1 i,j 2n. Let N 2n = {1, 2,..., 2n}. For I = (i 1,..., i k ) and J = (j 1,..., j k ) both belonging to N k 2n, we Let g IJ = (g is,j t ) 1 s,t k. We define IJ (g) = det g IJ. (2.13) For 1 k n, let I k = {2n+1 k, 2n+1 (k 1),..., 2n}, and J k = {1, 2,..., k}, and we take Then one can check that 1. For any n 1, n 2 N G, α k (n 1 gn 2 ) = α k (g). α k (g) = Ik,J k (g). (2.14) 2. α k (d n (t 1,..., t n )gd n (s 1,..., s n )) = k i=1 t 1 i s i α k (g). 3. Let w W G and w one of its representative in G. If α k (w ) 0 for all 1 k n, then w = w G 0. Combining these properties with the Bruhat decomposition of G, we have our lemma. Next we have Lemma There exists β l o[g] for 1 l m such that B G w G 0 λyzb M U = {g G α k (g) 0, β l (g) 0 for all 1 k n, 1 l m}, where α k is as defined in lemma Proof. Note that for any w W, B G wn G = B G wxun M J. For any X(x 1,..., x m ) X, we have X(x 1,..., x m ) = s 1 X(r 1,..., r m )s, where s = d m (s 1,..., s m ) T M

35 such that (x i, 1) if x i 0; (s i, r i ) = (1, 0) if x i = 0 29 From this we can see that B G wn G = B G wx(r)b M JU, (2.15) r {0,1} m and when w = w0 G, the union is disjoint. For 1 l m, we let J l = {1, 2,..., (n m),..., n m + l}, and we define β l (g) = In m+l 1,J l (g). (2.16) Then β l satisfies 1. β l (n 1 g) = β l (g) for any n 1 N G. 2. β l (gn 2 u) = β l (g) for any n 2 N M J and u U. 3. β l (d n (t 1,..., t n )gd m (s 1,..., s m )) = n m i=1 t 1 i l 1 j=1 t 1 n m+j l j=1 s j β l (g). 4. β l (w G 0 X(r)) = ±r l. The sign in front of r l depends on l, which is not important since we are only interested in β l. In fact, (1) is by the definition of I k while (3) and (4) are by direct calculation. For (2), note that if we only consider the first n column of (g ij ), multiplying elements in N M JU from the right corresponds to column operations adding multiples of column k 1 to column k 2 where 1 k 1 < k 2 n with k 1 n m. On the other hand elements in J l are consecutive from 1 to n m + l with n m missing, so In m+l 1,J l (g) is invariant under such column operations. So for g B G wx(r)b M JU with r {0, 1} m, α k (g) 0 and β l (g) 0 for all 1 k n and 1 l m if and only if w = w0 G (by lemma 2.3.3) and r = (1, 1,..., 1) (by the property of β l s), completing our proof.

36 Remark If we let ϖ i = e 1 + e e i, and ϖ j = e 1 + e e j be dominant weights of G and M with respect to B G and B M, then the properties of α k and β l actually shows that under the B G B M J 30 action, α i has the highest weight (ϖ k, 0) when 1 k n m, and (ϖ k, ϖ k (n m) ) when n m + 1 k n, and β l has the highest weight (ϖ n m+l 1, ϖ l). Now we can expressed K χ,ξ,ψ by α k and β l. First we have Lemma Let g = d n (t 1,..., t n )n G w G 0 λd m (s 1,..., s m )n M Ju B G w G 0 λb M JU, we have and By this we have α 1 (g) 1 if i = 1; t i = α i 1 αi 1 (g) if 2 i n m; β i (n m) αi 1 (g) if i > n m s j = β j α 1 n m+j 1(g) Lemma For g B G w G 0 λb M JU, we have n m 1 K χ,ξ,ψ (g) = (χ i χ 1 i+1 1 )(α i (g)) i=1 (χ n 1 )(α n (g)) m j=1 m j=1 for 1 j m (χ n m 1+j ξj 1 ) 1 2 (αn m+j 1 (g)) (χ 1 n m+jξ j 1 2 )(βj (g)). The proof of these are by direct calculation. By lemma we can see that when the assumptions in proposition are satisfied, the extension of K χ,ξ,ψ (g) by letting K χ,ξ,ψ (g) = 0 for g / B G w G 0 λb M JU is continuous, and so K χ,ξ,ψ (g) is continuous.

37 Uniqueness of the pairing for generic (χ, ξ) Each pairing l χ,ξ,ψ satisfying Condition A corresponds to an element of Hom BM J U(Ind G B G (χ), ξ 1 ψ 1 δ 1 2 BM J ψ U). (2.17) In this section we prove that Propostion For generic (χ, ξ), dimhom BM JU(Ind G B G (χ), ξ 1 ψ 1 δ 1 2 BM J ψ U) 1. By (2.15), the action of B G B M JU on G via (b 1, b 2 ) x = b 1 xb 1 2 has finite many orbits. According to 1.5 in [3], (see also 5.1(3) in [4] ), there is a numbering Z 1, Z 2,..., Z k of B G B M JU orbits on G, such that Z i = i j=1z j are open in G for every i. Note that Z 1 = B G w G 0 λb M JU. Given U, a union of Z i s, let I c (χδ 1 2 BG B G, U) be the set of functions f : U C which are locally constant, compactly supported modulo B G and satisfying f(bg) = χδ 1 2 BG (b)f(g). It is a B M JU-module. Since Z i 1 is open in Z i, by [7, Lemma 6.1.1], we have the exact sequence of B M JU-modules: 0 I(χδ 1 2 BG B G, Z i 1) I(χδ 1 2 BG B G, Z i) I(χδ 1 2 BG B G, Z i ) 0. Observe that 1. I(χδ 1 2 BG B G, Z k) = Ind G B G (χ) as B M JU-modules, and 2. dim Hom BM J U(I(χδ 1 2 BG B G, Z 1 ), ξ 1 ψ 1 δ 1 2 ψ BM J U) 1, so to prove proposition it suffices to prove the following lemma. Lemma Let U g = B G gb M JU. Suppose (χ, ξ) is generic, and U g Z 1, then dim Hom BM J U(I(χδ 1 2 BG B G, U g ), ξ 1 ψ 1 δ 1 2 ψ BM J U) = 0.

38 Proof. Let G g = B M JU g 1 B G g, then 32 Hom BM J U(I(χδ 1 2 BG B G, U g ), ξ 1 ψ 1 δ 1 2 BM J ψ U) =Hom Gg (g 1 (χδ 1 2 BG ) ξψδ 1 2 B M J ψ 1 U, δ g ) where δ g is the modulus character of G g. So we need to show that g 1 (χδ 1 2 BG ) ξψδ 1 2 B ψ 1 M J U δg 1 1 (2.18) when restricted to G g for any generic (χ, ξ). Recall from (2.15) that G = w W G,r {0,1} m B G wx(r)b M J. So when U g Z 1 we can assume g = wx(r) with either w w G 0 or r (1, 1,..., 1). To prove (2.18) we need to find b 1 B G and b 2 B M JU such that b 1 g = gb 2 (χδ 1 2 BG )(b 1 ) ξψδ 1 2 B M J ψ U δ g (b 2 ). (2.19) First suppose r (1, 1,..., 1). In this case we claim that T G (gt M g 1 ) contains a nontrivial torus. Let t = d m (t 1,... t m ) T M. Note that T G is stabilized by the adjoint action of W G, so it suffices to show that there exists a nontrivial torus T s of T M such that when t T s, X(r) 1 tx(r) T G. Note that X(r) 1 tx(r) = t X((1 t 1 )r 1, (1 t 2 )r 2,..., (1 t m )r m ), so when r j = 0 for some j, we can let T s = {t = d m (1,..., j-th t,..., 1)} be the torus we claimed. Then since (χ, ξ) is generic, one can find some b 2 T s and b 1 = gb 2 g 1 T M so that (2.19) is satisfied, completing the proof for this case. Now suppose r = (1,..., 1) F m and w w G 0, so X(r) = λ by our notation.

39 In this case there is a simple root α in G such that wn α w 1 N G. When α = e i e i+1 with 1 i n m 1, note that λ M J stabilizes ψ U, so ψ U (λ 1 n α (t)λ) = ψ U (n α (t)) 1 for some t F. 33 On the other hand, wn α (t)w 1 N G by our assumption. So let b 1 = wn α (t)w 1 and b 2 = λ 1 n α (t)λ we have (2.19). When α = e i e i+1 with i n m, we let r(t, i) = X(1,..., i -th 1, 1 + t,..., 1), where i = i (n m). Then for t 1 we have wn α (t)λ = wx(r(t, i))n α (t) =(d 1 m (r(t, i)) ) w wλd m (r(t, i))n α (t). So let b 1 = (d 1 m (r(t, i))n α (t)) w and b 2 = d m (r(t, i))n α (t). For generic (χ, ξ), we can always find some t F so that (2.19) is satisfied. When α = 2e n, we have wn α (t)λ = wλy(t)z( t)n α (t). So let b 1 = (n α (t)) w and b 2 = Y(t)Z( t)n α (t), we have (χδ 1 2 BG )(b 1 ) = 1, and we can find some t F so that ψ(z( t)) 1, so (2.19) is satisfied. 2.5 Rationality(I) In this section we prove the following proposition. Propostion For fixed ϕ 1 C c (G) and ϕ 2 C c (M J ), the function (χ, ξ) l χ,ξ,ψ (F χ (ϕ 1 ), F ξ (ϕ 2 )) extends to C n C m as a rational function of (q χ 1,..., q χn, q ξ 1,..., q ξm ). 2. The pairing l χ,ξ,ψ extends by meromorphic continuation to a pairing between Ind G B G (χ) and Ind MJ B (ξ, ψ) which is defined for all χ, ξ outside of a countable M J