A PRODUCT OF TENSOR PRODUCT L-FUNCTIONS OF QUASI-SPLIT CLASSICAL GROUPS OF HERMITIAN TYPE

A PRODUCT OF TENSOR PRODUCT L-FUNCTIONS OF QUASI-SPLIT CLASSICAL GROUPS OF HERMITIAN TYPE DIHUA JIANG AND LEI ZHANG Abstract. A family of global zeta integrals representing a product of tensor product (partial) L-functions: L S (s, π τ 1 )L S (s, π τ 2 ) L S (s, π τ r ) is established in this paper, where π is an irreducible cuspidal automorphic representation of a quasi-split classical group of Hermitian type and τ 1,, τ r are irreducible unitary cuspidal automorphic representations of GL a1,, GL ar, respectively. When r = 1 and the classical group is an orthogonal group, this family was studied by Ginzburg, Piatetski-Shapiro and Rallis in 1997. When π is generic and τ 1,, τ r are not isomorphic to each other, such a product of tensor product (partial) L-functions is considered by Ginzburg, Rallis and Soudry in 2011 with different kind of global zeta integrals. In this paper, we prove that the global integrals are eulerian and finish the explicit calculation of unramified local zeta integrals in a certain case (see Section 4 for detail), which is enough to represent the product of unramified tensor product local L-functions. The remaining local and global theory for this family of global integrals will be considered in our future work. 1. Introduction We study a finite product of tensor product (partial) automorphic L-functions for quasi-split unitary or orthogonal groups and general linear groups via global zeta integral method. Let G n be a quasi-split group, which is either U n,n, U n+1,n, SO 2n+1, or SO 2n, defined over a number field F. Let E be a quadratic extension of F when we discuss unitary groups and E be equal to F when we Date: March, 2013; revised in September, 2013. 2010 Mathematics Subject Classification. Primary 11F70, 22E50; Secondary 11F85, 22E55. Key words and phrases. Bessel Periods of Eisenstein Series, Global Zeta Integrals, Tensor Product L-functions, Classical Groups of Hermitian Type. The work of the first named author is supported in part by the NSF Grants DMS 1001672 and DMS 1301567. 1

2 DIHUA JIANG AND LEI ZHANG discuss orthogonal groups. Let A E be the ring of adeles of E and A be the ring of adeles of F. Take τ to be an irreducible generic automorphic representation of Res E/F (GL a )(A) = GL a (A E ) of isobaric type, i.e. (1.1) τ = τ 1 τ 2 τ r where a = r i=1 a i is a partition of a and τ i is an irreducible unitary cuspidal automorphic representation of GL ai (A E ). Let π be an irreducible cuspidal automorphic representation of G n (A). We consider a family of global zeta integrals (see Section 3 for definition), which represents a finite product of the tensor product (partial) automorphic L-functions: (1.2) L S (s, π τ) = L S (s, π τ 1 )L S (s, π τ 2 ) L S (s, π τ r ). It is often interesting and important in number theory and arithmetic to consider simultaneous behavior at a particularly given point s = s 0 of the L-functions L S (s, π τ i ) with i = 1, 2,, r. For instance, one may consider the nonvanishing at s = 1, the center of the symmetry 2 of the L-functions L S (s, π τ 1 ), L S (s, π τ 2 ),, L S (s, π τ r ), or particularly, take τ 1 = τ 2 = = τ r and consider the r-th power L S (s, π τ 1 ) r at a given value s = s 0 for all positive integers r. As remarked at the end of this paper, the arguments and the methods still work if one replaces the single variable s by multi-variable (s 1,, s r ). However, we focus on the case of single variable s in this paper. We use a family of the Bessel periods (discussed in Section 2) to define the family of global zeta integrals, following closely the formulation of Ginzburg, Piatetski-Shapiro and Rallis in [GPSR97], where the case when r = 1 and G n is an orthogonal group was considered. When π is generic, i.e. has a nonzero Whittaker-Fourier coefficient, and τ 1,, τ r are not isomorphic to each other, this family of tensor product L-functions was studied by Ginzburg, Rallis and Soudry in their recent book [GRS11]. However, the global zeta integrals studied in [GRS11] are formulated in a different way and can not cover the general situation considered in this paper. It is worthwhile to remark that the global zeta integrals here are the most general version of this kind, the origin of which goes back to the pioneering work of Piatetski- Shapiro and Rallis and of Gelbart and Piatetski-Shapiro ([GPSR87]). Other special cases of this kind were studied earlier by various authors, and we refer to the relevant discussions in [GPSR97] and [GRS11]. In addition to the potential application towards the simultaneous nonvanishing of the central values of the tensor product L-functions, the basic relation between the product of the tensor product (partial) L-functions and the family of global zeta integrals is also an important

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 3 ingredient in the proof of the nonvanishing of the certain explicit constructions of endoscopy correspondences as indicated for some special cases in the work of Ginzburg in [G08], and as generally formulated in the work of the first named author ([J11] and [J13]). We will come back to this topic in our future work ([JZ13]). In general, the meromorphic continuation to the whole complex plane of the product of the tensor product (partial) L-functions is known from the work of R. Langlands on the explicit calculation of the constant terms of Eisenstein series ([L71]). However, when π is not generic, i.e. has no nonzero Whittaker-Fourier coefficients, the Langlands conjecture on the standard functional equation and the finite number of poles for Re(s) 1 is still not known ([Sh10]). 2 According to the work of Arthur ([Ar13]) and also of C.-P. Mok ([Mk13]), any irreducible cuspidal automorphic representation π of G n (A) has a global Arthur parameter ψ, which determines an irreducible automorphic representation π ψ of the corresponding general linear group GL N (A E ), where the integer N depends on the type of G n. The mapping from π to π ψ is called the Arthur-Langlands transfer, which is the weak Langlands functorial transfer from G n to GL N. This means that the global transfer from π to π ψ is compatible with the corresponding local Langlands functorial transfers at all unramified local places of π. Hence we have an identity for partial L-functions L S (s, π τ) = L S (s, π ψ τ). The partial L-function on the right hand side is the Rankin-Selberg convolution L-function for general linear groups studied by Jacquet, Piatetski-Shapiro and Shalika ([JPSS83]). When π has a generic global Arthur parameter, the Arthur-Langlands transfer from G n to general linear groups is compatible with the corresponding local Langlands functorial transfer at all local places. Hence one may define the complete tensor product L-function by L(s, π τ) := L(s, π ψ τ), just as in ([Ar13] and [Mk13]). However, when the global Arthur parameter ψ is not generic, there exists irreducible cuspidal automorphic representation π with Arthur parameter ψ, whose local component π ν at some ramified local place ν may not be transferred to the corresponding ramified local component (π ψ ) ν under the local Langlands functorial transfer at ν. Hence it is impossible to define the local tensor product L-factors (and also γ- factors and ɛ-factors) of the pair (π ν, τ ν ) in terms of those of the pair ((π ψ ) ν, τ ν ) at such ramified local places ν. Therefore, it is still an open

4 DIHUA JIANG AND LEI ZHANG problem to define the local ramified L-factors (and also γ-factors and ɛ-factors) for an irreducible cuspidal automorphic representation π of G n (A) when π has a non-generic global Arthur parameter. At this point, it seems that the integral representation of Rankin-Selberg type for automorphic L-functions is the only available method to attack this open problem. For quasi-split classical groups of skew-hermitian type, some preliminary work has been done in [GJRS11], using Fourier-Jacobi periods. Further work is in progress, including the work of X. Shen in his PhD thesis in University of Minnesota, 2013, which has produced two preprints [Sn12] and [Sn13]. A parallel theory for this case will also be considered in future. In Section 2, we introduce a family of Eisenstein series, whose Bessel periods are needed to formulate the family of global zeta integrals as mentioned above. A basic analytic property of such global zeta integrals is stated in Proposition 2.1. We note that the construction of the global zeta integrals has two integers j and l involved, depending on the cuspidal data of the Eisenstein series and the structure of the Bessel periods. Section 3 finishes the first step in the global theory for the family of global zeta integrals, which proves that they are eulerian, i.e. they are expressed as an eulerian product of the corresponding local zeta integrals at all local places of F (Theorem 3.8). The argument is standard, although it is technically quite involved. Based on an explicit calculation of generalized Bruhat decomposition in [GRS11, Section 4.2], we calculate in Section 3.1 the Bessel-Fourier coefficients of the Eisenstein series used in the global zeta integrals. Then we use [GRS11, Section 4.4] to carry out a long calculation in Section 3.2, which proves Theorem 3.8. We note that Sections 2 and 3 are for both unitary groups and orthogonal groups. Following the general understanding of the Rankin-Selberg method, after expressing the local zeta integrals in terms of the expected local L-functions, which is the key part of the local theory for global zeta integrals, the global analytic properties of the global zeta integrals will be transferred to the expected complete (or partial) L-functions. Although for any pair (j, l) of integers, the global zeta integrals are eulerian (Theorem 3.8), it seems that only in the case where j = l + 1 the local zeta integral is better understood and is enough to reach the local L-factors of the tensor product type as we expected. The remaining cases will be considered in future. The local theory starts in Section 4. In Section 4.1, we reformulate the local zeta integrals from the eulerian product in Theorem 3.8 in terms of the uniqueness of local Bessel functionals and related them to

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 5 the corresponding twisted Jacquet modules. We show that the local zeta integrals converges absolutely when the real part of the complex parameter s is sufficiently large (Lemma 4.1). The twisted Jacquet modules are explicitly calculated following closely [GRS11, Chapter 5]. This is necessary for the development of the local theory at all finite local places. In Sections 4.2, 4.3, 4.4, and 4.5, we only consider the unramified case. In Section 4.2, we write down explicitly the unramified local L-factors of tensor product type for unitary groups in terms of the corresponding Satake parameters of the unramified representations. Section 4.3 shows that the unramified local zeta integrals are rational functions in q s E following the Bernstein rationality theorem. Starting with Section 4.4, we assume that j = l + 1 and are concentrated on the proof of Theorem 3.9, i.e. the explicit calculation of the unramified local zeta integrals in terms of the expected local L-factors. The main arguments used here can be viewed as natural extension of those used in [GPSR97] for orthogonal group case to unitary group case. Hence we only discuss the unitary group case, since the orthogonal groups case was treated in [GPSR97]. Sections 4.4 and 4.5 are quite technical and are devoted to the understanding of the denominator and numerator of the rational function from Section 4.3. The result is stated in Theorem 4.12, which is the main local result of the paper. The main global result in this paper is Theorem 4.13, which is stated at the end of Section 4. In order to carry out the complete understanding of the family of global zeta integrals, one has to develop the complete theory for the local zeta integrals at all local places, which is in fact our main concern and is considered in our work in progress. Finally, we would like to thank the referee for careful reading of the previous version of the paper and for many instructive and helpful comments. 2. Certain Eisenstein series and Bessel periods We introduce a family of Eisenstein series which will be used in the definition of a family of global zeta integrals, representing the family of the product of the tensor product L-functions as discussed in the introduction. The global zeta integrals are basically a family of Bessel periods of those Eisenstein series. We recall first the general notion of the Bessel periods of automorphic forms from [GPSR97], [GJR09], [BS09] and [GRS11]. Let F be an number field. Define E = F or E = F ( ρ), a quadratic extension of F, depending on that the classical group we considered is orthogonal or unitary, accordingly. It follows that the Galois group

6 DIHUA JIANG AND LEI ZHANG of E/F is either trivial or generated by a non-trivial automorphism x x. The ring of adeles of F is denoted by A, while the ring of adeles of E is denoted by A E. Let V be an E-vector space of dimension m with a non-degenerate quadratic form q V if E = F or a non-degenerate Hermitian form (also denoted by q V ) if E = F ( ρ). Let U(q V ) be the connected component of isometry group of (V, q V ) defined over F. It follows that U(q V ) is a special orthogonal group or a unitary group. Let m = Witt(V ) be the Witt index of V. Let V + be a maximal totally isotropic subspace of V and V be its dual, so that V has the following polar decomposition V = V + V 0 V, where V 0 = (V + V ) denotes the anisotropic kernel of V. We choose a basis {e 1, e 2,..., e m } of V + and a basis {e 1, e 2,..., e m } of V such that q V (e i, e j ) = δ i,j for all 1 i, j m. We assume in this paper that the algebraic F -group U(q V ) is F - quasi-split. Then the anisotropic kernel V 0 is at most two dimensional. More precisely, when E = F, if dim E V = m is even, then dim E V 0 is either 0 or 2, and if dim E V = m is odd, then dim E V 0 is 1; and when E = F ( ρ), dim E V 0 is 0 or 1 according to that dim E V = m is even or odd. When dim V 0 = 2, we choose an orthogonal basis {e (1) 0, e (2) 0 } of V 0 with the property that q V0 (e (1) 0, e (1) 0 ) = 1, q V0 (e (2) 0, e (2) 0 ) = c, where c F is not a square and q V0 = q V V0. When dim V 0 = 1, we choose an anisotropic basis vector e 0 for V 0. We put the basis in the following order (2.1) e 1, e 2,..., e m, e (1) 0, e (2) 0, e m,, e 2, e 1 if dim E V 0 = 2; (2.2) e 1, e 2,..., e m, e 0, e m,, e 2, e 1 if dim E V 0 = 1; and (2.3) e 1, e 2,..., e m, e m,, e 2, e 1 if dim E V 0 = 0. With the choice of the ordering of the basis vectors, the F -rational points U(q V )(F ) of the algebraic group U(q V ) are realized as an algebraic subgroup of GL m (E). Define n = [ m 2 ] and put G n = U(q V ). This agrees with the definition of G n which was given in the introduction. From now on, for any F -algebraic subgroup H of G n, the F -rational

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 7 points H(F ) of H are regarded as a subgroup of GL m (E). Similarly, the A-rational points H(A) of H are regarded as a subgroup of GL m (A E ). The corresponding standard flag of V (with respect to the given ordering of the basis vectors) defines an F -Borel subgroup B. We may write B = T N with T a maximal F -torus, whose elements are diagonal matrices, and with N the unipotent radical of B, whose elements are upper-triangular matrices. Let T 0 be the maximal F -split torus of G n contained in T. We define the root system Φ(T 0, G n ) with the set of positive roots Φ + (T 0, G n ) corresponding to the Borel subgroup given above. Take l an integer between 1 and m. Let V ± l be the totally isotropic subspace generated by {e ±1, e ±2,..., e ±l } and P l = M l U l be a standard maximal parabolic subgroup of G n, which stabilizes V + l. The Levi subgroup M l is isomorphic to GL(V + l ) G n l. Here GL(V + l ) = Res E/F (GL l ) and G n l = U(q Wl ) with q Wl = q V Wl and W l = (V + l V l ). Let l := [l 1 l 2... l p ] be a partition of l. Then P l = M l U l is a standard parabolic subgroup of G n, whose Levi subgroup M l = ResE/F GL l1 Res E/F GL l2 Res E/F GL lp G n l. 2.1. Bessel periods. Define N l to be the unipotent subgroup of G n consisting of elements of following type, (2.4) N l = n = z y x I m 2l y G n z Z l, where Z l is the standard maximal (upper-triangular) unipotent subgroup of Res E/F GL l. It is clear that N l = U [1 l ] where [1 l ] is the partition of l with 1 repeated l times. Fix a nontrivial character ψ 0 of F \A F and define a character ψ of E\A E by { ψ0 (x) if E = F ; (2.5) ψ(x) := ψ 0 ( 1tr 2 E/F ( x ρ ) if E = F ( ρ). Then take w 0 to be an anisotropic vector in W l and define a character ψ l,w0 of N l by l 1 (2.6) ψ l,w0 (n) := ψ( z i,i+1 + q Wl (y l, w 0 )), i=1 where y l is the last row of y in n N l as defined in (2.4), which is regarded as a vector in W l. z

8 DIHUA JIANG AND LEI ZHANG If l = m, ψ l,w0 is a generic character on the maximal unipotent group N = N m. We will not consider this case here and hence we always assume that l < m from now on. For κ F, we choose (2.7) w 0 = y κ = e m + ( 1) m+1 κ 2 e m, which implies that q(y κ, y κ ) = ( 1) m+1 κ and that the corresponding character is (2.8) l 1 ψ l,κ (n) = ψ l,w0 (n) = ψ( z i,i+1 + y l, m l + ( 1) m+1 κ 2 y l,m m l+1). i=1 The Levi subgroup M [1 l ] = (Res E/F GL 1 ) l G n l normalizes the unipotent subgroup N l, and acts on the set of the characters of N l (F ). Each orbit for this action contains a character of the form ψ l,κ, with κ F. The M [1 ](F )-orbits are classified by the Witt Theorem and l give all F -generic characters of N l (F ). The stabilizer of ψ l,w0 in the Levi subgroup M [1 l ] is the subgroup (2.9) L l,w0 = I l γ G n γj m 2l w 0 = J m 2l w 0 = H n l, I l where H n l is defined to be U(q Wl ) with q w 0 Wl = q w 0 V Wl, and J w 0 k is the k k matrix defined inductively by J k = ( ) 1 J k 1 and J1 = 1. Define (2.10) R l,w0 := H n l N l = U(q Wl )N w 0 l. Note that dim E V and dim E W l w 0 have the different parity. If l = 0, the unipotent subgroup N 0 is trivial and we have that R 0,w0 = U(q V w 0 ). When taking w 0 = y κ, we will use the notation ψ l,yκ = ψ l,κ, L l,yκ = L l,κ and R l,yκ = R l,κ, respectively. Let φ be an automorphic form on G n (A). Define the Bessel-Fourier coefficient (or Gelfand-Graev coefficient) of φ by (2.11) B ψ l,w 0 (φ)(h) := φ(nh)ψ 1 l,w 0 (n) dn. N l (F )\N l (A)

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 9 This defines an automorphic function on the stabilizer L l,w0 (A) = H n l (A). Take a cuspidal automorphic form ϕ on H n l (A) and define the (ψ l,w0, ϕ)-bessel period or simply Bessel period of φ by (2.12) P ψ l,w 0 (φ, ϕ) := B ψ l,w 0 (φ)(h)ϕ(h) dh. H n l (F )\H n l (A) We will apply this Bessel period to a family of Eisenstein series next. 2.2. Eisenstein series. We follow the notation of [MW95] to define a family of Eisenstein series. For some j with 1 j m, let P j = M j U j be a standard maximal parabolic F -subgroup of G n with the Levi subgroup M j = Res E/F (GL j ) G n j. When j = m, the group G n m is trivial and can be omitted, if dim E V 0 = 0, or dim E V 0 = 1 and E = F. Following [MW95, Page 5], the space X Mj of all continuous homomorphisms from M j (A) to C, which is trivial on the subgroup M j (A) 1 (defined in Chapter 1 [MW95]), can be identified with C by the mapping λ s s, which is normalized as in [Sh10]. Let τ be an irreducible unitary generic automorphic representation of GL j (A E ) of the following isobaric type: (2.13) τ = τ 1 τ 2 τ r, where j = [j 1 j 2 j r ] is a partition of j and τ i is an irreducible unitary cuspidal automorphic representation of GL ji (A E ). Let σ be an irreducible automorphic representation of G n j (A) (we do not assume that σ is cuspidal). Note that σ is irrelevant if j = m and the group G n m disappears. Following the definition of automorphic forms in [MW95, I.2.17], take an automorphic form (2.14) φ = φ τ σ A(U j (A)M j (F )\G n (A)) τ σ. For λ s X Mj, the Eisenstein series associated to φ(g) is defined by (2.15) E(φ, s)(g) = E(φ τ σ, λ s )(g) = λ s φ(δg). δ P j (F )\G n(f ) It is absolutely convergent for Re(s) large and uniformly converges for g over any compact subset of G n (A), has meromorphic continuation to s C and satisfies the standard functional equation. Recall that H n l is defined to be U(q Wl ) and that dim w 0 E V and dim E W l w0 have the different parity. Let π be an irreducible cuspidal

10 DIHUA JIANG AND LEI ZHANG automorphic representation of H n l (A) and take a cuspidal automorphic form (2.16) ϕ A 0 (H n l (F )\H n l (A)) π. The global zeta integral Z(s, φ τ σ, ϕ π, ψ l,w0 ) is defined to be the following Bessel period (2.17) Z(s, ϕ π, φ τ σ, ψ l,w0 ) := P ψ l,w 0 (E(φτ σ, s), ϕ π ). Because ϕ π is cuspidal, following a standard argument as in [CP04] and [BS09] for instance, one can easily prove the following. Proposition 2.1. The global zeta integral Z(s, φ τ σ, ϕ π, ψ l,w0 ) converges absolutely and uniformly in vertical strips in C, away from the possible poles of the Eisenstein series E(φ τ σ, s), and hence is meromorphic in s C with possible poles at the locations where the Eisenstein series has poles. We remark that after the Eisenstein series E(φ τ σ, s) is properly normalized, the functional equation for Z(s, φ τ σ, ϕ π, ψ l,w0 ) relating s to s follows from that for the Eisenstein series E(φ τ σ, s). Of course, it is an interesting problem to understand the poles of Z(s, φ τ σ, ϕ π, ψ l,w0 ) in terms of the structure of the global Arthur parameter ψ of π ([Ar13] and [Mk13]) and/or in terms of the explicit construction of the endoscopy transfer in [J13]. This is in fact a long term project outlined in [J13]. 3. The eulerian property of the global integrals We prove here that the global zeta integral Z(s, φ τ σ, ϕ π, ψ l,w0 ) will be expressed as an eulerian product of local zeta integrals. When j = n = [ m ], such global zeta integrals with generic π have been studied in 2 [GRS11, Chapter 10]. Hence we assume from now on that j < n and also l < m n. We first calculate the Bessel-Fourier coefficients of the Eisenstein series and state the result in Proposition 3.3. Then, by using cuspidality, we prove that after the Eisenstein series is fully unfolded, the global zeta integral ends up with one possible non-zero term (Proposition 3.6). Then by considering certain Fourier developments to the integrands, we show that the global zeta integral is eulerian (Theorem 3.8). Recall from (2.17) that Z(s, φ τ σ, ϕ π, ψ l,w0 ) is the (ψ l,w0, ϕ π )-Bessel period of the Eisenstein series E(φ τ σ, λ s )(g), which is given by (3.1) B ψ l,w 0 (E(φτ σ, s))(h)ϕ π (h) dh, H n l (F )\H n l (A)

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 11 where the Bessel-Fourier coefficient B ψ l,w 0 (E(φ τ σ, s))(h) is given, as in (2.11), by (3.2) B ψ l,w 0 (E(φτ σ, s))(h) := N l (F )\N l (A) E(φ τ σ, s)(nh)ψ 1 l,w 0 (n) dn. We first calculate the Bessel-Fourier coefficient B ψ l,w 0 (E(φ τ σ, s)). 3.1. Calculation of Bessel-Fourier coefficients. In order to calculate the Bessel-Fourier coefficient B ψ l,w 0 (E(φ τ σ, s)), i.e. the integral in (3.2), we assume that Re(s) is large, and unfold the Eisenstein series. This leads to consider the double cosets decomposition P j \G n /P l, whose set of representatives ɛ α,β is explicitly given in [GRS11, Section 4.2]. In our situation, we put it into three cases for discussion. Case (1): G n is not the F -split even special orthogonal group. In this case, the set of representatives ɛ α,β of the double coset decomposition P j \G n /P l is in bijection with the set of pairs of nonnegative integers (3.3) {(α, β) 0 α β j and j l + β α m}. Recall that m is the Witt index of (V, q V ) defining G n. In the next two cases, the set of pairs (α, β) is also given in (3.3). Case (2-1): G n is the F -split even special orthogonal group and l + β α < m = n. In this case, the set of representatives ɛ α,β of the double coset decomposition P j \G n /P l is in bijection with the set of pairs of nonnegative integers (3.4) {(α, β) 0 α β j and j l + β α n 1}. Case (2-2): G n is the F -split even special orthogonal group and l + β α = n. In this case, there are two double cosets corresponding to each pair (α, β), and hence we may choose representatives ɛ α,β and ɛ α,β = w q ɛ α,β w q of the two double cosets corresponding to such pairs (α, β).

12 DIHUA JIANG AND LEI ZHANG In all cases, we denote by P ɛ α,β l := ɛ 1 α,β P jɛ α,β P l the stabilizer in P l, whose elements have the following form as matrices in GL m (E): (3.5) g (α,β) l = a x 1 x 2 y 1 y 2 y 3 z 1 z 2 z 3 0 b x 3 0 y 4 y 5 0 z 4 z 2 0 0 c 0 0 y 6 0 0 z 1 d u v y 6 y 5 y 3 0 e u 0 y 4 y 2 0 0 d 0 0 y 1 c x 3 x 2 0 b x 1 0 0 a where the block sizes are determined by a, a GL α, b, b GL l α j+β, c, c GL j β, d, d GL β α, and e GL m 2(l+β α). In case i = 0, GL i disappears. The stabilizer in P j consists of elements of the following form, which are the indicated matrices conjugated by w t q where t = j β: (3.6) g (α,β) j = ɛ α,β gɛ 1 α,β = a y 1 z 1 x 1 y 2 z 2 x 2 y 3 z 3 0 d y 6 0 u y 5 0 v y 3 0 0 c 0 0 x 3 0 0 x 2 b y 4 z 4 x 3 y 5 z 2 0 e y 4 0 u y 2 0 0 b 0 0 x 1 c y 6 z 1 0 d y 1 0 0 a with the block sizes as before and wq t being the t-th power of the element w q. Also, when (V, q V ) is Hermitian, w q = I m ; when E = F and (V, q V ) is of odd dimension, w q = I m ; when E = F and anisotropic kernel (V 0, q V0 ) is of dimension two, take w q = diag(i m, wq, 0 I m ), where wq 0 = diag{1, 1}; and finally, when E = F and the anisotropic kernel (V 0, q V0 is a zero space, take w q = diag(i m 1, wq, 0 I m 1 ), where wq 0 = ( ) 1. Note that l, j < n = [ 1 m], where m = dim 2 E V and m is the Witt index of V. In Case (2-2), i.e. G n is the F -split even special orthogonal group and l + β α = m, we have two double cosets corresponding to each pair (α, β). For the double coset P j ɛ α,β P l, we get exactly the same form for the stabilizer as above. For the other double coset P j ɛ α,β P l, the stabilizer in P l consists of all elements of the form (g (α,β) l ) wq. w t q

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 13 To continue the calculation, we consider further double cosets decomposition P ɛ α,β l \P l /R l,w0. Recall that H n l = U(q Wl ), dim w 0 E V and dim E W l w0 have different parity, and R l,w0 = H n l N l with H n l = Ll,w0. By [GRS11, Section 5.1], we may choose a set of representatives of form: (3.7) η ɛ,γ := ɛ where ɛ is a representative in the quotient of Weyl groups γ ɛ W GLα GL l α t GL t \W GLl and γ is a representative P w\g n l /H n l, where P w := G n l ɛ 1 α,β P jɛ α,β. We are going to show that P w is the maximal parabolic subgroup of G n l as follows. In Case (1) or Case (2-1), i.e. when G n is not the F -split even special orthogonal group or when G n is the F -split even special orthogonal group with l + β α < n, then P w is the parabolic subgroup of G n l, which preserves the standard β α dimensional totally isotropic subspace V + l,β α of W l, where (3.8) V ± l,f = Span E { e±(l+1),..., e ±(l+f) }, for 1 f m l. In Case (2-2), i.e. when G n is the F -split even special orthogonal group with l+β α = n (with j, l < n), then, when w = ɛ α,β, P w is the parabolic subgroup of G n l, which preserves V + l,m l ; and when w = ɛ α,β, P w is the parabolic subgroup of G n l, which preserves w q V + l,m l. Denote the stabilizer in H n l of the double coset P wγh n l with η ɛ,γ as defined in (3.7) by (3.9) H ηɛ,γ n l = Hγ n l = H n l γ 1 P wγ = L l,w0 γ 1 P wγ. With the above preparation, we are ready to calculate the Bessel- Fourier coefficient B ψ l,w 0 (E(φ τ σ, λ))(h) by assuming that Re(s) is large

14 DIHUA JIANG AND LEI ZHANG so that we are able to unfold the Eisenstein series. B ψ l,w 0 (E(φ, s))(h) = E(φ, s)(nh)ψ 1 l,w 0 (n) dn N l (F )\N l (A) = λφ(ɛ α,β δnh)ψ 1 l,w 0 (n) dn, ɛ α,β E j,l N l (F )\N l (A) δ P ɛ α,β l (F )\P l (F ) where E j,l is the set of representatives of P j (F )\G n (F )/P l (F ). Set N α,β,l,w0 to be the set of representatives of P ɛ α,β l (F )\P l (F )/R l,w0 (F ) and deduce that the above is equal to λφ(ɛ α,β ηδnh)ψ 1 l,w 0 (n) dn, ɛ α,β η N α,β,l,w0 N l (F )\N l (A) δ R η l,w (F )\R l,w0 (F ) 0 where R η l,w 0 = R l,w0 η 1 P ɛ α,β l η. Since R l,w0 = H n l N l, by re-arranging the summation in δ and the integration of dn, we obtain that the above is equal to λφ(ɛ α,β ηδnh)ψ 1 l,w 0 (n) dn, ɛ α,β η δ H η n l (F )\H n l(f ) N η l (F )\N l(a) where N η l = N l η 1 P ɛ α,β l η. By factoring the integration of dn, we obtain that the Bessel-Fourier coefficient B ψ l,w 0 (E(φ τ σ, s))(h), when Re(s) is large, is equal to (3.10) λφ(ɛ α,β ηδunh)ψ 1 N η l (A)\N l(a) N η l (F )\N η l (A) l,w 0 (un) du dn. ɛ α,β ;η;δ In order to determine the vanishing of the summands in (3.10), we need the following two lemmas, which are the global version of Propositions 5.1 and 5.2 in [GRS11, Chapter 5]. Lemma 3.1. If α > 0, then the inner integral in the summands of (3.10) has the following property: λφ(ɛ α,β ηunh)ψ 1 N η l (F )\N η l (A) l,w 0 (un) du = 0 for all choices of data. Proof. Fix α, β and fix an ɛ W GLα GL l α t GL t \W GLl. If there exists a simple root subgroup U of Z l such that ɛuɛ 1 lies inside U α,l α t,t, then the subgroup ɛ α,β η ɛ,γ U(ɛ α,β η ɛ,γ ) 1 lies inside U j for every γ. Since

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 15 the automorphic function λφ is invariant on U j (A) and ψ l,w0 is not trivial on U(A), λφ(ɛ α,β ηzunh)ψ 1 l,w 0 (z) dz = U(F )\U(A) λφ(ɛ α,β ηunh) ψ 1 (x) dx E\A E is identically zero. If for each simple root subgroup U of GL l, ɛuɛ 1 does not lie inside U α,l α t,t, then according to [GRS11, Lemma 5.1], ɛ is uniquely determined modulo W Mα,l α t,t, and we can take (3.11) ɛ = I t I l α t Since α 0 (and l < m), we choose a nontrivial subgroup S of N l consisting of elements of form I l α I α y I m 2l y I α I α. I l α where y = (0 r (β α) y 2 y 3 )(wq t γ) 1, and y 2 and y 3 are of size α (m 2(l + β α)) and α (β α), respectively; and when G n is split, even orthogonal, l + β α = n and the representative w in the double coset of P j \G n /P l is ɛ wq α,β, we have that t = 1, otherwise, we always have that t = 0. Since w 0 is anisotropic, w 0 is not orthogonal to V 0 V l,β α and ψ l,w0 is not trivial on S(A). By (3.6), we have (ɛ α,β η ɛ,γ )S(ɛ α,β η ɛ,γ ) 1 lies inside U j and then λφ(ɛ α,β ηxunh)ψ 1 l,w 0 (x) dx S(F )\S(A) = λφ(ɛ α,β ηunh) ψ 1 l,w 0 (x) dx S(F )\S(A) is identically zero. This proves the lemma. By Lemma 3.1 and (3.10), when Re(s) is large, the Bessel-Fourier coefficient B ψ l,w 0 (E(φ τ σ, λ))(h) is equal to (3.12) λφ(ɛ 0,β ηδunh)ψ 1 N η l (A)\N l(a) N η l (F )\N η l (A) l,w 0 (un) du dn. ɛ 0,β ;η;δ

16 DIHUA JIANG AND LEI ZHANG In particular, we may choose the ɛ in (3.11), ( which is part ) of the representation η ɛ,γ in (3.7), to be of the form: ɛ = l t I. Note that ɛ is I t one of the representatives of W GLl t GL t \W GLl. The following lemma will help us to eliminate more terms in (3.12). Lemma 3.2. If β > max {j l, 0} and γw 0 is not orthogonal to V l,β for γ P w\g n l /H n l, then the inner integral in the summands of (3.12) has the property: λφ(ɛ 0,β η ɛ,γ unh)ψ 1 N η l (F )\N η l (A) l,w 0 (un) du = 0 for all choices of data. Proof. Consider the subgroup S w (depending on w = ɛ 0,β η ɛ,γ ) of N l consisting of elements of form I t I l t y I m 2l y, where y = (0 (l t) (m 2l β) y 5 )(wq t γ) 1 with t as defined before, and y 5 is of size (l t) β. By l t = l j + β > 0 and β > 0, y 5 is not trivial. Since γw 0 is not orthogonal to V l,β, ψ l,w 0 is not trivial on S w (A F ). By (3.6), φ is invariant on (ɛ 0,β η ɛ,γ )S w (A)(ɛ 0,β η ɛ,γ ) 1. It follows that the integral λφ(ɛ 0,β η ɛ,γ xunh)ψ 1 l,w 0 (x) dx S w(f )\S w(a) = λφ(ɛ 0,β η ɛ,γ unh) ψ 1 l,w 0 (x) dx I l t I β S w(f )\S w(a) is identically zero. Since the previous integral factors through this one, this finishes the proof. To summarize the above calculation, we recall that E j,l is the set of representatives of all double cosets in P j (F )\G n (F )/P l (F ) and N β,l,w0 is the set of representatives of P ɛ β l (F )\P l (F )/R l,w0 (F ) as defined before.

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 17 Proposition 3.3. For Re(s) large, the Bessel-Fourier coefficient of the Eisenstein series as in (3.2), B ψ l,w 0 (E(φ τ σ, λ))(h), is equal to λφ(ɛ β ηδunh)ψ 1 N η l (A)\N l(a) N η l (F )\N η l (A) l,w 0 (un) du dn, ɛ β where η δ ɛ β = ɛ 0,β Ej,l 0, which is the subset of E j,l consisting of elements with α = 0; η = diag(ɛ, γ, ɛ ) belongs to Nβ,l,w 0 0, which is the subset of N ( ) β,l,w0 I consisting of elements with α = 0, ɛ = l t, and t = I t j β, and has the property that if β > max {j l, 0}, then γw 0 is orthogonal to V l,β for γ P w(f )\G n l (F )/H n l (F ); δ belongs to H η n l (F )\H n l(f ). We will apply the formula in Proposition 3.3 to the calculation of the global zeta integral Z(s, φ τ σ, ϕ π, ψ l,w0 ) and use the cuspidality of ϕ π to prove that the global zeta integral Z(s, φ τ σ, ϕ π, ψ l,w0 ) is eulerian. 3.2. Global zeta integrals. By applying Proposition 3.3 to the global zeta integral in (3.1), we get (3.13) Z(s, φ τ σ, ϕ π, ψ l,w0 ) = B ψ l,w 0 (E(φτ σ, s))(h)ϕ π (h) dh H n l (F )\H n l (A) = ɛ β ;η;δ [H n l ] ϕ π (h) N η l (A)\N l(a) [N η l ] λφ(ɛ β ηδunh)ψ 1 l,w 0 (un) du dn dh where [H n l ] := H n l (F )\H n l (A) and [N η l ] := N η l (F )\N η l (A); and the summations ɛ β ;η;δ and other conditions for the representatives are given in Proposition 3.3. We combine the summation on δ and the integration dh and obtain that Z(s, φ τ σ, ϕ π, ψ l,w0 ) is equal to (3.14) ϕ π (h) λφ(ɛ β ηunh)ψ 1 n [N η l ] l,w 0 (un) du dn dh, ɛ β ;η H η n l (F )\H n l(a) where the integration n is over N η l (A)\N l(a). The following lemma is to make use of the cuspidality of ϕ π.

18 DIHUA JIANG AND LEI ZHANG Lemma 3.4. Let α = 0 and γ be a representative in P w\g n l /H n l. For a representative η = η ɛ,γ, if the stabilizer H η n l is a proper maximal parabolic subgroup of H n l, then the corresponding summand in (3.14) has the property: ϕ π (h) λφ(ɛ β η ɛ,γ unh)ψ 1 H η n l (F )\H n l(a) n [N η l ] l,w 0 (un) du dn dh = 0 for all choices of data. Proof. Let H η n l = M U, where U is the unipotent radical of the parabolic subgroup H η n l of H n l. Since φ is P j (F )-invariant, φ is leftinvariant with respect to the image under the adjoint action by ɛ 0,β η ɛ,γ of the unipotent radical U (A) of H η n l (A). Then we deduce that ϕ π (h) λφ(ɛ β η ɛ,γ unh)ψ 1 H η n l (F )\H n l(a) n [N η l ] l,w 0 (un) du dn dh = ϕ π (u h) du λφ(ɛ β η ɛ,γ unh)ψ 1 h [U ] n [N η l ] l,w 0 (un) du dn dh where is over M (F )U (A)\H h n l (A). By the cuspidality of π, we have that ϕ π (u h) du = 0, U (F )\U (A) and hence the whole integral is zero. This proves the lemma. To proceed with our calculation from (3.14) with Lemma 3.4, we discuss more explicitly each double coset. By Proposition 3.3, the representatives ɛ β have the restrictions that either β = max{0, j l} or β > max{0, j l} with γw 0 being orthogonal to V l,β for γ P w(f )\G n l (F )/H n l (F ). In order to understand the double cosets decomposition γ P w(f )\G n l (F )/H n l (F ), we recall the following descriptions. Lemma 3.5 (Proposition 4.4, [GRS11]). Let X be a non-trivial totally isotropic subspace of W l and P be the maximal parabolic subgroup of G n l preserving X. Then (1) If dim E X < Witt(W l ), then the set P \G n l /H n l consists of two elements. (2) Assume that Witt(w 0 ) = dim E X = Witt(W l ). (a) If G n l is unitary, then P \G n l /H n l consists of two elements.

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 19 (b) If G n l is orthogonal and dim W l 2 dim X + 2, then P \G n l /H n l consists of two elements. (c) If G n l is orthogonal and dim W l = 2 dim X + 1, then P \G n l /H n l consists of three elements. (3) If dim E X = Witt(W l ) and Witt(w 0 ) = dim E X 1, then P \G n l /H n l consists of one element. (4) If dim E W l = 2 dim E X, then Witt(w 0 ) = dim X 1, and, in particular, P \G n l /H n l consists of one element. We consider the case when G n l is not the F -split even orthogonal group or the case when G n l is the F -split even orthogonal group with l + β < n. In these cases, we must have that dim X = β. If l + β < m, then P w\g n l /H n l consists of two elements. It remains to consider that l + β = m. If l + β < n, we must have that l + β = m < n and hence G n l can not be the F -split even special orthogonal group. In this case l + β = m < n, if G n l is an F -quasisplit even unitary group, then Witt(W l yκ ) = Witt(W l ) 1 and P w\g n l /H n l has only one element; if G n l is an odd special orthogonal group, then { # P w\g 3, if Witt(w0 W l ) = m l, n l /H n l = 1, if Witt(w0 W l ) = m l 1; and if G n l is an F -quasisplit even special orthogonal group (with dim V 0 = 2) or an F -quasisplit odd unitary group, then { # P w\g 2, if Witt(w0 W l ) = m l, n l /H n l = 1, if Witt(w0 W l ) = m l 1. It remains to consider the case when G n l is an F -split even special orthogonal group with l + β = n. In this case, P w\g n l /H n l consists of two elements. Now we continue the calculation from Equation (3.14) and write Z β,η = ϕ(h) λφ(ɛ 0,β ηunh)ψ 1 H η n l (F )\H n l(a) n [N η l ] l,w 0 (un) du dn dh for each summand in (3.14). Then, we apply Lemmas 3.4 and 3.5 to find the nonvanishing summand in the summation in (3.14). For max{0, j l} β < m l, P w\g n l /H n l consists of two elements γ 1 and γ 2 such that γ 1 w 0 is orthogonal to V l,β and γ 2w 0 is not orthogonal to V l,β. If γw 0 is orthogonal to V l,β, the stabilizer Hγ n l = H η n l is a maximal parabolic subgroup of H n l, which preserves the isotropic subspace wqv t + l,β w 0.

20 DIHUA JIANG AND LEI ZHANG In this case, by Lemmas 3.2 and 3.4, there may be left with nonzero summands in the summation (3.14), which are with the representative ɛ β for β = max{0, j l} and with the representative η = η ɛ,γ having the property that γw 0 is not orthogonal to V l,β. For β = m l, there are six different cases. Also, we have that β = m l > max{0, j l}. If G n is the F -split even special orthogonal group, then there are two (P j, P l )-double cosets corresponding to the pair (0, β) and the chosen representatives are ɛ 0,β and ɛ 0,β. For these two cases, their stabilizer preserves two maximal isotropic subspace of W l with different orientations, and P w\g n l /H n l consists of one element in both cases with its stabilizer H γ n l = Hη n l being a maximal parabolic subgroup. Hence by Lemma 3.4, the corresponding summands are all zero. If G n is not the F -split even special orthogonal group and Witt(W l y κ ) = Witt(W l ) 1, there is only one double coset whose stabilizer is a maximal parabolic subgroup of H n l. Hence by Lemma 3.4, the corresponding summand is zero. If Witt(W l y κ ) = Witt(W l ) and G n is the odd unitary group or F -quasi-split even special orthogonal group, the stabilizers are similar to the case β < m l as discussed above. Hence by Lemmas 3.2 and 3.4, the corresponding summands are all zero. If G n is the odd special orthogonal group and Witt(W l y κ ) = Witt(W l ) 1, then P w\g n l /H n l consists of three elements and the representatives are chosen in [GRS11, (4.33)]. Two stabilizers are maximal parabolic subgroups of H n l, and the third representative γ satisfies the property that γw 0 is not orthogonal to V l,β. Hence by Lemmas 3.2 and 3.4, the corresponding summands are all zero. By the discussions above, we deduce that the corresponding summands are all zero, because of Lemmas 3.2 and 3.4. In conclusion, we are left with the case where β = max{0, j l} and γ with the property that the corresponding stabilizer is not a proper maximal parabolic subgroup of H γ n l, i.e. γw 0 is not orthogonal to V l,β. In this case, the representative η = η ɛ,γ is uniquely determined by β = max{0, j l}. In fact, if j l, then β = 0. It follows that η 1 = η ɛ,γ with γ = I m 2l and ( ) I (3.15) ɛ = l j ; I j

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 21 and if j > l, then β = j l. It implies that η 2 = η ɛ,γ with ɛ = I l and (3.16) γ = I m j I j l I V0 I j l I m j. Therefore, we are left with only one summand in the summation in (3.14) with the above representative, accordingly. Next we are going to write the only integral more explicitly (Proposition 3.6) and get ready to prove that it is eulerian in the next subsection. If j l, then β = 0. In this case we have that P w = G n l and H γ n l = H n l with ɛ and γ given above. Then the global zeta integral in (3.14) has the following expression: Z(s, φ τ σ, ϕ π, ψ l,w0 ) = Z 0,η1 = (3.17) [H n l ] ϕ π (h) N η l (A)\N l(a) [N η l ] λφ(ɛ 0,0 η 1 unh)ψ 1 l,w 0 (un) du dn dh. where [H n l ] := H n l (F )\H n l (A) and [N η l ] := N η l (F )\N η l (A). Recall that R l,w0 = H n l N l and R η l,w 0 = H η n l N η l. The stabilizers are, respectively, given by c 0 0 0 0 b y 4 z 4 0 (3.18) R η l,w 0 = e y 4 0 b 0 where c, c is of size j j, b, b of size (l j) (l j), and e of size (m 2l) (m 2l); and c 0 0 0 0 b y 4 z 4 0 (3.19) (ɛ 0,0 η ɛ,γ )R η l,w 0 (ɛ 0,0 η ɛ,γ ) 1 = e y 4 0 b 0 c with c Z j and b Z l j. (Here Z f is the maximal upper-triangular unipotent subgroup of GL f.) If j > l, then β = j l. In this case, ɛ = I l and γ is given in (3.16). The double coset decomposition P w\g n l /H n l produces two c

22 DIHUA JIANG AND LEI ZHANG representatives which, as given in [GRS11, Section 4.4], are γ = I m 2l and the γ as given in (3.16). For the representative γ = I m 2l, the corresponding stabilizer H γ n l is a proper maximal parabolic subgroup. Then, the corresponding integral in (3.14) is zero by Lemma 3.4. Now for the γ as given in (3.16), we have that the global zeta integral is expressed as Z(s, φ τ σ, ϕ π, ψ l,w0 ) = Z j l,η2 = ϕ π (h) H η n l (F )\H n l(a) N η l (A)\N l(a) (3.20) λφ(ɛ β η 2 unh)ψ 1 [N η l ] l,w 0 (un) du dn dh, where [N η l ] = N η l (F )\N η l (A). The stabilizers are given, respectively, c 0 0 y 6 0 d u v y (3.21) η ɛ,γ R η l,w 0 ηɛ,γ 1 6 = e u 0 d 0 where c, c is of size l l and c Z l, d, d of size (j l) (j l), and e of size (m 2j) (m 2j); and d y 6 u 0 v c 0 0 0 (3.22) (ɛ 0,β η ɛ,γ )R η l,w 0 (ɛ 0,β η ɛ,γ ) 1 = e 0 u c y 6. We conclude this subsection with the following proposition which summarizes the calculations discussed up to this point. Proposition 3.6. Take notation as above. If j l, then β = 0 and the global zeta integral has the following expression: Z(s, φ τ σ, ϕ π, ψ l,w0 ) = ϕ π (h) [H n l ] c N η l (A)\N l(a) d [N η l ] λφ(ɛ 0,0 ηunh)ψ 1 l,w 0 (un) du dn dh, where [H n l ] := H n l (F )\H n l (A) and [N η l ] := N η l (F )\N η l (A); and with η = η 1 given explicitly above. If j > l, then β = j l and the

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 23 global zeta integral has the following expression: Z(s, φ τ σ, ϕ π, ψ l,w0 ) = ϕ π (h) H η n l (F )\H n l(a) N η l (A)\N l(a) [N η l ] λφ(ɛ 0,β ηunh)ψ 1 l,w 0 (un) du dn dh, where [N η l ] = N η l (F )\N η l (A); and with η = η 2 given explicitly above. We are going to show that the global zeta integrals are eulerian based on Proposition 3.6. This is done for the two cases, separately. 3.3. Eulerian products: 0 < l < j case. We must have that β = j l. By Proposition 3.6, the global integral Z(s, φ τ σ, ϕ π, ψ l,w0 ) is equal to the following integral (3.23) ϕ π (h) λφ(ɛ 0,β ηunh)ψ 1 N η l (A)\N l(a) [N η l ] l,w 0 (un) du dn dh, h where h H η n l (F )\H n l(a); [N η l ] = N η l (F )\N η l (A); and η = η ɛ,γ is as given explicitly above. In order to show that the integral in (3.23) is an eulerian product of local zeta integrals, we first show that the integral in (3.23) can be expressed as an adelic integration of certain Bessel periods, which is stated in Proposition 3.7, and then we show the resulting integral in Proposition 3.7 factorizes as an eulerian product by means of the uniqueness of Bessel functionals, which is Theorem 3.8. First, we want to understand the Fourier coefficient of λφ: (3.24) λφ(ɛ 0,β ηuh)ψ 1 [N η l ] l,w 0 (u) du. We identify g Res E/F (GL j ) with its embedding ĝ of (g, I m 2j, g ) into the Levi subgroup Res E/F (GL j ) G n j of G n. Then, (ɛ 0,β η)n η l (ɛ 0,βη) 1 is the group Z l, consisting of elements z of the form ( ) (3.25) z Iβ y = Res z E/F (GL j ) with z Z l. By conjugating the element ɛ 0,β η across the variable u and changing the variable by (ɛ 0,β η)u(ɛ 0,β η) 1 ẑ, the Fourier coefficient in (3.24) reduces to (3.26) λφ(ẑ ɛ 0,β ηh)ψ 1 [Z l ] l,w 0 ((ɛ 0,β η) 1 ẑ (ɛ 0,β η)) dz.

24 DIHUA JIANG AND LEI ZHANG It follows from the choice of the representatives ɛ 0,β character has following expression: and η that the (3.27) ψ 1 l,w 0 ((ɛ 0,β η) 1 ẑ (ɛ 0,β η)) = ψ(z 1,2 + +z l 1,l +( 1) m+1 κ 2 y β,1), where z = (z e,f ) l l. If we write elements z of Z l as z = (z e,f ) j j, then this character can be written as (3.28) ψ Z l,κ(z ) := ψ(( 1) m+1 κ 2 z β,β+1 + z β+1,β+2 + + z j 1,j ). In this way, the Fourier coefficient in (3.26) can be written as (3.29) φ ψ Z l,κ λ (h) := λφ(ẑ h)ψ Z l,κ(z ) dz. [Z l ] Hence the global zeta integral Z(s, φ τ σ, ϕ π, ψ l,w0 ), which is expressed as in (3.23), is equal to the following integral (3.30) H η n l (F )\H n l(a) ϕ π (h) N η l (A)\N l(a) φ ψ Z l,κ λ (ɛ 0,β ηnh)ψ 1 l,w 0 (n) dn dh, with η = η ɛ,γ given explicitly above. Next, we want to understand the structure of the subgroup H η n l. By (3.9), H η n l = H n l γ 1 P wγ with η = η ɛ,γ, and P w = G n l P ɛ 0,β l is the parabolic subgroup of G n l, preserving the totally isotropic subspace as in (3.8). Denote by V + l,β P η w = P w ηh n l η 1 = P w γh n l γ 1. Then the elements of P η w are of form: I l d d 1 u v 1 v 1 0 0 v 1 (3.31) e 0 u 1 d 1 d I l with d 1 + ( 1) m+1 κv 2 1 = 0, where d 1 and v 1 are column vectors of size β 1; d, d are of size (β 1) (β 1); and e belongs to G n j. Note that P w η is the stabilizer of γy κ in P w. Hence we have (3.32) P η w = (GL(V + l,β 1 ) G n j) U η (V + l,β 1 ),

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 25 where U η (V + + l,β 1 ) is the subgroup of U(V l,β 1 ) consisting elements which fix the vector γy κ. Here U(V + l,β 1 ) is the unipotent radical of the parabolic subgroup P (V + l,β 1 ) of G n l preserving the totally isotropic subspace V + l,β 1. In order to precede our calculation, we need to know the structure of stabilizer in H n l. Let Q β 1,η be the parabolic subgroup of H n l, which preserves the totally isotropic subspace (η 1 V + l,β ) y κ of W l y κ and has the Levi decomposition Recall that the space W l y κ Q β 1,η = L β 1,η V β 1,η. has the polar decomposition W l y κ = V + l, m l 1 W 0 V l, m l 1, where W 0 is a non-degenerate subspace of W l y κ with the same anisotropic kernel as W l y κ and with dim E W 0 = dim E V 0 + 1 3. By taking as before that w 0 = y κ = e m +( 1) m+1 κ 2 e m, we obtain that W 0 = Span {y κ } V 0. Then it is easy to check that and (η 1 V + l,β ) y κ = Span {e m j+l+1,..., e m 1 } = V + m β,β 1, where H n j+1 := U(q Wj 1 ) with yκ It follows that L β 1,η = GL(V + m β,β 1 ) H n j+1, W j 1 y κ = V + l, m j W 0 V l, m j. GL(V + m β,β 1 ) = GL((η 1 V + l,β ) y κ ) = η 1 GL(V + l,β 1 )η Hη n l, and V β 1,η = η 1 U η (V + l,β 1 )η Hη n l. It is easy to check that Hence we have η 1 W j = V + l, m j V 0 V l, m j = y κ (W j 1 y κ ). U(q η 1 Wj ) = η 1 U(q Wj )η = η 1 G n j η H η n l. Putting together all these subgroups, we obtain the structure of H η n l : (3.33) H η n l = (GL(V + m β,β 1 ) U(q η 1 W j )) V β 1,η. Finally, we are ready to consider the partial Fourier expansion of cuspidal automorphic forms ϕ π on H n l (A). Let Z η l,β 1 be the maximal

26 DIHUA JIANG AND LEI ZHANG unipotent subgroup of GL(V + m β,β 1 ) consisting of elements of following type: η 1 I l d I m 2j+2 d I l η with d Z β 1. Then N η l,β 1 := Zη l,β 1 V β 1,η is a unipotent subgroup of H n l of the type as defined in (2.4) with the corresponding character defined as in (2.6) with y κ. Then, it is easy to check that the corresponding stabilizer H y κ n j+1 is equal to U(q ), which is isomorphic η 1 W j to G n j. Define C β 1,η := V β 1,η V β,η, which is also equal to {u V β 1,η u e m = e m } and is a normal subgroup of H η n l. It follows that C β 1,η \H η n l = P 1 β H y κ n j+1, where P 1 β is the mirabolic subgroup of Res E/F (GL β ) given by P 1 β = {( ) } d d1 Res 0 1 E/F (GL β ). Going back to the expression (3.30) of Z(s, φ τ σ, ϕ π, ψ l,w0 ), the inner integral (3.34) Φ(h) := φ ψ Z l,κ λ (ɛ 0,β ηnh)ψ 1 l,w 0 (n) dn N η l (A)\N l(a) as function in h, is left C β 1,η (A)-invariant. We recall that N l consists of elements of the form c x 1 x 2 x 3 y 6 x 4 x 5 I m j x 4 I j l y 6 I m 2 m x 3 I j l x 2 I m j x 1 c

L-FUNCTIONS OF CLASSICAL GROUPS OF HERMITIAN TYPE 27 where c Z l and the stabilizer N η l consists element of the form c 0 0 0 y 6 0 0 I m j 0 I j l y 6 I m 2 m 0. I j l 0 I m j 0 c Then η(n η l \N l)η 1 is isomorphic to a complementary subgroup consisting of elements of the form I l x 1 x 2 0 x 3 I j l 0 n 0 (x 1, x 2, x 3 ) := I m 2j x 2 I j l x, 1 and ψ l,κ (η 1 n 0 (x 1, x 2, x 3 )η) is not trivial on x 1. In detail, ψ l,κ Int η 1(n 0 (x 1, x 2, x 3 )) = ψ((x 1 ) l,j l ). The stabilizer (ɛ 0,β η)n η l (ɛ 0,βη) 1 in P j consists of elements of the form I j l y 6 c e c y 6. I j l The image of the domain of integration N η l \N l under the adjoint action of ɛ 0,β η is a subgroup U j,η of U j (the unipotent radical of the parabolic subgroup opposite P j ), consisting of elements of the form (3.35) I j l I l x 2 I m 2j x 1 x 3 x 2 I l x 1 I j l. Denote by ψ (m j+l,j l) the character over (ɛ 0,β η)n η l \N l(ɛ 0,β η) 1, given by ψ (m j+l,j l) (n) = ψ(n m j+l,j l ) where n m j+l,j l = (x 1 ) l,j l. I l

28 DIHUA JIANG AND LEI ZHANG Recall that η 1 C β 1,η η consists of elements of the form I l I β 1 0 u 0 0 1 0 0 0 I m 2j 0 u. 1 0 I β 1 It follows that N η l,β 1 = Zη l,β 1 V β 1,η = Z β C β 1,η. As a subgroup of P j, the stabilizer (ɛ 0,β η)n η l,β 1 (ɛ 0,βη) 1 consists of elements of the form d d 1 0 u 0 v 1 v 1 v 1 I l 0 I m 2j u, I l 0 1 d 1 d where d Z β 1. Note that (ɛ 0,β η)z β (ɛ 0,β η) 1 consists of elements of the above form with all matrices being zero except d and d 1 and (ɛ 0,β η)c β 1,η (ɛ 0,β η) 1 is equal to the subgroup where d = I β 1 and d 1 = 0. It follows that the expression in (3.30) of the global zeta integral Z(s, φ τ σ, ϕ π, ψ l,w0 ) is equal to (3.36) Φ(h) ϕ π (ch) dc dh, H η n l (F )C β 1,η(A)\H n l (A) [C β 1,η ] where [C β 1,η ] := C β 1,η (F )\C β 1,η (A), as before. C β 1,η (A)-invariant. We denote the inner integration by c ϕ C β 1,η π (h) = ϕ π (ch) dc. The integral in (3.36) becomes (3.37) [C β 1,η ] H η n l (F )C β 1,η(A)\H n l (A) I l Φ(h)ϕ C β 1,η π (h) dh. Note that Φ is Now we are in the standard step in the global unfolding process using partial Fourier expansion along the mirabolic subgroup Pβ 1. Both functions Φ(h) and ϕ C β 1,η (h) are automorphic on P 1 β (A) and ϕ C β 1,η π (h)