Unramified Unitary Duals for Split Classical p adic Groups; The Topology and Isolated Representations

Similar documents
UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS; THE TOPOLOGY AND ISOLATED REPRESENTATIONS

Marko Tadić. Introduction Let F be a p-adic field. The normalized absolute value on F will be denoted by F. Denote by ν : GL(n, F ) R the character

Colette Mœglin and Marko Tadić

TWO SIMPLE OBSERVATIONS ON REPRESENTATIONS OF METAPLECTIC GROUPS. In memory of Sibe Mardešić

SPHERICAL UNITARY REPRESENTATIONS FOR REDUCTIVE GROUPS

TWO SIMPLE OBSERVATIONS ON REPRESENTATIONS OF METAPLECTIC GROUPS. Marko Tadić

SQUARE INTEGRABLE REPRESENTATIONS OF CLASSICAL p-adic GROUPS CORRESPONDING TO SEGMENTS

Jacquet modules and induced representations

Weyl Group Representations and Unitarity of Spherical Representations.

ON THE RESIDUAL SPECTRUM OF HERMITIAN QUATERNIONIC INNER FORM OF SO 8. Neven Grbac University of Rijeka, Croatia

ON THE CLASSIFICATION OF RANK 1 GROUPS OVER NON ARCHIMEDEAN LOCAL FIELDS

A. I. BADULESCU AND D. RENARD

Geometric Structure and the Local Langlands Conjecture

Primitive Ideals and Unitarity

Classification of discretely decomposable A q (λ) with respect to reductive symmetric pairs UNIVERSITY OF TOKYO

Subquotients of Minimal Principal Series

JACQUET MODULES AND IRRREDUCIBILITY OF INDUCED REPRESENTATIONS FOR CLASSICAL p-adic GROUPS

BLOCKS IN THE CATEGORY OF FINITE-DIMENSIONAL REPRESENTATIONS OF gl(m n)

Unitarity of non-spherical principal series

0 A. ... A j GL nj (F q ), 1 j r

joint with Alberto Mínguez

14 From modular forms to automorphic representations

The Contragredient. Spherical Unitary Dual for Complex Classical Groups

On the Notion of an Automorphic Representation *

H(G(Q p )//G(Z p )) = C c (SL n (Z p )\ SL n (Q p )/ SL n (Z p )).

Discrete Series Representations of Unipotent p-adic Groups

(E.-W. Zink, with A. Silberger)

SPHERICAL UNITARY DUAL FOR COMPLEX CLASSICAL GROUPS

A partition of the set of enhanced Langlands parameters of a reductive p-adic group

ON THE STANDARD MODULES CONJECTURE. V. Heiermann and G. Muić

THE RESIDUAL EISENSTEIN COHOMOLOGY OF Sp 4 OVER A TOTALLY REAL NUMBER FIELD

ON RESIDUAL COHOMOLOGY CLASSES ATTACHED TO RELATIVE RANK ONE EISENSTEIN SERIES FOR THE SYMPLECTIC GROUP

Reducibility of generic unipotent standard modules

UNITARY DUAL OF GL(n) AT ARCHIMEDEAN PLACES AND GLOBAL JACQUET-LANGLANDS CORRESPONDENCE

On the unitary dual of classical and exceptional real split groups. Alessandra Pantano, University of California, Irvine MIT, March, 2010

AHAHA: Preliminary results on p-adic groups and their representations.

Whittaker models and Fourier coeffi cients of automorphic forms

ON THE RESIDUAL SPECTRUM OF SPLIT CLASSICAL GROUPS SUPPORTED IN THE SIEGEL MAXIMAL PARABOLIC SUBGROUP

arxiv: v2 [math.rt] 9 Jul 2014

Hermitian and Unitary Representations for Affine Hecke Algebras

ON THE MODIFIED MOD p LOCAL LANGLANDS CORRESPONDENCE FOR GL 2 (Q l )

On the Self-dual Representations of a p-adic Group

TRILINEAR FORMS AND TRIPLE PRODUCT EPSILON FACTORS WEE TECK GAN

PARABOLIC INDUCTION AND JACQUET MODULES OF REPRESENTATIONS OF O(2n, F)

CENTRAL CHARACTERS FOR SMOOTH IRREDUCIBLE MODULAR REPRESENTATIONS OF GL 2 (Q p ) Laurent Berger

Factorization of unitary representations of adele groups Paul Garrett garrett/

Fourier Coefficients and Automorphic Discrete Spectrum of Classical Groups. Dihua Jiang University of Minnesota

Local Langlands correspondence and examples of ABPS conjecture

Endoscopic character relations for the metaplectic group

SOME APPLICATIONS OF DEGENERATE EISENSTEIN SERIES ON Sp 2n

REDUCIBLE PRINCIPAL SERIES REPRESENTATIONS, AND LANGLANDS PARAMETERS FOR REAL GROUPS. May 15, 2018 arxiv: v2 [math.

Hodge Structures. October 8, A few examples of symmetric spaces

Traces, Cauchy identity, Schur polynomials

Cuspidality and Hecke algebras for Langlands parameters

On Cuspidal Spectrum of Classical Groups

COHOMOLOGY OF ARITHMETIC GROUPS AND EISENSTEIN SERIES: AN INTRODUCTION, II

On Certain L-functions Titles and Abstracts. Jim Arthur (Toronto) Title: The embedded eigenvalue problem for classical groups

Representations with Iwahori-fixed vectors Paul Garrett garrett/ 1. Generic algebras

LECTURE 4: REPRESENTATION THEORY OF SL 2 (F) AND sl 2 (F)

Representation Theory

REPRESENTATIONS OF FINITE GROUPS OF LIE TYPE: EXERCISES. Notation. 1. GL n

10 l-adic representations

MAT 445/ INTRODUCTION TO REPRESENTATION THEORY

On parabolic induction on inner forms of the general linear group over a non-archimedean local field

Fundamental Lemma and Hitchin Fibration

SYMMETRIC SUBGROUP ACTIONS ON ISOTROPIC GRASSMANNIANS

Exercises on chapter 1

REPRESENTATIONS OF S n AND GL(n, C)

What is the Langlands program all about?

Proof of a simple case of the Siegel-Weil formula. 1. Weil/oscillator representations

Notes on p-divisible Groups

COURSE SUMMARY FOR MATH 504, FALL QUARTER : MODERN ALGEBRA

U = 1 b. We fix the identification G a (F ) U sending b to ( 1 b

REPRESENTATIONS OF REDUCTIVE p-adic GROUPS. Under revision March Table of Contents

Math 121 Homework 5: Notes on Selected Problems

Math 210C. The representation ring

15 Elliptic curves and Fermat s last theorem

IN POSITIVE CHARACTERISTICS: 3. Modular varieties with Hecke symmetries. 7. Foliation and a conjecture of Oort

REPRESENTATION THEORY. WEEK 4

Unitarizable Minimal Principal Series of Reductive Groups

Holomorphy of the 9th Symmetric Power L-Functions for Re(s) >1. Henry H. Kim and Freydoon Shahidi

Classification of root systems

Dieudonné Modules and p-divisible Groups

1 Adeles over Q. 1.1 Absolute values

ALGEBRAIC GROUPS J. WARNER

Notes on nilpotent orbits Computational Theory of Real Reductive Groups Workshop. Eric Sommers

Citation Osaka Journal of Mathematics. 43(2)

LIFTING OF GENERIC DEPTH ZERO REPRESENTATIONS OF CLASSICAL GROUPS

LECTURE 2: LANGLANDS CORRESPONDENCE FOR G. 1. Introduction. If we view the flow of information in the Langlands Correspondence as

Invariant Distributions and Gelfand Pairs

Bruhat Tits buildings and representations of reductive p-adic groups

Quaternionic Complexes

Lemma 1.3. The element [X, X] is nonzero.

The Casselman-Shalika Formula for a Distinguished Model

In honour of Stephen Gelbart for his 60 th birthday. Marko Tadić

Math 249B. Nilpotence of connected solvable groups

Non separated points in the dual spaces of semi simple Lie groups

Irreducible subgroups of algebraic groups

QUIVERS AND LATTICES.

On the equality case of the Ramanujan Conjecture for Hilbert modular forms

Transcription:

ESI The Erwin Schrödinger International Boltzmanngasse 9 Institute for Mathematical Physics A-1090 Wien, Austria Unramified Unitary Duals for Split Classical p adic Groups; The Topology and Isolated Representations Goran Muić Marko Tadić Vienna, Preprint ESI 1937 (007) July 4, 007 Supported by the Austrian Federal Ministry of Education, Science and Culture Available via http://www.esi.ac.at

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS; THE TOPOLOGY AND ISOLATED REPRESENTATIONS GORAN MUIĆ AND MARKO TADIĆ Introduction The classification of irreducible unitary representations of reductive groups over local fields is fundamental problem of harmonic analysis with various possible applications, like those in the number theory and the theory of automorphic forms. The class of unramified unitary representations is especially important in mentioned applications. These representations occur in the following set-up that we fix in this paper. Let F be a non Archimedean local field of an arbitrary characteristic and O is its ring of integers. (When we work with the classical groups we are obliged to require that the characteristic of F is different than.) Let G be the group of the F points of a F split reductive group G. An irreducible (complex) representation π of G is unramified if it has a vector fixed under G(O). The set of equivalence classes Irr unr (G) of unramified irreducible representations of G is usually described by the Satake classification (see [Cr]). This classification is essentially the Langlands classification for those representations. We write Irr u,unr (G) Irr unr (G) for the subset consisting of unramified unitarizable representations. We equip that set with the topology of the uniform convergence of matrix coefficients on compacts ([F], [Di]; see also [T1], [T6]). The good understanding of unramified unitarizable representations is fundamental for the theory of automorphic forms since almost all components of cuspidal and residual automorphic representations are unramified and unitarizable. By the good understanding we mean the following: (1) To have an explicit classification of unramified unitary duals (with explicit parameters and with Satake parameters easily computed from them). The converse to (1) is not trivial and it is important. More precisely, from the point of view of the theory of automorphic forms, it would be important to have a way to decide from Satake parameters if a representation is unitarizable (even for general linear groups we need a simple algorithm). This leads () To have an effective way (an algorithm) for testing unitarity of an arbitrary unramified representation in Irr unr (G) given by its Satake parameter. (3) To understand the topology in terms of the classification, especially isolated points in Irr u,unr (G), which are exactly the isolated points in the whole unitary dual which are unramified representations (this would be in particular interesting to understand from point of view of automorphic spectra). If G = GL(n), then those tasks (and much more) are accomplished in the works of the second named author ([T4], [T5]; see also [T], [T3]) more than twenty years ago. In Section 4 of the present paper we give a simple solution (due to the second named author) to those problems based 1991 Mathematics Subject Classification. Primary E35. Both authors were partly supported by Croatian Ministry of Science, Education and Sports grant # 037-037794- 804. 1

GORAN MUIĆ AND MARKO TADIĆ entirely on [Ze], without use of a result of Bernstein on irreducibility of unitary parabolic induction proved in [Be] and used in earlier proof of the classification (see Theorem 4-1.) In this paper we give the solutions to the problems (1) (3) in the case of the split classical groups G = S n where S n is one of the groups Sp(n, F), SO(n + 1, F), O(n, F) (see Section 1 for the precise description of the groups). Regarding (3), we have an explicit description of isolated points, and () gives an algorithm for getting limit points for a given sequence in the dual. Further, algorithm from () gives parameters in (1) in the case of unitarizability (other direction is obvious). This paper is the end of a long effort ([M4], [M6]). The approach to the problem (1) is motivated and inspired by the earlier work [LMT], in creation of which ideas of E. Lapid played an important role (see also [T1] which is a special case of [LMT]). On the formal level, the formulation of the solution (see Theorem 0-8) to the problem (1) is dual to that of the one in [LMT], but in our unramified case it has a much more satisfying formulation. This is not surprising since we are dealing with very explicit representations. On the other hand, the proofs are more involved. For example, the proof of the unitarity of the basic building blocks (see Theorem 0-4 bellow) requires complicated arguments with the poles of degenerate Eisenstein series (see [M6]). The problems () and (3) were not yet considered for split classical groups. A characteristic of our approach is that at no point in the proofs, the explicit internal structure of representations plays a role. This is the reason that this can be considered as an external approach to the unramified unitary duals (of classical groups), which is a kind of a continuation of such approaches in [T3], [T], [LMT] etc. We expect that our approach has a natural Archimedean version (in a similar way that [LMT] covers non-archimedean and Archimedean cases, or as earlier paper [T3] have the Archimedean version [T], with the same description of unitary duals for general linear groups and the proofs along the same strategy). Now, we describe our results. They are stated in Section 5 in more detail than here. After being acquainted with the basic notation in Section 1, the reader may proceed to read Section 5 directly. We shall in the introduction use classical notation for induced representations. In the rest of the paper we shall use the notation adopted to the case of general linear and classical groups (which very often simplify substantially arguments in proofs). A part of the exposition bellow makes perfect sense also for Archimedean fields (we shall comment on this later). We fix the absolute value of F which satisfies d(ax) = a dx. Let χ be an unramified character of F and l Z >0. Then we consider the following induced representation of GL(l, F): Ind GL(l,F) P ( l 1 l 1 χ 1 χ l 1 χ) which has a character χ det as the unique irreducible quotient. We denote this character by: We introduce Langlands dual groups as follows: G = S n = SO(n + 1, F) G = S n = O(n, F) [ l 1, l 1 ](χ). Ĝ(C) = Sp(n, C) GL(N, C); N = n Ĝ(C) = O(n, C) GL(N, C); N = n G = S n = Sp(n, F) Ĝ(C) = SO(n + 1, C) GL(N, C); N = n + 1.

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 3 The local functorial lift σ GL(N,F) of σ Irr unr (S n ) to GL(N, F) is always defined and it is an unramified representation. (See (10-1) in Section 10 for the precise description.) It is an easy exercise to check that the map σ σ GL(N,F) is injective. This lift plays the key role in the solutions of the problems () and (3). In order to describe Irr u,unr (S n ) we need to introduce more notation. Let sgn u be the unique unramified character of order two of F and let 1 F be the trivial character of F. Let χ {1 F,sgn u }. Then we define α χ as follows: if S n = O(n, F), then α χ = 0 if S n = SO(n + 1, F), then α χ = 1 if S n = Sp(n, F), then α sgnu = 0 and α 1F = 1. We refer to Remark 5-3 for the explanation of this definition in terms of rank one reducibility. A pair (m, χ), where m Z >0 and χ is an unramified unitary character of F is called Jordan block. The following definition can be found in [M4] (see Definition 5-4 in Section 5): Definition 0-1. Let n > 0. We write Jord sn (n) for the collection of all sets Jord of Jordan blocks such that the following holds: χ {1 F,sgn u } and m (α χ + 1) Z for all (m, χ) Jord { n if S n = SO(n + 1, F) or S n = O(n, F); m = n + 1 if S n = Sp(n, F), (m,χ) Jord and, additionally, if α χ = 0, then card {k; (k, χ) Jord} Z. Let Jord Jord sn (n). Then, for χ {1 F,sgn u }, we let We let Jord χ = Jord χ = {k; (k, χ) Jord}. { Jord(χ); Jord χ is even; Jord(χ) { α χ + 1}; Jord χ is odd. We shall write Jord χ according to the character χ (the case l 1F = 0 or l sgnu = 0 is not excluded): { χ = 1F : a 1 < a < < a l1 F χ = sgn u : b 1 < b < < b lsgnu. Next, we associate to Jord Jord sn (n), the unramified representation σ(jord) of S n defined as the unique irreducible unramified subquotient of the representation parabolically induced from the representation ( l 1 F i=1 [ a i 1, a ) ( i 1 1 ] (1 F ) lsgn u j=1 [ b j 1, b ) j 1 1 ] (sgn u ). Recall that irreducible tempered (resp., square integrable) representations of a reductive group can be characterized by satisfying certain inequalities (resp., strict inequalities). The first author in [M4] defines negative (resp., strongly negative) irreducible representations as those ones which satisfy reverse inequalities (resp., strict inequalities). An unramified representation is strongly

4 GORAN MUIĆ AND MARKO TADIĆ negative if its Aubert dual is in discrete series. See [M4] for more details. We have the following result (see [M4]; Theorem 5-8 in Section 5): Theorem 0-. Let n Z >0. The map Jord σ(jord) defines a one to one correspondence between the set Jord sn (n) and the set of all strongly negative unramified representations of S n. The inverse mapping to Jord σ(jord) will be denoted by σ Jord(σ). An unramified representation is negative if its Aubert dual is tempered. Negative representations are classified in terms of strongly negative as follows ([M4]; Theorem 5-10 in Section 5): Theorem 0-3. Let σ neg Irr unr (S n ) be a negative representation. Then there exists a sequence of the pairs (l 1, χ 1 ),..., (l k, χ k ) (l i Z 1, χ i is an unramified unitary character of F ), unique up to a permutation and taking inverses of characters, and the unique strongly negative representation σ sn such that σ neg is a subrepresentation of parabolically induced representation ( Ind Sn [ l 1 1, l 1 1 ] (χ1) [ l k 1, l ) k 1 ] (χk) σ sn. Conversely, for a sequence of the pairs (l 1, χ 1 ),..., (l k, χ k ) (l i Z >0, χ i is an unramified unitary character of F ) and( a strongly negative representation σ sn, the unique ) irreducible unramified subquotient of Ind Sn [ l 1 1, l 1 1 ](χ1) [ l k 1, l k 1 ](χk) σ sn is negative and it is a subrepresentation. We let Jord(σ neg ) to be the multiset Jord(σ sn ) + k i=1 (multisets are sets where multiplicities are allowed). {(l i, χ i ), (l i, χ 1 i )} The proofs of Theorems 0- and 0-3 given in [M4] are obtained with the techniques of Jacquet modules enabling results to hold for all characteristic of F (different than two). The key result for this paper is the following deep result (see [M6]; see Theorem 5-11): Theorem 0-4. Every negative representation is unitarizable. Every strongly negative representation is a local component of a global representation appearing in the residual spectrum of a split classical group defined over a global field. The following theorem is a consequence of above results: Theorem 0-5. Let σ Irr unr (S n ) be a negative representation. Then its lift to GL(N, F) is given by: σ GL(N,F) (l,χ) Jord(σ) [ l 1, l 1 ](χ). Moreover, its Arthur parameter W F SL(, C) Ĝ(C) GL(N, C) is given by: (l,χ) Jord(σ) χ V l, where V l is the unique algebraic representation of SL(, C) of the dimension l.

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 5 In order to describe the whole Irr u,unr (S n ), we need to introduce more notation. We write M unr (S n ) for the set of pairs (e, σ neg ), where: e is a (perhaps empty) multiset consisting of a finite number of triples (l, χ, α) where l Z >0, χ is an unramified unitary character of F, and α R >0. σ neg Irr S nneg (this defines n neg ) is negative satisfying: n = l card e(l, χ) + n neg. (l,χ) For l Z >0 and an unramified unitary character χ of F, we denote by e(l, χ) the submultiset of e consisting of all positive real numbers α (counted with multiplicity) such that (l, χ, α) e. We attach σ Irr unr (S n ) to (e, σ neg ) in a canonical way. By definition, σ is the unique irreducible unramified subquotient of the following induced representation: (( (0-6) Ind Sn (l,χ,α) e [ l 1, l 1 ) ) ]( αχ) σ neg. (We remark that the definition of σ does not depend on the choice of ordering of elements in e.) In order to obtain unitary representations, we impose further conditions on e in the following definition (see Definition 5-13): Definition 0-7. Let M u,unr (S n ) be the subset of M unr (S n ) consisting of the pairs (e, σ neg ) satisfying the following conditions: (1) If χ {1 F,sgn u }, then e(l, χ) = e(l, χ 1 ) and 0 < α < 1 for all α e(l, χ). () If χ {1 F,sgn u } and l (α χ + 1) Z, then 0 < α < 1 for all α e(l, χ). (3) If χ {1 F,sgn u } and l (α χ + 1) Z, then 0 < α < 1 for all α e(l, χ). Moreover, if we write the exponents that belong to e(l, χ) as follows: 0 < α 1 α u 1 < β 1 β v < 1. (We allow u = 0 or v = 0.) Then we must have the following: (a) If (l, χ) Jord(σ neg ), then u + v is even. (b) If u > 1, then α u 1 1. (c) If v, then β 1 < < β v. (d) α i {1 β 1,..., 1 β v } for all i. (e) If v 1, then the number of indices i such that α i ]1 β 1, 1 ] is even. (f) If v, then the number of indices i such that α i ]1 β j+1, 1 β j [ is odd. The main result of the paper is the following explicit description of Irr u,unr (S n ) (see Theorem 5-14): Theorem 0-8. Let (e, σ neg ) M u,unr (S n ). Then Ind (( Sn (l,χ,α) e [ l 1, l 1 ]( αχ) ) ) σ neg is irreducible. Moreover, the map (( (e, σ neg ) Ind Sn (l,χ,α) e [ l 1, l 1 ) ) ]( αχ) σ neg is a one to one correspondence between M u,unr (S n ) and Irr u,unr (S n ).

6 GORAN MUIĆ AND MARKO TADIĆ This result is proved in Sections 7, 8, and 9. The preparation for the proof is done in the first two of these three sections. In Section, where we recall (from [T8]) some general principles for proving unitarity and non unitarity, we prove new criterion for non unitarity (see (RP) in Section ). In Section 6 we describe all necessary reducibility facts explicitly (most of them are already established in [M4]). Theorem 0-8 clearly solves problem (1) for the split classical groups. In Section 10 we describe a simple algorithm that has: INPUT: an arbitrary irreducible unramified representation σ Irr unr (S n ) given by its Satake parameter OUTPUT: tests the unitarity of σ and at the same time constructs the corresponding pair (e, σ neg ) if σ is unitary. The algorithm is based on the observation that, for a unitarizable σ, the Zelevinsky data (see [Ze]; or Theorem 1-6 here) of the lift σ GL(N,F) is easy to describe by the datum (e, σ neg ) of σ. (See Lemma 10-7.) This solves the problem () stated above. (We observe that this problem is almost trivial for GL(n, F). (See Theorem 4-1.)) The algorithm is very simple, and one can go almost directly to the algorithm in Section 10, to check if some irreducible unramified representation given in terms of Satake parameters is unitarizable (Definition 5-13 is relevant for the algorithm). In the appendix to the paper we give examples of use of the algorithm. The algorithm has ten steps (some of them quite easy), but usually only a few of them enter the test (see the appendix). It would be fairly easy to write a computer program (possible to handle classical groups of ranks exceeding tens of thousands) for determining unitarizability in terms of Satake parameters. Finally, we come to the problem (3). In Section 3 we show that Irr u,unr (S n ) is (naturally homeomorphic to) a closed (in fact, compact) subset of the complex manifold consisting of all Satake parameters for S n (see Theorem 3-5 and Theorem 3-7). The results of this section almost directly follow from [T6]. In Section 11 we determine the isolated points in Irr u,unr (S n ). To describe the result, we introduce more notation. Let σ Irr u,unr (S n ) be a strongly negative representation. Let χ {1 F,sgn u }. Then we write Jord(σ) χ for the set of l such that (l, χ) Jord(σ). If a Jord(σ) χ is not the minimum, then we write a for the greatest b Jord(σ) χ such that b < a. We have the following: a a is even (whenever a is defined). Now, we are ready to state the main result of Section 11. It is the following theorem (see Theorem 11-3): Theorem 0-9. A representation σ Irr u,unr (S n ) is isolated if and only if σ is strongly negative, and for every χ {1 F,sgn u } such that Jord(σ) χ the following holds: (1) a a 4, for all a Jord(σ) χ whenever a is defined. () If Jord χ {1}, then min Jord χ \ {1} 4. (We do not claim that 1 Jord χ in (). If 1 Jord χ, then () claims that minjord χ 4.) Since Irr u,unr (S n ) is an open subset of Irr u (S n ), the above theorem also classifies the isolated representations in the whole unitary dual Irr u (S n ), which are unramified.

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 7 As an example, let S 1 = Sp(, F) = SL(, F). Then the trivial representation 1 SL(,F) is strongly negative and Jord(1 SL(,F) ) = {(3,1 F )}. As it is well known, it is not isolated and this theorem confirms that. Let n and let S n = Sp(n, F). Then the trivial representation 1 Sp(n,F) is strongly negative and Jord(1 Sp(n,F) ) = {(n + 1,1 F )}. Clearly, 1 Sp(n,F) is isolated (as it is well known from [K]). We may consider the degenerate case n = 0. Then Sp(0, F) is the trivial group and 1 Sp(0,F) is its trivial representation. It is reasonable to call such representation strongly negative and let Jord(1 Sp(0,F) ) = {(1,1 F )}. Apart from that case, it is always Jord χ {1}. Similarly, if we let S n = SO(n + 1, F) (n > 0), then 1 SO(n+1,F) is strongly negative. We have Jord(1 SO(n+1,F) ) = {(n,1 F )}. As it is well known, it is not isolated for n = 1 and this theorem confirms that. It is isolated for n (as it is well known from [K]). We close this introduction by several comments. First recall that in [BbMo], D. Barbasch and A. Moy address the first of the three problems that we consider in our paper. Their related paper [BbMo] contains some very deep fundamental results on unitarizability, like the fact that the Iwahori-Matsumoto involution preserves unitarity in the Iwahori fixed vector case. Their Hecke algebra methods are opposite to our methods. Their approach is based on a careful study of internal structure of representations (on Iwahori fixed vectors), based on the Kazhdan-Lusztig theory [KLu]. Their main result Theorem A (on page 3 of [BbMo]) states that a parameter of any irreducible unitarizable unramified representation of a classical group is a complementary series from an induced from a tempered representation tensored with a GL-complementary series (in other words, that it comes from a complementary series starting with a representation induced by a negative representation (from Theorem 0-4) tensored with a GL-complementary series). They do not determine parameters explicitly (they observe that it is hard to describe parameters explicitly; see page 3 of their paper). They get the unitarizability of negative representations by local (Hecke algebra) methods, but do not relate them to the automorphic spectrum like Theorem 0-4 does. Summing up without going into details, description in [BbMo] covers partially 0-8 and in the paper there are complementary series in a number of cases, but the paper does not give the full picture: explicit parameterization of the unramified unitary duals. Our (quite different) approach gives explicit parameters (of different type), and these explicit parameters have a relatively simple combinatorial description (observe that all that we need for describing unitary duals are (one-dimensional) unramified unitary characters of general linear groups, parabolic induction, and taking irreducible unramified subquotients). Explicit parameters are fundamental for the harmonic analysis. For the difference of [BbMo] where they do not give complete proofs in all cases, but rather treat representative cases in detail, we give complete proofs here. Let us note that except motivations coming directly from the theory of automorphic forms, like applications to analytic properties of L-functions etc., a motivation for us to get explicit classification of unramified unitary duals was to be able to answer question () (which would be definitely important also for the study of automorphic forms), and to classify isolated points in (3) (which can be significant in the study of automorphic spectra). Recall that in two important cases where the unitarizability is understood (the case of general linear groups and the case of generic representations of quasi-split classical groups), the classification theorem is uniform for Archimedean case as well as for the non-archimedean case (see [T3], [T] and [LMT]). Moreover, the proofs are essentially the same (not only analogous). Therefore, it is natural to expect this to be the case for unramified unitarizable representations of classical split groups. Having this in mind, we shall comment briefly the Archimedean case. Assume

8 GORAN MUIĆ AND MARKO TADIĆ F = R or F = C. Let be the ordinary absolute value (resp., square of it) of F = R (resp., F = C), i.e. the modulus character of F (like in the non-archimedean case). If we fix some suitable maximal compact subgroup K of S n, then we may consider K spherical representations, and call them unramified. Then the above constructions and statements make sense. More precisely, define Jord sn (n) using only 1 F (i.e. all unramified characters χ of F which satisfy χ = χ 1 ). Call representations from {σ(jord); Jord Jord sn } strongly negative (define σ(jord) in the same way as in the non-archimedean case). Define negative representations as those ones which come as irreducible unramified representations of representations displayed in Theorem 0-4. Then Theorem 0-4 follows from [M6] (where is the uniform proof for non-archimedean and Archimedean case). Now, it is natural to ask if Theorem 0-8 is true also in that set up? We have not been able to check that using the results of [Bb]. But there is a number of facts which suggest this. The first is Theorem 0-4. Second is that number of arguments in the proof of Theorem 0-8 make sense in the Archimedean case (like in [LMT]). The complex case shows particular similarity (consult [T1]). We expect that the approach of this paper will be extended to the Archimedean case. We also expect that Theorem 0-9 describing isolated representations holds in the Archimedean case, with the similar proof. We plan to address Archimedean case in the future. At the end, let us note that one possible strategy to get answer to (1) (but not to () and (3)) would be to try to get Theorem 0-8 from the classification in [LMT], using Barbasch-Moy fundamental result that the Iwahori-Matsumoto involution preserves unitarity in the Iwahori fixed vector case (the proof of which is based on Kazhdan-Lusztig theory [KLu]). This much less direct approach would still leave a number of questions to be solved. Furthermore, we expect that the approach that we present in our paper has much more chances for generalization to (more) general case, then the one that we just discussed above (as it was the case for general linear groups, where first was obtained classification of unramified irreducible unitary representations using Zelevinsky classification, which very soon lead to the classification of general irreducible unitary representations in terms of Zelevinsky, as well as Langlands classification). The first named author would like to thank D. Ban for explaining to him the results of [Bn]. The second named author is thankful to D. Renard and A. Moy for discussions on the topics related (directly or indirectly) to this paper. We would like to thank J. Schwermer for several invitations to Schroedinger Institute for Mathematical Physics in Vienna where various versions of the paper were written. 1. Preliminary Results Let Z, R, and C be the ring of rational integers, the field of real numbers, and the field of complex numbers, respectively. Let F be a non Archimedean field of characteristic different from. We write O for the maximal compact subring of F. Let p the unique maximal ideal in O. Let be a fixed generator of p and let q be the number of elements in the corresponding residue field of O. We write ν for the normalized absolute value of F. Let χ be a character of F. We can uniquely write χ = ν e(χ) χ u where χ u is a unitary character and e(χ) R. Let G be an l group (see [BeZ]). We will consider smooth representations of G on complex vector spaces. We call them shortly representations. If σ is a representation of G the we write V σ for its space. Its contragredient representation will be denoted by σ and the corresponding non degenerate cannonical pairing will be denoted by, : V eσ V σ C. If σ 1 and σ are representations of G, then we write Hom G (σ 1, σ ) for the space of all G intertwining maps σ 1 σ. We say that σ 1 and σ are equivalent, σ 1 σ, if there is a bijective ϕ Hom G (σ 1, σ ). Let Irr(G) be the set of

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 9 equivalence classes of irreducible admissible representations of G. Let R(G) be the Grothendieck group of the category M adm.fin.leng. (G) of all admissible representations of finite length of G. If σ is an object of M adm.fin.leng. (G), then we write s.s.(σ) for its semi simplification in R(G). Frequently, in computations we will write shortly σ instead of s.s.(σ). If G is the trivial group, then we write its unique irreducible representation as 1. Next, we shall fix the notation for the general linear group GL(n, F) of type n n with entries in F. Let I n the identity matrix in GL(n, F). Let t g be the transposed matrix of g GL(n, F). The transposed matrix of g GL(n, F) with respect to the second diagonal will be denoted by τ g. If χ is a character of F and π is a representation of GL(n, F), then the representation (χ det) π of GL(n, F) will be written as χπ. We fix the minimal parabolic subgroup Pmin GLn of GL(n, F) consisting of all upper triangular matrices in GL(n, F). A standard parabolic subgroup P of GL(n, F) is a parabolic subgroup containing Pmin GLn. There is a one to one correspondence between the set of all ordered partitions of n, α = (n 1,..., n k ) (n i Z >0 ), and the set of standard parabolic subgroups of GL(n, F) attaching to a partition α a parabolic subgroup P α consisting of all block upper triangular matrices: p = (p ij ) 1 i,j k, p ij is a matrix of type n i n j, p ij = 0 (i > j). The parabolic subgroup P α admits a Levi decomposition P α = M α N α, where M α = {diag(g 1,..., g k ); g i GL(n i, F) (1 i k)} N α = {p P α ; p ii = I ni (1 i k)} Let π i be a representation of GL(n i, F) (1 i k). Then we consider π 1 π k as a representation of M α as usual: π 1 π k (diag(g 1,..., g k )) = π 1 (g 1 ) π k (g k ), and extend it trivially accros N α to the representation of P α denoted by the same letter. Then we form (normalized) induction written as follows (see [BeZ1], [Ze]): π 1 π k = i n,α (π 1 π k ) := Ind GL(n,F) P α (π 1 π k ) In this way obtain the functor M adm.fin.leng. (M α ) in,α M adm.fin.leng. (GL(n, F)) and a group homomorphism R(M α ) in,α R(GL(n, F)). Next, if π is a representation of GL(n, F), then we form the normalized Jacquet module r α,n (π) of π (see [BeZ1]). It is a representation of M α. In this way obtain a functor M adm.fin.leng. (GL(n, F)) rα,n M adm.fin.leng. (M α ) and a group homomorphism R(GL(n, F)) rα,n R(M α ). The functors i n,α and r α,n are related by the Frobenius reciprocity: Hom GL(n,F) (π, i n,α (π 1 π k )) Hom Mα (r α,n (π), π 1 π k ). We list some additional basic properties of induction: π 1 (π π 3 ) (π 1 π ) π 3, π 1 π and π π 1 have the same composition series, if π 1 π is irreducible, then π 1 π π π 1, χ(π 1 π ) (χπ 1 ) (χπ ), for a character χ of F, π 1 π π 1 π. We take GL(0, F) to be the trivial group (we consider formally the unique element of this group as a 0 0 matrix and the determinant map GL(0, F) F ). We extend formally as follows:

10 GORAN MUIĆ AND MARKO TADIĆ π 1 = 1 π := π for every representation π of GL(n, F). The listed properties hold in this extended setting. We also let r (0), 0 (1) = 1. Now, we fix the basic notation for the split classical groups. Let 00... 01 J n = 00... 10 : GL(n, F). 10... 0 The symplectic group (of rank n 1) is defined as follows: { Sp(n, F) = g GL(n, F); g [ 0 Jn J n 0 ] tg = [ ]} 0 Jn. J n 0 Next, the split orthogonal groups special odd-orthogonal groups (both of rank n 1) are defined by SO(n, F) = { g SL(n, F); g J n tg = J n } O(n, F) = { g GL(n, F); g J n tg = J n } We take Sp(0, F), SO(0, F), O(0, F) to be the trivial groups (we consider their unique element formally as 0 0 matrix). In the sequel, we fix one the following three series of the groups: S n = Sp(n, F), n 0 S n = O(n, F), n 0 S n = SO(n + 1, F), n 0. Let n > 0. Then the minimal parabolic subgroup P Sn min of S n consisting of all upper triangular matrices is fixed. A standard parabolic subgroup P of S n is a parabolic subgroup containing Pmin Sn. There is a one to one correspondence between the set of all finite sequences of positive integers of total mass n and the set of standard parabolic subgroups of S n defined as follows. For α = (m 1,..., m k ) of the total mass m := l i=1 m i n, we let { P Pα Sn := (m1,...,m k, (n m), m k,m k 1,...,m 1 ) S n ; S n = Sp(n, F), O(n, F) P (m1,...,m k, (n m)+1, m k,m k 1,...,m 1 ) S n ; S n = SO(n + 1, F). (The middle term (n m) is omitted if m = n). The parabolic subgroup Pα Sn admits a Levi decomposition P α = Mα SnNSn α, where M Sn α N Sn α = {diag(g 1,..., g k, g, τ g 1 k,...,τ g1 1 ); g i GL(m i, F) (1 i k), g S n m } = {p Sn α ; p ii = I ni i} Let π i be a representation of GL(n i, F) (1 i k). Let σ be a representation of S n m. Then we consider π 1 π k σ as a representation of Mα Sn as usual: π 1 π k σ(diag(g 1,..., g k, g, τ g 1 k,...,τ g 1 1 )) = π 1(g 1 ) π k (g k ) σ(g), and extend it trivially accros Nα Sn to the representation of Pα Sn denoted by the same letter. Then we form (normalized) induction written as follows (see [T9]): π 1 π k σ = I n,α (π 1 π k σ) := Ind Sn (π Pα Sn 1 π k σ)

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 11 In this way obtain a functor M adm.fin.leng. (M Sn α ) In,α M adm.fin.leng. (S n ) and a group homomorphism R(M Sn R(S n ). Next, if π is a representation of S n, then we form the normalized Jacquet module Jacq α,n (π) of π. It is a representation of Mα Sn. In this way obtain a func- α ) In,α tor M adm.fin.leng. (S n ) Jacq α,n M adm.fin.leng. (Mα Sn ) and a group homomorphism R(S n ) Jacq α,n R(Mα Sn ). Here Frobenius reciprocity tells: Further Hom Sn (π, I n,α (π 1 π k σ)) Hom M Sn (Jacq α α,n(π), π 1 π k σ). π 1 (π σ) (π 1 π ) σ, π σ π σ π σ and π σ have the same composition series, if π σ is irreducible, then π σ π σ. We remark that the third listed property follows for example from the general result ([BeDeK], Lemma 5.4), but in our case there is a proof that is simpler and based on the following result of Waldspurger (see [MœViW]): σ σ, S n = SO(n + 1, F), O(n, F) σ σ x, S n = Sp(n + 1, F), [ ] [ ] 0 Jn 0 Jn where x GL(n, F) satisfies x tx = ( 1), and σ J n 0 J n 0 x (g) = σ(x 1 gx), g S n. Now, the stated property is obvious for S n = SO(n+1, F) and O(n, F). Let S n = Sp(n+1, F). Then (π σ) y π σ x, where y = diag(i m, x, I m ) (π is a representation of GL(m, F)). If π σ = m ρ ρ is a decomposition into irreducible representations in R(S n ), then π σ = m ρ ρ, and, by a result of Waldspurger, we have the following: (1-1) π σ = π σ x = (π σ) y = m ρ ρ y = m ρ ρ = π σ = π σ. In this paper we mostly work with unramified representations. Let n 1. If G is one of the groups GL(n, F), Sp(n, F), O(n, F), or SO(n + 1, F), then we let K be its maximal compact subgroup of the form GL(n, O), Sp(n, O), O(n, O), or SO(n + 1, O), respectively. We say that σ Irr(G) is unramified if it has a non zero vector invariant under K. Unramified representations of G are classified using the Satake classification. To explain the Satake classification, we let P min = M min N min be the minimal parabolic subgroup of G as described above: M min = {diag(x 1,..., x n )}; G = GL(n, F) M min = {diag(x 1,..., x n, x 1 n,..., x 1 )}; G = Sp(n, F), O(n, F) M min = {diag(x 1,..., x n, 1, x 1 n 1,..., x 1)}; G = SO(n + 1, F). Let W = N G (M min )/M min be the Weyl group of G. It acts on M min by the conjugation: w.m = wmw 1, w W, m M min. This action extend to the action on the characters χ of M min in the usual way: w(χ)(m) = χ(w 1 mw), w W, m M min. 1

1 GORAN MUIĆ AND MARKO TADIĆ Explicitly, using above description of M min, we fix the isomorphism M min (F ) n (considering only the first n coordinates). If G = GL(n, F), then W acts on M min as the group of the permutations of n letters. If G is one of the groups Sp(n, F), O(n, F), or SO(n + 1, F), then W acts on M min as a group generated by the group of the permutations of n letters and the following transformation: (x 1, x,..., x n ) (x 1 1, x,..., x n ). We have the following classification result (see [Cr]; [R] for O(n, F)): Theorem 1-. (i) Assume that χ 1,..., χ n is a sequence of unramified characters of F. Then the induced representation Ind G P min (χ 1 χ n ) contains the unique unramified irreducible subquotient, say σ G (χ 1,..., χ n ). (ii) Assume that χ 1,..., χ n and χ 1,..., χ n are two sequences of unramified characters of F. Then σ G (χ 1,..., χ n ) σ G (χ 1,..., χ n) if and only if there is w W such that χ 1 χ n = w(χ 1 χ n ). In another words, if and only if there is a permutation α of and a sequence ǫ 1,..., ǫ n {±1} such that χ i = χǫ i α(i), i = 1,..., n. (ǫ 1 = 1,..., ǫ n = 1 for G = GL(n, F).) (iii) Assume that σ Irr(G) is an unramified representation. Then there exists a sequence (χ 1,..., χ n ) of unramified characters of F such that σ σ G (χ 1,..., χ n ). Every such sequence we call a supercuspidal support of σ. We let Irr unr (G) be the set of equivalence classes of irreducible unramified representations of G. We consider the trivial representation of the trivial group to be unramified. We let { Irr unr (GL) = n 0 Irr unr (GL(n, F)) (1-3) Irr unr (S) = n 0 Irr unr (S n ). There is another more precise classification of the elements of Irr unr (GL) that we describe (see [Ze]). In order to write down the Zelevinsky s classification we will introduce some notation. Let χ be an unramified character of F, and let n 1, n R, n n 1 Z 0. We shall denote by [ν n 1 χ, ν n χ] or [n 1, n ] (χ) the set {ν n 1 χ, ν n 1+1 χ,..., ν n χ}, and call a segment of unramified characters. To such a segment = [ν n 1 χ, ν n χ], Zelevinsky has attached a representation which is the unique irreducible subrepresentation of ν n 1 χ ν n 1+1 χ ν n χ. This representation is a character We find convenient to write it as follows: ν (n 1+n )/ χ 1 GL(n n 1 +1,F). (1-4) or [n 1, n ] (χ). We let (1-5) e( ) = e([n 1, n ] (χ) ) = (n 1 + n )/ + e(χ). Related to Theorem1-, we see = [n 1, n ] (χ) = σ GL(n 1+n +1,F) (ν n 1 χ, ν n 1+1 χ,..., ν n χ). The segments 1 and of unramified characters are linked if and only 1 is a segment but 1 and 1. We consider the empty set as a segment of unramified characters. It is not linked to any other segment. We let = 1 Irr GL(0, F).

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 13 Now, we give the Zelevinsky s classification. Theorem 1-6. (i) Let 1,..., k be a sequence of segments of unramified characters. Then 1 k is reducible if and only there are indices i, j, such that the segments i and j are linked. Moreover, if 1 k is irreducible, then it belongs to Irr unr (GL). (ii) Conversely, if σ Irr unr (GL), then there is, up to a permutation, a unique sequence of segments of unramified characters 1,..., k such that σ 1 k. The similar classification exists in the case of the classical groups (see [M4]). We recall it in Sections 5 and 6. We end this section by the following remark: Remark 1-7. It follows from Theorem 1- (ii) that every unramified representation σ Irr unr (S) is self dual. Also, if π Irr unr (GL), then there exists a unique unramified irreducible subquotient, say σ 1 of π σ. The representation σ 1 is self dual, and it is also a subquotient of π σ. (See the basic properties for the induction for the split classical groups listed above.). Some General Results on Unitarizability Let G be a connected reductive p adic group or O(n, F) (n 0). We recall that a contragredient representation π of G will be denoted by π. We write π for the complex conjugate representation of the representation of π. We remind the reader that this means the following. In the representation space V π we change the multiplication to α. new v := ᾱ. old v, α C, v V π In this way we obtain V π. We let π(g)v = π(g)v, g G, v V π = V π. It is easy to see the following: π π. The Hermitian contragredient of the representation of π is defined as follows: π + := π. Moreover, we have the following: (H-IC) Let P = MN be a parabolic subgroup of G. Then Ind G P (σ)+ Ind G P (σ+ ). A representation π Irr(G) is said to be Hermitian if there is a non degenerate G invariant Hermitian form, on V π. This means the following: (-1) (-) (-3) (-4) αv + βw, u = α v, u + β w, u v, w = w, v π(g)v, π(g)v = v, w, v, w = 0, w V π, implies v = 0. Since π is irreducible, the Hermitian form, is unique up to a non zero real scalar. Let Irr + (G) be the set of equivalence classes of irreducible Hermitian representations of G. Since we work with unramified representations, we let (-5) { Irr +,unr (GL) = n 0 Irr +,unr (GL(n, F)) Irr +,unr (S) = n 0 Irr +,unr (S n ). We list the following basic properties of Hermitian representations: (H-Irr) If π Irr(G), then π Irr + (G) if and only if π π + (H-Ind) Let P = MN be a parabolic subgroup of G. Let σ Irr + (M). Then there is a non trivial G invariant Hermitian form on Ind G P (σ). In particular, if Ind G P (σ) is irreducible, then it is Hermitian.

14 GORAN MUIĆ AND MARKO TADIĆ SIn addition, a Hermitian representation π Irr(G) is said to be unitarizable if the form, is definite. Let Irr u (G) be the set of equivalence classes of irreducible unitarizable representations of G. We have the following: Irr u (G) Irr + (G) Irr(G). In this paper we classify unramified unitarizable representations (see (1-3)) { Irr u,unr (GL) = n 0 Irr u,unr (GL(n, F)) (-6) Irr u,unr (S) = n 0 Irr u,unr (S n ). Now, we recall principles used in the construction and the classification of unitarizable unramified representations. Some of them are already well known (see [T8]), but some of them are new (see (NU-RP)). Bellow, P = MN denotes a parabolic subgroup of G, and σ denotes an irreducible representation of M. (UI) Unitary parabolic induction: unitarizability of σ implies that parabolically induced representation Ind G P (σ) is unitarizable. (UR) Unitary parabolic reduction: suppose that σ is a Hermitian representation such that parabolically induced representation Ind G P (σ) is irreducible and unitarizable, then σ is (irreducible) unitarizable representation. (D) Deformation (or complementary series): let X be a connected set of characters of M such that each representation Ind G P (χσ), χ X, is Hermitian and irreducible. Now, if Ind G P (χ 0σ) is unitarizable for some χ 0 X, then the whole family Ind G P (χσ), χ X consists of unitarizable representations. (ED) Ends of deformations: suppose that Y is a set of characters of M, and X a dense subset of Y satisfying (D); then each irreducible subquotient of any Ind G P (χσ), χ Y, is unitarizable. Sometimes we get important irreducible unitarizable representations in the following way. Denote by Z the center of G. Let k be a global field, P k the set of places of k, k v the corresponding completion of k, A k the ring of adeles of k, ω a unitary character of Z(A k ) and L(ω, G(k)\G(A k )) the representation of G(A k ) v Pk G(k v ) by right translations on the space of square integrable functions on G(A k ) which transforms for the action of Z(A k ) according to ω. Suppose that π is an irreducible representation of G = G(F). (RS) Residual automorphic spectrum factors: if F k v for some global field k and v P k, and there exists an irreducible (non-cuspidal) subrepresentation Π of L(ω, G(A k )) such that π is isomorphic to a (corresponding) tensor factor of Π, then π is unitarizable. It is evident that π as above is unitarizable. But this construction is technically much more complicated than the above four. It requires computation of residues of Eisenstein series. The last principle is not necessary to use for the classification of Irr u,unr (GL), but we use it [M6] in order to prove the unitarity of basic building blocks of Irr u,unr (S). (See Theorem 5-11 in Section 5.) In addition, the following simple remark is useful for proving non unitarity. Obviously, Cauchy- Schwartz inequality implies that matrix coefficients of unitarizable representations are bounded. Now, (D) implies directly the following: Remark -7. (unbounded matrix coefficients) Let X be a connected set of characters of M such that each representation Ind G P(χσ), χ X, is Hermitian and irreducible and that Ind G P (χ 0σ) has an unbounded matrix coefficient for some χ 0 X. Then all Ind G P(χσ), χ X, are not unitarizable.

UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 15 In addition, we will use the following two criteria for proving non unitarity. The criteria are very technical. In a special case they were already applied in [LMT]. We present them here in the most general form. Let P = M N be a maximal parabolic subgroup of G. Assume that the Weyl group W(M) = N G (M)/M has two elements. (It has one or two elements always.) We write w 0 for a representative of the nontrivial element in W(M). Assume that σ Irr(M) is an irreducible unitarizable representation such that w 0 (σ) σ. Then there is a standard normalized intertwining operator (at least in the cases that we need) N(δP s σ) : IndG P(δP s σ) IndG P(δ s P σ). We have the following: (N-1) N(δP s σ)n(δ s P σ) = id (N-) N(δP s σ) is Hermitian, and therefore holomorphic, for s 1R. Let, σ be M invariant definite Hermitian form on V σ. Then (-8) f 1, f s = f 1 (k), N(δPσ)f s (k) σ dk K is a Hermitian form on Ind G P (δp s π). It is non degenerate whenever IndG P (δp s σ) is irreducible and N(δP s σ) is holomorphic. Now, we make the following two assumptions: (A-1) If Ind G P (δs P σ) is reducible at s = 0, then N(σ) is non trivial. (A-) If Ind G P(δP s σ) is irreducible at s = 0, let s 1 > 0 be the first point of reducibility (this must exist because of Remark -7). We assume that N(δP s σ) is holomorphic and non trivial for s ]0, s 1 ]. (Then (N-1) implies that N(δ s P σ) is holomorphic for s ]0, s 1[). We assume that N(δP s σ) has a pole at s = s 1 of an odd order. If (A-1) holds, then (N-1) implies that Ind G P (σ) is a direct sum of two non trivial (perhaps reducible) representations on which N(σ) acts as id and id, respectively. Now, since Ind G P (δs P σ) is irreducible and N(δP s σ) is holomorphic for s > 0, s close to 0, we conclude that, s is not definite. Hence Ind G P (δp s σ) is not unitarizable for s > 0, s close to 0. If (A-) holds, then we write k for the order of pole of N(δP s σ) at s = s 1. We realize the family of representations Ind G P (δs P σ) (s C) on the compact picture, say X. Let f X such that F s := (s s 1 ) k N(δ s P σ)f is holomorphic and non zero at s = s 1. We see that F s is real analytic near s 1. Let h X. We compute using (N-1) h, F s s = h(k), N(δPπ)F s s (k) σ dk K (-9) = (s s ) k h(k), f(k) σ dk. Now, we apply some elementary results from the Linear algebra (see ([Vo], Theorem 3., Proposition 3.3)) to our situation. First, we may assume that f belongs to some fixed K isotypic component, say E, of X. Since X is an admissible representation of K, we see dime <. We consider the restriction of the family of Hermitian forms, s on E. We write this restriction as follows (, ) t, where t = s s 1. Let E = E 0 E 1 E N = {0} be the filtration of E defined as follows. The space E n is the space of vectors e E for which there is a neighborhood U of 0 and a (real) analytic function f e : U E satisfying (i) f e (0) = e (ii) e E the function s (f e (s), e ) t vanishes at 0 to order at least n. K

16 GORAN MUIĆ AND MARKO TADIĆ Let F = F s1. Since t F t+s1 is a real analytic function from a neighborhood U of 0 into E, (-9) implies F E k. Moreover, since also f E, we see that (-9) applied to f = h implies that F E k+1. We conclude (-10) E k /E k+1 0. Next, we define a Hermitian form (, ) n on E n by the formula (e, e ) n 1 = lim t 0 t n (f e(s), f e (s)) s. (It is easy to see that this definition is independent of the choices of f e and f e.) The radical of the form (, ) n is exactly E n+1. We write (p n, q n ), for the signature character on E n /E n+1. It is proved in ([Vo], Proposition 3.3) that for t small positive, (, ) t has a signature and for t small negative ( n even ( n p n + n odd p n, n q n, n odd q n ) p n + n even Now, we are ready to show the the non unitarity of Ind G P (δp s σ) for s s 1 small positive. It is enough to show the Hermitian form, s is not definite. Without a loss of generality we may assume that, s (s ]0, s 1 [) is positive definite. Then it is positive definite on Ind G P (δ s 1 P σ)/ kern(δ(s 1) P σ). Thus, if the unitarity exists immediately after s 1, then the form, s is positive definite for s > s 1 close to s 1. In particular, (, ) t is positive definite for t > 0 close to 0. Hence q n = p n + q n = 0. Since k is odd, we see that n n odd n even p k = q k = 0. This contradicts (-10). We have proved the following non unitarity criteria. (RP) Let P = MN be a self-dual maximal parabolic subgroup of G. We write w 0 for the representative of the nontrivial element in W(M). Assume that σ Irr(M) is an irreducible unitarizable representation such that w 0 (σ) σ. Then we have the following: (i) If (A-1) holds (i.e. Ind G P (σ) is reducible and N(σ) is non trivial), then IndG P (δs Pσ) is not unitarizable for s > 0, s close to 0. (ii) If (A-) holds (i.e. Ind G P (δs P σ) is irreducible at s = 0, s 1 > 0 is the first reducibility point, N(δP s σ) is holomorphic and non trivial for s ]0, s 1] and N(δP s σ) has a pole at s = s 1 of an odd order), then Ind G P (δs P σ) is not unitarizable for s > s 1, s close to s 1. q n ). 3. Topology of Unramified Dual Let G be a connected reductive p adic group or O(n, F) (n 0). The topology on the non unitary dual Irr(G) is given by the uniform convergence of matrix coefficients on compacta ([T1], [T6]; see also [F], [Di]). Then Irr u (G) is closed subset in Irr(G). We supply Irr u (G) with the relative topology. Now, we assume that G is one of the groups GL(n, F), O(n, F), SO(n + 1, F) or Sp(n, F). Let K be the maximal compact subgroup introduced in the paragraph before Theorem 1-. The Weyl group W of G acts naturally on the analytic manifold D n = (C ) n. The space of W orbits

D W n UNRAMIFIED UNITARY DUALS FOR SPLIT CLASSICAL p ADIC GROUPS 17 has the structure of analytic manifold. The manifold D n parameterizes unramified principal series of G as follows: Ind G P min (χ 1 χ n ) (χ 1 ( ),..., χ n ( )). Therefore, the manifold Dn W parameterizes unramified principal series of G, up to association. Let Irr I (G) be the set of equivalence classes of irreducible representations σ of G for which there exists a representation in unramified principal series, say Ind G P min (χ 1 χ n ), such that σ is an irreducible subquotient of Ind G P min (χ 1 χ n ). The principal series is determined, up to association, uniquely by this condition. We have a well defined map (3-1) ϕ G : Irr I (G) Dn W defined by ϕ G (σ) = W orbit of a n tuple (χ 1 ( ),..., χ n ( )). We call ϕ G (σ) the infinitesimal character of σ. The fibers of ϕ G are finite. Its restriction to Irr unr (G) induces a bijection (3-) ϕ G : Irr unr (G) D W n, ϕ G (σ G (χ 1,..., χ n )) = W orbit of the n tuple (χ 1 ( ),..., χ n ( )). Now, we recall some results from [T6]. Lemma 3-3. Suppose that G is connected (later we shall discuss the case of O(n, F)). Then the set Irr I (G) is a connected component of Irr(G). Therefore it is open and closed there. The map ϕ G given by (3-1) is continuous and closed. Next, ([T6], Lemma 5.8) implies the following: Lemma 3-4. Suppose that G is connected. Then Irr unr (G) is an open subset of Irr I (G). We have the following description of topology on Irr unr (G): Theorem 3-5. Suppose that G is connected. Then the map (3-) is a homeomorphism. Proof. As it is continuous and bijective, it is enough to show that it is closed. So, let Z be a closed set in Irr unr (G). We must show that ϕ G (Z) is closed. In order to prove that, let Cl(ϕ G (Z)) be its closure. We must show that Cl(ϕ G (Z)) = ϕ G (Z). Let x Cl(ϕ G (Z)). Then there exists a sequence (x m ) m 1 in ϕ G (Z) such that lim m x n = x. (We remark that Dn W is a complex analytic manifold.) We write x m = W orbit of the n tuple (s m,1,..., s m,n ) D n (m 1); x = W orbit of the n tuple (s 1,..., s n ) D n. After passing to a subsequence and making appropriate identification, we may assume lim m s m,i = s i for i = 1,..., n. We may take unramified characters χ m,i and χ 1,..., χ n, m 1, i = 1,..., n, such that χ m,i ( ) = s m,i and χ i ( ) = s i. Clearly, we have the following: ϕ G (σ G (χ m,1,..., χ m,n )) = x m (m 1); ϕ G (σ G (χ 1,..., χ n )) = x. Now, Proposition 5. of [T6] tells that passing to a subsequence, we may assume that the characters of σ G (χ m,1,..., χ m,n ) converge pointwise, and that there are irreducible subquotients σ 1,..., σ l of Ind G P min (χ 1 χ n ) and k 1,..., k l Z >0 such that the pointwise limit is a character of l i=1 k iσ i. Since the representations σ G (χ m,1,..., χ m,n ) are unramified, among the representations σ 1,..., σ l is σ G (χ 1,..., χ n ). Therefore, one of equivalent descriptions of topology in [T6], implies σ G (χ 1,..., χ n ) Z. Hence x = ϕ G (σ G (χ 1,..., χ n )) ϕ G (Z). This shows that ϕ G (Z) is closed.