A spectral gap property for subgroups of finite covolume in Lie groups

A spectral gap property for subgroups of finite covolume in Lie groups Bachir Bekka and Yves Cornulier Dedicated to the memory of Andrzej Hulanicki Abstract Let G be a real Lie group and H a lattice or, more generally, a closed subgroup of finite covolume in G. We show that the unitary representation λ G/H of G on L 2 (G/H) has a spectral gap, that is, the restriction of λ G/H to the orthogonal of the constants in L 2 (G/H) does not have almost invariant vectors. This answers a question of G. Margulis. We give an application to the spectral geometry of locally symmetric Riemannian spaces of infinite volume. 1 Introduction Let G a locally compact group. Recall that a unitary representation (π, H) of G almost has invariant vectors if, for every compact subset Q of G and every ε > 0, there exists a unit vector ξ H such that sup x Q π(x)ξ ξ < ε. If this holds, we also say that the trivial representation 1 G is weakly contained in π and write 1 G π. Let H be a closed subgroup of G for which there exists a G-invariant regular Borel measure µ on G/H (see [BHV, Appendix B] for a criterion of the existence of such a measure). Denote by λ G/H the unitary representation of G given by left translation on the Hilbert space L 2 (G/H, µ) of the square integrable measurable functions on the homogeneous space G/H. If µ is finite, we say that H has finite covolume in G. In this case, the space C1 G/H of the constant functions on G/H is contained in L 2 (G/H, µ) and is G-invariant as well as its orthogonal complement L 2 0(G/H, µ) = { ξ L 2 (G/H, µ) : 1 G/H } ξ(x)dµ(x) = 0.

In case µ is infinite, we set L 2 0(G/H, µ) = L 2 (G/H, µ). Denote by λ 0 G/H the restriction of λ G/H to L 2 0(G/H, µ) (in case µ is infinite, λ 0 G/H = λ G/H). We say that λ G/H (or L 2 (G/H, µ)) has a spectral gap if λ 0 G/H has no almost invariant vectors. In the terminology of [Marg91, Chapter III. (1.8)], H is called weakly cocompact. By a Lie group we mean a locally compact group G whose connected component of the identity G 0 is open in G and is a real Lie group. We prove the following result which has been conjectured in [Marg91, Chapter III. Remark 1.12]. Theorem 1 Let G be a Lie group and H a closed subgroup with finite covolume in G. Then the unitary representation λ G/H on L 2 (G/H) has a spectral gap. It is a standard fact that L 2 (G/H) has a spectral gap when H is cocompact in G (see [Marg91, Chapter III, Corollary 1.10]). When G is a semisimple Lie group, Theorem 1 is an easy consequence of Lemma 3 in [Bekk98]. Our proof is by reduction to these two cases. The crucial tool for this reduction is Proposition (1.11) from Chapter III in [Marg91] (see Proposition 5 below). From Theorem 1 and again from this proposition, we obtain the following corollary. Corollary 2 Let G be a second countable Lie group, H a closed subgroup with finite covolume in G and σ a unitary representation of H. Let π = Ind G Hσ be the representation of G induced from σ. If 1 H is not weakly contained in σ, then 1 G is not weakly contained in π. Observe that, by continuity of induction, the converse result is also true: if 1 H σ, then 1 G π. From the previous corollary we deduce a spectral gap result for some subgroups of G with infinite covolume. Recall that a subgroup H of a topological group G is called co-amenable in G if there is a G-invariant mean on the space C b (G/H) of bounded continuous functions on G/H. When G is locally compact, this is equivalent to 1 G λ G/H ; this property has been extensively studied by Eymard [Eyma72] for which he refers to as the amenability of the homogeneous space G/H. Observe that a normal subgroup H in G is co-amenable in G if and only if the quotient group G/H is amenable. 2

Corollary 3 Let G be a second countable Lie group and H a closed subgroup with finite covolume in G. Let L be a closed subgroup of H. Assume that L is not co-amenable in H. Then λ G/L has a spectral gap. Corollary 3 is a direct consequence of Corollary 2, since the representation λ G/L on L 2 (H/L) is equivalent to the induced representation Ind G Hλ H/L. Here is a reformulation of the previous corollary. Let G be a Lie group and H a closed subgroup with finite covolume in G. If a subgroup L of H is co-amenable in G, then L is co-amenable in H. Observe that the converse (if L is co-amenable in H, then L is co-amenable in G) is true for any topological group G and any closed subgroup H which is co-amenable in G (see [Eyma72, p.16]). Using methods from [Leuz03] (see also [Broo86]), we obtain the following consequence for the spectral geometry of infinite coverings of locally symmetric Riemannian spaces of finite volume. Recall that a lattice in the locally compact group G is a discrete subgroup of G with finite covolume. Corollary 4 Let G be a semisimple Lie group with finite centre and maximal compact subgroup K and let Γ be a torsion-free lattice G. Let Ṽ be a covering of the locally symmetric space V = K\G/Γ. Assume that the fundamental group of Ṽ is not co-amenable in Γ. (i) We have h(ṽ ) > 0 for the Cheeger constant h(ṽ ) of Ṽ. (ii) We have λ 0 (Ṽ ) > 0, where λ 0(Ṽ ) is the bottom of the L2 -spectrum of the Laplace Beltrami operator on Ṽ. There is in general no uniform bound for h(ṽ ) or λ 0(Ṽ ) for all coverings Ṽ. However, it was shown in [Leuz03] that, when G has Kazhdan s Property (T), such a bound exists for every locally symmetric space V = K\G/Γ. Observe also that if, in the previous corollary, the fundamental group of Ṽ is co-amenable in Γ and has infinite covolume, then h(ṽ ) = λ 0(Ṽ ) = 0, as shown in [Broo81]. 2 Proofs of Theorem 1 and Corollary 4 The following result of Margulis (Proposition (1.11) in Chapter III of [Marg91]) wil be crucial. 3

Proposition 5 ([Marg91]) Let G be a second countable locally compact group, H a closed subgroup of G, and σ a unitary representation of H. Assume that λ G/H has a spectral gap and that 1 H is not weakly contained in σ. Then 1 G is not weakly contained in Ind G Hσ. In order to reduce the proof of Theorem 1 to the semisimple case, we will use several times the following proposition. Proposition 6 Let G a separable locally compact group and H 1 and H 2 be closed subgroups of G with H 1 H 2 and such that G/H 2 and H 2 /H 1 have G- invariant and H 2 -invariant regular Borel measures, respectively. Assume that the H 2 -representation λ H2 /H 1 on L 2 (H 2 /H 1 ) and that the G-representation λ G/H2 on L 2 (G/H 2 ) have both spectral gaps. Then the G-representation λ G/H1 on L 2 (G/H 1 ) has a spectral gap. Proof Recall that, for any closed subgroup H of G, the representation λ G/H is equivalent to the representation Ind G H1 H induced by the identity representation 1 H of H. Hence, we have, by transitivity of induction, λ G/H1 = Ind G H 1 1 H1 = Ind G H 2 (Ind H 2 H 1 1 H1 ) = Ind G H 2 λ H2 /H 1. We have to consider three cases: First case: H 1 has finite covolume in G, that is, H 1 has finite covolume in H 2 and H 2 has finite covolume in G. Then λ 0 G/H 1 = λ 0 G/H 2 Ind G H 2 λ 0 H 2 /H 1. By assumption, λ 0 H 2 /H 1 and λ 0 G/H 2 do not weakly contain 1 H2 and 1 G, respectively. It follows from Proposition 5 that Ind G H 2 λ 0 H 2 /H 1 does not weakly contain 1 G. Hence, λ 0 G/H 1 does not weakly contain 1 G. Second case: H 1 has finite covolume in H 2 and H 2 has infinite covolume in G. Then λ G/H1 = λ G/H2 Ind G H 2 λ 0 H 2 /H 1. By assumption, λ 0 H 2 /H 1 and λ G/H2 do not weakly contain 1 H2 and 1 G. As above, using Proposition 5, we see that λ G/H1 does not weakly contain 1 G. Third case: H 1 has infinite covolume in H 2. By assumption, λ H2 /H 1 does not weakly contain 1 H2. By Proposition 5 again, it follows that λ G/H1 = Ind G H 2 λ H2 /H 1 does not weakly contain 1 G. For the reduction of the proof of Theorem 1 to the case where G is second countable, we will need the following lemma. 4

Lemma 7 Let G be a locally compact group and H a closed subgroup with finite covolume. The homogeneous space G/H is σ-compact. Proof Let µ be the G-invariant regular probability measure on the Borel subsets of G/H. Choose an increasing sequence of compact subsets K n of G/H with lim n µ(k n ) = 1. The set K = n K n has µ-measure 1 and is therefore dense in G/H. Let U be a compact neighbourhood of e in G. Then UK = G/H and UK = n UK n is σ-compact. Proof of Theorem 1 Through several steps the proof will be reduced to the case where H is a lattice in G and where G is a connected semisimple Lie group with trivial centre and without compact factors. First step: we can assume that G is σ-compact and hence second-countable. Indeed, let p : G G/H be the canonical projection. Since every compact subset of G/H is the image under p of some compact subset of G (see [BHV, Lemma B.1.1]), it follows from Lemma 7 that there exists a σ-compact subset K of G such that p(k) = G/H. Let L be the subgroup of G generated by K U for a neighbourhood U of e in G. Then L is a σ-compact open subgroup of G. We show that L H has a finite covolume in L and that λ G/H has a spectral gap if λ L/L H has a spectral gap. Since LH is open in G, the homogeneous space L/L H can be identified as L-space with LH/H. Therefore L H has finite covolume in L. On other hand, the restriction of λ G/H to L is equivalent to the L-representation λ L/L H, since LH/H = p(l) = G/H. Hence, if λ L/L H has a spectral gap, then λ G/H has a spectral gap. Second step: we can assume that G is connected. Indeed, let G 0 be the connected component of the identity of G. We show that G 0 H has a finite covolume in G 0 and that λ G/H has a spectral gap if λ G 0 /G 0 H has a spectral gap. The subgroup G 0 H is open in G and has finite covolume in G as it contains H. It follows that G 0 H has finite index in G since G/G 0 H is discrete. Hence λ G/G 0 H has a spectral gap. On the other hand, since G 0 H is closed in G, the homogeneous space G 0 /G 0 H can be identified as a G 0 -space with G 0 H/H. Therefore G 0 H has finite covolume in G 0. The restriction of λ G 0 H/H to G 0 is equivalent to the G 0 -representation λ G 0 /G 0 H. 5

Suppose now that λ G 0 /G 0 H has a spectral gap. Then the G 0 H-representation λ G 0 H/H has a spectral spectral, since L 2 0(G 0 H/H) = L 2 0(G 0 /G 0 H) as G 0 -representations. An application of Proposition 6 with H 1 = H and H 2 = G 0 H shows that λ G/H has a spectral gap. Hence, we can assume that G is connected. Third step: we can assume that H is a lattice in G. Indeed, let H 0 be the connected component of the identity of H and let N G (H 0 ) be the normalizer of H 0 in G. Observe that N G (H 0 ) contains H. By [Wang76, Theorem 3.8], N G (H 0 ) is cocompact in G. Hence, λ G/NG (H 0 ) has a spectral gap. It follows from the previous proposition that λ G/H has a spectral gap if λ NG (H 0 )/H has a spectral gap. On the other hand, since H 0 is a normal subgroup of H, we have L 2 0(N G (H 0 )/H) = L 2 0((N G (H 0 )/H 0 )/(H/H 0 )), as N G (H 0 )-representations. Hence, λ NG (H 0 )/H has a spectral gap if and only if λ N/H has a spectral gap, where N = N G (H 0 )/H 0 and H = H/H 0. The second step applied to the Lie group N/H shows that λ N/H has a spectral gap if λ N 0 /(N 0 H) has a spectral gap. Observe that N 0 H is a lattice in the connected Lie group N 0, since H is discrete and since H has finite covolume in N G (H 0 ). This shows that we can assume that H is a lattice in the connected Lie group G. Fourth step: we can assume that G is a connected semisimple Lie group with no compact factors. Indeed, let G = SR be a Levi decomposition of G, with R the solvable radical of G and S a semisimple subgroup. Let C be the maximal compact normal subgroup of S. It is proved in [Wang70, Theorem B, p.21] that HCR is closed in G and that HCR/H is compact. Hence, by the previous proposition, λ G/H has a spectral gap if λ G/HCR has a spectral gap. The quotient G = G/CR is a connected semisimple Lie group with no compact factors. Moreover, H = HCR/CR is a lattice in G since HCR/CR = H/H CR is discrete and since HCR has finite covolume in G. Observe that λ G/HCR is equivalent to λ G/H as G-representation. Fifth step: we can assume that G has trivial centre. Indeed, let Z be the centre of G. It is known that ZH is discrete (and hence closed) in G (see 6

[Ragh72, Chapter V, Corollary 5.17]). Hence, ZH/H is finite and λ ZH/H has a spectal gap. By the previous proposition, λ G/H has a spectral gap if λ G/ZH has a spectral gap. Now, G = G/Z is a connected semisimple Lie group with no compact factors and with trivial centre, H = ZH/Z is a lattice in G and λ G/ZH is equivalent to λ G/H. Sixth step: by the previous steps, we can assume that H is a lattice in a connected semisimple Lie group G with no compact factors and with trivial centre. In this case, the claim was proved in Lemma 3 of [Bekk98]. This completes the proof of Theorem 1. Proof of Corollary 4 The proof is identical with the proof of Theorems 3 and 4 in [Leuz03]; we give a brief outline of the arguments. Let Λ be the fundamental group of Ṽ. First, it suffices to prove Claims (i) and (ii) for G/Γ instead of K\G/Γ (see Section 4 in [Leuz03]). So we assume that Ṽ = G/Λ. Equip G with a right invariant Riemannian metric and G/Λ with the induced Riemannian metric. Observe that G/Λ has infinite volume, since Λ is of infinite index in Γ. Claim (ii) is a consequence of (i), by Cheeger s inequality 1 4 h(g/λ)2 λ 0 (G/Λ). Recall that the Cheeger constant h(g/λ) of G/Λ is the infimum over all numbers A( Ω)/V (Ω), where Ω is an open submanifold of G/Λ with compact closure and smooth boundary Ω, and where V (Ω) and A( Ω) are the Lebesgue measures of Ω and Ω. To prove Claim (i), we proceed exactly as in [Leuz03]. By Corollary 3, there exists a compact neighbourhood H of the identity in G and a constant ε > 0 such that ( ) ε ξ sup h H λ G/Λ (h)ξ ξ, for all ξ L 2 (G/Λ). Let Ω be an open submanifold of G/Λ with compact closure and smooth boundary Ω. By [Leuz03, Proposition 1], we can find an open subset Ω of G/Λ compact closure and smooth boundary such that, for all h H, ( ) V (U h ( Ω)) CV ( Ω) A( Ω) V (Ω), where the constant C > 0 only depends on H. Here, h denotes the distance d G (e, g) of h to the group unit and, for a subset S of G/Λ, U r (S) is the 7

tubular neighbourhood U r ( Ω) = {x G/Λ : d G/Λ (x, S) r} By Inequality ( ), applied the characteristic function χ eω of Ω, there exists h H such that where X = ε 2 V ( Ω) λ G/Λ (h)χ eω χ eω 2 = V (X), { x G/Λ : x Ω, hx / Ω } { x G/Λ : x / Ω, hx Ω }. One checks that X U h ( Ω). It follows from Inequalities ( ) and ( ) that ε 2 C A(Ω) ε2. Hence, 0 < h(g/λ). V (Ω) C References [Bekk98] B. Bekka. On uniqueness of invariant means. Proc. Amer. Math. Soc. 126, 507-514 (1998). [BHV] B. Bekka, P. de la Harpe, A. Valette. Kazhdans Property (T), Cambridge University Press 2008. [Broo81] [Broo86] R. Brooks. The fundamental group and the spectrum of the Laplacian. Comment. Math. Helv. 56, 581-598 (1981). R. Brooks. The spectral geometry of a tower of coverings. J. Diff. Geometry 23, 97-107 (1986). [Eyma72] P. Eymard. Moyennes invariantes et représentations unitaires, Lecture Notes in Mathematics, Vol. 300. Springer-Verlag, 1972. [Leuz03] E. Leuzinger. Kazhdans property (T), L 2 -spectrum and isoperimetric inequalities for locally symmetric spaces. Comment. Math. Helv. 78, 116-133 (2003). [Marg91] G.A. Margulis. Discrete subgroups of semisimple Lie groups, Springer-Verlag, 1991. 8

[Ragh72] M.S. Raghunathan. Discrete subgroups of Lie groups, Springer- Verlag, 1972. [Wang70] H.C.Wang. On the the deformation of a lattice in a Lie group. Amer. J. Math.. 85,189-212 (1963). [Wang76] S.P.Wang. Homogeneous spaces with finite invariant measure. Amer. J. Math.. 98,311-324 (1976). Address IRMAR, UMR 6625 du CNRS, Université de Rennes 1, Campus Beaulieu, F-35042 Rennes Cedex, France E-mail : bachir.bekka@univ-rennes1.fr, yves.decornulier@univ-rennes1.fr 9