On Square-Integrable Representations of Classical p-adic Groups

Canad. J. Math. Vol. 52 3, 2000 pp. 539 581 On Square-Integrable Representations of Classical p-adic Groups Chris Jantzen Abstract. In this paper, we use Jacquet module methods to study the problem of classifying discrete series for the classical p-adic groups Sp2n, FandSO2n + 1, F. 1 Introduction 1.1 Introduction One of the central questions in the representation theory of p-adic groups is to determine the discrete series. This paper studies the problem of determining the noncuspidal discrete series for the classical groups Sp 2n F andso 2n+1 F. Let S n F denotesp 2n F or SO 2n+1 F we treat the two families simultaneously. Now, a noncuspidal discrete series representation occurs as a subquotient of a parabolically induced representation. Here, we constrain where one needs to look for such discrete series representations. Ultimately, we hope that such an analysis can be used to help prove exhaustion for the noncuspidal discrete series. First, we reduce the problem of classifying the discrete series to classifying those squareintegrable representations supported on sets of the form S ρ, β; σ = {ν α ρ} α β+z {σ}, where ρ = ρ is an irreducible unitary supercuspidal representation of GL n F, ν = det, σ an irreducible supercuspidal representation of S r F, and β = 0or 1 2. In general, if π is an irreducible representation not necessarily square-integrable supported on S ρ, β; σ as above, we define χ 0 π. This is a subquotient of the normalized Jacquet module taken with respect to the smallest standard parabolic subgroup admitting a nonzero Jacquet module; it is minimal with respect to an appropriate ordering. This is used to produce δ 0 π, which has the form δ 0 π = δ[ν b 1 ρ, ν a 1 δ[ν b k ρ, ν a k σ, where a 1 a 2 a k δ[ν b ρ, ν a denotes the generalized Steinberg representation of GL a b+1n F whose minimal Jacquet module is ν a ρ ν a 1 ρ ν b ρ. If m = a 1 b 1 +1n+ +a k b k +1n+r,letP = MN denote the standard parabolic subgroup of S m FwithLevifactor Then, we show that M = GL a1 b 1 +1nF GL ak b k +1nF S r F. Received by the editors November 20, 1998. AMS subject classification: 22E50. c Canadian Mathematical Society 2000. π Ind G P δ0 π. 539

540 Chris Jantzen Further, we show that π is square-integrable if and only if a i + b i > 0foralli. Thus, every square-integrable representation supported on S ρ, β; σ with ρ, β, σ as above, is a subrepresentation of an induced representation of this form. We note that not every δ 0 having the form described above occurs as δ 0 π for a square-integrable π. However,we expect that with a couple of additional conditions on a i, b i, that will be the case. We now discuss the contents section by section. The next section introduces notation and recalls some general results that will be needed later. Section 2.1 reviews some results of Zelevinsky on induced representations for general linear groups. In Section 2.2, we define χ 0 π, δ 0 πforπ an irreducible representation of GL m F supported on a set of the form {ν α ρ} α β+z,whereρ is an irreducible unitary supercuspidal representation. We also establish some of the basic properties of χ 0 π, δ 0 π. In Section 2.3, we show how these can be used to show that the only irreducible squareintegrable representations of GL m F are the generalized Steinberg representations, a result originally due to Bernstein. The connection between δ 0 π and the Langlands data subrepresentation version of the Langlands classification is established in Section 2.4. We use Section 3.1 to review some background material on the representation theory of S n F. In Section 3.2, we recall a result which allows us to reduce the problem of classifying the discrete series of S r F to that of classifying the square-integrable representations supported on sets of the form S ρ, β; σ. In Section 3.3, we give a conjecture which, when coupled with recent work of Mœglin, leads to an expected parameterization of such square-integrable representations, at least for pairs ρ, σ with generic reducibility. The definitions and basic properties of χ 0 π, δ 0 π mentioned above are discussed in detail in Section 4.1. In Section 4.2, we give the criterion for square-integrability mentioned above. In Section 4.3, we use this to determine which sets S ρ, β; σ support squareintegrable representations. Section 4.4 gives some basic constraints on δ 0 π. In the fifth chapter, we give an example to show how these results may be applied. We restrict our attention to the case where Ind G P ν 1 2 ρ σ is reducible, where P is the standard parabolic subgroup of S n+r F withlevifactorm = GL n F S r F. By results in Section 4.3, only S ρ, 1 2 ; σ will support square-integrable representations, so we restrict our attention to representations supported on this set. The goal of this chapter is to classify those irreducible, square-integrable representations whose δ 0 has k = 2. The case k = 1is already known cf. [Tad5]; we discuss this case in Section 5.1. In Section 5.2, weshow that if π is an irreducible, square-integrable representation and k = 2, then δ 0 πhasoneofthe following forms: 1. δ 0 π = δ[ν d ρ, ν c δ[ν b ρ, ν a σ,or 2. δ 0 π = δ[ν c ρ, ν b δ[ν d ρ, ν a σ for a, b, c, d 1 2 + Z with a > b > c > d 1 2.Further,anythingofform1.or2.above actually occurs as δ 0 π for some irreducible, square-integrable representation π. Thisfollows from the discussion in Section 5.3. We note that the irreducible, square-integrable representations appearing in Ind G P δ[ν d ρ, ν c δ[ν b ρ, ν a σ are classified by the results in [Tad5] where P and G are clear from context. In Section 5.3, we do a corresponding analysis for Ind G P δ[ν c ρ, ν b δ[ν d ρ, ν a σ, though our approach is somewhat different. I would like to take this opportunity to thank a few people. In particular, conversations withhenrykim,alanroche,paulsally,andmarkotadić have been quite helpful during

Classical p-adic Groups 541 various stages of the research for this paper. My thanks go out to all of them, and to the referee as well. 1.2 Notation and Preliminaries In this section, we introduce notation and recall some results that will be needed in the rest of the paper. This largely follows the setup used in [Tad1]. Let F be a p-adic field with char F = 0. Let denote the absolute value on F, normalized so that ϖ = q 1, ϖ a uniformizer. As in [B-Z], we let ν = det on GL n F with the value of n clear from context. Define on GLFasin[B-Z]:ifρ 1,...,ρ k are representations of GL n1 F,...,GL nk F, let ρ 1 ρ k denote the representation of GL n1 + +n k F obtained by inducing ρ 1 ρ k from the standard parabolic subgroup of GL n1 + +n k F with Levi factor GL n1 F GL nk F. Frequently, we work in the Grothendieck group setting. That is, we work with the semisimplified representation. So, for any representation π and irreducible representation ρ, letmρ, π denote the multiplicity of ρ in π. We write π = π 1 + + π k if mρ, π = mρ, π 1 + + mρ, π k for every irreducible ρ. Similarly, we write π π 0 if mρ, π mρ, π 0 foreverysuchρ. Forclarity,weuse= when defining something or working in the Grothendieck group; = is used to denote an actual equivalence. We now turn to symplectic and odd-orthogonal groups. Let J n = 1 denotethen n antidiagonal matrix above. Then, 1... 1 1 SO 2n+1 F = {X SL 2n+1 F T XJ 2n+1 X = J 2n+1 }, { } Sp 2n F = X GL 2n F T J J X X =. J J We use S n F todenoteeitherso 2n+1 F orsp 2n F. In either case, the Weyl group is W = {permutations and sign changes on n letters}. We take as minimal parabolic subgroup in S n F thesubgroupp consisting of upper triangular matrices. Let α = n 1,...,n k be an ordered partition of a nonnegative integer m n into positive integers. Let M α S n Fbethesubgroup X 1... M α = X k X τ Xk... τ X1 X i GL ni F, X S n m F,

542 Chris Jantzen where τ X = J T X 1 J.ThenP α = M α P is a parabolic subgroup of S n and every parabolic subgroup is of this form up to conjugation. For α = n 1,...,n k, let ρ 1,...,ρ k be representations of GL n1 F,...,GL nk F, respectively, and σ arepresentationofs n m F. Let ρ 1 ρ k σ denote the representation of S n F obtained by inducing the representation ρ 1 ρ k σ of M α extended trivially to P α. If m = n, wewriteρ 1 ρ k 1, where 1 denotes the trivial representation of S 0 F. We recall some structures which will be useful later cf. Section1of[Zel1]andSection 4 of [Tad3]. Let R GL n F resp., R S n F denote the Grothendieck group of the category of all smooth finite-length GL n F-modules resp., S n F-modules. Set R = n 0 R GL n F and R[S] = n 0 R S n F.Theoperators and lift naturally to : R R R and : R R[S] R[S]. With these multiplications, R becomesan algebraandr[s] amoduleoverr. Let π be an irreducible representation of S n F. Then, there is a standard Levi M and an irreducible supercuspidal representation ρ 1 ρ k σ of M with ρ i an irreducible supercuspidal representation of GL ni Fandσ an irreducible supercuspidal representation of S n m F such that π is a subquotient of ρ 1 ρ k σ. We say that the multiset {ρ 1,...,ρ k ; σ} is in the support of π. Further,M and ρ 1 ρ k σ areuniqueupto conjugation cf. Theorem2.9, [B-Z]. ByPropositions4.1and4.2of [Tad3], ρ 1 ρ i 1 ρ i ρ i+1 ρ k σ = ρ 1 ρ i 1 ρ i ρ i+1 ρ k σ, where denotes contragredient. Thus, if {ρ 1,...,ρ i 1,ρ i,ρ i+1,...,ρ k ; σ} is in the support of π,sois{ρ 1,...,ρ i 1, ρ i,ρ i+1,...,ρ k ; σ}. Therefore, every {ρ 1,...,ρ k ; σ},withρ i = ρ i or ρ i, is in the support of π. Further, these exhaust the support of π. More generally, we extend the definition of support as in [Tad5]: If π is a finite-length representation and {ρ 1,...,ρ k ; σ} is in the support of π for every irreducible subquotient π of π,wesaythat {ρ 1,...,ρ k ; σ} is in the support of π. We recall some notation of Bernstein-Zelevinsky [B-Z]. If P = MU is a standard parabolic subgroup of G and ξ arepresentationofm, weleti GM ξ denote the representation obtained by normalized parabolic induction. Similarly, is π is a representation of G, we let r MG π denote the normalized Jacquet module of π with respect to P. Next,weintroducesomeconvenientshorthandforJacquetmodulescf. [Tad3]. If π is a representation of some S n F andα is a partition of m n, lets α π denote the Jacquet module with respect to M α. Note that, by abuse of notation, we also allow s α to be applied to representations M β when M β > M α cf. Section 2.1, [B-Z]. Further, we define s GL as in [Tad5]: for π ρ 1 ρ k σ with ρ i asupercuspidal representation of GL ni F andσ a supercuspidal representation of S n m F, we set s GL π = s n1 + +n k π. We will occasionally use similar notation for representations of GL n F. If α = n 1,...,n k is a partition of m n, GL n F has a standard parabolic subgroup with Levi factor L α = GLn1 F GL nk F GL n m F L α consists of block-diagonal matrices; the corresponding parabolic subgroup of block upper triangular matrices. If π is a representation of GL n F, we let r α π denote the Jacquet module of π with respect to L α.

Classical p-adic Groups 543 Finally, suppose π is a representation of S n F. Consider M min = {M standard Levi r MG π 0butr LG π = 0 L < M}. Note that if π has supercuspidal support in the sense above, these are all conjugate. Then, formally set s min π = r MG π. M M min If π has supercuspidal support of parabolic rank m,thens min π R } R {{} R[S]. We m may define r min similarly for representations of GL n F. 2 The Case of GL n F 2.1 Background Material We now review some results on induced representations for GL n F. This section is all based on the work of Zelevinsky [Zel1]. First, if ρ is an irreducible supercuspidal representation of GL r F andm n mod 1, we define the segment [ν m ρ, ν n ρ] = {ν m ρ, ν m+1 ρ,...,ν n ρ}. We note that the induced representation ν m ρ ν m+1 ρ ν n ρ has a unique irreducible subrepresentation, which we denote by ζ[ν m ρ, ν n, and a unique irreducible quotient, which we denote by δ[ν m ρ, ν n. Lemma 2.1.1 Let ρ 1, ρ 2 be irreducible unitary supercuspidal representations of GL r1 F, GL r2 F. Suppose m 1 n 1,m 2 n 2 satisfy m 1 n 1 mod 1, m 2 n 2 mod 1. Then, δ[ν m 1 ρ 1,ν n 1 ρ 1 ] δ[ν m 2 ρ, ν n 2 is reducible if and only if all of the following hold: 1. ρ 1 = ρ2 2. m 1 n 1 m 2 n 2 3. either a m 1 < m 2 and m 2 1 n 1 n 2 1,orb m 1 > m 2 and m 1 1 n 2 n 1 1. ζ[ν m 1 ρ 1,ν n 1 ρ 1 ] ζ[ν m 2 ρ 2,ν n 2 ρ 2 ] is irreducible if and only if the same conditions hold. Proof This is a special case of Theorem 4.2 [Zel1]. Next, consider a representation of the form χ = ρ 1 1 ρ k 1 1 ρ 1 2 ρ k 2 2 ρ 1 m ρk m m with ρ j 1 an irreducible representation of GL j r F foralli, j. Byashuffleofχ, wemeanthe i usual: a permutation on χ such that for all i, ρ 1 i,...,ρ k i i appear in that order. That is, the relative orders in the parenthesized pieces are preserved. Further, if χ is a representation of a standard Levi M of GL n F andshχ isashuffleofχ, thenshχ is a representation of a standard Levi subgroup of GL n FwhichwedenotebyshM. We have the following:

544 Chris Jantzen Lemma 2.1.2 shuffling 1. Suppose π is an irreducible representation of GL n F such that r MG π χ, whereχ has the form ν α 1,1 ρ 1 ν α 1, j 1 ρ1 ν α 2,1 ρ 2 ν α 2, j 2 ρ2 ν α m,1 ρ m ν α m, jm ρm, where a ρ 1,...,ρ m are irreducible unitary supercuspidal representations of GL r1 F,..., GL rm F, b α i,k R with α i,1 α i,2 α i, ji mod 1 for all i, and such that c if ρ i = ρk,thenα i,1 α k,1 mod 1. Then, for every shuffle shχ of χ, wehaver shmg π shχ. Further,ifr shmg π shχ for any such shuffle, we necessarily have r MG π χ, and therefore r shmg π shχ for every such shuffle. 2. i GM χ = i GshM shχ for any such shuffle. Proof See Lemma 5.4 and Section 10 of [Jan3]. Lemma 2.1.3 Let ρ 1,α 1,...,ρ m,α m be pairs with ρ 1,...,ρ m irreducible unitary supercuspidal representations of GL r1 F,...,GL rm F; α 1,...,α m R such that ρ i = ρ j implies α i α j mod 1. Letτρ i,α i be an irreducible representation of a general linear group supported on {ν α ρ i } α αi +Z. Let M be the standard Levi subgroup of G = GL n F which admits τρ 1,α 1 τρ m,α m as a representation. Then, 1. τρ 1,α 1 τρ m,α m is irreducible. 2. mult τρ 1,α 1 τρ m,α m, r MG τρ1,α 1 τρ m,α m = 1. Further, if τ ρ i,α i is an irreducible representation of a general linear group supported on {ν α ρ i } α αi +Z,then mult τ ρ 1,α 1 τ ρ m,α m, r MG τρ1,α 1 τρ m,α m = 0 unless τ ρ i,α i = τρ i,α i for all i. 3. If π is an irreducible representation of GL n F and r MG π τρ 1,α 1 τρ m,α m, then π = τρ 1,α 1 τρ m,α m. Proof The first claim is an immediate consequence of [Zel1, Proposition 8.5]. Claims 2. and 3. follow fairly easily see Corollary 5.6 and [Jan3, Section 10] for details.

Classical p-adic Groups 545 2.2 A Basic Lemma for GL n F Let ρ be an irreducible unitary supercuspidal representation of GL n F, 0 α 0 < 1. Suppose π is a representation of GL mn F of finite length, supported on {ν α ρ} α α0 +Z. Then, we make the following definition: Definition 2.2.1 Let χ 0 π denote the lowest element of r min π with respect to the lexicographic order. χ 0 π is unique up to multiplicity. Lemma 2.2.2 χ 0 π has the form χ 0 π = ν a 1 ρ ν a 1 1 ρ ν b 1 ν a k ρ ν a k 1 ρ ν b k ρ, with a 1 a 2 a k and a i α 0 + Z for all i. Proof Write χ 0 = ν α 1 ρ ν α 2 ρ. Clearly, α i α 0 + Z for all i. Letj 1 be the smallest integer such that α j+1 α 1. Suppose j > 1. Then, we claim that α 2 = α 1 1. To see this, observe that if α 2 <α 1 1 χ 0 = ν α 1 ρ ν α 2 ρ ν α 3 ρ r min π ν α 1 ρ ν α 2 ρ ν α 3 ρ r 2n,n,...,n π χ 0 = ν α 2 ρ ν α 1 ρ ν α 3 ρ r min π, since ν α 1 ρ ν α 2 ρ = ν α 2 ρ ν α 1 ρ is irreducible. However, χ 0 <χ 0 in the lexicographic order, contradicting the definition of χ 0 π. Thus, α 2 = α 1 1. Next, suppose j > 2. Then, we claim that α 3 = α 1 2 = α 2 1. First, if α 3 <α 2 1, then the same argument as above tells us that χ 0 = ν α 1 ρ ν α 3 ρ ν α 2 ρ ν α 4 ρ r min π. Again, χ 0 <χ 0 lexicographically, contradicting the definition of χ 0.Thus,α 3 = α 1 1 = α 2 or α 3 = α 1 2 = α 2 1. However, if α 3 = α 1 1 = α 2,wehave χ 0 = ν α 1 ρ ν α1 1 ρ ν α1 1 ρ ν α 4 ρ r min π δ[ν α 1 1 ρ, ν α 1 ν α1 1 ρ ν α 4 ρ r 3n,n,...,n π χ 0 = ν α 1 1 ρ ν α 1 ρ ν α 1 1 ρ ν α 4 ρ r min π,

546 Chris Jantzen noting that δ[ν α 1 1 ρ, ν α 1 ν α 1 1 ρ is the only irreducible representation of GL 3n F containing ν α 1 ρ ν α 1 1 ρ ν α 1 1 ρ in its minimal Jacquet module and again, we have χ 0 <χ 0 lexicographically, a contradiction. Thus, we are left with α 3 = α 1 2, as claimed. We now move to the more general step. Suppose j > i. Inductively, we may assume χ 0 π = ν α 1 ρ ν α 1 1 ρ ν α 1 i+2 ρ ν α i ρ ν α i+1 ρ. We want to show that α i = α 1 i +1. First,ifα i <α 1 i +1,wecanusethesame argument that we used to show α 2 = α 1 1toget χ 0 = ν α 1 ρ ν α1 1 ρ ν α1 i+3 ρ ν α i ρ ν α1 i+2 ρ ν α i+1 ρ r min π. However, χ 0 <χ 0 lexicographically, contradicting the definition of χ 0. Similarly, if α i = α 1 i +2,wecanusethesameargumentweusedtoshowα 3 = α 2 1toget χ 0 = ν α 1 ρ ν α1 i+4 ρ ν α1 i+2 ρ ν α1 i+3 ρ ν α1 i+2 ρ ν α i+1 ρ r min π. Again, χ 0 < χ 0 lexicographically, a contradiction. Thus, α i {α 1 1,α 1 2,..., α 1 i +3} {α 1 i +1}. Now, suppose α i = α 1 k with k i 3. Then, χ 0 = ν α 1 ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 k 2 ρ ν α 1 i+3 ρ ν α 1 i+2 ρ ν α 1 k ρ ν α i+1 ρ r min π ν α 1 ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 k 2 ρ ν α 1 i+3 ρ ν α 1 i+2 ρ ν α 1 k ρ ν α i+1 ρ r n,...,n,2n,n,...,n π χ 0 = ν α 1 ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 k 2 ρ ν α 1 i+3 ρ ν α 1 k ρ ν α 1 i+2 ρ ν α i+1 ρ r min π similarly commuting ν α1 k ρ around ν α1 i+3 ρ,...,ν α1 k 2 ρ ν α 1 ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 k ρ ν α 1 k 2 ρ ν α 1 i+3 ρ ν α 1 i+2 ρ ν α i+1 ρ r min π.

Classical p-adic Groups 547 Next, ν α 1 ρ ν α 1 k+1 ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 k ρ ν α 1 k 2 ρ ν α 1 i+2 ρ ν α i+1 ρ r min π ν α 1 ρ ν α 1 k+1 ρ δ[ν α 1 k 1 ρ, ν α 1 k ν α 1 k ρ ν α 1 k 2 ρ ν α 1 i+2 ρ ν α i+1 ρ r n,...,n,3n,n,...,n π or ν α 1 ρ ν α 1 k+1 ρ ζ[ν α 1 k 1 ρ, ν α 1 k ν α 1 k ρ ν α 1 k 2 ρ ν α 1 i+2 ρ ν α i+1 ρ r n,...,n,3n,n,...,n π ν α 1 ρ ν α 1 k+1 ρ ν α 1 k ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 k 2 ρ ν α 1 i+2 ρ ν α i+1 ρ r min π or ν α 1 ρ ν α 1 k+1 ρ ν α 1 k 1 ρ ν α 1 k ρ ν α 1 k ρ ν α 1 k 2 ρ ν α 1 i+2 ρ ν α i+1 ρ r min π. We can rule out the second of these possibilities since it is lexicographically lower than χ 0. Thus, we have shifting parentheses ν α 1 ρ ν α 1 k+2 ρ ν α 1 k+1 ρ ν α 1 k ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 i+2 ρ ν α i+1 ρ r min π ν α 1 ρ ν α 1 k+2 ρ δ[ν α 1 k ρ ν α 1 k+1 ν α 1 k ρ ν α 1 k 1 ρ ν α 1 i+2 ρ ν α i+1 ρ r n,...,n,3n,n,...,n π χ 0 = ν α 1 ρ ν α 1 k+2 ρ ν α 1 k ρ ν α 1 k+1 ρ ν α 1 k ρ ν α 1 k 1 ρ ν α 1 i+2 ρ ν α i+1 ρ r min π. However, χ 0 <χ 0 lexicographically, contradicting the definition of χ 0. Thus, we cannot have α i = α 1 k with k i 3. The only possibility remaining is α i = α 1 i +1,as needed. Thus, by induction, we have α i = α 1 i +1for1 i j. Now, in the statement of the lemma, we have a 1 = α 1 and b 1 = α 1 j + 1. Repeating this argument to deal with a 2, b 2 through a k, b k finishes the lemma.

548 Chris Jantzen Definition 2.2.3 With notation as above, if χ 0 π = ν a 1 ρ ν a 1 1 ρ ν b 1 ρ ν a k ρ ν a k 1 ρ ν b k ρ, set δ 0 π = δ[ν b 1 ρ, ν a 1 δ[ν b k ρ, ν a k. Corollary 2.2.4 Let π be an irreducible representation with χ 0 π, δ 0 π be as above. Let M = GL a1 b 1 +1nF GL ak b k +1nF. Then, π i GM δ0 π. Proof First, observe that by central character considerations, there is a direct summand V 0 of the space of r Mmin Gπ such that the semisimplification of V 0 consists of copies of χ 0 π. By Frobenius reciprocity, this implies π i GMmin χ0 π. By Lemma 5.5 of [Jan3], there is an irreducible subquotient θ of i MMmin χ0 π such that π i GM θ. We claim that θ = δ 0 π. Suppose not. Consider any χ r Mmin Mθ. Then, χ<χ 0 π lexicographically since χ 0 π is the highest term in r Mmin M i MMmin χ0 π. However, by Frobenius reciprocity, θ r MG π. This implies χ r Mmin Gπ, contradicting the definition of χ 0 π. Thus, θ = δ 0 π. 2.3 A Result of Bernstein We now give an application of the results of the previous section. A well-known theorem of Bernstein cf. [Zel1, Theorem 9.3] says that an irreducible representation of a p-adic general linear group is essentially square-integrable if and only if it has the form δ[ν α ρ, ν α+k for some irreducible unitary supercuspidal representation ρ and some nonnegative integer k. A proof of this fact may be obtained fairly easily at this point. We first recall the Casselman criterion. Let π be an irreducible square-integrable representation of GL n F. Suppose χ = ν α 1 ρ 1 ν α k ρ k r min π, with ρ i an irreducible unitary supercuspidal representation of GL ni F, α i n 1 + + n k = n. Then, R, and n 1 α 1 > 0 n 1 α 1 + n 2 α 2 > 0. n 1 α 1 + n 2 α 2 + + n k 1 α k 1 > 0 n 1 α 1 + n 2 α 2 + + n k 1 α k 1 + n k α k = 0. Conversely, if π is an irreducible representation such that the inequalities above hold for every χ r min π, then π is square-integrable.

Classical p-adic Groups 549 Theorem 2.3.1 Bernstein π is an irreducible square-integrable representation of GL n F if and only if π has the form π = δ[ν m ρ, ν m,whereρis an irreducible unitary supercuspidal representation of GL k F,m 1 2 Z with m 0,andn= 2m +1k. Proof As noted in [Zel1, Theorem 9.3], the Casselman criterion implies δ[ν m ρ, ν m is square-integrable. In the other direction, we first claim that if π is square-integrable, suppπ mustbe contained in a set of the form {ν α ρ} α α0 +Z for some irreducible unitary supercuspidal ρ and some α 0 R. This follows easily from Lemma 2.1.2. For example, suppose suppπ {ν α ρ 1 } α α0 +Z {ν β ρ 2 } β β0 +Z, but not completely in either. Here, we allow the possibility that ρ 1 = ρ2 or α 0 β 0 mod 1, but not both. By Lemma 2.1.2, we could obtain and χ 1 = ν α 1 ρ 1 ν α k ρ 1 ν β 1 ρ 2 ν β l ρ 2 r min π χ 2 = ν β 1 ρ 2 ν β l ρ 2 ν α 1 ρ 1 ν α k ρ 1 r min π for some α 1,...,α k,β 1,...,β l. One of the Casselman criterion inequalities for χ 1 gives n 1 α 1 + + n 1 α k > 0. Similarly, one of the Casselman criterion inequalities for χ 2 gives n 2 β 1 + +n 2 β l > 0. Adding these gives n 1 α 1 + +n 1 α k +n 2 β 1 + +n 2 β l > 0. However, by the Casselman criterion, we must have n 1 α 1 + + n 1 α k + n 2 β 1 + + n 2 β l = 0, a contradiction. Thus, suppπ must be contained in a set of the form {ν α } α α0 +Z, as claimed. Since suppπ {ν α } α α0 +Z, we can apply Lemma 2.2.2. Let χ 0 be as in Lemma 2.2.2. If k = 1 notation as in Lemma 2.2.2, then the Casselman criterion inequalities require a 1 = b 1. By Corollary 2.2.4, e.g., wethengetπ = δ[ν a 1 ρ, ν a 1, as needed. Suppose k 2. The Casselman criterion inequalities require a 1 > b 1. So, suppose a i > b i for i = 1,..., j 1anda j b j note that if such a j did not exist, we would have a1 +a 1 1 + + b 1 + + ak +a k 1 + + b k > 0, contradicting the Casselman criterion. Since a j a 1,...,a j 1 and b j b 1,...,b j 1,wehave δ[ν b i ρ, ν a i δ[ν b j ρ, ν a j = δ[ν b j ρ, ν a j δ[ν b i ρ, ν a i is irreducible for all i j 1. From this fact and Corollary 2.2.4, we have π δ[ν b 1 ρ, ν a 1 δ[ν b j 2 ρ, ν a j 2 {δ[ν b j 1 ρ, ν a j 1 δ[ν b j ρ, ν a j } δ[ν b j+1 ρ, ν a j+1 δ[ν b k ρ, ν a k = δ[ν b 1 ρ, ν a 1 δ[ν b j 2 ρ, ν a j 2 {δ[ν b j ρ, ν a j δ[ν b j 1 ρ, ν a j 1 } δ[ν b j+1 ρ, ν a j+1 δ[ν b k ρ, ν a k

550 Chris Jantzen. = δ[ν b j ρ, ν a j δ[ν b 1 ρ, ν a 1 δ[ν b j 2 ρ, ν a j 2 δ[ν b j 1 ρ, ν a j 1 δ[ν b j+1 ρ, ν a j+1 δ[ν b k ρ, ν a k. Therefore, by Frobenius reciprocity, r min π ν a j ρ ν a j 1 ρ ν b j ρ ν a 1 ρ ν a 1 1 ρ ν b 1 ρ ν b k ρ k. In particular, since a j b j,weseethat na j + na j 1 + + nb j 0, contradicting the Casselman criterion. Thus, if π is square-integrable, we must have k = 1, as needed. 2.4 Connection with the Langlands classification In this section, we establish the connection between δ 0 π and the Langlands data for π. We shall also give a lemma which will be needed later. Let us briefly review the Langlands classification for general linear groups. First, if δ is an essentially square-integrable representation of GL n F, then there is an εδ R such that ν εδ δ is unitarizable. Suppose δ 1,...,δ k are irreducible, essentially square-integrable representations of GL n1 F,...,GL nk F withεδ 1 εδ k. We allow weak inequalities since we are assuming δ i is essentially square-integrable; if we allowed δ i essentially tempered, we would have strict inequalities. The formulations are equivalent. Then, δ 1 δ k has a unique irreducible subrepresentation Langlands subrepresentation. Further, any irreducible representation of a general linear group may be realized this way. We favor the subrepresentation version of the Langlands classification over the quotient version since π δ 1 δ k tells us δ 1 δ k appears in the appropriate Jacquet module for π. For notational convenience, let δρ, m = δ[ν m+1 2 ρ, ν m+1 2 for m Z with m 0. Then, ν α δρ, m = δ[ν m+1 2 +α ρ, ν m+1 2 +α. Write δ 0 π = ν α 1 δρ, m 1 ν α 2 δρ, m 2 ν α k δρ, m k. In particular, α i = a i+b i 2 and m i = a i b i +1.Wethenhavethefollowing: Proposition 2.4.1 Let δ 0 π be as above and ν α 1 δρ, m 1,ν α 2 δρ, m 2,...,ν α k δρ, m k be apermutationofν α 1 δρ, m 1,ν α 2 δρ, m 2,...,ν α k δρ, mk with α 1 α 2 α k. Then, Further, is the Langlands data for π. ν α 1 δρ, m 1 ν α 2 δρ, m 2 ν α k δρ, m k = ν α 1 δρ, m 1 ν α 2 δρ, m 2 ν α k δρ, m k. δ 0 π = να 1 δρ, m 1 ν α 2 δρ, m 2 ν α k δρ, m k

Classical p-adic Groups 551 Proof First, we focus on showing ν α 1 δρ, m 1 ν α 2 δρ, m 2 ν α k δρ, m k = ν α 1 δρ, m 1 ν α 2 δρ, m 2 ν α k δρ, m k. Suppose that ν α 1 δρ, m 1 = ν α i δρ, m i. For j < i, wehavea j a i definition of δ 0 π. Further, since α i α j,wemusthaveb j b i. Thus, by Lemma 2.1.1, ν α j δρ, m j ν α i δρ, m i = ν α i δρ, m i ν α j δρ, m j is irreducible. Thus, we can commute ν α i δρ, m i to the front as follows: ν α 1 δρ, m 1 ν α i 2 δρ, m i 2 ν α i 1 δρ, m i 1 ν α i δρ, m i ν α i+1 δρ, m i+1 ν α k δρ, m k = ν α 1 δρ, m 1 ν α i 2 δρ, m i 2 ν α i δρ, m i ν α i 1 δρ, m i 1 ν α i+1 δρ, m i+1 ν α k δρ, m k = ν α 1 δρ, m 1 ν α i δρ, m i ν α i 2 δρ, m i 2 ν α i 1 δρ, m i 1 ν α i+1 δρ, m i+1 ν α k δρ, m k. = ν α i δρ, m i ν α 1 δρ, m 1 ν α i 2 δρ, m i 2 ν α i 1 δρ, m i 1 ν α i+1 δρ, m i+1 ν α k δρ, m k = ν α 1 δρ, m 1 ν α 1 δρ, m 1 ν α i 2 δρ, m i 2 ν α i 1 δρ, m i 1 ν α i+1 δρ, m i+1 ν α k δρ, m k. Next, we identify ν α 2 δρ, m 2 among the remaining terms. We can then use the same argument to commute it into the second position, giving ν α 1 δρ, m 1 ν α k δρ, m k = ν α 1 δρ, m 1 ν α 2 δρ, m 2. Iterating this procedure, after k 1steps,weobtain ν α 1 δρ, m 1 ν α k δρ, m k = ν α 1 δρ, m 1 ν α k δρ, m k, as claimed. The claim regarding the Langlands data is now straightforward. By Corollary 2.2.4, we have π ν α 1 δρ, m 1 ν α k δρ, m k = ν α 1 δρ, m 1 ν α k δρ, m k, which has a unique irreducible subrepresentation whose Langlands data is δ 0π. Thus, δ 0 π must be the Langlands data for π.

552 Chris Jantzen The lemma below will be needed in Section 4.2. Suppose We let χ 1 = ν α 1 ρ ν α 2 ρ ν α j ρ χ 2 = ν β 1 ρ ν β 2 ρ ν β k ρ. m.l.s. χ 1,χ 2 = ν γ 1 ρ ν γ 2 ρ ν γ j+k ρ with ν γ 1 ρ ν γ 2 ρ ν γ j+k ρ the shuffle of χ1 and χ 2 which is minimal with respect to the lexicographic order m.l.s. for minimal lexicographic shuffle. Lemma 2.4.2 Let π 1, π 2 be finite-length representations supported on {ν α ρ} α α0 +Z. Then, χ 0 π 1 π 2 = m.l.s. χ 0 π 1,χ 0 π 2. Proof First, by the characterization of the minimal Jacquet module of an induced representation via shuffles, we have m.l.s. χ 0 π 1,χ 0 π 2 r min π 1 π 2. Thus, χ 0 π 1 π 2 m.l.s. χ 0 π 1,χ 0 π 2 lexicographically. On the other hand, by the characterization of the minimal Jacquet module of an induced representation via shuffles, we have χ 0 π 1 π 2 = sh 0 χ 1,χ 2 for some χ 1 r min π 1, χ 2 r min π 2, and a shuffle sh 0.Bydefinition,χ 0 π 1 χ 1 and χ 0 π 2 χ 2 lexicographically. Therefore, sh 0 χ0 π 1,χ 0 π 2 sh 0 χ 1,χ 2 lexicographically. Thus, m.l.s. χ 0 π 1,χ 0 π 2 sh 0 χ0 π 1,χ 0 π 2 sh 0 χ 1,χ 2 = χ 0 π 1 π 2 lexicographically. Combining the inequalities gives the lemma. 3 Supercuspidal Supports 3.1 Background material In this section, we give some additional background material for S n F. In particular, we review the Casselman criterion, the Langlands classification, as well as some additional structures on R, R[S] which we need later. First, we review the Casselman criterion for the temperedness resp., square-integrability of representations of S n F. Let π be an irreducible representation of S n F and

Classical p-adic Groups 553 ν α 1 ρ 1 ν α k ρk σ s min π, with ρ i an irreducible unitary supercuspidal representation of GL mi F, σ an irreducible supercuspidal representation of S m Fandα i R. Then, if π is tempered, m 1 α 1 0, m 1 α 1 + m 2 α 2 0,.. m 1 α 1 + m 2 α 2 + + m k α k 0. Conversely, if the corresponding inequalities hold for every element of s min π, then π is tempered. The criterion for square-integrability is the same except that the weak inequalities are replaced by strict inequalities. Next, we review the Langlands classification for S n F. As in Section 2.4, if δ is an irreducible, essentially square-integrable representation of GL n F, we have εδ R such that ν εδ δ is unitarizable. Let δ 1,...,δ k be irreducible essentially square-integrable representations of GL n1 F,...,GL nk F satisfying εδ 1 εδ k < 0andτ atempered representation of S n m F. Then, δ 1 δ k τ has a unique irreducible subrepresentation which we denote by Lδ 1,...,δ k ; τ. Equivalently, we could formulate the Langlands classification with δ 1,...,δ k essentially tempered and εδ 1 < <εδ k < 0. Further, any irreducible representation may be realized this way. As with general linear groups, we favor the subrepresentation version of the Langlands classification for the following reason: In the subrepresentation version, δ 1 δ k τ s n1,...,n k Lδ1,...,δ k ; τ. The Langlands classification is done in its general form in [Sil1] and [B-W]; the Casselman criterion in [Cas]. The discussion above is largely based on [Tad1]. The reader is referred there for more details. We now turn to some structures we will need later. Definition 3.1.1 1. If τ is a representation of GL n F, set m τ = n r i τ. i=0 2. If π is a representation of S n F, set µ π = n s i π. i=0 Observethatwemayliftm to a map m : R R R. With multiplication given by and comultiplication given by m, R has the structure of a Hopf algebra cf. [Zel1, Section 1.7]. In particular, if we define : R R R R R R bytaking τ 1 τ 2 τ 1 τ 2 = τ 1 τ 1 τ 2 τ 2 and extending bilinearly, we have m τ 1 τ 2 = m τ 1 m τ 2. Now, define s: R R R R by taking the map s: τ 1 τ 2 τ 2 τ 1 and extending it bilinearly. For notational convenience, write m: R R R for multiplication. Set

554 Chris Jantzen M = m 1 m s m.ifwedefine : R R R R[S] R R[S] by taking τ 1 τ 2 τ θ = τ 1 τ τ 2 θ and extending bilinearly, we have the following: Theorem 3.1.2 Tadić For τ R, θ R[S],wehave µ τ θ = M τ µ θ. In other words, with and µ, R[S] acquires the structure of an M -Hopf module over R cf. [Tad2]. We note that there is a corresponding result for the groups O 2n F [Ban]. Therefore, we expect that once certain issues related to disconnectedness are addressed, it will be possible to bring O 2n F into this discussion as well. Finally, we shall make use of the following: Definition 3.1.3 Suppose ρ is an irreducible, unitary, supercuspidal representation of GL n F andσ and irreducible, supercuspidal representation of S r F. For α 0, we say that ρ, σsatisfiescαifν α ρ σ is reducible and ν β σ is irreducible for all β R\{±α}. It is well-known that if ρ ρ,thenν β ρ σ is irreducible for all β R. Ifρ = ρ,then there is an α 0suchthatν α ρ σ is reducible cf. [Tad5]. Further, it is then the case that ν β ρ σ is irreducible for all β R \{±α}cf. [Sil2]. If σ is generic and ρ, σ satisfiescα, then α {0, 1 2, 1} cf. [Sha1], [Sha2]. If σ is nongeneric, one can have ρ, σ satisfying Cα forα>1cf. [Re]. In general, it is expected and we shall assume that α 1 2 Z. Assuming certain conjectures, this has recently been verified in [Mœ2] and [Zh]. The problem of determining α for a given pair ρ, σ is difficult. However, we note that in the case σ = 1, much is known. In particular, when n 2andρ = ρ is tamely ramified cf. [Adl], the value of α has been explicitly calculated in [M-R] for a large collection of such ρ using a criterion from [Sha2]. 3.2 Reducing the Problem Based on Supercuspidal Supports Let ρ be an irreducible unitary supercuspidal representation of GL n F, β R. Set Sρ, β = {ν α ρ, ν α ρ} α β+z. If ρ = ρ, wemaytake0 β 1 2 ;otherwise0 β<1. Suppose ρ 1,ρ 2,...,ρ m are irreducible, unitary, supercuspidal representations of GL n1 F,...,GL nm F, and β 1,β 2,..., β m R, with0 β i 1 2 if ρ i = ρ i,0 β i < 1 if not. Further, assume that Sρ 1,β 1,...,Sρ m,β m aredisjoint.set S ρ 1,β 1, ρ 2,β 2,...,ρ m,β m = Sρ 1,β 1 Sρ 2,β 2 Sρ m,β m. If σ is an irreducible supercuspidal representation of S r F, set S ρ 1,β 1, ρ 2,β 2,...,ρ m,β m ; σ = Sρ 1,β 1 Sρ 2,β 2 Sρ m,β m {σ}. We note that every irreducible representation of S n F has supercuspidal support on a set of this form.

Classical p-adic Groups 555 We now recall some results from [Jan3]. Suppose π is an irreducible representation supported on S ρ 1,β 1, ρ 2,β 2,...,ρ m,β m ; σ. Then, there exist irreducible representations τ 1,τ 2,...,τ m 1 of GL k1 F, GL k2 F,...,GL km 1 F and an irreducible representation θ m of S km +rfsuchthat π τ 1 τ 2 τ m 1 θ m 2. τ i is supported on Sρ i,β i andθ m is supported on S ρ m,β m ; σ. Further, θ m is unique. Similarly, one could single out ρ 1,β 1,...,ρ m 1,β m 1, resp., to produce θ 1,...,θ m 1, resp., supported on S ρ 1,β 1 ; σ,...,s ρ m 1,β m 1 ; σ,resp. Write ρ i,β i π = θ i. Theorem 3.2.1 Let Irr ρ 1,β 1,...,ρ m,β m ; σ denote the set of all irreducible representations of all S n F,n 0, supported on S ρ 1,β 1,...,ρ m,β m ; σ.then,themap implements a bijective correspondence π ρ 1,β 1 π ρ m,β m π Irr ρ 1,β 1,...,ρ m,β m ; σ Irr ρ 1,β 1 ; σ Irr ρ m,β m ; σ. Further, π is square-integrable resp., tempered if and only if ρ 1,β 1 π,...,ρ m,β m π are all square-integrable resp., tempered. Remark 3.2.2 The correspondence described above also respects contragredience, duality, Langlands data, induction, and Jacquet modules in a sense made precise in [Jan3]. For our purposes, the key feature is that it respects square-integrability. 3.3 A Conjecture of Tadić Suppose that ρ 1,β 1, ρ 2,β 2,...,ρ m,β m,σ are as in Section 3.2. We let R ρ 1,β 1, ρ 2,β 2,...,ρ m,β m denote the subalgebra of R generated by the representations supported in S ρ 1,β 1, ρ 2,β 2,...,ρ m,β m.then, R ρ 1,β 1, ρ 2,β 2,...,ρ m,β m = R ρ1,β 1 R ρ 2,β 2 R ρ m,β m as Hopf subalgebras of R cf. Remark 8.7 of [Zel1]. On irreducible representations, this tensor product decomposition is determined by the appropriate Jacquet module cf. Lemma 2.1.3 2.. If we let R ρ 1,β 1, ρ 2,β 2,...,ρ m,β m ; σ denote the additive subgroup of R[S] generated by representations supported in S ρ 1,β 1, ρ 2,β 2,...,ρ m,β m ; σ,then R ρ 1,β 1, ρ 2,β 2,...,ρ m,β m ; σ = R ρ 1,β 1 ; σ R ρ 2,β 2 ; σ R ρ m ; β m ; σ as R ρ 1,β 1, ρ 2,β 2,...,ρ m,β m = R ρ1,β 1 R ρ 2,β 2 R ρ m,β m M - Hopf modules cf. [Jan3, Proposition 10.10]. On irreducible representations, this tensor product decomposition corresponds to that described in Theorem 3.2.1 above.

556 Chris Jantzen Conjecture 3.3.1 Suppose ρ 1,σ 1, ρ 2,σ 2 bothsatisfycαsamevalueofα, 0 β 1 2.Then, R ρ 1,β; σ 1 = R ρ2,β; σ 2 as R ρ 1,β = R ρ2,β M -Hopf modules. A similar result should hold if ρ i ρ i for i = 1, 2. Of course, if two irreducible representations correspond under the isomorphism, they should have supercuspidal support of the same parabolic rank. We also note that this isomorphism should send tempered resp., square-integrable representations to tempered resp., square-integrable ones and commute with and ˆ =duality operator cf. [Aub], [S-S]. That R ρ 1,β = R ρ2,β is conjectured in [Zel2] or more precisely, is an immediate consequence of a conjecture in [Zel2]. That it holds follows from the results in chapter 7 of [B-K]. The conjecture above was suggested by Marko Tadić. It is not difficult to describe the conjectured isomorphism. For concreteness, suppose that ρ 1,σ 1, ρ 2,σ 2 bothsatisfyc1/2. For R ρ 1,β = R ρ2,β,wewant and δ[ν β+x ρ 1,ν β+y ρ 1 ] δ[ν β+x ρ 2,ν β+y ρ 2 ] δ[ν β+x ρ 1,ν β+y ρ 1 ] δ[ν β+x ρ 2,ν β+y ρ 2 ] for all x, y Z with x y. This gives a bijective correspondence between irreducible essentially square-integrable representations in R ρ 1,β and those in R ρ 2,β.Thisimmediately extends to a bijective correspondence between irreducible essentially tempered representations in R ρ 1,β and those in R ρ 2,β since any irreducible essentially tempered representation in R ρ i,β may be written as an irreducible product of irreducible essentially square-integrable representations in R ρ i,β. Finally, in general, two irreducible representations correspond if their Langlands data correspond. Equivalently, two irreducible representations π 1 R ρ 1,β, π 2 R ρ 2,β correspond if δ 0 π 1 and δ 0 π 2 correspond cf. Chapter 2. We cannot be quite as explicit about the isomorphism R ρ 1,β; σ 1 = R ρ2,β; σ 2, but we can describe an inductive procedure. Suppose we know the map for irreducible representations whose supercuspidal support has parabolic rank n 1. Let π 1 R ρ 1,β; σ 1, π2 R ρ 2,β; σ 2 be irreducible representations with supercuspidal support of parabolic rank n. If π 1, π 2 are nontempered, π 1 π 2 if their Langlands data correspond. This is a question of whether a collection of irreducible essentially tempered representations from R ρ 1,β and R ρ 2,β correspond and whether a tempered representation from R ρ 1,β; σ 1 and one from R ρ2,β; σ 2 both having supercuspidal support of parabolic rank < n correspond. Thus, if π 1, π 2 are nontempered, we can check if π 1 π 2. If π 1, π 2 are tempered but ˆπ 1, ˆπ 2 are nontempered, we can simply check whether ˆπ 1 ˆπ 2. Thus, the only problem is when π 1, π 2 and ˆπ 1,ˆπ 2 are all tempered. By [Jan3, Corollary 4.2], this forces β = 0and π 1 = ρ 1 ρ 1 ρ }{{} 1 σ π 2 = ρ 2 ρ 2 ρ 2 σ }{{} 2, n n

Classical p-adic Groups 557 noting that the hypothesis C1/2 forces both of these to be irreducible by [Gol]. These should correspond under the isomorphism, finishing the inductive procedure. The same argument works for Cαwithα 0orC0 when β 0orinthecasewhereρ i ρ i. In the case where α = 0andβ = 0, we have ρ i σ i = T 1 ρ i ; σ i T 2 ρ i ; σ i. For isomorphism purposes, these cannot be distinguished, giving rise to two such isomorphisms. While it is easy enough to describe the expected isomorphism, it is likely to be very difficult to show that it respects and µ. It is included here mainly for motivation; which we take up momentarily. As for evidence for this conjecture, we point to the fact that the conditions Cα, α = 0, 1 2,... generally seem to be enough to determine how induced representations supported on S ρ, β; σ decompose, especially when Jacquet module methods are employed cf. [Tad3], [Tad4], [Tad5], [Jan1], [Jan2]. To see the significance of this conjecture to the problem at hand, suppose, e.g., ρ, σ satisfies C1/2. Then, theconjecturegivesabijective correspondencebetweenirreducible square-integrable representations supported on S ρ, β; σ and those supported on S ρ,β; σ foranyotherρ,σ satisfying C1/2. Now, if we let ρ be the trivial representation of F and σ the trivial representation of SO 1 F, we have ρ,σ satisfying C1/2. In this case, Mœglin [Mœ1] has parameterized the irreducible square-integrable representations based on the results of Kazhdan-Lusztig [K-L]. Thus, we can expect an analogous parameterization for any pair ρ, σ satisfying C1/2. 4 Basic Results 4.1 A Basic Lemma for S n F Let ρ be an irreducible unitary supercuspidal representation of GL n F, σ an irreducible supercuspidal representation of S r F. Definition 4.1.1 Let π be an irreducible representation supported on S ρ, β; σ.set Xπ = {χ s min π χ = ν α 1 ρ σ has α 1 + + α m minimal for s min π}. Then, let χ 0 π Xπ which is minimal in the lexicographic ordering. Lemma 4.1.2 Assume β 1 2 Z.Then,χ 0π has the form χ 0 π = ν a 1 ρ ν a 1 1 ρ ν b 1 ρ ν a k ρ ν a k 1 ρ ν b k ρ σ, with a 1 a 2 a k. Proof Take τ σ s GL π irreducible such that χ 0 π s min τ σ. Since β 1 2 Z,we may apply Lemma 2.2.2 for τ to finish the proof. Definition 4.1.3 With notation as above, if β 1 2 Z and χ 0 π = ν a 1 ρ ν a 1 1 ρ ν b 1 ρ ν a k ρ ν a k 1 ρ ν b k ρ,

558 Chris Jantzen set δ 0 π = δ[ν b 1 ρ, ν a 1 δ[ν b k ρ, ν a k σ. Lemma 4.1.4 Suppose χ 0 π, δ 0 π as above, β 1 2 Z.LetM= GL a 1 b 1 +1nF GL ak b k +1nF S r F. Then, π i GM δ0 π. Proof The proof parallels that of Corollary 2.2.4. 4.2 A criterion for square-integrability Theorem 4.2.1 Suppose π is an irreducible representation with χ 0 π = ν a 1 ρ ν b 1 ρ ν a m ρ ν b m ρ σ. 1. π is nontempered if and only if a i + b i < 0 for some i. 2. π is tempered but not square-integrable if and only if a i + b i 0 for all i and equality occurs for at least one i. 3. π is square-integrable if and only if a i + b i > 0 for all i. Proof 3 Suppose not say π is square-integrable but a i + b i 0forsomei. Fixi to be the smallest value of i for which a i + b i 0. Now, for j < i we know that a j a i and a j + b j > 0 a i + b i. Therefore, b j > b i.thus, δ[ν b j ρ, ν a j δ[ν b i ρ, ν a i = δ[ν b i ρ, ν a i δ[ν b j ρ, ν a j by irreducibility. Using these equivalences, we may commute δ[ν b i ρ, ν a i forward: π δ[ν b 1 ρ, ν a 1 δ[ν b i 1 ρ, ν a i 1 δ[ν b i ρ, ν a i δ[ν b i+1 ρ, ν a i+1 δ[ν b m ρ, ν a m σ = δ[ν b 1 ρ, ν a 1 δ[ν b i ρ, ν a i δ[ν b i 1 ρ, ν a i 1 δ[ν b i+1 ρ, ν a i+1 δ[ν b m ρ, ν a m σ. = δ[ν b i ρ, ν a i δ[ν b 1 ρ, ν a 1 δ[ν b i 1 ρ, ν a i 1 Therefore, by Frobenius reciprocity, δ[ν b i+1 ρ, ν a i+1 δ[ν b m ρ, ν a m σ. s min π ν a i ρ ν b i ρ ν a 1 ρ ν b i ρ. However, since a i + b i 0, this violates the Casselman criterion for square-integrability. Thus, π square-integrable implies a i + b i > 0foralli.

Classical p-adic Groups 559 2 The proof that π tempered implies a i + b i 0foralli is essentially the same as that used in 3 above. We now argue that if π is not square-integrable, then a i + b i = 0forsomei. If π is tempered but not square-integrable, we have π δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k δ for some square-integrable δ. Wehavek 1. For convenience, we will use δ[ν β i ρ, ν α i and δ[ν α i ρ, ν α i interchangeably when i k. Writeδ 0 δ = δ[ν β k+1 ρ, ν α k+1 δ[ν β l ρ, ν α l. Note that π δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k δ δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k δ[ν β k+1 ρ, ν α k+1 δ[ν β l ρ, ν α l σ. Let δ[ν β 1 ρ, ν α 1,...,δ[ν β l ρ, ν α l be the permutation of δ[ν β 1 ρ, ν α 1,..., δ[ν β l ρ, ν α l satisfying 1. α 1 α 2 α l 2. if α i = α i+1,thenβ i+1 β i. We claim that δ[ν β 1 ρ, ν α 1 δ[ν β l ρ, ν α l = δ[ν β 1 ρ, ν α 1 δ[ν β l ρ, ν α l. Since the proof of this claim is very similar to the proof of Proposition 2.4.1, we omit the details. To apply that argument here, one also needs the following observation: By 1 above, β j + α j > 0for j > k. As a consequence, π δ[ν β 1 ρ, ν α 1 δ[ν β l ρ, ν α l σ. Therefore, by Frobenius reciprocity we see that s min π ν α 1 ρ ν β 1 ρ ν α l ρ ν β l ρ σ. We shall show that this is, in fact, χ 0 π. We now check that χ 0 π = ν α 1 ρ ν β 1 ρ ν α l ρ ν β l ρ σ. In fact, we show more we check that χ 0 δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k δ = ν α 1 ρ ν β 1 ρ ν α l ρ ν β l ρ σ.

560 Chris Jantzen To do this, consider τ σ s GL δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k δ with τ irreducible and χ 0 δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k δ smin τ σ. To calculate this, we write MGL δ[ν β ρ, ν α α+1 = δ[ν i+1 ρ, ν β δ[ν i ρ, ν α 1 as in [Tad5]. It follows from Theorem 3.2 that s GL δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k δ i=β = M GL δ[ν α 1 ρ, ν α 1 M GL δ[ν α k ρ, ν α k s 0 GLδ σ, where we use s 0 GL δtodenotethatpartofs GLδ attached to the general linear group i.e., s GL δ = s 0 GL δ σ. For δ[ν α j ρ, ν α j with j k, thei = α j,α j +1terms in M GL δ[ν β j ρ, ν α j which both give rise to a copy of δ[ν α j ρ, ν α j minimize [ i +1+ i +2+ + α j ]+[i +i +1+ + α j ]. Thus, τ δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k s 0 GL δ. If we let s Xδ GL δ = {τ s 0 GLδ τ irreducible and s min τ σ Xδ}, we must clearly have τ δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k s Xδ GL δ. Now, by Lemma 2.4.2 writing χ 0 δ = χ 0 0 δ σ as above χ 0 δ[ν α 1 ρ, ν α 1 δ[ν α k ρ, ν α k s Xδ GL δ σ = m.l.s. ν α 1 ρ ν α 1 ρ,...,ν α k ρ ν α k ρ,χ 0 0δ σ = ν α 1 ρ ν β 1 ρ ν α l ρ ν β l ρ σ, by the construction of ν α 1 ρ ν β 1 ρ ν α l ρ ν β l ρ σ, as needed. Since π is assumed not to be square-integrable, we have k 1and α 1 + α 1 = 0. Therefore, β i + α i = 0forsomei. Thisfinishesthecase2. 1 Writeπ = L δ[ν β 1 ρ, ν α 1,...,δ[ν β k ρ, ν α k ; T.Write Then, δ 0 T = δ[ν β k+1 ρ, ν α k+1 δ[ν β l ρ, ν α l σ. π δ[ν β 1 ρ, ν α 1 δ[ν β k ρ, ν α k δ[ν β k+1 ρ, ν α k+1 δ[ν β l ρ, ν α l σ. Wecannowargueaswedidtoshowa i + b i = 0forsomei in 2. The converse directions now follow immediately.

Classical p-adic Groups 561 4.3 Supports for Square-Integrable Representations Proposition 4.3.1 Suppose ρ is an irreducible unitary supercuspidal representation of GL n F and σ an irreducible supercuspidal representation of S r F. Let S ρ, β; σ be as in Section 3. 1. If ρ ρ, there are no irreducible square-integrable representation supported on S ρ, β; σ for any β. 2. Suppose ρ = ρ and ρ, σ satisfies Cα. Then, there are irreducible square-integrable representations supported on S ρ, β; σ if and only if β α mod 1. Proof Claim 1 follows from [Tad4, Theorem 6.2]. In the case where β / 1 2 Z,thesecond claim also follows from [Tad4, Theorem 6.2]. We consider 2. when β 1 2Z. First, assume that ρ, σsatisfiescαandβ α mod 1. Now, suppose there were an irreducible square-integrable representation π supported on S ρ, β; σ.write χ 0 π = ν a 1 ρ ν a 1 1 ρ ν b 1 ρ ν a 2 ρ ν a 2 1 ρ ν b 2 ρ ν a k ρ ν a k 1 ρ ν b k ρ σ, noting that a i, b i β β mod 1. By Lemma 4.1.4, we have π δ[ν b 1 ρ, ν a 1 δ[ν b k 1 ρ, ν a k 1 δ[ν b k ρ, ν a k σ. Observe that, by [Tad3, Theorem 13.2], δ[ν b k ρ, ν a k σ = δ[ν a kρ, ν b k σ is irreducible. Thus, π δ[ν b 1 ρ, ν a 1 δ[ν b k 1 ρ, ν a k 1 δ[ν a k ρ, ν b k σ. Therefore, by Frobenius reciprocity, s min π ν a 1 ρ ν a 1 1 ρ ν b 1 ρ ν a k 1 ρ ν a k 1 1 ρ ν b k 1 ρ ν b k ρ ν b k 1 ρ ν a k ρ σ. However, it follows from Theorem 4.2.1 that we have b k + b k 1 + + a k < 0 < a k +a k 1 + + b k, contradicting the construction of χ 0 π. Thus, there can be no irreducible square-integrable representation supported on S ρ, β; σ,asclaimed. The converse direction of 2. follows immediately from the fact that ν α σ has a squareintegrable subrepresentation. We note that in the case where σ is generic, the above proposition follows from [Mu].

562 Chris Jantzen 4.4 Some Constraints on χ 0 π Lemma 4.4.1 Suppose ρ, σ satisfies Cαandπ is an irreducible representation supported on S ρ, β; σ with β α mod 1. If χ 0 π = ν a 1 ρ ν b 1 ρ ν a k ρ ν b k ρ σ, then for each 1 i k, we have b i α. Further,ifα>0, there is at most one i for which b i = α. Proof First, we define a representation we will need in the proof. If α 1, let θ α = ν α ρ δν α ρ; σ, which is irreducible cf. [Tad3]. If α = 1 2,letθ α denote the irreducible subquotient common to ν 1 2 ρ δν 1 2 ρ; σ andδ[ν 1 2 ρ, ν 1 2 σ θ α is needed only for α>0. We observe that θ α is the only irreducible representation containing ν α ρ ν α ρ σ in its minimal Jacquet module. Further, s min θ α ν α ρ ν α ρ σ. Let i bethelargestvalueforwhichb i > 0. Then, [ν b i ρ, ν a i ρ] [ν b j ρ, ν a j ρ]forall j > i. Therefore, commuting arguments give If b i α,weget π δ[ν b 1 ρ, ν a 1 δ[ν b i 1 ρ, ν a i 1 δ[ν b i ρ, ν a i δ[ν b i+1 ρ, ν a i+1 δ[ν b k ρ, ν a k σ = δ[ν b 1 ρ, ν a 1 δ[ν b i 1 ρ, ν a i 1 δ[ν b i+1 ρ, ν a i+1 δ[ν b k ρ, ν a k δ[ν b i ρ, ν a i σ. π ν a 1 ρ ν b 1 ρ ν a i 1 ρ ν b i 1 ρ ν a i+1 ρ ν b i+1 ρ ν a k ρ ν b k ρ ν a i ρ ν bi+1 ρ ν b i ρ σ = ν a 1 ρ ν b 1 ρ ν a i 1 ρ ν b i 1 ρ ν a i+1 ρ ν b i+1 ρ ν a k ρ ν b k ρ ν a i ρ ν b i+1 ρ ν b i ρ σ since ν b i ρ σ = ν b i ρ σ is irreducible. However, by Frobenius reciprocity, this contradicts the minimality of χ 0 πabove.thus,b i = α. Now, suppose b i = α and j < i isthelargestvalueforwhichb j > 0. Then, a commuting argument gives π δ[ν b 1 ρ, ν a 1 δ[ν b k ρ, ν a k σ = δ[ν b 1 ρ, ν a 1 δ[ν b k 2 ρ, ν a k 2 δ[ν b j ρ, ν a j δ[ν b i ρ, ν a i σ,

Classical p-adic Groups 563 where a m = a m if m < j, a m+1 if j m < i, a m+2 if i m, and similarly for b m. If b j >α= b i, we can use a commuting argument to get π δ[ν b 1 ρ, ν a 1 δ[ν b k 2 ρ, ν a k 2 δ[ν b i ρ, ν a i δ[ν b j ρ, ν a j σ π ν a 1 ρ ν b 1 ρ ν a k 2 ρ ν b k 2 ρ ν a i ρ ν b i ρ ν a j ρ ν b j+1 ρ ν b j ρ σ ν a 1 ρ ν b 1 ρ ν a k 2 ρ ν b k 2 ρ ν a i ρ ν b i ρ ν a j ρ ν b j+1 ρ ν b j ρ σ since ν b j ρ σ = ν b j ρ σ is irreducible. This contradicts the minimality of χ 0 πabove. If b j = α = b i,observethatδ[ν α ρ, ν a i δ[ν α ρ, ν a j ν a i ρ ν a j+1 ρ ν a j ρ ν a j ρ ν α ρ ν α ρ. Thus, arguing as above, we get π ν a 1 ρ ν b 1 ρ ν a k 2 ρ ν b k 2 ρ ν a i ρ ν a j+1 ρ ν a j ρ ν a j ρ ν α ρ ν α ρ σ s p,...,p π ν a 1 ρ ν b 1 ρ ν a k 2 ρ ν b k 2 ρ ν a i ρ ν a j+1 ρ ν a j ρ ν a j ρ ν α+1 ρ ν α+1 ρ θ α s min π ν a 1 ρ ν b 1 ρ ν a k 2 ρ ν b k 2 ρ ν a i ρ ν a j+1 ρ ν a j ρ ν a j ρ ν α+1 ρ ν α+1 ρ ν α ρ ν α ρ σ, again contradicting the minimality of χ 0 π above. This finishes the proof. The following refinement is of interest when α 1. Lemma 4.4.2 Suppose ρ, σ satisfiescα and π is an irreducible representation supported on S ρ, α; σ. If now using b i for lower ends χ 0 π = ν a 1 ρ ν b 1 ρ ν a k ρ ν b k ρ σ, then there is a β with α +1 β>0 such that each of { β, β 1,..., α} appears exactly once among b 1, b 2,...,b k and there are no other negative b i s.

564 Chris Jantzen Proof Let [ d 1, c 1 ],...,[ d k, c k ]bethepermutationof[ b 1, a 1 ],...,[ b k, a k ]having d 1 d k and if d i = d i+1,thenc i c i+1. Then, we claim that π δ[ν d 1 ρ, ν c 1 δ[ν d k ρ, ν c k σ. To see this, observe that if a i a j and b i b j,thenδ[ν b i ρ, ν a i δ[ν b j ρ, ν a j = δ[ν b j ρ, ν a j δ[ν b i ρ, ν a i by irreducibility. One can get from δ[ν b 1 ρ, ν a 1 δ[ν b k ρ, ν a k to δ[ν d 1 ρ, ν c 1 δ[ν d k ρ, ν c k through a sequence of such transpositions, hence equivalences are preserved. Therefore, π δ[ν d 1 ρ, ν c 1 δ[ν d k ρ, ν c k σ, as claimed. If d k 0, we are done: β = α + 1. So, suppose d k < 0. By Lemma 4.4.1, d k α. Then, we need to check that d k = α. Ifd k > α, we would have π δ[ν d 1 ρ, ν c 1 δ[ν d k 1 ρ, ν c k 1 δ[ν dk+1 ρ, ν c k ν d k ρ σ = δ[ν d 1 ρ, ν c 1 δ[ν d k 1 ρ, ν c k 1 δ[ν d k+1 ρ, ν c k ν d k ρ σ, which, by Frobenius reciprocity, contradicts the minimality of δ 0 π switching signs on d k lowers the exponent total. Thus, if d k < 0, we have d k = α. Now, consider d k 1. If d k 1 0, we are done: β = α. So, suppose d k 1 < 0. We have d k 1 d k = α. Further, by Lemma 4.4.1, we cannot have d k 1 = d k = α, so d k 1 > α. We need to check that d k 1 = α + 1. Suppose this were not the case. Then, π δ[ν d 1 ρ, ν c 1 δ[ν d k 2 ρ, ν c k 2 δ[ν dk 1+1 ρ, ν c k 1 ν d k 1 ρ δ[ν d k ρ, ν c k σ = δ[ν d 1 ρ, ν c 1 δ[ν d k 2 ρ, ν c k 2 δ[ν dk 1+1 ρ, ν c k 1 δ[ν d k ρ, ν c k ν d k 1 ρ σ = δ[ν d 1 ρ, ν c 1 δ[ν d k 2 ρ, ν c k 2 δ[ν d k 1+1 ρ, ν c k 1 δ[ν d k ρ, ν c k ν d k 1 ρ σ by the irreducibility of ν d k 1 ρ σ. Again, by Frobenius reciprocity, this contradicts the minimality of δ 0 π. Finally, suppose we have d i = d i+1 +1foralli < j with j k 2. If d j 0, we are done: β = d j+1. So, suppose d j < 0. An argument similar to that in the preceding paragraph shows that we cannot have d j > d j+1 + 1. Thus, it remains to show that we cannot have