The structure of the critical set in the general mountain pass principle

ANNALES DE LA FACULTÉ DES SCIENCES DE TOULOUSE GUANGCAI FANG The structure of the critical set in the general mountain pass principle Annales de la faculté des sciences de Toulouse 6 e série, tome 3, n o 3 (1994), p 345-362 <http://wwwnumdamorg/item?id=afst_1994_6_3_3_345_0> Université Paul Sabatier, 1994, tous droits réservés L accès aux archives de la revue «Annales de la faculté des sciences de Toulouse» (http://picardups-tlsefr/~annales/) implique l accord avec les conditions générales d utilisation (http://wwwnumdamorg/legalphp) Toute utilisation commerciale ou impression systématique est constitutive d une infraction pénale Toute copie ou impression de ce fichier doit contenir la présente mention de copyright Article numérisé dans le cadre du programme Numérisation de documents anciens mathématiques http://wwwnumdamorg/

Annales de la Faculté des Sciences de Toulouse Vol III, no 3, 1994 The structure of the critical set in the general mountain pass principle GUANGCAI FANG(1) RÉSUMÉ - Nous etudions la structure de l ensemble critique engendré par le principe general du col de Ghoussoub-Preiss Ce faisant, nous étendons et simplifions les resultats de structure de Pucci-Serrin établis dans le contexte du principe du col classique de Ambrosetti-Rabinowitz Le result at principal est le suivant : si les deux points base sont non critiques, alors 1 une des trois assertions suivantes sur 1 ensemble critique correspondant est vraie : 1) l ensemble des maxima locaux stricts contient un sous-ensemble fermé qui separe les points de base ; 2) il existe un point selle de type passe-montagne ; 3) il y a au moins un couple d ensembles disjoints non vides de points selle qui sont aussi points limites de minima locaux Ces ensembles sont connectes par 1 cnscmblc des minima locaux mais pas par celui des points selle ABSTRACT - We study the structure of the critical set generated by the general mountain pass principle of Ghoussoub-Preiss In the process, we extend and simplify the structural results of Pucci-Serrin in the context of the classical mountain pass theorem of Ambrosetti-Rabinowitz The main result states that if the two "base points" are not critical, then one of the following three assertions about the corresponding critical set holds true: (1~ the set of proper local maxima contains a closed subset that separates the two base points; (2) there is a saddle point of mountain-pass type; (3) there is at least one pair of nonempty closed disjoint sets of saddle points that are also limiting points of local minima; these sets are connected through the set of local minima but not through the set of saddle points (*) Reçu le 02 novembre 1993 (1) Department of Mathematics, University of British Columbia, Vancouver, BC, Canada V6T 1 Z4 This is a part of the author s PhD dissertation in preparation at the University of British Columbia under the supervision of Dr N Ghoussoub - 345 -

0 Introduction In this paper, we shall use the information obtained in a recent result of Ghoussoub-Preiss [GP] about the location of min-max critical points, to describe the structure of the critical set in the Mountain pass theorem of Ambrosetti-Rabinowitz [AR] We will not be using Morse theory and the functionals are only supposed to be C1 We will use systematically the methodology initiated in [GP] to reprove, simplify and extend various related results established by Hofer [H], Pucci-Serrin [PS1, 2, 3] and Ghoussoub-Preiss [GP] To state the refined version of the Mountain pass theorem, following: we need the DEFINITION (a)2014 A closed subset H of a Banach space X is said to separate two points u and v in X, if u and v belong to two disjoint connected components of X ~ H We will also need the following compactness condition DEFINITION (b)2014 A C1-function 03C6 : X -- IR is said to verify the Palais-Smale condition around the set F at the level c (in short every sequence in X verifying if has a convergent subsequence We will denote by Kc the set of critical points of p at the level c We let also r~ be the set of all continuous paths joining two points u and v in X, that is: where C( ~0,1~; X ~ is the space of all X -valued continuous functions on ~0,1~ We shall make a systematic use of the following

Under Let 1 Structure of the critical set in the separating set F THEOREM 1 (Ghoussoub-Preiss) - p : X ~ IR be a C1-function on a Banach space X For two points u and v in X, consider the number and suppose F is a closed subset of X that separates u and v and such that If ~p verifies then F n Kc is non empty Remark (a~ In the case where F = ~~p > c~, max~ ~p(u~, ~p(v) } that is when c, the above theorem reduces to the well known mountain pass theorem of Ambrosetti-Rabinowitz To classify the various types of critical points, we use the following notations: (Recall that x is said to be a saddle point if in each neighborhood of z there exist two points y and z such that ~p(y~ ~p(z~~ THEOREM 2 - the hypothesis of Theorem 1, assume F n Pc contains no compact set that separates u and v, then either F n Mc ~ 0 or Proof - Suppose that F n M~ = ~ = F n Sc Since F separates u and v we can use a result of Whyburn [Ku, chap VIII, 57; III, theorem 1] to find a closed connected subset F C F that also separates u and v Note that F n Kc = F n Pc and the latter is relatively open in F while F n Kc is

Theorem With closed Since F is connected, then either F ~ Pc = 0 or FnPc = F But the first case is impossible since by Theorem 1 we have F = F n K~ 7~ ~ Hence F ~ Pc which is impossible by assumption and the claim is proved 0 COROLLARY 32014 Under the hypothesis of Theorem 1, assume 03C6 verifies and that u, v ~ Mc If Pc does not contain a Compact subset that separates u and v, then Sc ~ 0 In particular, dimensional, tlten 5 c ~ 0 c, 03C6 verifies (PS)c and X is infinite Proof First observe that Ke is the disjoint union of Sc, Mc and Pc By the condition, we know that K~ is compact Suppose Sc 0 = For each x E Mc, there exists a such that C Lc Let Then M~ C N C Lc Since u, v ~ M~ and M~ is compact, we may assume that u, v ~ N Now put Fo = It is clear that infx~f0 03C6(x) > c and that Fo separates u, v Moreover, Fo n (Mc U = ~ By Theorem 2, P~ n Fo and hence P~ must contain a compact subset that separates u, v A contradiction that completes the proof 0 2 and Corollary 3 improve earlier results by and they are due to Ghoussoub-Preiss ~GP~ The next Theorem 4 appears in ~GP~ and improve the results of Hofer ~H~ and Pucci-Serrin ~PS2~ that appear in Corollary 5 - Remark ~b~ Pucci and Serrin DEFINITION (c) (Hofer)- Say that a point x in Kc is of mountainpass type if for any neighborhood N of x, the set {x E N ~ 5p~x~ c~ is nonempty and not path connected We denote by Hc the set of critical points of mountain pass type at the level c THEOREM 4 - the hypothesis of Theorem 1, we have: (1) Either F ~ Mc ~ Ø (2) If F ~ Pc contains no compact set that separates u and v, then either F n M~ ~ ~ or F n Kc contains a saddle point of mountain pass type Here is an immediate corollary:

Let With COROLLARY 5 - the hypothesis of Theorem 1, assume then: (1~ either M~ ~ Mc ~ 0 or Hc ~ ~; (2) if X is infinite dimensional, then either M~ ~ Mc ~ ~ or Kc contains a saddle point of mountain pass type Proof - It is enough to apply Theorem 4 to the set F = 8~c,p > c~ and to notice that no local minimum can be on such a set To prove Theorem 4, we shall need the following topological lemma Throughout this paper, we denote by e) the open ball in the Banach space X which is centered at ae and with radius e > 0 - LEMMA 6 Fo be a closed subset of X that separates two distinct points u and v Let Zz (i = 1, 2,, n~ be n mutually disjoint open subsets of X such that u, Zi Let G be an open subset of X ~Fo and denote by Yi = Then the following holds: (i) the set Fl = ~ (~ni=1 ~Yi) separates u and v (ii) if Ai (i = 1, 2,, n~ are n nonempty connected components of G and for each i (1 i n ) TZ C (Zi n aaz) is a relatively open subset of ayi such that Ti n 8L 0 for = any connected component L of G with L ~ Ai, then the set F2 = U ayi,tz) also separates u and v Proof (i) Since G C X ~Fo, we have Clearly Fi is closed and u, f ~ Fi We need only to show that for any 96 r~,

If g~~0,1~~ n (Fo B fi) 7" 0, we are done Otherwise so that if g ([0,1]) n (~ni=1 ~Yi) = Ø, then g([0,1]) C ~ni=1 YZ C ~ni=1 ZZ which contradicts that u, v ~ Z2 (ii) We first prove the following claims: for i, j = 1, 2,, n, we have: (a) Ti ~ Yi ~ ~Yi, Ti ~ G = Ø and Ai ~ F2 = Ø; (b) Tj ~ Yi = Ø and Ti ~ Tj = Ø if i ~ j; (c) ZZ n (8G B Ti) c ay2 B TZ (a) Since G is open, it is clear from the definition of TZ that Ti C Zi n 8G so that TZ C YZ n ay2 and Ti n G ~ for = i = 1, 2,, n On the other hand, Ai ~ Yj cazn{z3bg) (c) Since G is open, we have that for any x E ZZ n 8G B Ti, G, hence x E Zi B G and x E YZ Moreover, for any x E 8G B TZ and any E > 0 there is y E B { x, E) n G Clearly y ~ Y2 so that x E ~Yi Since Ti ~ Zi n { ag B Ti ) Ø, = we have that x E ayi B Ti Back to the proof of the Lemma, we note first that the set F2 is closed and is equal to Clearly u, v tf F2 and we need only to show that for any g E g ~[0, i]~ n F2 ~ ~ Suppose not, and take go E r~ such that go ~ [0,1] ~ n F2 = 0 We shall work toward a contradiction First by (1), we have

Let il be the first i E {1,, n} such that go ([0, 1]) n Ti ~ 0 We shall find a gzl E r~ such that To do this, we define the following times: We shall show the following: Indeed, it is clear that 0 sl s2 tl t2 Since u, v ~ Zi, we have 0 sl and t2 1 On the other hand, go(t2) ~ ZZ1 since the latter is open, while go(t1) E ayil n Ti1 since go ([0,1]) n F2 = Ø, hence (a) yields that go(tl ) E ~Yi1 n Til Ti1 = C Modulo a similar reasoning for Zi1 sl, s2, (d) and (e) are therefore verified To prove (f), we note first that go(t) E G for t E (si s2 ) LJ (tl, ~2 ), since otherwise go(t) E Yzl which contradicts (4) and (5) So, for any t E (tl, t2)~ go(t) E U for some connected component U of G If, we have that Ti1 n 8U 0 and since go(tl ) = E Ti1, we see that g0(t1) ~ 8U Hence there must be t3 E (t l t) such that go(t3) E 8U C 8G B By Ti1 (c) we see that F2 which is a contradiction So U = Ail and consequently, go(t) E A21 for all t E (tl, t2), and (f) is proved Since now Ail is path connected, then for sl s21 s2 t1 t21 t2, we can use a path in AZ1 to join go(sz1 ) and go(t21 ) In this way, we get a path gi1 E r~ such that g21 ([0,1]) nti1 = 0 and gi1 ([0,1]) n11 = 0 for 1 i ii, since by (a), Ail n Ti = 0 for all i = 1, 2,,, n On the other hand, since Ail n F2 ~, = we get that gi1 ([0,1]) n F2 = 0 and (3) is established Next, let i2 be the first i E {1,,, n~ such that gil ~~0,1~) n TZ ~ ~ Clearly ii i2 n In the same way, we can construct gz2 E r~ such that

By iterating a finite number of times, we will get a g~ e I v such that for But this contradicts assertion (i) and the lemma is proved D Proof of Theorem 4 (1) Suppose F n Kc contains no critical points of mountain-pass type and F n Me = 0 We claim that: there exist finitely many components of Ge, Indeed, otherwise we could find a sequence xi in F n Kc and a sequence of different components of G~ such that (x2, Ci) -~ 0 But then any limit point of the sequence x2 would be a critical point for p of mountainpass type belonging to F, thus contradicting our initial assumption Hence (*) is verified Let now Mi = F n Ci Since any point of would be a critical point of Mountain-pass type, we may find for each i = 1,, p an open set Ni such that: Since F n Me = 0, we may also assume that Observe that for each i (1 i n), for any x E Mi there must be such that n U = 0 for any component U of Ge with U ~ G~ Put

For Then Ti C N2 n 8Ci and is relatively open in 8fi Now By Lemma 6, F separates u and v, hence F n contradiction since Mi C T2 and F n Me 0 = which is clearly a (2) Since F separates u and v, we can again use the result of Whyburn mentioned above to get a closed connected subset F also separating u and v and such that F 8U = = av where U and V are two components containing u and v respectively Assume F n 0 The set K = F n Pc is an open subset relative to F If K is not closed, then any a? E KBK is a saddle point since F n Me 0 Moreover, = if H is any open neighborhood of a? not intersecting Me and such that ~p c on H, then both sets U n H and V n H meet the set ~Sp c~ This shows that x is a saddle point of mountain pass type Assume now K is closed Then it is a clopen set in the connected space F Hence either K = F or K = 0 In the first case F is then contained in Pc and since it separates u and v, we get a contradiction In the second case, note that by part (1) F n Kc contains a point of mountain pass type Such a point is necessarily a saddle point since F n Me == 0 and F n Pc This clearly finishes the proof of Theorem 4 D For the sequel, we shall need the following concept 2 Structure of the critical set at the level c DEFINITION (d) 2014 A, B two disjoint subsets of X and any nonempty subset C of X, we say that A, B are connected through C if there is no F C CUAUB both relatively closed and open such that A C F and F~B = 0 When A and B are connected through C, we also say that C connects A and B or that the space C U A U B is connected between A and B We refer to Kuratowski [Ku, pp142-148] for details The following theorem is an improvement on the results of Pucci and Serrin [PS3] that are stated in Corollary 8 below

THEOREM 7 - With the hypothesis of Theorem I, we further assume u, v / Kc and that p verifies (PS)c Then one of the following three assertions concerning the set Kc must be true: (I) Pc contains a compact subset that u separates and v; Fig 1 Kc contains a saddle point of mountain-pass type; Fig 2 (iii) there are finitely many components pf Gc, say Cy (I = 1, 2,, n) such that Sc S and S n S( $ for (I # j, I I, j n ) = = where S Sc = n Ci; moreover there are at least two of them Si1c, Si2c, % n Si2c are (ii # 12, 1 ii, 12 n ) such that the sets Mc n nonempty and connected through Mc

With COROLLARY 8 - Fig 3 the hypothesis of Theorem 1, assume further that ~p verifies and that c If ~ ~ does not separate 0 and e, then at least one of the following two cases occurs: ~a~ K~ contains a saddle point of mountain-pass type; ~,Q~ M~ intersects at least two components of S~ Moreover, if Me has only a finite number of components, then either (a~~ K~ contains a saddle point of mountain-pass type, or (03B2 ) at least one component of Mc intersects two or more components o f To prove Theorem 7, we shall need the following two topological lemmas The first is staightforward LEMMA 9 Let M be a subset of a Banach space X Suppose M = Ml U M2 and Ml n M2 = Ø If M1 is both open and closed relative to the subspace M, then there exist open sets D1, D2 of X such that Proof Since Ml is both relatively - open and closed, so is M2 there exist open sets Ei, E2 such that Hence

- y~~ Let Set H1 ne2) = and H2 E2, ( E1 n E2) = Then M1 C Hl, M2 C H2 and Hl n H2 0 Since Ei = is open, for each x E 8E2 n El, 3 > 0 such that the ball B (x, centered at x with radius is contained in Let ~ x dist(x, 8Ei = n E2) Then ~ x > > 0 Set Clearly, M1 C D1, M2 C D2 and D1, D2 are open We now claim that D1 n D2 = ~ Indeed, if not, say z E D1 n D2, then there exist B(x, E~/4~, B(y, Ey/4~ such that z E B(x, E~/4~ n B(y, Ey/4~ Then On the other hand, E~ ~ ~ ~ - and Ey ~ ~ x - which imply that lemma > A contradiction which completes the proof of the n) be n mutually disjoint compact If n~, the sets,52 n M and S ~ n Mare not connected through M, then there are n mutually disjoint open sets N2 (i = 1, 2,, n~ such that - LEMMA 10 S Z (i = 1, 2,, subsets of a Banach space X and let M be any nonempty subset of X for all i, j (i ~ j; i, j 1, 2, =, Pr~oof For each i (i = 1, 2,, n), we denote by MZ the compact set,sz n M Since by assumption none of the pairs M2, M3 (i ~ j; i, j = ~ ~ ~ 1, 2,, n) are connected through M, there exist by Lemma 9 open sets Oij and Pij (Oij Pji, = i ~ j; i, j = 1, 2,, n) such that MZ C Oi j, Mj ~ Pij, Oij ~ Pij = Ø (i ~ j; i, j = 1, 2,, n) and

For each i (i = 1, 2,, n), let Then Put for each i (i = 1, 2,, n) Then by (22)-(25), we have It is not generally true that Ms U M C lemma, we let Oz In order to prove the Then By (25) and (26), we see that M~~ is both open and closed relative to Ms U M Again by Lemma 9, there exist two open sets D and D~ such that

Now for each i (i = 1, 2, have =, n) put OD OZ n D" By ~2?) and (29), we By the compactness of Si and M% we may introduce Let 62 = (1/4) min{ai, 03B41 i = 1, 2,, n} and Then Q, G OiD and Siq n M = 0 By (211), we see that 2014 $2 ~ 362 By the compactness we may also introduce Put Then By (210), we have that R n Ni) = 0 and Ni n Nj ~ (i ~ j; = i, j = 1, 2,, n~ Furthermore Hence

By (214) and (216), we also have that Now let N 1 = Nl U { x E X S q ) b3 ~ U R and By (28), (212) and (215) it follows that By (210), (217) and (218), we see that So (219) and (220) imply that Ni satisfy (21) This completes the proof of the lemma 0 3 Proof of Theorem 7 Suppose assertions (ii) and (iii) are not true and let us prove (i) The critical set Kc is the disjoint union of Sc, Me and Pc Also by the (PS) condition, Kc is compact It is also clear that Sc is closed and compact We will assume that 5c 7~ 0 since otherwise we conclude by Corollary 3 We start with the following: Claim 1 There exist finitely many components of G~, say CZ (i = 1, 2,, n~ and rh > 0 such that Indeed, if not, we could find a sequence xi in Sc and a sequence (Ci) i of different components of Gc such that dist(xi, Ci) ~ 0 But then any limit point of the sequence x2 would be a saddle point of mountain-pass type for

This In p, thus contradicting our assumption that assertion (ii) is false Claim 1 is hence proved We clearly may assume that for all i 1, 2, =, n Next for each i = 1, 2,, n, let S ~,S~ = n Ci They and mutually disjoint Also we have that all are compact Claim ~ There are n mutually disjoint open sets N2 (i = 1, 2,, n~ such that Indeed, we have two cases to consider Case 1 Me ~ = - is a trivial case By the initial assumption that Kc, for each i (i = 1, 2,, n~ there exists an open neighborhood NZ are mutually disjoint compact sets, in such a way that they are also mutually disjoint of S ~ such that u, v / Ni Since the we may take the Case 2 0 2014 this case we are in a situation where we have n mutually disjoint compact sets S ~ (i, = 1, 2,,?t) and a nonempty set Me Moreover all the pairs,5~ n Me,,S~ n Me (i 7~ j; i, j 1; 2, t, n) = are not connected through Me since assertion (33) is assumed false Applying Lemma 10, we can then find n mutually disjoint open sets N2 such that (33) is verified Since u, v ~ Kc, we may clearly assume that Ni Claim 2 is proved in both cases In order to finish the proof of Theorem 7, we still need the following Claim 3 There exists a closed set F such that F separates u, v while We let for each i (i = 1, 2,,, n):

Then observe that for each i (1 i n) there must be B(x, > 0) such that for any connected component U of Gc with U, = 0 Otherwise a? is a saddle point of mountain-pass type and this contradicts that assertion (ii) is assumed false Put Clearly and T ~ is open relative to ayi Gc with U ~ C2 Now let Also ~1 8U = ~ for any component U of Clearly, inf - c Since Fn Lc separates u, v and in view of Claim 1, Claim 2, (35 and (37), we see that we can apply Lemma 6 with Az C2, = G = G~, Zi Yi y{, Ti T~ = = for all i = = 1, 2,, n to conclude that F separates u, v On the other hand, since Me n = ~, we have by (33) and (35), that 8Yie n Me 0 = Therefore by (32) and (36), we have 1 n U M~) ~ Hence F = n (M~ U S~) _ ~ and Claim 3 is thus proved Finally by Theorem 2, we see that F n Pc and hence Pc must contain a compact subset that separates u, v which implies assertion (i) This finishes the proof of the theorem D Remark (c) replaced by Theorem 7 is still true if instead of ~p verifies only The proof also shows that the condition ~, 1? ~ K~ can be S ~ U Me We have the following surprising corollary concerning the cardinality of the critical set Kc generated by the Mountain pass theorem COROLLARY 11 - Suppose dim(x) > 2 Then under ~he hypothesis of Theorem 7, one of the following three assertions must be true: ~1~ k~ has a saddle point of mountain-pass type; (2) the cardinality of Pc is at least the continuum; (3) the cardinality of M~ is at least the continuum

Proof If K~ does - not contain a saddle point of mountain-pass type, then either assertion (i) or assertion (iii) in Theorem 7 is true Let us first assume that assertion (iii) is true Then there exist two disjoint nonempty closed subsets of Ke, say, M; and M; which are connected through Me Clearly dist(m1c, M2c) = d > 0 For any 0 03C3 d, let Mu ~ x = E X ~ ~ Then M~ ~1 M~ = ~, M~ C M~ We claim that 8Mu n Me 7~ 0 Otherwise, there will be two disjoint open sets Mu and XBMu such that M; c Mu, Mu n M; = 0, M~ u M; u M; c Mu u This contradicts that M;,, M~ are connected through M~ Now let m~ E 8Mu n M~ Then we have a map f : ~ E ( 0, d) - m~ E M~ Clearly inj ective which implies assertion (3) f is If instead, assertion (i) of Theorem 7 is true, then Pc separates u and v Let Xo be a subspace of X of dimension 2 containing u, v and let Q Pc n Xo Then Q = separates u, v in Xo Clearly, the cardinality of Q is at least the continuum Since Q C Pc, this implies assertion (2) of the corollary References [AR] AMBROSETTI (A) and RABINOWITZ (PH) 2014 critical point theory and applications, J Funct Anal 14 (1973), pp 349-381 Dual variational methods in [GP] GHOUSSOUB (N) and PREISS (D) A general mountain 2014 pass principle for locating and classifying critical points, Analyse Nonlinéaire 6, n 5 (1989), pp 321-330 [Ku] KURATOWSKI (K) 2014 1968 Topology, Vol II, Academic Press, New York and London, [PS1] PUCCI (P) and SERRIN (J) Extensions of the mountain 2014 pass theorem, J Funct Analysis 59 (1984), pp 185-210 [PS2] PUCCI (P) and SERRIN (J) A mountain 2014 pass theorem, J Diff Eq 60 (1985), pp 142-149 [PS3] PUCCI (P) and SERRIN 2014 (J) The structure of the critical set in the mountain pass theorem, Trans AMS 91, n 1 (1987), pp 115-132 2014 [H] HOFER (H) A geometric description of the neighbourhood of a critical point given by the mountain pass theorem, J London Math Soc 31 (1985), pp 566-570 - 362 -