ORBITAL STABILITY OF SOLITARY WAVES FOR A 2D-BOUSSINESQ SYSTEM

Electronic Journal of Differential Equations, Vol. 05 05, No. 76, pp. 7. ISSN: 07-669. URL: http://ejde.math.txstate.edu or http://ejde.math.unt.edu ftp ejde.math.txstate.edu ORBITAL STABILITY OF SOLITARY WAVES FOR A D-BOUSSINESQ SYSTEM ALEX M. MONTES, JOSÉ R. QUINTERO Abstract. In this article, using a variational approach, we establish the nonlinear orbital stability of ground state solitary waves for a D Boussinesq- Benney-Luke system that models the evolution of three dimensional long water waves with small amplitude in the presence of surface tension.. Introduction The focus of the present work is the two-dimensional Boussinesq-Benney-Luke type system I µ η t + Φ µ 3 Φ + ɛ η Φ = 0, I µ Φ t + η µσ η + ɛ Φ = 0,. that arises in the study of the evolution of small amplitude long water waves in the presence of surface tension see Quintero and Montes []. Here µ, ɛ are small positive parameters, σ is the Bond number associated with the surface tension and the functions ηt, x, y and Φt, x, y denote the wave elevation and the potential velocity on the bottom z = 0, respectively. The aspect that we study about the system. is the orbital stability of solitary wave solutions. It is well know that the study of this kind of states of motion is very important to understand the behavior of many physical systems. A special feature on the Boussinesq system. is that the Benney-Luke equation see [3] and the Kadomtsev-Petviashvili KP equation can be derived up to some order with respect to µ and ɛ from system.. Moreover, for small wave speed and large surface tension, is showed in [] see also [6] that a suitable renormalized family of solitary waves of the Boussinesq system. converges to a nontrivial solitary wave for the KP-I equation. We will use this fact in the stability analysis. One of the main characteristics behind water wave systems is the existence of a Hamiltonian structure which characterizes travelling waves as critical points of the action functional and also provides relevant information for the stability of travelling 00 Mathematics Subject Classification. 35B35, 76B5, 35Q35. Key words and phrases. Solitary waves; orbital stability; ground state solutions; Boussinesq system. c 05 Texas State University - San Marcos. Submitted June, 05. Published June 4, 05.

A. M. MONTES, J. R. QUINTERO EJDE-05/76 waves. In our particular Boussinesq system., the Hamiltonian structure is given by ηt = BH η, B = I µ 0 Φ t Φ, 0 where the Hamiltonian H is defined as η H = Φ Φ + η + µ 3 Φ + µσ η + ɛη Φ dx dy. R On the other hand, by Noether s Theorem, there is a functional Q named the Charge which is conserved in time for classical solutions defined formally as η Q = Φ B η η x, = ηφ Φ Φ x + µη x Φ dx dy. We will see that travelling waves of wave speed c for the Boussinesq system. corresponds to stationary solutions of the modulated system ηt η = BH Φ c, t Φ where H c Y = HY + cqy. In other words, solutions of the system R H Y + cq Y = 0. We note from the Hamiltonian structure that the well definition of the functionals H and H c require having η, Φ H R. These conditions already characterize the natural space energy space to look for travelling waves solutions of the system Bousinesq-Benney-Luke, as shown in the preliminary section. It is important to mention that using the conservation in time of the Hamiltonian, A. Montes et. al. see [7] established the existence of global solutions for the Cauchy problem associated with the system. and the initial condition in the energy space. On the other hand, J. Quintero and A. Montes in [] showed the existence of solitary waves travelling wave solutions in the energy space by using a variational approach in which weak solutions correspond to critical points of an energy under a special constrain. Regarding the stability issue, we need to recall that M. Grillakis, J. Shatah and W. Strauss in [4] gave a general result used to establish orbital stability of solitary waves for a class of abstract Hamiltonian systems. In this case, solitary waves of least energy Y c are minimums of the action functional H c and the stability analysis depends on the positiveness of the symmetric operator H c Y c in a neighborhood of the solitary wave Y c, except possibly in two directions, and also the strict convexity of the real function d c = inf{h c Y : Y M c }, where M c is a suitable set. The verification of the positiveness of H c Y c is much simpler for one-dimensional spatial problems since the spectral analysis for the operator H c Y c is reduced to studying the eigenvalues of a ordinary differential equation which at ± infinity becomes to a ordinary differential equation with constant coefficients see [, 8, ]. The key fact to obtain stability in those cases is that in the one dimensional case solitary waves are unique up to translations, and d can be rescaled allowing to establish the strict convexity d c in a direct way see [, 8, ]. In the two-dimensional spatial case, we have a harder task to overcome using Grillakis et al. approach since the spectral analysis is not straightforward for our problem. In order to avoid making the spectral analysis required

EJDE-05/76 ORBITAL STABILITY 3 in Grillakis et al. work, we used a direct approach to prove orbital stability of ground state solitary wave solutions of the system. in the case of wave speed c near, using strongly the variational characterization of d, as done for other D models: see Shatah for nonlinear Klein Gordon equations [4], Quintero for the D Benney-Luke equation [9] and also in the case of a D Boussinesq-KdV type system [0], Saut for the KP equation [], Fukuizumi for the nonlinear Schrödinger equation with harmonic potential [3] and Liu for the generalized KP equation [5], among others. This article is organized as follows. In section, we present preliminaries related with the existence of solitons solitary wave solutions for the system Boussinesq- Benney-Luke and the link between solitons for the system. and the KP equation. In section 3, we prove the strict convexity of d for c 0,, but near. In section 4, we establish the orbital stability result.. Preliminaries To simplify the computation, we rescale the parameters µ and ɛ from the system. by defining ηt, x, y = ɛ η t µ, x µ, y µ t, Φt, x, y = µ ɛ Φ, µ x µ, y µ. So, by a solitary wave solution for the system. we mean a solution for the rescaled system of the form ηt, x, y = u x ct, y, Φt, x, y = v x ct, y, where c denotes the speed of the wave. Then, one sees that the solitary wave profile u, v should satisfy the system 3 v v + c I u x u v = 0, u σ u c I v x +. v = 0. Our stability analysis of the solitary wave solutions will be perform in the following appropriate spaces. Recall that the standard Sobolev space H k R, k Z +, is the Hilbert space defined as the closure of C0 R with inner product u, v H k = D α u D α v dx. R α k We denote V the closure of C0 R with respect to the norm given by v V := v + v dx dy = v x + vy + vxx + vxy + v yy dx dy. R R Note that V is a Hilbert space with respect to the inner product v, w V = v x, w x H R + v y, w y H R. Also, we define the energy space X = H R V, which is a Hilbert space with respect to the norm u, v X = u H R + v V = u + u + v + v dx dy. R

4 A. M. MONTES, J. R. QUINTERO EJDE-05/76 We can see that solutions u, v of system. are critical points of the functional J c = H c given by J c u, v = I c u, v + Gu, v, where the functionals I c and G are defined on the space X by I c u, v = I u, v + I,c u, v, I u, v = u + σ u + v + 3 v dx dy, R I,c u, v = c uv x + u x v dx dy, R Gu, v = u v dx dy. R In fact, note that I c, G C X, R and its derivatives in u, v in the direction of U, V are given by I cu, v, U, V = uu + σ u U + v V + 3 v V dx dy R c uv x + v x U + u x V + vu x dx dy, R G u, v, U, V = v U + u v V dx dy. R Then we see that J cu, u σ u ci v = v x + v 3 v v + ci u, x u v meaning that critical points of the functional J c satisfy the solitary wave system.... Existence of solitary waves. Quintero and Montes [] established the existence of solitary wave solutions for the Boussinesq-Benney-Luke system. for σ > 0 and 0 < c < min{, 8σ 3 }, by using the Concentration-Compactness principle and the existence of a local compact embedding result. The strategy was to consider the following minimization problem I c := inf{i c u, v : u, v X with Gu, v = }.. The existence of solitary waves is consequence of the following results [], which we will use throughout this work. Next, we assume that σ > 0 and 0 < c < min{, 8σ 3 }. Lemma.. The functional I c is nonnegative and there are positive constants C σ, c < C σ, c defined as C σ, c = min { c, σ c, 3 c }, C σ, c = max { + c, 4σ 3 + c, σ + c } such that C σ, ci c u, v u, v X C σ, ci c u, v..3 Furthermore, I c is finite and positive. Theorem.. If u 0, v 0 is a minimizer for problem., then u, v = ku 0, v 0 is a nontrivial solution of. for k = 3 I c.

EJDE-05/76 ORBITAL STABILITY 5 Theorem.3. If {u m, v m } is a minimizing sequence for., then there is a subsequence which we denote the same, a sequence of points x m, y m R, and a minimizer u 0, v 0 X of., such that the translated functions ũ m, ṽ m = u m + x m, + y m, v m + x m, + y m converge to u 0, v 0 strongly in X... Link between solitary waves for. and the KP equation. Assuming σ > 3/8, c is close to, and balancing the effects of nonlinearity and dispersion, Quintero and Montes [] established that a renormalized family of solitons of the Boussinesq-Benney-Luke system converges to a nontrivial soliton for a KP-I type equation. More precisely, set σ > 0, ɛ > 0, µ = ɛ, c = ɛ and for a given couple u, v X define the functions z and w by ux, y = ɛ / zx, Y, vx, y = wx, Y, X = ɛ / x, Y = ɛy..4 Then a simple calculation shows that I u, v = ɛ / I,ɛ z, w, I cɛ u, v = ɛ / I ɛ z, w, where I, I,ɛ, I ɛ and G ɛ are given by I,c u, v = ɛ / I,ɛ z, w, Gu, v = G ɛ z, w, I ɛ z, w = I,ɛ z, w + I,ɛ z, w, I,ɛ z, w = ɛ z + σzx + ɛzy + ɛ wx + w y dx dy R + w 3 xx + ɛwxy + ɛ w yy dx dy, R I,ɛ z, w = c ɛ zw x + z x w xx + ɛw yy dx dy, R G ɛ z, w = z wx + ɛw y dx dy. R Note that if σ > 3/8 then there is a family {u c, v c } c such that Then, if we denote I c u c, v c = I c, Gu c, v c =, 0 < c <. I ɛ := inf{i ɛ z, w : z, w X with G ɛ z, w = }, there is a correspondent family {z ɛ, w ɛ } ɛ such that I ɛ = I ɛ z ɛ, w ɛ, G ɛ z ɛ, w ɛ =, I c = ɛ / I ɛ..5 We have the following results see []. Lemma.4. Let σ > 3/8. Then we have where lim I ɛ = lim I ɛ z ɛ, w ɛ = J 0 > 0,.6 ɛ 0 + ɛ 0 + J 0 = inf{j 0 w : w V, G 0 w = }, J 0 w = w x + wy + σ 3 w xx dx dy, R

6 A. M. MONTES, J. R. QUINTERO EJDE-05/76 G 0 w = wx 3 dx dy. R Lemma.5. Let σ > 3/8. Then we have lim zɛ x w ɛ = 0 in L R. ɛ 0 + Moreover, there is a nontrivial distribution w 0 V such that Furthermore, lim xw ɛ = x w 0 in L R. ɛ 0 + z ɛ L R + x z ɛ L R = O, yy w ɛ L R = Oɛ, x w ɛ L R + xx w ɛ L R = O. Using the previous lemmas, Quintero et al. showed that there are nontrivial distributions w 0 V, z 0 H R such that as ɛ 0 +, w ɛ w 0 in V, z ɛ z 0 in H R, and x w 0 being a solution of the solitary wave equation for the KP-I type equation ux σ 3 u xxx + 3uu x x + u yy = 0. We shall use Lemmas.4 and.5 in our proof of stability. 3. Variational approach for stability Recall that the solitary waves for the Boussinesq-Benney-Luke system. are characterized as critical points of the functional defined on X by In particular, if we have J c u, v = I c u, v + Gu, v. K c u, v = J cu, v, u, v K c u, v = I c u, v + 3Gu, v = J c u, v + Gu, v. Thus, on any critical point u, v of the functional J c we have that Now, define the set J c u, v = 3 I cu, v, 3. J c u, v = Gu, v, 3. I c u, v = 3 G u, v. 3.3 M c = {u, v X : K c u, v = 0, u, v 0}. Note that M c is just the artificial constrain for minimizing the functional J c on X. We will see that the analysis of the orbital stability of ground states solutions depends upon some properties of the function d defined by dc = inf{j c u, v : u, v M c }.

EJDE-05/76 ORBITAL STABILITY 7 A ground state solution is a solitary wave which minimizes the action functional J c among all the nonzero solutions of.. Moreover, the set of ground state solutions G c = {u, v M c : dc = J c u, v} can be characterized as G c = {u, v X \ {0} : dc = 3 I cu, v = Gu, v} M c. We note that there is a simple relationship between d and d, and so regarding the convexity of them. In fact, dc = inf{j c u, v : u, v M c } = inf{h c u, v : u, v M c } = d c. In the next lemmas we present important variational properties of dc. Lemma 3.. Let 0 < c < and σ > 3/8. Then dc exist and is positive. dc = inf{ 3 I cu, v : K c u, v 0, u, v 0}. Proof. Let u, v M c, then we have that J c u, v = 3 I cu, v 0. This implies that dc exists. Now, Using the Young inequality and that the embedding H R L q R is continuous for q, we see that there is a constant C > 0 such that Gu, v C u 3 H R + v 3 H R. 3.4 Thus, using.3 we see that J c u, v = 3 I cu, v = Gu, v C u, v 3 X C I c u, v 3/. Then follows that 3 I cu, v C, and this implies that dc C > 0. For u, v X such that K c u, v 0 we have that Gu, v < 0. Define α [0, by α = I cu, v 3Gu, v. Then a direct computation shows that K c αu, v = 0. In other words, αu, v M c. So that, Hence, we obtain dc J c αu, v = α 3 I cu, v 3 I cu, v. dc inf { 3 I cu, v : K c u, v 0 }. If u, v M c, we see that J c u, v = 3 I cu, v and inf { 3 I cu, v : K c u, v 0, u, v 0 } inf { J c u, v : u, v M c } = dc. Then the statement of the lemma follows.

8 A. M. MONTES, J. R. QUINTERO EJDE-05/76 Lemma 3.. Let 0 < c < and σ > 3/8. Then If {u m, v m } is a minimizing sequence of dc, then there is a subsequence, which we denote the same, a sequence of points x m, y m R, and u c, v c X \{0} such that the translated functions u m + x m, + y m, v m + x m, + y m converge to u c, v c strongly in X, u c, v c M c, dc = J c u c, v c and u c, v c is a solution of.. Moreover, where I c = inf{i c u, v : Gu, v =, u, v X }. Let {u m, v m } be a sequence in X such that dc = 4 7 I3 c, 3.5 3 I cu m, v m dc and J c u m, v m d dc. Then there exist a subsequence of {u m, v m } which denote the same, a sequence x m, y m R and u c, v c M c such that the translated functions u m + x m, + y m, v m + x k, + y k converge to u c, v c strongly in X and d = dc = 3 I cu c, v c. Proof. The first part of this result is consequence of the Theorem., Theorem.3 and the following argument. Let u, v X \ {0} be such that K c u, v = 0, then Consider the couple Then Gz, w =. Thus, I c I c z, w = I c u, v = 3 Gu, v = 3 Gu, v = 3J cu, v. z, w = G 3 u, v I cu, v = u, v. G /3 u, v /3 3 Ic /3 u, v = /3 3 /3. 3J c u, v So that, we concluded 4 7 I3 c dc. Now, suppose that u, v 0 such that Gu, v =. Take t such that K c tu, tv = 0. In this case, I c u, v + 3t = 0. Therefore Then we obtain, t = 4 9 I c u, v. dc J c tu, tv = t I c u, v + t = 4 7 I3 c u, v. Thus, we have shown that dc 4 7 I c 3. This proves 3.5. Now, we show the second part. Since K c = I c + 3G then we see that J c u m, v m = 3 I cu m, v m + K c u m, v m d dc.

EJDE-05/76 ORBITAL STABILITY 9 Then for m large enough we have that K c u m, v m 0. This fact implies that the sequence {u m, v m } is a minimizing sequence for dc. Then using the part we have that there exist a subsequence of {u m, v m }, which denote the same, a sequence x m, y m R and u c, v c M c such that u m + x m, + y m, v m + x k, + y k u c, v c in X. In particular K c u c, v c = 0 and d = dc = 3 I cu c, v c. Lemma 3.3. Let 0 < c < and σ > 3/8. Then If 0 < c < c < and u, v G c, then we have that dc and I,c u, v are uniformly bounded functions on [c, c ]. If c < c and u ci, v ci G ci, we have the following inequalities c c dc dc I,c u c, v c + oc c, c c c dc dc + I,c u c, v c + oc c. c 3 If 0 < c < c <, u c, v c G c and I,c u c, v c 0, then c c dc dc + I,c u c, v c. 3c In particular, d is a strictly decreasing function on c,. Proof. Let c, c be such that 0 < c < c < and let u, v X be such that Gu, v 0. Define t c by t c = I c u, v 3 Gu, v. Then we have that K c t c u, v = 0 and J c t c u, v = t c 3 I c u, v. Using.3 we see that there exist C > 0 that depends only on σ such that for all c [c, c ], dc J c t c u, v = 4 Ic 3 u, v 7 G u, v C u, v 6 X G u, v. Now, let z, w G c, then we have that I c z, w + 3Gz, w = 0. Moreover, C σ, c, c z, w X I c z, w = 3 Gz, w C z, w 3 X. Then we conclude that Thus, we have shown that C σ, c, c z, w X C σ, c, c 3 I cz, w /. dc C σ, c, c. C σ, c, c Hence, if u, v G c we see that I c u, v and Gu, v are uniformly bounded on [c, c ] since dc = 3 I cu, v = Gu, v, which implies that I,c u, v is also uniformly bounded because K c u, v = 0 and I u, v = u, v X.

0 A. M. MONTES, J. R. QUINTERO EJDE-05/76 Let z, w be defined by z, w = tu c, v c. We want t such that K c z, w = 0. Note that K c z, w = t I c u c, v c + 3t 3 Gu c, v c = t I c u c, v c c c I,c u c, v c c + 3t 3 Gu c, v c = t 3tGu c, v c 3Gu c, v c c c I,c u c, v c. c Thus, t has to be such that tgu c, v c = Gu c, v c + c c 3c I,c u c, v c or equivalently t = + c c I,c u c, v c 3c Gu c, v c = c c I,c u c, v c. 3c dc Then for this t, we conclude that K c z, w = 0. Now, dc J c w, z = t I c u c, v c + tgu c, v c = t I c u c, v c + c c I,c u c, v c + tgu c, v c c = t dc c c I,c u c, v c. 3c But we have that t = c c I,c u c, v c 3c dc = c c I,c u c, v c + O c c. 3c dc Then we see that t dc c c I,c u c, v c 3c = dc c c I,c u c, v c + O c c, c which implies the desired result, c c dc dc I,c u c, v c + oc c. c Now, let z, w be defined by z, w = tu c, v c. As before, we want t such that K c z, w = 0. In this case, t = c c I,c u c, v c = + c c I,c u c, v c. 3c Gu c, v c 3c dc Since K c z, w = 0, we see that dc J c z, w = t dc + c c I,c u c, v c. 3c Then, as above, we have that t = + c c I,c u c, v c + O c c. 3c dc

EJDE-05/76 ORBITAL STABILITY Using this we conclude that t dc + c c I,c u c, v c 3c = dc + c c c I,c u c, v c + O c c, which implies the other inequality. 3 Assume that K c u c, v c = 0. Hence we see that Gu c, v c 0. Now, if I,c u c, v c 0 then for c < c we have that Thus, we obtain K c u c, v c = K c u c, v c + c c c I,c u c, v c 0. dc 3 I c u c, v c = I c u c, v c + c c I,c u c, v c 3 c dc + c c 3c I,c u c, v c. This also implies that dc < dc, provided that 0 < c < c <. Convexity of d. Now, we prove that the function d is strictly convex on, with > 0 near. To do this, we compute d and analyze the behavior of d and d near. We have the following results. Lemma 3.4. If u c, v c G c, then we have that d c = I,cu c, v c. 3.6 c Proof. Note that d can be computed by taking approptiate limits in part of Lemma 3.3 Theorem 3.5. Let σ > 3/8 and u c, v c G c. Then we have that lim dc = 0 and I,c u c, v c < 0 for cnear. c Proof. From Equations.4-.6 and 3.5 we obtain the first part. Now, using the same notation as Section. we have ɛi,ɛ z ɛ, w ɛ = c z ɛ x w ɛ + ɛ x z ɛ xx w ɛ + ɛ yy w ɛ dx dy R = c z ɛ x w ɛ x w ɛ dx dy R cɛ x z ɛ xx w ɛ + ɛz ɛ yy w ɛ dx dy c x w ɛ dx dy. R R Then using Lemma.5 we see that lim ɛ 0 ɛi,ɛ z ɛ, w ɛ < 0, + meaning that for ɛ near 0 + we have I,ɛ z ɛ, w ɛ < 0, which implies that for c near, we ahve I,c u c, v c < 0.

A. M. MONTES, J. R. QUINTERO EJDE-05/76 Theorem 3.6. Let σ > 3/8. Then there exist 0 < < enough near such that d is a decreasing function on,. Furthermore, lim c d c = 0. Proof. Using 3.6 and Theorem 3.5 we have that d is a decreasing function for c near and we also have that lim c u c, v c X = 0 for any u c, v c X such that dc = 3 I cu c, v c, since from.3 we see that u c, v c X CσI c u c, v c = Cσdc. Thus, from 3.6 and definition of I,c we conclude that d c u c L R v c x L R + u c x L R v c L R 3 u c, v c X. Therefore, lim c d c = 0. From the previous results we have the following lemma. Lemma 3.7. Let σ > 3/8, then d and d are strictly convex for c near. We will use the following result by Shatah [4]. Lemma 3.8. Suppose that h is a strictly convex function in a neighborhood of. Then given ε > 0, there exist Nε > 0 such that for c ε = ε, If c ε < < c and c < ε/, hc ε hc c ε c If c < < c ε and c < ε/, hc ε hc c ε c h hc c h hc c Nε. + Nε. Theorem 3.9. Let σ > 3/8. If 0 < < with near and u c0, v c0 G c0, then for c close to, there exist ρc > 0 such that ρ = 0 and c c0 dc d I,c0 u c0, v c0 + ρc. Proof. Let c <, c close to. Then by Lemma 3.8, for c < < c we see that From Lemma 3.3 we have dc d + dc dc c c d dc c Nc. c c 0 I,c0 u c0, v c0 + oc. Then we obtain dc dc c c dc d c Nc I, u c0, v c0 + oc c Nc. Using the continuity of d, we have as c that dc d c I, u c0, v c0 Nc. As a consequence of this inequality follows that c c0 dc d I,c0 u c0, v c0 + c Nc.

EJDE-05/76 ORBITAL STABILITY 3 Now, let < c be c close to. If c < < c, then by using Lemma 3.8, Then from Lemma 3.3, Thus, we obtain dc d dc dc c c d dc c + Nc. c0 c I,c0 u c0, v c0 + oc. dc dc c c dc d c + Nc I, u c0, v c0 + oc c + Nc. Again, using the continuity of d, we have as c that dc d c I, u c0, v c0 + Nc. As a consequence of this inequality holds c c0 dc d I,c0 u c0, v c0 + c Nc, and the result follows. 4. Orbital stability of the solitary waves We first consider the modulated system associated with the system. on X. In other words, we assume that the solution ηt, Φt of the system. has the form ηt, x, y = zt, x ct, y, Φt, x, y = wt, x ct, y Then we see that zt, wt satisfies the modulated system I z t c I z x 3 w + w + z w = 0, I w t c I w x + z σ z + w = 0. Observe that the modulated Hamiltonian for this system has the form H c z, w = J cz, w = Hz, w + I,cz, w, We also observe that H c is conserved in time on solutions since I z t = w H c z, w = c I z x + 3 w w z w, I w t = z H c z, w = c I w x + z σ z + w. Now we introduce the regions R i c, i =,, in the energy space X by R c = {z, w X : H c z, w < dc, 3 I cz, w < dc} R c = {z, w X : H c z, w < dc, 3 I cz, w > dc}, and have the following result. 4. Lemma 4.. R c, R c are invariant regions under the flow for the modulated system 4..

4 A. M. MONTES, J. R. QUINTERO EJDE-05/76 Proof. Let u 0, v 0 R c. Suppose that zt, wt satisfies the modulated system 4. with initial condition z0 = u 0, w0 = v 0. By characterization of dc and definition of R c, we must have that K c u 0, v 0 > 0. In fact, suppose that K c u 0, v 0 0. Then we see that dc 3 I cu 0, v 0. Moreover, if zt, wt R c for some t > 0, we have that K c zt, wt > 0. Now, suppose that there exists a minimum t 0 such that K c zt, wt > 0 for t [0, t 0 and K c zt 0, wt 0 = 0. Observe that dc 3 I czt 0, wt 0 lim inf t t 3 I czt, wt + 3 K czt, wt 0 lim inf J c zt, wt t t 0 lim inf H c zt, wt t t 0 H c z 0, w 0 < dc. This is a contradiction, which shows that R c is invariant under the flow for the modulated system 4.. In a similar fashion we have that R c is also invariant under the flow for the modulated system 4.. The following lemma will be used to obtain stability with respect to the ground state solutions. Lemma 4.. Let σ > 3/8 and 0 < < be near. If Ut = ηt, Φt is a global solution of the Boussinesq-Benney-Luke system. with initial condition U0 = U 0 X, then for every M, there is δm such that if Then we have d + M U 0 U c0 X < δm. 3 I Ut d, for all t R. M Proof. Let M be fixed and define c = M and c = + M. Now, let z i t, w i t be defined by the formulas ηt, x, y = z i t, x c i t, y, Φt, x, y = w i t, x c i t, y, i =,. Then the couple z i t, w i t satisfies the modulated system 4. with initial condition z i 0, w i 0 = U0. For this solution we have that the modulated Hamiltonian is conserved in time, in other words H ci Ut = H ci U0. Now, using hypothesis and inequality.3 we conclude for small δ that I ci U c0 = I ci U0 + Oδ.

EJDE-05/76 ORBITAL STABILITY 5 Since d is a strictly decreasing function such that d = 3 I U c0, we can choose δ small enough in such a way that We also have that dc < 3 I U0 < dc. J ci U0 = J ci U c0 + Oδ = J c0 U c0 + c i I,c0 U c0 + Oδ = d + c i I,c0 U c0 + Oδ dc i ρc i + Oδ, where we have make used of Theorem 3.9. Next, let δ be small enough such that δ < min{ρ, ρ c0 + }. M M This implies H ci U0 = J ci U0 < dc i. 4. Then, using Lemma 4., we have for all t R that H ci Ut < dc i, d + M 3 I Ut d M Finally we establish the main result in this work. Theorem 4.3 Orbital stability. Let σ > 3/8 and 0 < < be near. Then the ground state solitary wave solutions of the Boussinesq-Benney-Luke system. are stable in the following sense: Given ε > 0, there exist δε > 0 such that if U 0 X satisfies U 0 U c0 X < δε, then there exist a unique solution Ut of the Boussinesq-Benney-Luke system. with initial condition U 0 such that inf Ut V X < ε, for all t R. V G c0 Proof. We will argue by contradiction. Suppose that there exist a positive number ε 0, and sequences {t k } R and {U k 0 } X, such that lim k U k 0 U c0 X = 0, inf U k t k V X > ε 0, V G c0 where U k denotes the unique solution of system. with initial condition U k 0 = U0 k. Now, from the Lemma 4. and the assumption, given m > 0 we have the existence of δm and a subsequence k m such that and U km 0 U c0 X < δm d + k m 3 I U k m t km d, k m.

6 A. M. MONTES, J. R. QUINTERO EJDE-05/76 meaning that there exist a subsequence of {U k t k }, which we denote the same, such that d + k 3 I U k t k d. k In particular, we have that 3 I U k t k d as k. Now, we consider c = + k and V k, t defined by U k t, x, y = V k, t, x c t, y. Then as in proof of previous lemma see 4., we obtain that H c U k t k = J c U k t k < dc < d < d. k On the other hand, J c U k t k = J c0 U k t k c c 0 + I,c0 U k t k = J c0 U k t k + I,c0 U k t k. k But note that lim k since we have that I,c0 U k t k lim k k k U k t k X lim k Using these facts, we conclude that U k t k X = 3 I, U k t k d. J c0 U k t k d d. Then by Corollary 3., there exist U c0 G c0 such that as k, U k t k U c0 in X, 3 I U k t k d = d, k C = 0, also J c0 U k t k d. But this contradicts the assumption of instability inf V G c0 U k t k V X > ε 0. Acknowledgments. A. M. Montes was supported by the Universidad del Cauca Colombia under the project I.D. 398. J. R. Quintero was supported by the Mathematics Department at Universidad del Valle Colombia under the project CI 700. A. M. and J. Q. are supported by Colciencias grant No 4878.

EJDE-05/76 ORBITAL STABILITY 7 References [] J. Bona, P. Souganidis, W. Strauss; Stability and instability of solitary waves of Korteweg de Vries type equation. Proc. Roy. Soc. London. Ser A 4 987, 395-4. [] A. de Bouard, J. C. Saut; Remarks on the stability of generalized KP solitary waves. Comtemp. Math. 00 996, 75-84. [3] R. Fukuizumi; Stability and instability of standing waves for the nonlinear Schrödinger equation with harmonic potential. Discrete contin. Dynam. Syst. 7 00, 55-544. [4] M. Grillakis, J. Shatah, W. Strauss; Stability Theory of Solitary Waves in Presence of Symmetry, I. Functional Anal. 74 987, 60-97. [5] Y. Liu, X-P. Wang; Nonlinear stablitity of solitary waves of a generalized Kadomtsev- Petviashvili equation. Comm. Math. Phys. 83 997, 53-66. [6] A. M. Montes; Boussinesq-Benney-Luke type systems related with water wave models. Doctoral Thesis, Universidad del Valle Colombia, 03. [7] A. M. Montes, J. R. Quintero; On the nonlinear scattering and well-posedness for a D- Boussinesq-Benney-Luke type system. Submmited to Journal Diff. Eq. 03. [8] J. R. Quintero; Nonlinear stability of a one-dimensional Boussinesq equation. Dynamic of PDE 5 003, 5-4. [9] J. R. Quintero; Nonlinear stability of solitary waves for a -D Benney-Luke equation. Discrete Contin. Dynam. Systems 3 005, 03-8. [0] J. R. Quintero; The Cauchy problem and stability of solitary waves for a D Boussinesq-KdV type system. Diff. and Int. Eq. 3 00, 35-360. [] J. R. Quintero, A. M. Montes; Existence, physical sense and analyticity of solitons for a D Boussinesq-Benney-Luke System. Dynamics of PDE 0 03, 33-34. [] J. R. Quintero, J. C. Muñoz; Solitons and nonlinear stability/instability for a D Benney- Luke-Paumond Equation. In preparation 04. [3] J. R. Quintero, R. L. Pego; Two-dimensional solitary waves for a Benney-Luke equation. Physica D 45 999, 476-496. [4] J. Shatah; Stable standing waves of nonlinear Klein-Gordon equations. Comm. Math. Phys. A. 9 983, 33-37. Alex M. Montes Universidad del Cauca, Carrera 3 3N-00, Popayán, Colombia E-mail address: amontes@unicauca.edu.co José R. Quintero Universidad del Valle, A. A. 5360, Cali, Colombia E-mail address: jose.quintero@correounivalle.edu.co