On Critical p-laplacian Systems

Adv. Nonlinear Stud. 07; ao esearch Article Zhenyu Guo Kanishka Perera* and Wenming Zou On Critical -Lalacian Systems DOI: 0.55/ans-07-609 eceived February 3 06; revised July 3 07; acceted July 3 07 Abstract: We consider the critical -Lalacian system u λa u a u v b = μ u u + αγ u α u v x Ω v λb u a v b v = μ v v + γ u α v v x Ω u v in D 0 (Ω) where u := div( u u) is the -Lalacian oerator defined on D ( ) := u L ( ) : u L ( )} endowed with the norm u D := ( u dx) N 3 < < N λ μ μ 0 γ = 0 a b α > satisfy a + b = α + = := N N the critical Sobolev exonent Ω is N or a bounded domain in and D 0 (Ω) is the closure of C 0 (Ω) in D ( ). Under suitable assumtions we establish the existence and nonexistence of a ositive least energy solution of this system. We also consider the existence and multilicity of the nontrivial nonnegative solutions. Keywords: Nehari Manifold -Lalacian Systems Least Energy Solutions Critical Exonent MSC 00: 35B33 35J0 58E05 Communicated by: Zhi-Qiang Wang Introduction Equations and systems involving the -Lalacian oerator have been extensively studied in the recent years (see e.g. [ 3 5 7 0 3 6 7 9 0 3 6] and the references therein). In the resent aer we study the critical -Lalacian system u λa u a u v b = μ u u + αγ u α u v x Ω v λb u a v b v = μ v v + γ u α v v x Ω u v in D 0 (Ω) where u := div( u u) is the -Lalacian oerator defined on D ( ) := u L ( ) : u L ( )} (.) Zhenyu Guo: School of Sciences Liaoning Shihua University 300 Fushun; and Deartment of Mathematical Sciences Tsinghua University 00084 Beijing P.. China e-mail: guozy@63.com *Corresonding author: Kanishka Perera: Deartment of Mathematical Sciences Florida Institute of Technology Melbourne FL 390 USA e-mail: kerera@fit.edu Wenming Zou: Deartment of Mathematical Sciences Tsinghua University 00084 Beijing P.. China e-mail: wzou@math.tsinghua.edu.cn

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems endowed with the norm u D := ( u dx) N 3 < < N λ μ μ 0 γ = 0 a b α > satisfy a + b = α + = := N N the critical Sobolev exonent Ω is N or a bounded domain in and D 0 (Ω) is the closure of C 0 (Ω) in D ( ). The case for = was thoroughly investigated by Peng Peng and Wang [] recently; some uniqueness synchronization and non-degenerated roerties were verified there. Note that we allow the owers in the couling terms to be unequal. We consider the two cases (H) Ω = λ = 0 μ μ > 0; (H) Ω is a bounded domain in λ > 0 μ μ = 0 γ =. Let S := inf u D u Ω dx (.) 0 (Ω)\0} ( Ω u dx) be the shar constant of imbedding for D 0 (Ω) L (Ω) (see e.g. []). Then S is indeendent of Ω and is attained only when Ω =. In this case a minimizer u D ( ) satisfies the critical -Lalacian equation u = u u x. (.3) Damascelli Merchán Montoro and Sciunzi [4] recently showed that all solutions of (.3) are radial and radially decreasing about some oint in when N N+ <. Vétois [5] considered a more general equation and extended the result to the case < < N N+. Sciunzi [4] extended this result to the case < < N. By exloiting the classification results in [4 8] we see that for < < N all ositive solutions of (.3) are of the form U εy (x) := [N( N ) N ] ε N ( ) ε > 0 y (.4) ε + x y and U εy dx = U εy In case (H) the energy functional associated with system (.) is given by dx = S N. (.5) I(u v) = ( u + v ) (μ u + μ v + γ u α v ) (u v) D where D := D ( ) D ( ) endowed with the norm (u v) D = u D + v D. In this case (.) with α = and = is well studied by Chen and Zou [ ]. Define N = (u v) D : u = 0 v = 0 u = (μ u + αγ u α v ) v = (μ v + γ u α v )}. It is easy to see that N = 0 and that any nontrivial solution of (.) is in N. By a nontrivial solution we mean a solution (u v) such that u = 0 and v = 0. A solution is called a least energy solution if its energy is minimal among energies of all nontrivial solutions. A solution (u v) is ositive if u > 0 and v > 0 and semitrivial if it is of the form (u 0) with u = 0 or (0 v) with v = 0. Set A := inf (uv) N I(u v) and note that A = inf (uv) N N ( u + v ) = inf (uv) N N (μ u + μ v + γ u α v ). Consider the nonlinear system of equations μ k Our main results in this case are the following. + αγ k α l = μ l + γ k α l = k > 0 l > 0. (.6)

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 3 Theorem.. If (H) holds and γ < 0 then A = N (μ (N )/ + μ (N )/ )S N/ and A is not attained. Theorem.. Let (H) and the following conditions hold: N (C) < < N α > and 0 < γ 3 (3 ) min μ α ( α ) μ ( α α ) }; (.7) (C) N N+ < < N α < and γ N (N ) max μ α ( α ) μ ( α α ) }. (.8) Then A = N (k 0 + l 0 )S N/ and A is attained by ( k 0 U εy l 0 U εy ) where (k 0 l 0 ) satisfies (.6) and k 0 = mink : (k l) satisfies (.6)}. (.9) Theorem.3. Assume that N N+ < < N α < and (H) holds. If γ > 0 then A is attained by some (U V) where U and V are ositive radially symmetric and decreasing. Theorem.4 (Multilicity). Assume that N N+ < < N α < and (H) holds. There exists γ (0 N (N ) max μ α ( α ) μ ( α α ) such that for any γ (0 γ ) there exists a solution (k(γ) l(γ)) of (.6) satisfying I( k(γ)u εy l(γ)u εy ) > A and ( k(γ)u εy l(γ)u εy ) is a (second) ositive solution of (.). For the case (H) we have the following theorem. }] Theorem.5. If (H) holds N and 0 < λ < λ (a a b b ) (Ω) where λ (Ω) > 0 is the first Dirichlet eigenvalue of in Ω then system (.) has a nontrivial nonnegative solution. Proof of Theorem. Lemma.. Assume that (H) holds and < γ < 0. If A is attained by a coule (u v) N then (u v) is a critical oint of I i.e. (u v) is a solution of (.). Proof. Define N := (u v) D : u N := (u v) D : u 0 v 0 G (u v) := u (μ u + αγ u α v ) = 0} 0 v 0 G (u v) := v (μ v + γ u α v ) = 0}. Obviously N = N N. Suose that (u v) N is a minimizer for I restricted to N. It follows from the standard minimization theory that there exist two Lagrange multiliers L L such that I (u v) + L G (u v) + L G (u v) = 0.

4 Z. Guo K. Perera and W. Zou Critical -Lalacian Systems Noticing that we get that and I (u v)(u 0) = G (u v) = 0 I (u v)(0 v) = G (u v) = 0 G (u v)(u 0) = ( ) μ u + ( α) G (u v)(0 v) = G (u v)(u 0) = α αγ u α v > 0 γ u α v > 0 G (u v)(0 v) = ( ) μ v + ( ) G (u v)(u 0)L + G (u v)(u 0)L = 0 G (u v)(0 v)l + G (u v)(0 v)l = 0 αγ u α v γ u α v G (u v)(u 0) + G (u v)(0 v) = ( ) u 0 G (u v)(u 0) + G (u v)(0 v) = ( ) v 0. We claim that u > 0. Indeed if u = 0 then by (.) we have u S ( u ) Thus a desired contradiction comes out u 0 almost everywhere in. Similarly v > 0. Hence Define the matrix Then which means that L = L = 0. = 0. G (u v)(u 0) = G (u v)(u 0) > G (u v)(0 v) G (u v)(0 v) = G (u v)(0 v) > G (u v)(u 0). M := ( G (u v)(u 0) G (u v)(u 0) G (u v)(0 v). G (u v)(0 v)) det(m) = G (u v)(u 0) G (u v)(0 v) G (u v)(0 v) G (u v)(u 0) > 0 Proof of Theorem.. It is standard to see that A > 0. By (.4) we know that ω μi u = μ i u u in where i =. Set e = ( 0... 0) and (u (x) v (x)) = (ω μ (x) ω μ (x + e )) := μ ( N)/ i U 0 satisfies where is a ositive number. Then v 0 weakly in D ( ) and v 0 weakly in L ( ) as +. Hence lim u α + v dx = lim u α α + v ( ) v dx lim ( u + v dx) ( = 0. α v dx)

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 5 Therefore for > 0 sufficiently large the system has a solution (t s ) with u dx = μ u dx = t μ u v dx = μ v dx = s μ v α dx + t dx + t α lim ( t + s ) = 0. + Furthermore ( t u s v ) N. Then by (.5) we obtain that A = inf (uv) N I(u v) I( t u s v ) = N (t s s u dx + s v dx) = N (t μ N + s μ N )S N which imlies that A N (μ (N )/ + μ (N )/ )S N/. For any (u v) N u dx μ u dx μ S ( u dx). Therefore u dx μ (N )/ S N/. Similarly Then A N (μ (N )/ + μ (N )/ )S N/. Hence N v dx μ S N. A = N N (μ αγ uα v dx γ uα v dx + μ N )S N. (.) Suose by contradiction that A is attained by some (u v) N. Then ( u v ) N and I( u v ) = A. By Lemma. we see that ( u v ) is a nontrivial solution of (.). By the strong maximum rincile we may assume that u > 0 v > 0 and so u α v dx > 0. Then u dx < μ u dx μ S ( u dx) which yields that Similarly Therefore N u dx > μ S N. N v dx > μ S N. A = I(u v) = N ( u + v ) dx > N N (μ + μ N )S N which contradicts (.).

6 Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 3 Proof of Theorem. Proosition 3.. Assume that c d satisfy μ c + αγ c α d μ d + γ c α d c > 0 d > 0. (3.) If N < < N α > and (.7) holds then c + d k + l where k l satisfy (.6). Proof. Let y = c + d x = c d y 0 = k + l and x 0 = k l. By (3.) and (.6) we have that y y (x + ) μ x (x + ) μ + γ x α + αγ x α := f (x) y0 = f (x 0 ) := f (x) y0 = f (x 0 ). Thus f (x) = f (x) = αγ(x + ) x α [ ( )μ (μ x + αγ x α ) αγ γ(x + ) (μ + γ x α ) α [( )x x + x (α )] αx α + ( )μ ]. γ αγ Let x = ( ( )μ ) /( ) x = α It follows from (.7) that and g (x) = ( )μ x + x (α ) αγ g (x) = ( )x α αx α + ( )μ. γ max g αγ (x) = g (x ) = ( )( x (0+ ) ( ) )μ (α ) 0 min g (x) = g (x ) = ( α α x (0+ ) ) + ( )μ 0. γ That is f (x) is strictly decreasing in (0 + ) and f (x) is strictly increasing in (0 + ). Hence where f = f } := x (0 + ) : f (x) = f (x)}. y maxf (x) f (x)} min x (0+ ) (maxf (x) f (x)}) = min (maxf (x) f (x)}) = y f =f } emark 3.. From the roof of Proosition 3. it is easy to see that system (.6) under the assumtion of Proosition 3. has only one real solution (k l) = (k 0 l 0 ) where (k 0 l 0 ) is defined as in (.9). 0

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 7 Define the functions F (k l) := μ k F (k l) := μ l l(k) := ( αγ ) k(l) := ( γ ) α Then F (k l(k)) 0 and F (k(l) l) 0. + αγ k α l k > 0 l 0 + γ k α l k 0 l > 0 k α ( μ k ) 0 < k μ l α ( μ l ) α 0 < l μ. (3.) Lemma 3.. Assume that N N+ < < N α < and γ > 0. Then F (k l) = 0 F (k l) = 0 k l > 0 (3.3) has a solution (k 0 l 0 ) such that F (k l(k)) < 0 for all k (0 k 0 ) (3.4) that is (k 0 l 0 ) satisfies (.9). Similarly (3.3) has a solution (k l ) such that F (k(l) l) < 0 for all l (0 l ) (3.5) that is (k l ) satisfies (.6) and l = minl : (k l) is a solution of (.6)}. Proof. We only rove the existence of (k 0 l 0 ). It follows from F (k l) = 0 k l > 0 that Substituting this into F (k l) = 0 we have By setting l = l(k) for all k (0 μ ). μ ( α αγ ) ( μ k ) α + γ k ( )α ( αγ ) f(k) := μ ( α αγ ) ( μ k ) α + γ k ( )α ( k ( )( α) αγ ) ( μ k ) = 0. (3.6) k ( )( α) ( μ k ) the existence of a solution of (3.6) in (0 μ /( ) ) is equivalent to f(k) = 0 ossessing a solution in (0 μ /( ) ). Since α < we get that lim f(k) = f k 0 + (μ ) = γ μ α > 0 which imlies that there exists k 0 (0 μ /( ) ) such that f(k 0 ) = 0 and f(k) < 0 for k (0 k 0 ). Let l 0 = l(k 0 ). Then (k 0 l 0 ) is a solution of (3.3) and (3.4) holds. emark 3.. From N N+ < < N and α < we get that < <. It can be seen from N < < N and α > that < <. Lemma 3.3. Assume that N N+ < < N α < and (.8) holds. Let (k 0 l 0 ) be the same as in Lemma 3.. Then (k 0 + l 0 ) maxμ μ } < and F (k l(k)) < 0 for all k (0 k 0 ) F (k(k) l) < 0 for all l (0 l 0 ). (3.7)

8 Z. Guo K. Perera and W. Zou Critical -Lalacian Systems Proof. ecalling (3.) we obtain that From l ( k) = ( μ αγ ) l (k) = ( αγ ) = ( μ αγ ) α (k k (μ l (( α μ ) ) = l (μ ) = 0 k α l (k) > 0 l (k) < 0 (μ k μ k ) ( α α k μ k ) k for k (0 ( α μ ) ) for k (( α μ ) ) [( α μ k ) ( α μ k ) μ ). ) (μ k α( α) )( μ ( ) and k (( α μ ) )/( μ /( ) ) we have k ( α) = ( ( )μ ) )/(. Then by (.8) we get that min l (k) = k (0μ /( ) ] Therefore l (k) > for k (0 μ /( ) ] with min k (( α μ )/( ) μ /( ) ] = ( ( )μ αγ. ( α) k = ( ( )μ ) l (k) = l ( k) ) ( α ) k )] = 0 which imlies that l(k) + k is strictly increasing on [0 μ /( ) ]. Noticing that k 0 < μ /( ) we have μ = l(μ ) + μ > l(k 0 ) + k 0 = l 0 + k 0 that is μ (k 0 + l 0 ) ( )/ <. Similarly μ (k 0 + l 0 ) ( )/ <. To rove (3.7) by Lemma 3. it suffices to show that (k 0 l 0 ) = (k l ). It follows from (3.4) and (3.5) that k k 0 and l 0 l. Suose by contradiction that k > k 0. Then l(k ) + k > l(k 0 ) + k 0. Hence l + k(l ) = l(k ) + k > l(k 0 ) + k 0 = l 0 + k(l 0 ). Following the arguments as in the beginning of the current roof we have that l + k(l) is strictly increasing for l [0 μ /( ) ]. Therefore l > l 0 which contradicts l 0 l. Then k = k 0 and similarly l 0 = l. emark 3.3. For any γ > 0 condition (.8) always holds for dimension N large enough. Proosition 3.4. Assume that N N+ < < N α < and (.8) holds. Then k + l k 0 + l 0 F (k l) 0 F (k l) 0 (3.8) k l 0 (k l) = (0 0) has a unique solution (k l) = (k 0 l 0 ). Proof. Obviously (k 0 l 0 ) satisfies (3.8). Suose that ( k l) is any solution of (3.8) and without loss of generality assume that k > 0. We claim that l > 0. In fact if l = 0 then k k 0 + l 0 and F ( k 0) = μ k ( )/ 0. Thus μ k μ (k 0 + l 0 ) a contradiction to Lemma 3.3.

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 9 Suose by contradiction that k < k 0. It can be seen that k(l) is strictly increasing on (0 ( μ α ) )/( ] and strictly decreasing on Since 0 < [( μ α ) μ ] and k(0) = k(μ ) = 0. k < k 0 = k(l 0 ) there exist 0 < l < l < μ /( ) such that k(l ) = k(l ) = k and F ( k l) < 0 k < k(l) l < l < l. (3.9) It follows from F ( k l) 0 and F ( k l) 0 that l l( k) and l l or l l. By (3.7) we see F ( k l( k)) < 0. By (3.9) we get that l < l( k) < l. Therefore l l. On the other hand set l 3 := k 0 + l 0 k. Then l 3 > l 0 and moreover k(l 3 ) + k 0 + l 0 k = k(l 3 ) + l 3 > k(l 0 ) + l 0 = k 0 + l 0 that is k(l 3 ) > k. By (3.9) we have l < l 3 < l. Since k + l k 0 + l 0 we obtain that l k 0 + l 0 k = l 3 < l. This contradicts l l. Proof of Theorem.. ecalling (.4) and (.6) we see that ( k 0 U εy l 0 U εy ) N is a nontrivial solution of (.) and Then A I( k 0 U εy l 0 U εy ) = N (k 0 + l 0 )S N. (3.0) Let (u n v n )} N be a minimizing sequence for A i.e. I(u n v n ) A as n. Define c n = ( u n Sc n Sd n dx) and d n = ( v n u n dx = (μ u n μ cn + αγ α c v n dx = (μ v n μ dn + γ α c dx). + αγ u n α v n ) dx n d n (3.) + γ u n α v n ) dx n d Dividing both sides of these inequalities by Sc n and Sd n resectively and denoting we deduce that μ cn + αγ c n = α c n c n d S n = d n μ d n S dn n. (3.) + γ α c n dn that is F ( c n d n ) 0 and F ( c n d n ) 0. Then for N < < N and α > Proosition 3. and emark 3. ensure that c n + d n k + l = k 0 + l 0 whereas for N N+ < < N and α < Proosition 3.4 guarantees that c n + d n k 0 + l 0. Therefore c n + d n (k 0 + l 0 )S = (k 0 + l 0 )S N. (3.3) Noticing that I(u n v n ) = N ( u n + v n ) by (3.0) (3.) we have S(c n + d n ) NI(u n v n ) = NA + o() (k 0 + l 0 )S N + o(). Combining this with (3.3) we get that c n + d n (k 0 + l 0 )S (N )/ as n. Thus Hence A = lim I(u n v n ) lim n n N S(c n + d n ) = N (k 0 + l 0 )S N. A = N (k 0 + l 0 )S N = I( k 0 U εy l 0 U εy ).

0 Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 4 Proofs of Theorems.3 and.4 For (H) holding and γ > 0 define where N := (u v) D \ (0 0)} : A := inf I(u v) (uv) N ( u + v ) = (μ u + μ v + γ u α v )}. It follows from N N that A A. By the Sobolev inequality we see that A > 0. Consider u = μ u u + αγ u α u v v = μ v v + γ u α v v u v H0 (B(0 )) where B(0 ) := x : x < }. Define and N () := (u v) H(0 ) \ (0 0)} : B(0) A () := ( u + v ) = B(0) inf I(u v) (uv) N () x B(0 ) x B(0 ) where H(0 ) := H0 (B(0 )) H 0 (B(0 )). For ε [0 minα } ) consider (μ u + μ v + γ u α v )} (4.) Define and I ε (u v) := u = μ u ε (α ε)γ u + ε u α ε u v ε x B(0 ) v = μ v ε ( ε)γ v + ε u α ε v ε v x B(0 ) u v H0 (B(0 )). B(0) ( u + v ) ε B(0) N ε := (u v) H(0 ) \ (0 0)} : G ε (u v) := A ε := B(0) (μ u ε + μ v ε + γ u α ε v ε ) B(0) ( u + v ) (μ u ε + μ v ε + γ u α ε v ε ) = 0} inf I ε (u v). (uv) N ε Lemma 4.. Assume that N N+ < < N α <. For ε (0 minα } ) there holds A ε < min inf (u0) N ε I ε (u 0) inf (0v) N ε I ε (0 v)}. (4.) (4.3) Proof. From minα } it is easy to see that < ε <. Then we may assume that u i is a least energy solution of u = μ i u ε u u H0 (B(0 )) i =. Therefore I ε (u 0) = a := inf I ε (u 0) I ε (0 u ) = a := inf I ε (0 v). (u0) N ε (0v) N ε

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems We claim that for any s there exists a unique t(s) > 0 such that ( t(s)u t(s)su ) N ε. In fact t(s) ε = = B(0) ( u + s u ) B(0) (μ u ε + μ su ε + γ u α ε su ε ) qa + qa s qa + qa s ε + s ε B(0) γ u α ε u ε where q := ( ε) ε = (N ε+ε) N as ε 0. Noticing that t(0) = we have εn+ε that is Then and so lim s 0 t (s) s ε s = ( ε) B(0) γ u α ε u ε ( ε)a t (s) = ( ε) B(0) γ u α ε u ε ( ε)a s ε s( + o()) as s 0. t(s) = B(0) γ u α ε u ε ( ε)a s ε ( + o()) as s 0 t(s) ε Since q = ε we have A ε I ε ( t(s)u t(s)su ) = B(0) γ u α ε u ε a s ε ( + o()) as s 0. = ( ε )(qa + qa s ε + s ε = a ( q ) s ε < a = inf I ε (u 0) (u0) N ε Similarly A ε < inf (0v) N ε I ε (0 v). B(0) B(0) γ u α ε u ε + o( s ε ) as s is small enough. γ u α ε u ε )t ε Noticing the definition of ω μi in the roof of Theorem. similarly to Lemma 4. we obtain that A < min inf (u0) N I(u 0) inf (0v) N = mini(ω μ 0) I(0 ω μ )} = min N N μ S N N I(0 v)} N μ S N }. (4.4) Proosition 4.. For any ε (0 minα } ) system (4.) has a classical ositive least energy solution (u ε v ε ) and u ε v ε are radially symmetric decreasing. Proof. It is standard to see that A ε > 0. For (u v) N ε with u 0 and v 0 we denote by (u v ) its Schwartz symmetrization. By the roerties of the Schwartz symmetrization and γ > 0 we get that B(0) ( u + v ) B(0) (μ u ε + μ v ε + γ u α ε v ε ).

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems Obviously there exists t (0 ] such that ( t u t v ) N ε. Therefore I ε ( t u t v ) = ( ε )t ε ( ε) B(0) B(0) ( u + v ) ( u + v ) = I ε (u v). (4.5) Therefore we may choose a minimizing sequence (u n v n ) N ε of A ε such that (u n v n ) = (u n v n) and I ε (u n v n ) A ε as n. By (4.5) we see that u n v n are uniformly bounded in H0 (B(0 )). Passing to a subsequence we may assume that u n u ε v n v ε weakly in H0 (B(0 )). Since H 0 (B(0 )) ε L (B(0 )) is comact we deduce that B(0) (μ u ε ε + μ v ε ε + γ u ε α ε v ε ε ) = lim which imlies that (u ε v ε ) we get that B(0) n B(0) (μ u n ε + μ v n ε + γ u n α ε v n ε ) = ( ε) ε lim n I ε(u n v n ) = ( ε) ε A ε > 0 = (0 0). Moreover u ε 0 v ε 0 are radially symmetric. Noticing that B(0) ( u ε + v ε ) ( u ε + v ε ) lim B(0) n B(0) ( u n + v n ) (μ u ε ε + μ v ε ε + γ u ε α ε v ε ε ). Then there exists t ε (0 ] such that ( t ε u ε t ε v ε ) N ε and therefore A ε I ε ( t ε u ε t ε v ε ) = ( ε )t ε ε lim n ( ε) B(0) B(0) = lim n I ε (u n v n ) = A ε which yields that t ε = (u ε v ε ) N ε I(u ε v ε ) = A ε and B(0) ( u ε + v ε ) = lim n B(0) ( u ε + v ε ) ( u n + v n ) ( u n + v n ). That is u n u ε v n v ε strongly in H0 (B(0 )). It follows from the standard minimization theory that there exists a Lagrange multilier L satisfying Since I ε(u ε v ε )(u ε v ε ) = G ε (u ε v ε ) = 0 and G ε(u ε v ε )(u ε v ε ) = ( ε ) I ε(u ε v ε ) + LG ε(u ε v ε ) = 0. B(0) (μ u ε ε + μ v ε ε + γ u ε α ε v ε ε ) < 0 we get that L = 0 and so Iε(u ε v ε ) = 0. By A ε = I(u ε v ε ) and Lemma 4. we have u ε 0 and v ε 0. Since u ε v ε 0 are radially symmetric decreasing by the regularity theory and the maximum rincile we obtain that (u ε v ε ) is a classical ositive least energy solution of (4.).

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 3 Proof of Theorem.3. We claim that A () A for all > 0. (4.6) Indeed assume <. Since N ( ) N ( ) we get that A ( ) A ( ). On the other hand for every (u v) N ( ) define (u (x) v (x)) := (( ) u( x) ( ) v( x)). Then it is easy to see that (u v ) N ( ). Thus we have A ( ) I(u v ) = I(u v) for all (u v) N ( ) which means that A ( ) A ( ). Hence A ( ) = A ( ). Obviously A A (). Let (u n v n ) N be a minimizing sequence of A. We assume that u n v n H0 (B(0 n)) for some n > 0. Therefore (u n v n ) N ( n ) and A = lim I(u n v n ) lim A ( n ) = A () n n which comletes the roof of the claim. By recalling (4.) and (4.3) for every (u v) N () there exists t ε > 0 with t ε as ε 0 such that ( t ε u t ε v) N ε. Then lim su ε 0 It follows from (4.6) that A ε lim su I ε ( t ε u t ε v) = I(u v) for all (u v) N (). ε 0 lim su A ε A () = A. (4.7) ε 0 According to Proosition 4. we may let (u ε v ε ) be a ositive least energy solution of (4.) which is radially symmetric decreasing. By (4.3) and the Sobolev inequality we have A ε = ε ( ε) B(0) ( u ε + v ε ) C > 0 for all ε (0 minα } ] (4.8) where C is indeendent of ε. Then it follows from (4.7) that u ε v ε are uniformly bounded in H0 (B(0 )). We may assume that u ε u 0 v ε v 0 u to a subsequence weakly in H0 (B(0 )). Hence (u 0 v 0 ) is a solution of u = μ u u + αγ u α u v x B(0 ) v = μ v v + γ u α v v x B(0 ) u v H0 (B(0 )). Suose by contradiction that u ε + v ε is uniformly bounded. Then by the dominated convergent theorem we get that lim ε 0 B(0) u ε ε = B(0) u 0 lim ε 0 B(0) v ε ε = B(0) v 0 lim ε 0 B(0) uε α ε v ε ε = B(0) u α 0 v 0. Combining these with Iε(u ε v ε ) = I (u 0 v 0 ) similarly to the roof of Proosition 4. we see that u ε u 0 v ε v 0 strongly in H0 (B(0 )). It follows from (4.8) that (u 0 v 0 ) = (0 0) and moreover u 0 0 v 0 0. Without loss of generality we may assume that u 0 0. By the strong maximum rincile we obtain that u 0 > 0 in B(0 ). By the Pohozaev identity we have a contradiction 0 < B(0) ( u 0 + v 0 )(x ν)dσ = 0

4 Z. Guo K. Perera and W. Zou Critical -Lalacian Systems where ν is the outward unit normal vector on B(0 ). Hence u ε + v ε as ε 0. Let K ε := maxu ε (0) v ε (0)}. Since u ε (0) = max B(0) u ε (x) and v ε (0) = max B(0) v ε (x) we see that K ε + as ε 0. Setting we have and (U ε V ε ) is a solution of Since U ε (x) := K ε u ε (K a ε ε x) V ε (x) := K ε v ε (K a ε ε x) a ε := ε maxu ε (0) V ε (0)} = max U ε = μ U ε ε V ε = μ V ε ε U ε (x) dx = K max x B(0Kε aε ) U ε (x) (α ε)γ + ε Uα ε ε ( ε)γ + ε Uα ε ε a ε (N ) ε max x B(0Kε aε ) V ε (x)} = (4.9) V ε ε x B(0 K a ε ε ) V ε ε x B(0 K a ε ε ). u ε (y) dy = K (N )ε ε u ε (x) dx u ε (x) dx we see that (U ε V ε )} n is bounded in D. By ellitic estimates we get that u to a subsequence (U ε V ε ) (U V) D uniformly in every comact subset of as ε 0 and (U V) is a solution of (.) that is I (U V) = 0. Moreover U 0 V 0 are radially symmetric decreasing. By (4.9) we have (U V) = (0 0) and so (U V) N. Thus A I(U V) = ( ) ( U + V ) dx lim inf ε 0 ( ) B(0K aε ε ) = lim inf ε 0 ( ε ) lim inf ε 0 ( ε ) = lim inf ε 0 A ε. ( U ε + V ε ) dx B(0K aε ε ) B(0) ( U ε + V ε ) dx ( u ε + v ε ) dx It follows from (4.7) that A I(U V) lim inf ε 0 A ε A which means that I(U V) = A. By (4.4) we get that U 0 and V 0. The strong maximum rincile guarantees that U > 0 and V > 0. Since (U V) N we have I(U V) A A. Therefore I(U V) = A = A (4.0) that is (U V) is a ositive least energy solution of (.) with (H) holding which is radially symmetric decreasing. This comletes the roof.

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 5 emark 4.. If (H) and (C) hold then it can be seen from Theorems. and.3 that ( k 0 U εy l 0 U εy ) is a ositive least energy solution of (.) where (k 0 l 0 ) is defined by (.9) and U εy is defined by (.4). Proof of Theorem.4. To rove the existence of (k(γ) l(γ)) for γ > 0 small recalling (3.) we denote F i (k l γ) by F i (k l) i = in this roof. Let k(0) = μ /( ) and l(0) = μ /( ). Then Obviously we have which imlies that F (k(0) l(0) 0) = F (k(0) l(0) 0) = 0. k F (k(0) l(0) 0) = μ k > 0 l F (k(0) l(0) 0) = k F (k(0) l(0) 0) = 0 l F (k(0) l(0) 0) = μ l > 0 det ( kf (k(0) l(0) 0) l F (k(0) l(0) 0) k F (k(0) l(0) 0) l F (k(0) l(0) 0) ) > 0. By the imlicit function theorem we see that k(γ) l(γ) are well defined and of class C in ( γ γ ) for some γ > 0 and F (k(γ) l(γ) γ) = F (k(γ) l(γ) γ) = 0. Then ( k(γ)u εy l(γ)u εy ) is a ositive solution of (.). Noticing that N (k(γ) + l(γ)) = k(0) + l(0) = μ + μ N lim γ 0 we obtain that there exists γ (0 γ ] such that It follows from (4.4) and (4.0) that k(γ) + l(γ) > minμ N μ N } for all γ (0 γ ). I( k(γ)u εy l(γ)u εy ) = N (k(γ) + l(γ))s N > min N μ S N N μ S N } N N > A = A = I(U V) that is when (H) is satisfied ( k(γ)u εy l(γ)u εy ) is a different ositive solution of (.) with resect to (U V). 5 Proof of Theorem.5 In this section we consider the case (H). Proosition 5.. Let q r > satisfy q + r and set Then S qr (Ω) = S (Ω) = inf uv W 0 (Ω) uv=0 inf u W 0 (Ω) u=0 S qr (Ω) = Ω ( u + v ) dx ( Ω u q v r dx) Ω u dx ( Ω u dx) q + r (q q r r ). S (Ω). (5.) Moreover if u 0 is a minimizer for S (Ω) then (q / u 0 r / u 0 ) is a minimizer for S qr (Ω).

6 Z. Guo K. Perera and W. Zou Critical -Lalacian Systems Proof. For u = 0 in W 0 (Ω) and t > 0 taking v = t / u in the first quotient gives S qr (Ω) [t r + t q ] Ω u dx ( Ω u dx) and minimizing the right-hand side over u and t shows that S qr (Ω) is less than or equal to the right-hand side of (5.). For u v = 0 in W 0 (Ω) let w = t/ v where t = Ω u dx Ω v dx. Then Ω u dx = Ω w dx and hence u q w r dx u dx = w dx by the Hölder inequality so Ω Ω Ω Ω ( u + v ) dx ( Ω u q v r dx) = Ω (t r u + t q w ) dx ( Ω u q w r dx) t r Ω u dx ( Ω u dx) [t r + t q ]S (Ω). + t q Ω w dx ( Ω w dx) The last exression is greater than or equal to the right-hand side of (5.) so minimizing over (u v) gives the reverse inequality. By Proosition 5. S ab (Ω) = λ (Ω) S α = (a a b b ) (α α ) S (5.) where λ (Ω) > 0 is the first Dirichlet eigenvalue of in Ω. When (H) is satisfied we will obtain a nontrivial nonnegative solution of system (.) for λ < S ab (Ω). Consider the C -functional Φ(w) = Ω [ u + v λ(u + ) a (v + ) b ] dx Ω (u + ) α (v + ) dx w W where W = D 0 (Ω) D 0 (Ω) with the norm given by w = u + v for w = (u v) denotes the norm in L (Ω) and u ± (x) = max±u(x) 0} are the ositive and negative arts of u resectively. If w is a critical oint of Φ 0 = Φ (w)(u v ) = ( u + v ) dx and hence (u v ) = 0 so w = (u + v + ) is a nonnegative weak solution of (.) with (H) holding. Ω Proosition 5.. If 0 = c < S N/ α /N and λ < S ab(ω) then every (PS) c sequence of Φ has a subsequence that converges weakly to a nontrivial critical oint of Φ. Proof. Let w j } be a (PS) c sequence. Then Φ(w j ) = Ω [ u j + v j λ(u + j )a (v + j )b ] dx Ω (u + j )α (v + j ) dx = c + o()

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 7 and Φ (w j )w j = [ u j + v j λ(u + j )a (v + j )b ] dx (u + j )α (v + j ) dx Ω Ω = o( w j ) (5.3) so N [ u j + v j λ(u + j )a (v + j )b ] dx = c + o( w j + ). (5.4) Ω λ Since the integral on the left-hand side is greater than or equal to ( S ab (Ω) ) w j λ < S ab (Ω) and > it follows that w j } is bounded in W. So a renamed subsequence converges to some w weakly in W strongly in L s (Ω) L t (Ω) for all s t < and a.e. in Ω. Then w j w strongly in W q 0 (Ω) Wr 0 (Ω) for all q r < by Boccardo and Murat [6 Theorem.] and hence w j w a.e. in Ω for a further subsequence. It then follows that w is a critical oint of Φ. Suose w = 0. Since w j } is bounded in W and converges to zero in L (Ω) L (Ω) equation (5.3) and the Hölder inequality give o() = ( u j + v j ) dx (u + j )α (v + j ) dx w j ( w j ). If w j 0 then Φ(w j ) 0 contradicting c for a renamed subsequence. Then (5.4) gives contradicting c < S N/ α /N. Ω c = w j N Ω = 0 so this imlies w j N Sα + o() N + o() S α N + o() ecall (.4) and (.5) and let η : [0 ) [0 ] be a smooth cut-off function such that η(s) = for s 4 and η(s) = 0 for s ; set u ερ(x) = η( x ρ )U ε0(x) for ρ > 0. We have the following estimates for u ερ (see [5 Lemma 3.]): u ερ N dx S Sα + C( ε ρ ) N (5.5) C ε log( ρ ε ) Cε if N = uερ dx C ε Cρ ( ε N ρ ) if N > (5.6) u ερ dx S N C( ε N ρ ) (5.7) where C = C(N ). We will make use of these estimates in the roof of our last theorem. Proof of Theorem.5. In view of (5.) Φ(w) ( λ S ab (Ω) ) w Sα w

8 Z. Guo K. Perera and W. Zou Critical -Lalacian Systems so the origin is a strict local minimizer of Φ. We may assume without loss of generality that 0 Ω. Fix ρ > 0 so small that Ω B ρ (0) su u ερ and let w ε = (α / u ερ / u ερ ) W. Note that Φ(w ε ) = ( u ερ λα a b u ερ ) α α u ερ as + and fix 0 > 0 so large that Φ( 0 w ε ) < 0. Then let and set Γ = γ C([0 ] W) : γ(0) = 0 γ() = 0 w ε } c := inf max Φ(γ(t)) > 0. γ Γ t [0] By the mountain ass theorem Φ has a (PS) c sequence w j }. Since t t 0 w ε is a ath in Γ By (5.5) (5.7) c max Φ(t 0w ε ) = t [0] N ( u ερ λ(α a b ) u ερ N N ) =: (α α ) u ερ N S ε. (5.8) S ε S + λ(αa b ) C ε log ε + O(ε ) (α α ) (S + O(ε )) = S α ( λα a α b CS log ε + O())ε if N = and S ε S N λ(αa b ) C ε + O(ε N ) (α α ) (S N + O(ε N )) N N = S α ( λα a α b CS N + O(ε N ))ε if N > so S ε < S α if ε > 0 is sufficiently small. So c < S N/ α /N by (5.8) and hence a subsequence of w j} converges weakly to a nontrivial critical oint of Φ by Proosition 5. which then is a nontrivial nonnegative solution of (.) with (H) holding. Funding: The first and third authors acknowledge the suort of the NSFC (grant nos. 37 7386). eferences []. A. Adams and J. J. F. Fournier Sobolev Saces nd ed. Pure Al. Math. (Amsterdam) 40 Elsevier/Academic Press Amsterdam 003. [] S. N. Armstrong and B. Sirakov Nonexistence of ositive suersolutions of ellitic equations via the maximum rincile Comm. Partial Differential Equations 36 (0) no. 0 047. [3] C. Azizieh and P. Clément A riori estimates and continuation methods for ositive solutions of -Lalace equations J. Differential Equations 79 (00) no. 3 45. [4] M.-F. Bidaut-Véron Local and global behavior of solutions of quasilinear equations of Emden Fowler tye Arch. ation. Mech. Anal. 07 (989) no. 4 93 34. [5] L. Boccardo and D. Guedes de Figueiredo Some remarks on a system of quasilinear ellitic equations NoDEA Nonlinear Differential Equations Al. 9 (00) no. 3 309 33. [6] L. Boccardo and F. Murat Almost everywhere convergence of the gradients of solutions to ellitic and arabolic equations Nonlinear Anal. 9 (99) no. 6 58 597. [7] Y. Bozhkov and E. Mitidieri Existence of multile solutions for quasilinear systems via fibering method J. Differential Equations 90 (003) no. 39 67.

Z. Guo K. Perera and W. Zou Critical -Lalacian Systems 9 [8] J. Byeon L. Jeanjean and M. Mariş Symmetry and monotonicity of least energy solutions Calc. Var. Partial Differential Equations 36 (009) no. 4 48 49. [9] J. Carmona S. Cingolani P. J. Martínez-Aaricio and G. Vannella egularity and Morse index of the solutions to critical quasilinear ellitic systems Comm. Partial Differential Equations 38 (03) no. 0 675 7. [0] C. Chen On ositive weak solutions for a class of quasilinear ellitic systems Nonlinear Anal. 6 (005) no. 4 75 756. [] Z. Chen and W. Zou Positive least energy solutions and hase searation for couled Schrödinger equations with critical exonent Arch. ation. Mech. Anal. 05 (0) no. 55 55. [] Z. Chen and W. Zou Positive least energy solutions and hase searation for couled Schrödinger equations with critical exonent: Higher dimensional case Calc. Var. Partial Differential Equations 5 (05) no. 43 467. [3] P. Clément J. Fleckinger E. Mitidieri and F. de Thélin Existence of ositive solutions for a nonvariational quasilinear ellitic system J. Differential Equations 66 (000) no. 455 477. [4] L. Damascelli S. Merchán L. Montoro and B. Sciunzi adial symmetry and alications for a roblem involving the ( ) oerator and critical nonlinearity in Adv. Math. 65 (04) 33 335. [5] M. Degiovanni and S. Lancelotti Linking solutions for -Lalace equations with nonlinearity at critical growth J. Funct. Anal. 56 (009) no. 3643 3659. [6] L. Ding and S.-W. Xiao Multile ositive solutions for a critical quasilinear ellitic system Nonlinear Anal. 7 (00) no. 5 59 607. [7] H. Dong and D. Kim Global regularity of weak solutions to quasilinear ellitic and arabolic equations with controlled growth Comm. Partial Differential Equations 36 (0) no. 0 750 777. [8] M. Guedda and L. Véron Local and global roerties of solutions of quasilinear ellitic equations J. Differential Equations 76 (988) no. 59 89. [9] T.-S. Hsu Multile ositive solutions for a critical quasilinear ellitic system with concave-convex nonlinearities Nonlinear Anal. 7 (009) no. 7 8 688 698. [0] L. Leadi and H. amos Quoirin Princial eigenvalue for quasilinear cooerative ellitic systems Differential Integral Equations 4 (0) no. 07 4. [] S. Peng Y.-f. Peng and Z.-Q. Wang On ellitic systems with Sobolev critical growth Calc. Var. Partial Differential Equations 55 (06) no. 6 Paer No. 4. [] K. Perera P. Pucci and C. Varga An existence result for a class of quasilinear ellitic eigenvalue roblems in unbounded domains NoDEA Nonlinear Differential Equations Al. (04) no. 3 44 45. [3] K. Perera and I. Sim -Lalace equations with singular weights Nonlinear Anal. 99 (04) 67 76. [4] B. Sciunzi Classification of ositive D ( )-solutions to the critical -Lalace equation in Adv. Math. 9 (06) 3. [5] J. Vétois A riori estimates and alication to the symmetry of solutions for critical -Lalace equations J. Differential Equations 60 (06) no. 49 6. [6] Z. Yang Existence of entire exlosive ositive radial solutions for a class of quasilinear ellitic systems J. Math. Anal. Al. 88 (003) no. 768 783.