Lecture notes. Fundamental inequalities: techniques and applications

Size: px

Start display at page:

Download "Lecture notes. Fundamental inequalities: techniques and applications"

Nelson Anthony
6 years ago
Views:

1 Lecture notes Fundmentl nequltes: technques nd pplctons Mnh Hong Duong Mthemtcs Insttute, Unversty of Wrwck Eml: Jnury 4, 07

2 Abstrct Inequltes re ubqutous n Mthemtcs (nd n rel lfe. For exmple, n optmzton theory (prtculrly n lner progrmmng nequltes re used to descrbed constrnts. In nlyss nequltes re frequently used to derve pror estmtes, to control the errors nd to obtn the order of convergence, just to nme few. Of prtculr mportnce nequltes re Cuchy-Schwrz nequlty, Jensen s nequlty for convex functons nd Fenchel s nequltes for dulty. These nequltes re smple nd flexble to be pplcble n vrous settngs such s n lner lgebr, convex nlyss nd probblty theory. The m of ths mn-course s to ntroduce to undergrdute students these nequltes together wth useful technques nd some pplctons. In the frst secton, through vrety of selected problems, students wll be fmlr wth mny technques frequently used. The second secton dscusses ther pplctons n mtrx nequltes/nlyss, estmtng ntegrls nd reltve entropy. No dvnced mthemtcs s requred. Ths course s tught for Wrwck s tem for Interntonl Mthemtcs Competton for Unversty Students, 4th IMC 07. The competton s plnned for students completng ther frst, second, thrd or fourth yer of unversty educton.

3 3 Contents Fundmentl nequltes: Cuchy-Schwrz nequlty, Jensen s nequlty for convex functons nd Fenchel s dul nequlty 4. Cuchy-Schwrz nequlty Convex functons Jensen s nequlty Convex conjugte nd Fenchel s nequlty Some technques to prove nequltes Applctons to mtrx nequltes, estmtng ntegrls nd reltve entropy. Mnkowsk s nequlty for determnnts Hdmrd s Inequlty Hermte-Hdmrd nequlty Reltve entropy

4 Fundmentl nequltes: Cuchy-Schwrz nequlty, Jensen s nequlty for convex functons nd Fenchel s dul nequlty In ths secton, we revew three bsc nequltes tht re Cuchy-Schwrz nequlty, Jensen s nequlty for convex functons nd Fenchel s nequlty for dulty. For smplcty of presentton, we only consder smplest underlyng spces such s R n or fnte set. These nequltes, however, cn be stted n much more complex stutons. Mny technques for provng nequltes re presented v selected exmples nd exercses n Secton Cuchy-Schwrz nequlty Let u, v be two vectors of n nner product spce (over R, for smplcty. The Cuchy- Schwrz nequlty sttes tht u, v u v. Proof. If v = 0, the nequlty s obvous true. Let v 0. For ny λ R, we hve 0 u + λv = u + λ u, v + λ v. Consder ths s qudrtc functon of λ. Therefore we hve u, v u v, whch mples the Cuchy-Schwrz nequlty.. Convex functons Defnton. (Convex functon. Let X R n be convex set. A functon f : X R s cll convex f for ll x, x X nd t [0, ] f(tx + ( tx tf(x + ( tf(x. A functon f : X R s cll strctly convex f for ll x, x X nd t (0, f(tx + ( tx < tf(x + ( tf(x. Exmple.. Exmples of convex functons: ffne functons: f(x = x + b, for, b R n, exponentl functons: f(x = e x for ny R,

5 5 ( n Euclden-norm: f(x = x = x. Verfyng convexty of dfferentble functon Note tht n Defnton., the functon f does not requre to be dfferentble. The followng crter cn be used to verfy the convexty of f when t s dfferentble. ( If f : X R s dfferentble, then t s convex f nd only f f(x f(y + f(y (x y for ll x, y X. ( If f : X R s twce dfferentble, then t s convex f nd only f ts Hessn f(x s sem-postve defnte for ll x X. Some mportnt propertes of convex functons Lemm.. Below re some mportnt propertes of convex functons. ( If f nd g re convex functons, then so re m(x = mx{f(x, g(x} nd s(x = f(x + g(x. ( If f nd g re convex functons nd g s non-decresng, then h(x = g(f(x s convex. Proof. These propertes cn be drectly proved by verfyng the defnton..3 Jensen s nequlty Theorem.3 (Jensen s nequlty. Let f be convex functon, 0 α ; =,..., n such tht α =. Then for ll x,..., x n, we hve ( f α x α f(x. ( Proof. We prove by nducton. The cses n =, re obvous. Suppose tht the sttement s true for n =,..., K. Suppose tht α,..., α K re non-negtve numbers such tht K α =. We need to prove tht ( K f α x K α f(x.

6 6 Snce K α =, t lest one of the coeffcents α must be strctly postve. Assume tht α > 0. Then by the conductng ssumptons, we obtn ( K ( f α x = f α x + ( α where we hve used the fct tht K K = α x α ( K α α f(x + ( α f x α α f(x + ( α = K α f(x, = α α =. K = = α α f(x Remrk.4 (Jensen s nequlty- probblstc form. Jensen s nequlty cn lso be stted usng probblstc form. Let (Ω, A, µ be probblty spce. If g s rel-vlued functon tht s µ ntegrble nd f f s convex functon on the rel lne, then ( f g dµ f g dµ. Ω Ω Exmple. (Exmples of Jensen s nequlty. For ll rel numbers x,..., x n, t holds ( x n x. Proof. Snce f(x = x s convex, we hve ( n ( x = f n x n f(x = n x, whch s the desred sttement. Arthmetc-Geometrc (AM-GM Inequlty. Let (x n nd (λ n be rel number stsfyng x 0, λ 0, λ =.

7 7 Then, wth the conventon 0 0 =, λ x n x λ. ( In prtculr, tkng λ =... = λ n = yelds n x n /n x... x n. Proof. By tkng the logrthmc both sdes, ( s equvlent to ( λ ln(x ln λ x. Ths s exctly Jensen s nequlty pplyng to the convex functon f(x = ln(x..4 Convex conjugte nd Fenchel s nequlty The convex conjugte of functon f : R d R s, f : R d R, defned by f (y = sup x R d {x y f(y}. Fenchel s nequlty: for x, y R d, we hve f(x + f (y x y. Exmple.3. Exmples of Fenchel s nequlty f(x = x, then f (y = sup{x y x } = y. Fenchel s nequlty reds x R d ( x + y x y. f(x = p x p where p >. Then f (y = sup{x y p x p } = q y q where + =. p q x R d Fenchel s nequlty becomes: for x, y R d nd p, q > such tht + =, we hve p q p x p + q y q x y.

8 8.5 Some technques to prove nequltes In prctses, three nequltes ntroduced n prevous sectons often do not pper n stndrd forms. It s crucl to recognze them. In ths secton, through exercses we wll lern some technques to prove nequltes. Exercse. Let, b R, b > 0 for =,..., n. Prove tht b ( n n b. Proof. By the Cuchy-Schrz nequlty we hve ( ( ( ( = b b. b b Exercse (Problem 6, IMC 05. Prove tht n= Proof. By the AM-GM Inequlty we hve whch mples tht Dvdng both sdes by n(n + yelds Hence by summng up over n we obtn n(n + <. (n + = n + (n + > n(n +, n= (n + n(n + >. ( < n. n(n + n + n(n + < n= ( n =. n + Exercse 3. Let A, B, C be three ngles of trngle. Prove tht 3 sn A + sn B + sn C 3.

9 9 Proof. Consder the functon f(x = sn x. Snce f (x = sn (x 0, f s concve. Therefore, sn A + sn B + sn C ( A + B + C sn = sn π =. Exercse 4 (Problem 3, IMC 06. Let n be postve nteger nd,..., n ; b,..., b n be rel number such tht + b > 0 for =,..., n. Prove tht ( n b b b b + b. (3 ( + b Proof. We notce the smlr form of both sdes of (3. For A, B R we hve Applyng (4 for A =, B = b, we get LHS = b b + b = AB B A + B = A B A + B ( b = + b b + b. (4 Smlrly now pplyng (4 for A = b, B = n b, we obtn ( n b RHS = n. + b Therefore (3 s equvlent to ( n By Cuchy-Schrz nequlty we hve ( ( b = whch mples (5 s desred. b n + b b ( + b + b b + b (5 b + b Exercse 5 (Problem, IMC 00. Let 0 < < b. Prove tht b (x + e x dx e e b ( + b,

10 0 Proof. By the AM-GM Inequlty x + x > 0 for ny 0 < x b, we hve b (x + e x dx b xe x dx = e x b = b (x + e x dx. Exercse 6 (Problem 6, IMC 00. Let n be n nteger nd let f n (x = sn x sn(x... sn( n x. Prove tht f n (x 3 f n (π/3. Proof. Let g(x = sn x sn(x /. We hve ( g(x = sn x sn(x / 4 = 4 sn x sn x sn x 3 cos x 3 3 sn x + 3 cos x ( 3 4 = = g(π/ Note we hve use the AM-GM nequlty tht 3 sn x + 3 cos x = sn x + sn x + sn x + ( 3 cos x 4 4 sn x sn x sn x 3 cos x. Therefore, let K = 3 [ ( / n ], we hve f n (x = sn x sn(x... sn( n x / ( 3/4 = ( sn x sn(x / ( sn(x sn(4x / sn(4x sn(8x / K ( K/... ( sn( n x sn( n x / sn( n x = g(x g(x /... g( n x K ( sn( n x g(π/3g(x g(π/3 /... g( n π/3 K = f n (π/3 / sn( n π/3 K/ ( K/ = f n (π/3 3 fn (π/3. 3 Ths s the desred nequlty. K/ Exercse 7 (IMO 995. Let, b, c be postve rel numbers such tht bc =. Prove tht 3 (b + c + b 3 (c + + c 3 ( + b 3.

11 Proof. Let x =, y =, z =. Then x, y, z re postve rel numbers nd xyz =. We b c hve 3 (b + c = = x y + z. Smlrly b 3 (c + = x 3 ( y + z y z + x, c 3 ( + b = z x + y. By Cuchy-Schrz nequlty (see lso Exercse nd the Arthmetc-Geometrc Inequlty we hve x y + z + y z + x + z x + y (x + y + z (x + y + z = x + y + z 3 3 xyz = 3. Exercse 8 (Problem, The 6th Annul Vojtech Jrnk Interntonl Competton 06. Let, b, c be postve rel number such tht + b + c =. Prove tht ( + bc Proof. By the AM-GM nequlty we hve ( b c( + c + 78 b + bc = + 3bc + 3bc + 3bc 4 4 b3 c 3, nd Therefore, ( + bc ( + b + c 3 7 = bc,.e., 3 bc 7. ( b c( + c + ( 4 b = 64 bc 4 b3 c 3 (4 4 b3 c 3 ( = c3 b 3 Exercse 9. Let x, y R, y > 0. Prove tht e x + y(ln y x y. Proof. Let f(x = e x. Then for y > 0, we hve f (y = sup x R {x y e x } = y(ln y. By Fenchel s nequlty we hve f(x + f (y = e x + y(ln y x y s desred.

12 Exercse 0 (IMO 00. Let, b, c be postve rel numbers. Prove tht + 8bc + b b + 8c + c c + 8b. Proof. Snce the expresson on the LHS does not chnge when we replce (, b, c by (k, kb, kc for rbtrry k R, we cn ssume tht + b + c =. Snce x x s convex for x > 0, pplyng Jensen s nequlty we obtn + 8bc + Next we show tht b b + 8c + c c + 8b ( + 8bc + b(b + 8c + c(c + 8b = 3 + b 3 + c 3 + 4bc. (6 3 +b 3 +c 3 +4bc = (+b+c 3 = 3 +b 3 +c 3 +3( b+ c+b c+b +c +c b+6bc, (7 whch s equvlent to show tht ( b + c + b c + b + c + c b 6bc. Ths s ndeed true becuse of the AM-GM nequlty ( b + c + b c + b + c + c b 6 6 b c b b c c c b = 6bc. The desred nequlty follows from (6 nd (7. Applctons to mtrx nequltes, estmtng ntegrls nd reltve entropy Ths secton dscusses severl pplctons of nequltes ntroduced n prevous sectons. These pplctons nclude mtrx nequltes n Sectons. nd., ntegrl estmtons n Secton.3 nd reltve entropy n Secton.4.. Mnkowsk s nequlty for determnnts Lemm.. [Vl03, Lemm 5.3] Let A nd B be two non-negtve symmetrc n n mtrces, nd λ [0, ]. Assume tht A s nvertble. Then det(λa + ( λb /n λ(det A /n + ( λ(det B /n, (8 nd det(λa + ( λb (det A λ (det B λ. (9

13 3 Proof. tht We frst prove (8. Snce det(λa = λ n det A, to prove (8 t s suffcent to prove det(a + B /n (det A /n + (det B /n. (0 Ths nequlty s known s Mnkowsk s nequlty. Snce A+B = A / (I+A / BA / A / = A / (I + CA /, where C = A / BA / s non-negtve symmetrc mtrx, nd det(mn = (det M(det N we hve [ det(a+b /n = (det A /n det(i+c /n, (det A /n +(det B /n = (det A /n +(det C ]. /n We next show tht det(i + C /n + (det C /n for ll C non-negtve nd symmetrc, whch wll mply (0. Snce C s non-negtve symmetrc, t hs nonnegtve egenvlues λ,..., λ n nd n n det(i + C /n = ( + λ /n nd (det C /n = λ /n. We need to prove n ( + λ /n + n λ /n. Ths nequlty ndeed holds true snce usng the Arthmetc-Geometrc nequlty. we hve + n λ /n n = ( + λ /n ( /n ( λ /n λ + λ n + λ n We now prove (9. From (8 nd the Arthmetc-Geometrc nequlty we hve λ + λ =. det(λa + ( λb /n λ(det A /n + ( λ(det B /n (det A λ/n (det B ( λ/n, whch mples tht tht s (9 s expected.. Hdmrd s Inequlty Theorem. (Hdmrd s nequlty. det(λa + ( λb (det A λ (det B ( λ, det A n, ( where { } n re (rel vectors columns of A nd s the Euclden norm.

14 4 Proof. The nequlty s obvously true If A s sngulr. Therefore, ssume tht the column of A re lnerly ndependent. By dvdng ech column by ts length, the nequlty s equvlent to the specl cse where ech column hs length. Suppose tht {b } n re unt column vectors nd B hs the {b } s column. We need to show tht det B. Indeed, let C = B T B. Then C s non-negtve symmetrc whose dgonl entres re ll. Thus trc = n. Let λ,..., λ n 0 be the egenvlues of C. By the rthmetc-geometrc nequlty we hve (det B = det C =.e., det B s requred. n ( λ n n ( n λ = n trc =, Corollry.3. Let A be n n postve defnte mtrx. Then det A n A. ( Proof. Sne A s postve defnte, there exsts B such tht A = B T B. columns of B. We hve Let b re the det A = (det B b = A. (3 Proposton.4. [Dn0, Theorem.8] Let A nd B be n n postve defnte mtrces. Then n(det A det B m n tr(a m B m (4 for ny postve nteger m. Proof. Let λ,..., λ n > 0 be egenvlues of A nd suppose tht A = P T ΛP where P s n orthonorml mtrx nd Λ = dg(λ,..., λ n. Let b (m,..., b nn (m denote the dgonl elements of (P T BP m. Snce the trce opertor s nvrnt under permutton, we hve n tr(am B m = n tr(p T Λ m P B m = n tr(λm P T B m P = n tr(λm (P T BP m = λ m b (m. n

15 5 Usng the lst dentty nd the rthmetc-geometrc nequlty, we hve n tr(am B m (λ m n On the other hnd from ( we hve (det A det B m n = (det Λ m det P T B m P n λ m n Together wth (5 we obtn the clmed nequlty. (b (m n (5 (b (m n.3 Hermte-Hdmrd nequlty Theorem.5. Let f : [, b] R be convex functon. Then ( + b f b b f(x dx Proof. Usng chnge of vrble x = ( λ + λb we get f( + f(b. (6 Smlrly we lso get b b f(x dx = 0 f(( λ + λb dλ. So we hve b b b b f(x dx = ( f(x dx = f(( λ + λb dλ f(λ + ( λb dλ. 0 f(λ + ( λb dλ. (7 Snce f s convex on [, b] we hve ( + b ( λ + ( λb + ( λ + λb f = f f(λ + ( λb + f(( λ + λb [ (λf( ( ] + ( λf(b + ( λf( + λf(b = f( + f(b. Integrtng wth respect to λ from 0 to nd usng (7 we obtn (6.

16 6 Remrk.6. We cn pply Hermte-Hdmrd nequlty for two sub-ntervls [, +b nd [ +b, b] to obtn ( f( + f ( 3 + b f 4 ( + 3b f 4 +b f(x dx b b f(x dx b +b ( f +b +b + f(b. Summng up both sdes we obtn [ ( 3 + b ( + 3b ] [ ] b f + f f(x dx ( + b f( + f + f(b We cn get further mprovements by repetedly decomposng to smller sub-ntervls. Exercse. Let 0 < x < y be postve rel numbers. Prove the followng geometrc, logrthmc, nd rthmetc men nequltes xy ] y x ln(y ln(x x + y. (8 Proof. Let f : R R, f(x = e x. Then f s convex. Accordng to Hermte-Hdmrd nequlty for rbtrry rel numbers < b we hve e +b b Tkng = ln x, b = ln y yelds b e x dx = eb e b e + e b. ln x+ln y e = xy y x ln y ln x x + y. Exercse. Let n be postve nteger. Prove tht n k= n k k + ln(n! k= ( k + k. (9 k + Proof. Let f : [0, R, f(x =. Then f s convex snce f (x = > 0. +x (+x 3 Applyng the Hermte-Hdmrd nequlty for f wth = 0 nd b = k, k s postve nteger, we get + k k k 0 + x dx = k ln(k + ( +, + k

17 7.e., k + k ln(k + Summng up over k from to n we obtn s clmed. n k= n k + k k= ( k + ln(k + = ln(n! k. + k n k= ( k + k + k.4 Reltve entropy Defnton.7 (Reltve entropy. Let P nd Q be two probblty mesures on spce X. Suppose tht P s bsolutely contnuous wth respect to Q. The reltve entropy of P wth respect to Q s gven by In prtculr, H(P Q = X ( dp dp log dq dq dq.. (Dscrete probblty mesures If X s fnte set wth X = n. Let P = (p,..., p n nd Q = (q,..., q n wth p, q 0, n p = n q = then H(P Q = ( p p log. q. (Contnuous probblty mesures Let P (dx = p(x dx nd Q(dx = q(x dx re two probblty mesure on X = R d. Then ( p(x H(P Q = p(x log dx. R q(x d Lemm.8 (Log-Sum nequlty. Let,..., n nd b,..., b n be non-negtve rel numbers. Prove tht ( n log log b n b. (0 In prtculr, f n = n b = then (comprng wth Exercse below ( log 0. b

18 8 Proof. Consder x f(x = x log x for x > 0. Then f s convex. Let α = By Jensen s nequlty we hve.e., f( α x α f(x, ( ( α x log α x Substtutng vlues of α nd x, we obtn n n b log ( n n b whch s equvlent to the log-sum nequlty s requred. α x log x. ( n log b n b, Lemm.9 (Postvty of the reltve entropy. Prove tht H(P Q 0. b n b, x = b. Proof. Snce the functon x ϕ(x = x log x s convex for x > 0, by Jensen s nequlty.4, we hve ( dp ( dp H(P Q = ϕ dq ϕ dq X dq dq = ϕ( = 0. X Lemm.0 (Convexty of the reltve entropy. Let P, P, Q, Q be probblty mesures on fnte set X. Prove tht for ny 0 λ, we hve H(λP + ( λp λq + ( λp λh(p Q + ( λh(p Q. ( Proof. For ech x X, pplyng the Log-Sum nequlty for = λp (x, = ( λp (x, b = λq (x, b = ( λp (x we hve ( λp + ( λp ( λp ( ( λp (λp + ( λp log λp log + ( λp log λq + ( λq λq ( λq Summng over x X we obtn (. = λp log p q + ( λp log p q. Lemm. (Dulty formul for the reltve entropy. Let P = (p,..., p n, Q = (q,..., p n be probblty mesures on fnte set X = {,..., n}. Prove tht { ( } H(P Q = sup x p log e x q. x=(x,...,x n

19 9 Proof. Let f(p := H(P Q = n p log p q. We wll fnd the convex conjugte f (x of f(p. f (x = sup{ p x p p log p q } where the sup s tken over ll S := {p = (p,..., p n : p 0, n p = }. Snce S s compct nd the functon nsde the supremum s contnuous, the supremum s cheved t some p whch stsfes x log p q + λ = 0, for some (Lgrnge multplers λ R. Solvng ths equlty, we get p = q e x +λ. Snce n p =, we must hve = q e x +λ = e λ ( n whch gves λ = log q e x. Therefore, q e x, f (x = = (λ q e x +λ x q e x +λ (x + λ q e x +λ ( = (λ = log q e x. Hence H(P Q = f(p = sup x {x p f (x} = sup x=(x,...,x n { n ( x n } p log ex q. Remrk.. The contnuous counter-prt of the dulty formul for the reltve entropy s { } H(P Q = f dp log e f dq. sup f C b (X X X Lemm.3 (Pnsker s nequlty. Let P, Q be probblty mesures on fnte set X. Let P Q = x X p(x q(x be the totl vrton dstnce between P nd Q. Prove tht H(P Q P Q.

20 0 Proof. Let A := {x X : p(x q(x}. It follows tht P Q = (p(x q(x + ( (q(x p(x = (p(x q(x = (P (A Q(A. x A x X A x A By the Log-Sum nequlty we hve H(P Q = x X where we hve used tht p(x log p(x q(x = p(x log p(x q(x + p(x log p(x q(x x A x X A x A p(x p(x log x A x A q(x + p(x log log b x X A = P (A log P (A P (X A + P (X A log Q(A Q(X A (P (A Q(A = P Q, + ( log b ( b for 0, b. x X A p(x x X A q(x References [Dn0] F. M. Dnnn. Mtrx nd opertor nequltes. Journl of Inequltes n Pure nd Appled Mthemtcs, Volume, Issue 3, Artcle 34, 00. [Vl03] Cédrc Vlln. Topcs n optml trnsportton, volume 58 of Grdute Studes n Mthemtcs. Amercn Mthemtcl Socety, Provdence, RI, 003.

Lecture notes. Fundamental inequalities: techniques and applications

Lecture notes. Fundamental inequalities: techniques and applications Lecture notes Fundmentl inequlities: techniques nd pplictions Mnh Hong Duong Mthemtics Institute, University of Wrwick Emil: m.h.duong@wrwick.c.uk Februry 8, 207 2 Abstrct Inequlities re ubiquitous in