A REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS. S. S. Dragomir 1. INTRODUCTION

Transcription

1 TAIWANESE JOURNAL OF MATHEMATICS Vol. 14, No. 1, , February 2010 This aer is available olie at htt:// A REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS FOR f-divergence MEASURES S. S. Dragomir Abstract. A refiemet of the discrete Jese s iequality for covex fuctios defied o a covex subset i liear saces is give. Alicatio for f- divergece measures icludig the Kullbac-Leibler ad Jeffreys divergeces are rovided as well. 1. INTRODUCTION The Jese iequality for covex fuctios lays a crucial role i the Theory of Iequalities due to the fact that other iequalities such as that arithmetic meageometric mea iequality, Hölder ad Miowsi iequalities, Ky Fa s iequality etc. ca be obtaied as articular cases of it. Let C be a covex subset of the liear sace X ad f a covex fuctio o C. If = 1,..., is a robability sequece ad x =x 1,...,x C, the 1.1 f i x i i f x i, is well ow i the literature as Jese s iequality. I 1989, J. Pecarić ad the author obtaied the followig refiemet of 1.1: xi1 + + x i f i x i i1... i+1 f +1 i 1,...,i +1 =1 xi1 + + x i i1... i f i 1,...,i =1 i f x i, Received Jauary 21, 2008, acceted Aril 11, Commuicated by Se-Ye Shaw Mathematics Subject Classificatio: 26D15, 94A17. Key words ad hrases: Covex fuctios, Jese s iequality, f-divergece measures, Kullbac- Leibler divergece, Jeffreys divergece. 153

2 154 S. S. Dragomir for 1 ad,x as above. If q 1,..., 0 with =1, the the followig refiemet obtaied i 1994 by the author [6] also holds: xi1 + + x i 1.3 f i x i i1... i f i 1,...,i =1 i 1,...,i =1 i f x i, i1... i f q 1 x i1 + + x i where 1 ad, x are as above. For other refiemets ad alicatios related to Ky Fa s iequality, the arithmetic mea-geometric mea iequality, the geeralised triagle iequality etc., see [3-8]. The mai aim of the reset aer is to establish a differet refiemet of the Jese iequality for covex fuctios defied o liear saces. Natural alicatios for the geeralised triagle iequality i ormed saces ad for the arithmetic meageometric mea iequality for ositive umbers are give. Further alicatios for f-divergece measures of Csiszár with articular istaces for the total variatio distace, χ 2 -divergece, Kullbac-Leibler ad Jeffreys divergeces are rovided as well. The followig result may be stated. 2. GENERAL RESULTS Theorem 1. Let f : C R be a covex fuctio o the covex subset C of the liear sace X, x i C, i > 0, i {1,...,} with i =1. The [ f j x j mi 1 f ] jx j x + f x 1 [ 1 1 f ] jx j x + f x [ max 1 f ] jx j x + f x 1 j f x j.

3 Jese s Iequality with Alicatios for f-divergece Measures 155 I articular, 2.2 Proof. f 1 x j [ 1 mi 1 f x ] j x + f x 1 1 [ 2 1 f x ] j x + f x 1 [ 1 max 1 f x ] j x + f x 1 1 f x j. For ay {1,...,}, we have j x j x = which imlies that j x j = j j j x j =1 1 j j x j 2.3 jx j x = 1 1 j j x j C for each {1,...,}, sice the right side of 2.3 is a covex combiatio of the elemets x j C, j {1,...,}\{}. Taig the fuctio f o 2.3 ad alyig the Jese iequality, we get successively f jx j x = f 1 1 = 1 1 j j x j 1 j j f x j j f x j f x

4 156 S. S. Dragomir for ay {1,...,}, which imlies f jx j x + f x 1 j f x j for each {1,...,}. Utilisig the covexity of f, we also have f jx j x + f x 1 [ f 1 ] jx j x + x = f j x j 1 for each {1,...,}. Taig the miimum over i 2.5, utilisig the fact that mi α 1 α max α ad the taig the maximum i 2.4, we deduce the desired iequality 2.1. After settig x j = y j q ly l ad j =,j {1,...,}, Theorem 1 becomes the followig corollary: Corollary 1. Let f : C R be a covex fuctio o the covex subset C, 0 C, y j X ad > 0,j {1,...,} with =1. If y j q ly l C for ay j {1,...,}, the 2.6 { [ } f 0 mi 1 f q l y l y ]+ f y q l y l 1 { [ } 1 1 f q l y l y ]+ f y q l y l 1 { [ } max 1 f q l y l y ]+ f y q l y l 1 f y j q l y l.

5 Jese s Iequality with Alicatios for f-divergece Measures 157 I articular, if y j 1 y l C for ay j {1,...,}, the 2.7 { [ ] } f mi 1f y l y +f y 1 y l 1 [ } { f y l y ]+ f y 1 y l 1 { [ } 1 max f y l y ]+f y 1 y l 1 1 f y j 1 y l. The above results ca be alied for various covex fuctios related to celebrated iequalities as metioed i the itroductio. Alicatio 1. If X, is a ormed liear sace ad 1, the the fuctio f : X R, f x = x is covex o X. Now, o alyig Theorem 1 ad Corollary 1 for x i X, i > 0, i {1,...,} with i =1, we get: 2.8 j x j mi 1 1 j x j x + x j x j x + x max 1 1 j x j x + x j x j ad 2.9 max j x { [ ] x l x l. } l x l

6 158 S. S. Dragomir I articular, we have the iequality: 1 x j 1 mi 1 1 x j x + x x j x + x 1 max 1 1 x j x + x 1 x j ad 2.11 [ ] max x 1 x l x j 1 x l. If we cosider the fuctio h t :=1 t 1 t + t, 1, t [0, 1, the we observe that h t =1+t 1 1 t t 1 t, which shows that h is strictly icreasig o [0, 1. Therefore, { } = m +1 m 1 m, where m := 2.12 mi mi. By 2.9, we the obtai the followig iequality: [ ] m +1 m 1 m j x j l x l. max x l x l Alicatio 2. Let x i, i > 0, i {1,...,} with i =1. The followig iequality is well ow i the literature as the arithmetic mea-geometric mea iequality: 2.13 j x j x j j.

7 Jese s Iequality with Alicatios for f-divergece Measures 159 The equality case holds i 2.13 iff x 1 = = x. Alyig the iequality 2.1 for the covex fuctio f :0, R, f x = l x ad erformig the ecessary comutatios, we derive the followig refiemet of 2.13: i x i max 1 jx j x x jx j x 1 mi jx j x 1 I articular, we have the iequality: 1 x i max x j x 1 1 x x x 1 x i i. x 1 1 j x x 1 1 mi x 1 j x 1 x 1 1 x i. 3. APPLICATIONS FOR f-divergences Give a covex fuctio f :[0, R, the f-divergece fuctioal i 3.1 I f, q:= q i f, where = 1,...,, q =q 1,...,q are ositive sequeces, was itroduced by Csiszár i [1], as a geeralized measure of iformatio, a distace fuctio o the set of robability distributios P. As i [1], we iterret udefied exressios by f 0 = lim f t, t 0+ a a 0f = lim 0 qf q 0+ q 0f q i 0 =0, 0 f t = a lim, a > 0. t t

8 160 S. S. Dragomir The followig results were essetially give by Csiszár ad Körer [2]: i If f is covex, the I f, q is joitly covex i ad q; ii For every, q R +, we have 3.2 I f, q f j q. j If f is strictly covex, equality holds i 3.2 iff 1 q 1 = 2 q 2 = = q. If f is ormalized, i.e., f 1 = 0, the for every, q R + with i = q i, we have the iequality 3.3 I f, q 0. I articular, if, q P, the 3.3 holds. This is the well-ow ositive roerty of the f-divergece. The followig refiemet of 3.3 may be stated. Theorem 2. For ay, q P, we have the iequalities [ 1 I f, q max 1 f + f 1 [ f + f 1 [ 1 mi 1 f + f 1 rovided f :[0, R is covex ad ormalized o [0,. ] ] ] 0, The roof is obvious by Theorem 1 alied for the covex fuctio f :[0, R ad for the choice x i = i q i, i {1,...,} ad the robabilities q i, i {1,...,}. If we cosider a ew divergece measure R f, q defied for, q P by 3.5 R f, q:= f 1

9 Jese s Iequality with Alicatios for f-divergece Measures 161 ad call it the reverse f divergece, we observe that 3.6 R f, q=i f r, t with r = 1 1 1,...,1 1 q1, t = 1 1,...,1 q 1 2. With this otatio, we ca state the followig corollary of the above roositio. Corollary 2. For ay, q P, we have 3.7 I f, q R f, q 0. The roof is obvious by the secod iequality i 3.4 ad the details are omitted. I what follows, we oit out some articular iequalities for various istaces of divergece measures such as: the total variatio distace, χ 2 -divergece, Kullbac- Leibler divergece, Jeffreys divergece. The total variatio distace is defied by the covex fuctio f t = t 1, t R ad give i: j 3.8 V, q := 1 q = j. j The followig imrovemet of the ositivity iequality for the total variatio distace ca be stated as follows. Proositio 1. For ay, q P, we have the iequality: 3.9 V, q 2 max 0. The roof follows by the first iequality i 3.4 for f t = t 1,t R. The K. Pearso χ 2 -divergece is obtaied for the covex fuctio f t = 1 t 2,t R ad give by 3.10 χ 2, q:= Proositio 2. For ay, q P, 3.11 χ 2, q max 2 j 1 = { 2 1 j 2. } 4 max 2 0.

10 162 S. S. Dragomir Proof. O alyig the first iequality i 3.4 for the fuctio f t = 1 t 2,t R, we get { 1 2 } 2 χ 2, q max { } 2 = max. 1 Sice [ +1 ] 2 = 1 4, the for each {1,...,}, which roves the last art of The Kullbac-Leibler divergece ca be obtaied for the covex fuctio f : 0, R, f t =t l t ad is defied by 3.12 KL, q := j l j = j l j. Proositio 3. For ay, q P, we have: [ { KL, q l max 1 }] 0. Proof. The first iequality is obvious by Theorem 2. Utilisig the iequality betwee the geometric mea ad the harmoic mea, x α y 1 α 1 α x + 1 α y, x,y > 0, α [0, 1] we have 1 1 1, 1 for ay {1,...,}, which imlies the secod art of Aother divergece measure that is of imortace i Iformatio Theory is the Jeffreys divergece

11 3.14 J, q:= Jese s Iequality with Alicatios for f-divergece Measures 163 j 1 l j = which is a f-divergece for f t =t 1 l t, t > 0. j l Proositio 4. For ay, q P, we have: { [ ]} J, q max l 1 [ ] 2 max Proof. have J, q j Writig the first iequality i Theorem 2 for f t =t 1 l t, we max = max = max { [ { l { l 1 1 [ l 1 l ]}, rovig the first iequality i Utilisig the elemetary iequality for ositive umbers, we have l b l a b a [ 1 l l 1 = = a + b, a,b > 0 l 1 1 l 1 1 l 1 1 l 1 1 ] ] + [ 1 ] = 2q 2 0, + 2 for each {1,...,}, givig the secod iequality i }, l }

12 164 S. S. Dragomir ACKNOWLEDGMENT The author would lie to tha the aoymous referee for some suggestios that have bee imlemeted i the fial versio of the aer. REFERENCES 1. I. Csiszár, Iformatio-tye measures of differeces of robability distributios ad idirect observatios, Studia Sci. Math. Hug., , I. Csiszár ad J. Körer, Iformatio Theory: Codig Theorems for Discrete Memoryless Systems, Academic Press, New Yor, S. S. Dragomir, A imrovemet of Jese s iequality, Bull. Math. Soc. Sci. Math. Roumaie, , S. S. Dragomir, Some refiemets of Ky Fa s iequality, J. Math. Aal. Al., , S. S. Dragomir, Some refiemets of Jese s iequality, J. Math. Aal. Al., , S. S. Dragomir, A further imrovemet of Jese s iequality, Tamag J. Math., , S. S. Dragomir, A ew imrovemet of Jese s iequality, Idia J. Pure ad Al. Math., , S. S. Dragomir, J. Pecarić ad L. E. Persso, Proerties of some fuctioals related to Jese s iequality, Acta Math. Hug., , J. Pecarić ad S. S. Dragomir, A refiemets of Jese iequality ad alicatios, Studia Uiv. Babeş-Bolyai, Mathematica, , S. S. Dragomir School of Egieerig ad Sciece, Victoria Uiversity, P. O. Box 14428, Melboure City Mail Cetre, VIC 8001, Australia sever.dragomir@vu.edu.au