TAIWANESE JOURNAL OF MATHEMATICS Vol. 14, No. 1,. 153-164, February 2010 This aer is available olie at htt://www.tjm.sysu.edu.tw/ A REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS FOR f-divergence MEASURES S. S. Dragomir Abstract. A refiemet of the discrete Jese s iequality for covex fuctios defied o a covex subset i liear saces is give. Alicatio for f- divergece measures icludig the Kullbac-Leibler ad Jeffreys divergeces are rovided as well. 1. INTRODUCTION The Jese iequality for covex fuctios lays a crucial role i the Theory of Iequalities due to the fact that other iequalities such as that arithmetic meageometric mea iequality, Hölder ad Miowsi iequalities, Ky Fa s iequality etc. ca be obtaied as articular cases of it. Let C be a covex subset of the liear sace X ad f a covex fuctio o C. If = 1,..., is a robability sequece ad x =x 1,...,x C, the 1.1 f i x i i f x i, is well ow i the literature as Jese s iequality. I 1989, J. Pecarić ad the author obtaied the followig refiemet of 1.1: xi1 + + x i+1 1.2 f i x i i1... i+1 f +1 i 1,...,i +1 =1 xi1 + + x i i1... i f i 1,...,i =1 i f x i, Received Jauary 21, 2008, acceted Aril 11, 2008. Commuicated by Se-Ye Shaw. 2000 Mathematics Subject Classificatio: 26D15, 94A17. Key words ad hrases: Covex fuctios, Jese s iequality, f-divergece measures, Kullbac- Leibler divergece, Jeffreys divergece. 153
154 S. S. Dragomir for 1 ad,x as above. If q 1,..., 0 with =1, the the followig refiemet obtaied i 1994 by the author [6] also holds: xi1 + + x i 1.3 f i x i i1... i f i 1,...,i =1 i 1,...,i =1 i f x i, i1... i f q 1 x i1 + + x i where 1 ad, x are as above. For other refiemets ad alicatios related to Ky Fa s iequality, the arithmetic mea-geometric mea iequality, the geeralised triagle iequality etc., see [3-8]. The mai aim of the reset aer is to establish a differet refiemet of the Jese iequality for covex fuctios defied o liear saces. Natural alicatios for the geeralised triagle iequality i ormed saces ad for the arithmetic meageometric mea iequality for ositive umbers are give. Further alicatios for f-divergece measures of Csiszár with articular istaces for the total variatio distace, χ 2 -divergece, Kullbac-Leibler ad Jeffreys divergeces are rovided as well. The followig result may be stated. 2. GENERAL RESULTS Theorem 1. Let f : C R be a covex fuctio o the covex subset C of the liear sace X, x i C, i > 0, i {1,...,} with i =1. The [ f j x j mi 1 f ] jx j x + f x 1 [ 1 1 f ] jx j x + f x 1 2.1 [ max 1 f ] jx j x + f x 1 j f x j.
Jese s Iequality with Alicatios for f-divergece Measures 155 I articular, 2.2 Proof. f 1 x j [ 1 mi 1 f x ] j x + f x 1 1 [ 2 1 f x ] j x + f x 1 [ 1 max 1 f x ] j x + f x 1 1 f x j. For ay {1,...,}, we have j x j x = which imlies that j x j = j j j x j =1 1 j j x j 2.3 jx j x = 1 1 j j x j C for each {1,...,}, sice the right side of 2.3 is a covex combiatio of the elemets x j C, j {1,...,}\{}. Taig the fuctio f o 2.3 ad alyig the Jese iequality, we get successively f jx j x = f 1 1 = 1 1 j j x j 1 j j f x j j f x j f x
156 S. S. Dragomir for ay {1,...,}, which imlies 2.4 1 f jx j x + f x 1 j f x j for each {1,...,}. Utilisig the covexity of f, we also have 2.5 1 f jx j x + f x 1 [ f 1 ] jx j x + x = f j x j 1 for each {1,...,}. Taig the miimum over i 2.5, utilisig the fact that mi α 1 α max α ad the taig the maximum i 2.4, we deduce the desired iequality 2.1. After settig x j = y j q ly l ad j =,j {1,...,}, Theorem 1 becomes the followig corollary: Corollary 1. Let f : C R be a covex fuctio o the covex subset C, 0 C, y j X ad > 0,j {1,...,} with =1. If y j q ly l C for ay j {1,...,}, the 2.6 { [ } f 0 mi 1 f q l y l y ]+ f y q l y l 1 { [ } 1 1 f q l y l y ]+ f y q l y l 1 { [ } max 1 f q l y l y ]+ f y q l y l 1 f y j q l y l.
Jese s Iequality with Alicatios for f-divergece Measures 157 I articular, if y j 1 y l C for ay j {1,...,}, the 2.7 { [ ] } f0 1 1 1 mi 1f y l y +f y 1 y l 1 [ } { 1 1 1 1 2 f y l y ]+ f y 1 y l 1 { [ } 1 max 1 1 1 f y l y ]+f y 1 y l 1 1 f y j 1 y l. The above results ca be alied for various covex fuctios related to celebrated iequalities as metioed i the itroductio. Alicatio 1. If X, is a ormed liear sace ad 1, the the fuctio f : X R, f x = x is covex o X. Now, o alyig Theorem 1 ad Corollary 1 for x i X, i > 0, i {1,...,} with i =1, we get: 2.8 j x j mi 1 1 j x j x + x 1 1 1 j x j x + x max 1 1 j x j x + x j x j ad 2.9 max j x { [ 1 1 + ] x l x l. } l x l
158 S. S. Dragomir I articular, we have the iequality: 1 x j 1 mi 1 1 x j x + x 1 2 1 1 2.10 x j x + x 1 max 1 1 x j x + x 1 x j ad 2.11 [ ] 1 1 +1 max x 1 x l x j 1 x l. If we cosider the fuctio h t :=1 t 1 t + t, 1, t [0, 1, the we observe that h t =1+t 1 1 t 1 + 1 t 1 t, which shows that h is strictly icreasig o [0, 1. Therefore, { } 1 1 + = m +1 m 1 m, where m := 2.12 mi mi. By 2.9, we the obtai the followig iequality: [ ] m +1 m 1 m j x j l x l. max x l x l Alicatio 2. Let x i, i > 0, i {1,...,} with i =1. The followig iequality is well ow i the literature as the arithmetic mea-geometric mea iequality: 2.13 j x j x j j.
Jese s Iequality with Alicatios for f-divergece Measures 159 The equality case holds i 2.13 iff x 1 = = x. Alyig the iequality 2.1 for the covex fuctio f :0, R, f x = l x ad erformig the ecessary comutatios, we derive the followig refiemet of 2.13: i x i max 1 jx j x x 1 2.14 jx j x 1 mi jx j x 1 I articular, we have the iequality: 1 x i max x j x 1 1 x 1 1 1 x x 1 x i i. x 1 1 j x x 1 1 mi x 1 j x 1 x 1 1 x i. 3. APPLICATIONS FOR f-divergences Give a covex fuctio f :[0, R, the f-divergece fuctioal i 3.1 I f, q:= q i f, where = 1,...,, q =q 1,...,q are ositive sequeces, was itroduced by Csiszár i [1], as a geeralized measure of iformatio, a distace fuctio o the set of robability distributios P. As i [1], we iterret udefied exressios by f 0 = lim f t, t 0+ a a 0f = lim 0 qf q 0+ q 0f q i 0 =0, 0 f t = a lim, a > 0. t t
160 S. S. Dragomir The followig results were essetially give by Csiszár ad Körer [2]: i If f is covex, the I f, q is joitly covex i ad q; ii For every, q R +, we have 3.2 I f, q f j q. j If f is strictly covex, equality holds i 3.2 iff 1 q 1 = 2 q 2 = = q. If f is ormalized, i.e., f 1 = 0, the for every, q R + with i = q i, we have the iequality 3.3 I f, q 0. I articular, if, q P, the 3.3 holds. This is the well-ow ositive roerty of the f-divergece. The followig refiemet of 3.3 may be stated. Theorem 2. For ay, q P, we have the iequalities [ 1 I f, q max 1 f + f 1 [ 1 1 3.4 1 f + f 1 [ 1 mi 1 f + f 1 rovided f :[0, R is covex ad ormalized o [0,. ] ] ] 0, The roof is obvious by Theorem 1 alied for the covex fuctio f :[0, R ad for the choice x i = i q i, i {1,...,} ad the robabilities q i, i {1,...,}. If we cosider a ew divergece measure R f, q defied for, q P by 3.5 R f, q:= 1 1 1 1 f 1
Jese s Iequality with Alicatios for f-divergece Measures 161 ad call it the reverse f divergece, we observe that 3.6 R f, q=i f r, t with r = 1 1 1,...,1 1 q1, t = 1 1,...,1 q 1 2. With this otatio, we ca state the followig corollary of the above roositio. Corollary 2. For ay, q P, we have 3.7 I f, q R f, q 0. The roof is obvious by the secod iequality i 3.4 ad the details are omitted. I what follows, we oit out some articular iequalities for various istaces of divergece measures such as: the total variatio distace, χ 2 -divergece, Kullbac- Leibler divergece, Jeffreys divergece. The total variatio distace is defied by the covex fuctio f t = t 1, t R ad give i: j 3.8 V, q := 1 q = j. j The followig imrovemet of the ositivity iequality for the total variatio distace ca be stated as follows. Proositio 1. For ay, q P, we have the iequality: 3.9 V, q 2 max 0. The roof follows by the first iequality i 3.4 for f t = t 1,t R. The K. Pearso χ 2 -divergece is obtaied for the covex fuctio f t = 1 t 2,t R ad give by 3.10 χ 2, q:= Proositio 2. For ay, q P, 3.11 χ 2, q max 2 j 1 = { 2 1 j 2. } 4 max 2 0.
162 S. S. Dragomir Proof. O alyig the first iequality i 3.4 for the fuctio f t = 1 t 2,t R, we get { 1 2 } 2 χ 2, q max 1 1 + 1 1 { } 2 = max. 1 Sice 1 1 4 [ +1 ] 2 = 1 4, the 2 1 4 2 for each {1,...,}, which roves the last art of 3.11. The Kullbac-Leibler divergece ca be obtaied for the covex fuctio f : 0, R, f t =t l t ad is defied by 3.12 KL, q := j l j = j l j. Proositio 3. For ay, q P, we have: [ { 1 1 3.13 KL, q l max 1 }] 0. Proof. The first iequality is obvious by Theorem 2. Utilisig the iequality betwee the geometric mea ad the harmoic mea, x α y 1 α 1 α x + 1 α y, x,y > 0, α [0, 1] we have 1 1 1, 1 for ay {1,...,}, which imlies the secod art of 3.13. Aother divergece measure that is of imortace i Iformatio Theory is the Jeffreys divergece
3.14 J, q:= Jese s Iequality with Alicatios for f-divergece Measures 163 j 1 l j = which is a f-divergece for f t =t 1 l t, t > 0. j l Proositio 4. For ay, q P, we have: { [ ]} 1 3.15 J, q max l 1 [ ] 2 max 0. + 2 Proof. have J, q j Writig the first iequality i Theorem 2 for f t =t 1 l t, we max = max = max { [ 1 1 1 1 { l { l 1 1 [ 1 1 1 l 1 l ]}, rovig the first iequality i 3.15. Utilisig the elemetary iequality for ositive umbers, we have l b l a b a [ 1 l l 1 = = 2 1 2 a + b, a,b > 0 l 1 1 l 1 1 l 1 1 l 1 1 ] ] + [ 1 ] 1 2 1 2 1 1 + = 2q 2 0, + 2 for each {1,...,}, givig the secod iequality i 3.15. 1 }, l }
164 S. S. Dragomir ACKNOWLEDGMENT The author would lie to tha the aoymous referee for some suggestios that have bee imlemeted i the fial versio of the aer. REFERENCES 1. I. Csiszár, Iformatio-tye measures of differeces of robability distributios ad idirect observatios, Studia Sci. Math. Hug., 2 1967, 299-318. 2. I. Csiszár ad J. Körer, Iformatio Theory: Codig Theorems for Discrete Memoryless Systems, Academic Press, New Yor, 1981. 3. S. S. Dragomir, A imrovemet of Jese s iequality, Bull. Math. Soc. Sci. Math. Roumaie, 344 82 1990, 291-296. 4. S. S. Dragomir, Some refiemets of Ky Fa s iequality, J. Math. Aal. Al., 1632 1992, 317-321. 5. S. S. Dragomir, Some refiemets of Jese s iequality, J. Math. Aal. Al., 1682 1992, 518-522. 6. S. S. Dragomir, A further imrovemet of Jese s iequality, Tamag J. Math., 251 1994, 29-36. 7. S. S. Dragomir, A ew imrovemet of Jese s iequality, Idia J. Pure ad Al. Math., 2610 1995, 959-968. 8. S. S. Dragomir, J. Pecarić ad L. E. Persso, Proerties of some fuctioals related to Jese s iequality, Acta Math. Hug., 701-2 1996, 129-143. 9. J. Pecarić ad S. S. Dragomir, A refiemets of Jese iequality ad alicatios, Studia Uiv. Babeş-Bolyai, Mathematica, 241 1989, 15-19. S. S. Dragomir School of Egieerig ad Sciece, Victoria Uiversity, P. O. Box 14428, Melboure City Mail Cetre, VIC 8001, Australia E-mail: sever.dragomir@vu.edu.au