ENTROPY BOUNDS USING ARITHMETIC- GEOMETRIC-HARMONIC MEAN INEQUALITY. Guru Nanak Dev University Amritsar, , INDIA

Transcription

1 Internatonal Journal of Pure and Appled Mathematc Volume 89 No , ISSN: prnted veron; ISSN: on-lne veron url: do: PAjpam.eu ENTROPY BOUNDS USING ARITHMETIC- GEOMETRIC-HARMONIC MEAN INEQUALITY Om Parkah 1, Pryanka Kakkar 2 1,2 Department of Mathematc Guru Nanak Dev Unverty Amrtar, , INDIA Abtract: In the preent communcaton, e have developed ne nequalte ung arthmetc-geometrc-harmonc mean nequalty and conequently, appled ourfndngtothefeld of entropy theory. It oberved that ourreult provde the tronger upper bound to Shannon entropy a found by Smc[10] and Tapu and Popecu[7]ung Jenen nequalty. Moreover, e have extended our fndng to the feld of codng theory by provdng the comparon of varou codeord length. AMS Subject Clafcaton: 26B25, 94A17, 94A24, 94A29 Key Word: entropy, arthmetc-geometrc-harmonc mean nequalty, decreang functon, Jenen nequalty, mean codeord length, probablty dtrbuton 1. Introducton In today orld, many development n computatonal mathematc have made t poble to compute a much larger number of mathematcal entte than could be calculated earler. Hoever, mot of thee entte can only be expreed n term of nequalte and a uch the mportance of the nequalte ha ncreaed Receved: September 12, 2013 Correpondence author c 2013 Academc Publcaton, Ltd. url:.acadpubl.eu

2 720 O. Parkah, P. Kakkar gnfcantly large enough. Moreover, a varety of reearcher ncludng phycal centt, engneer, nformaton theoretcan, tattcan and computer centt have developed nequalte n order to olve varou optmzatonal problem n ther repectve dcplne. It ell knon that the Arthmetc-Geometrc-Harmonc AGH mean nequalty a fundamental relatonhp and a ueful tool for problem olvng and developng relaton th other mathematcal dcplne. The mportance of th nequalty may be gnfcant becaue of t applcaton toard the oluton of many optmzaton problem ncludng utlty theory, fnancal mathematc etc. The AGH mean nequalty defned a follo: Let X = x 1,x 2,...,x n be a equence of potve number and let P = p 1,p 2,...,p n betheprobabltydtrbutonaocatedthx uchthat n p = 1. Then n n n 1 A n = p x x p p = G n = H n 1.1 x here A n the arthmetc mean, G n the geometrc mean and H n the harmonc mean Refer Kapur [3]. Author lke Smc [10], [11], [12], Tapu and Popecu [7], Dragmor [9] orked on Jenen nequalty, etablhed ne loer and upper bound for t and appled ther fndng n nformaton theory. The objectve of the preent paper to ntroduce ne nequalte and to extend ther applcaton to the feld of entropy and codng theory. The organzaton of the paper a follo: In Secton 2, e ue the monotoncty of a functon and AGH mean nequalty to derve ne nequalte. Secton 3 provde the harper upper bound to Shannon [2] entropy ung the ne nequalte. Thee upper bound are mlar to reult acheved by Smc [10] and Tapu and Popecu [7] ung Jenen nequalty. A a conequence of the development of ne nequalte, e have provded the comparon of dfferent mean codeord length. 2. Ne Inequalte ung Arthmetc-Geometrc-Harmonc Mean Inequalty Theorem 2.1. Let f be a monotoncally decreang functon on nterval I = [a,b], then n n f A n = f p x = f G n 2.1 x p

3 ENTROPY BOUNDS USING ARITHMETIC th equalty gn f and only f all the member of X = x 1,x 2,...,x n are equal. Proof. Ung nequalty 1.1, the proof drectly follo from the defnton of monotoncally decreang functon. Remark. The nequalty 2.1 revere f the functon f monotoncally ncreang. Theorem 2.1 may be refned n the form of follong nequalty: Theorem 2.2. For a monotoncally decreang functon f on nterval I, e have n n f p x mn f x p 1 r< n. pr x r +p x pr+p 2.2 here 1 r < n., r, Proof. Let x r,x X here 1 r < n and let p r,p be ther correpondng probablte. Snce x r,x I, therefore prxr+px p r+p I. So, Ung theorem 2.1, e have n n f p x = f pr x r +p x p x + p r +p, r, n x p. pr x r +p x pr+p, r, Snce x r,x X are arbtrary, therefore e have n n f p x mn 1 r< n f x p pr x r +p x pr+p, r, Note. There equalty gn n 2.2 for n = 2. Theorem 2.3. For a monotoncally decreang functon f defned on nterval I hch atfe the relaton fxy = fx+fy, e have pr x 0 max f x pr r +f x p r +p x pr+p f 1 r< n

4 722 O. Parkah, P. Kakkar n x p n f p x 2.3 here 1 r < n. Proof. Let x r,x X here 1 r < n and let p r,p be ther correpondng probablte. Snce x r,x I, therefore prxr+px p r+p I. So, Ung theorem 2.2, e have n n f p x = f pr x r +p x p x + p r +p, r, n x p. pr x r +p x pr+p, r, n pr = f x p x r +p x pr+p +f f x pr r f xp p r +p that, pr f x pr r +f xp f x r +p x pr+p n x p n f p x Snce x r,x X are arbtrary, the aerton of theorem 2.3 follo. A more general form of Theorem 2.4 may be elaborated a follo: Theorem 2.4. For a monotoncally decreang functon f defned on nterval I hch atfe the relaton fxy = fx+fy, e have pr 0 max f x pr r +f xp f x r +p x pr+p 1 r< n max f xpr r +f xp +f xpt 1 r<<t n pr x r +p x +p t x pr+p +p t t f +p t n n f p x x p t

5 ENTROPY BOUNDS USING ARITHMETIC here 1 r < < t n. 2.4 Proof. Let u uppoe that maxmum of the expreon f x pr r +f x p prx pr+p f r+p x p r+p obtaned at r = u, = v,u,v 1,2,..,n. We have to ho that for any 1,2,...,n u,v, pu f x pu u +f xpv v f x u +p v x pu+p v v p u +p v pu x x pu u +f x pv v +f x p u +p v x v +p x pu+p v+p f p u +p v +p f that, pu x u +p v x v +p x p u +p v +p pu+p v+p x p +f pu x u +p v x pu+p v v p u +p v No pu x u +p v x v +p x pu+p v+p pu x f x p u +p v x pu+p v v +f p u +p v +p p u +p v pu x u +p v x v +p x pu+p v+p ff f p u +p v +p x p pu x u +p v x pu+p v v p u +p v ff x p pu+pv+p pu x u +p v x v p u +p v pu+pv pu+pv+p pu x u +p v x v +p x p u +p v +p that, hch alay true. ff x p pu+pv+p pu x u +p v x v p u +p v p u +p v pu x u +p v x v p u +p v +p p u +p v + pu+pv pu+pv+p p p u +p v +p x

6 724 O. Parkah, P. Kakkar So, the frt part of the theorem proved a maxmum of the expreon pr f x pr r +f xp +f xpt t f x r +p x +p t x pr+p +p t t +p t greater than or equal to f x pu u pu +f xpv v +f xp f x u +p v x v +p x pu+p v+p p u +p v +p Next, e have to ho that max 1 r<<t n f x pr r pr +f xp +f xpt t f x r +p x +p t x pr+p +p t t +p t n x p n f p x Chooe arbtrary x u,x v,x X and e ll ho that pu f x pu u +f xpv v +f xp f x u +p v x v +p x pu+p v+p p u +p v +p that, n f p x n n x p n f p x x p f x pu u f xpv v f xp pu x u +p v x v +p x pu+p v+p + f p u +p v +p No n f p x n x p f x pu u f xpv v f xp pu x u +p v x v +p x pu+p v+p + f p u +p v +p

7 ENTROPY BOUNDS USING ARITHMETIC n ff f p x n, u,v, x p. pu x u +p v x v +p x pu+p v+p p u +p v +p hch true a proved n Theorem 2.2. Snce the nequalty true for any u,v, {1,2,...,n}, max 1 r<<t n f x pr r Hence the theorem. pr +f xp +f xpt t f x r +p x +p t x t +p t n x p n f p x pr+p +p t We no gve the generalzaton of Theorem 2.4 n the follong theorem: Theorem 2.5. For a monotoncally decreang functon f defned on nterval I hch atfe the relaton fxy = fx+fy, e have n n 0 S 2 S 3... S n 1 f p x here... S 2 = max 1 r 1 <r 2 n S n 1 = max 1 r 1 <r 2 <...<r n 1 n x p f x pr 1 r 1 +f x p r2 r 2 f pr1 x r1 +p r2 x r2 p r1 +p r2 n 1 f x pr r f n 1 p r x r n 1 p r Proof. The proof mlar a proved n Theorem 2.5. pr1 +p r2 n 1 pr Theorem 2.6. Let f be a monotoncally decreang functon on nterval I = [a,b], then n n n 1 p f A n = f p x = f G n = f H n x p x 2.5 th equalty gn f and only f all the member of X = x 1,x 2,...,x n are equal.

8 726 O. Parkah, P. Kakkar Proof. By ung the defnton of monotonc decreang functon, the proof follo from the nequalty Applcaton of Inequalte n Entropy and Codng Theory We ll prove the tandard reult already extng n nformaton theory a ell a ne bound for the entropy by applyng above nequalte. Theorem 3.1. For any probablty dtrbuton P = {p 1,p 2,...,p n }, e have HP logn 3.1 here HP = n p logp Shannon [2] meaure of entropy for n varable th equalty f and only f all p = 1 n. Proof. Snce f x = logx a monotoncally decreang functon n the nterval [1, [, theorem 2.1 gve log n n p x log x p that, n n log p x p logx 3.2 Agan ubttutng x = 1 p,1 n n 3.2, e get the dered reult. Remark. On ubttutng the monotoncally decreang functon fx = exp x, x,x 1 herex > 0 n theorem 2.1, e agan arrve at the tandard reult n p x n xp, that, arthmetc mean greater than or equal to geometrc mean. In the next theorem, e ll further mprove the nequalty 3.1. Theorem 3.2. For any probablty dtrbuton P = {p 1,p 2,...,p n }, e have 2pr 2p 0 max p r log +p log logn HP r< n

9 ENTROPY BOUNDS USING ARITHMETIC Proof. Snce f x = logx a monotoncally decreang functon n the nterval [1, [ hch atfe the relaton fxy = fx+fy, ubttutng t n the theorem 2.3, e get pr+p 0 max 1 r< n log n logx pr r logx p +log x p n +log p x Agan, ubttutng x = 1 p,1 n n 3.4, e have pr x r +p x 2 0 max p r log logp + log 1 r< n n logn p logp that, 2pr 2p 0 max p r log +p log logn HP 1 r< n 3.4 Note. 1. Proceedng a above, the reult 3.3 can alo be acheved ung theorem The reult 3.3 have been acheved by Smc [9] but by ung Jenen [5] nequalty. The nequalty 3.3 can further be mproved by mean of the follong theorem: Theorem 3.3. For any probablty dtrbuton P = {p 1,p 2,...,p n }, e have n 0 HP logn max log n 1 pr n 1 1 r 1 <r 2 <...<r n 1 n n 1 p p pr r r 3.5 Proof. Applyng theorem 2.5 th fx = logx, x = 1 p, = 1,2,...,n, e get the dered reult.

10 728 O. Parkah, P. Kakkar Note. The reult 3.4 have been acheved by Tapu and Popecu [7] but by ung Jenen [5] nequalty. Theorem 3.4. For any probablty dtrbuton P = {p 1,p 2,...,p n }, e have R 2 P HP logn 3.6 th equalty f and only f all p = 1 n here R 2P = log n p2 the Reny [1] entropy of order 2. Proof. Applyng theorem 2.6 th fx = logx, x = 1 p, = 1,2,...,n, and after mplfcaton, e get the dered reult. Theorem 3.5. For any probablty dtrbuton P = {p 1,p 2,...,p n }, e have n α n L α = 1 α log D p D 1 α α l p l = L, α < 1,α L α = 1 α 1 log D n pα Dα 1l n pα n pα l n pα = Lα,α > 1,α here L α the Campbell [6] mean codeord length, L α Kapur [4] mean codeord length, Lα Parkah and Kakkar [8] mean codeord length. Proof. Let f be a monotoncally decreang functon, then ung theorem 2.1, e have n n f p D 1 α α l D 1 α α l p, α < Lettng fx = log D x n 3.9, e get the dered reult 3.7. Note. The nequalty gn revere for α > 1. Let f be a monotoncally decreang functon. Agan, ung theorem 2.1, e have n f pα Dα 1l n α 1l p α n p n D α, α > pα Lettng fx = log D x n 3.10, e get n log pα Dα 1l n α 1l p α n p D n log D D α pα

11 ENTROPY BOUNDS USING ARITHMETIC After mplfcaton, e get the dered reult 3.8. Note. The nequalty gn revere for α < 1. Acknoledgment The author are thankful to Councl of Scentfc and Indutral Reearch, Ne Delh for provdng the fnancal atance for the preparaton of the manucrpt. Reference [1] A. Reny, On meaure of entropy and nformaton,proc. Fourth Berkeley Symp. Math. Statt. Prob., , [2] C. E. Shannon, A mathematcal theory of communcaton, Bell Sytem Tech. J., , [3] J.N. Kapur, Inequalte: Theory Applcaton and Meaurement, Mathematcal Scence Trut Socety, Inda [4] J.N. Kapur, Entropy and Codng, MSTS, Inda [5] J.L.W.V. Jenen, Sur le foncton convexe et le negalte entre le valeur moyenne, Acta Math., , [6] L.L. Campbell, A codng theorem and Reny entropy, Inform. Control, , [7] N. Tapu, P.G. Popecu, A ne entropy upper bound, Appled Mathematc Letter, , [8] O. Parkah, P. Kakkar, Development of to ne mean codeord length, Informaton Scence, , [9] S.S. Dragomr, An nequalty for logarthmc mappng and applcaton for the Shannon entropy, Computer and Mathematc th Applcaton, , [10] S. Smc, Jenen nequalty and ne entropy bound, Appled Mathematc Letter, ,

12 730 O. Parkah, P. Kakkar [11] S. Smc, Bet poble global bound for Jenen nequalty, Appled Mathematc and Computaton, , [12] S. Smc, On a global upper bound for Jenen nequalty, J. Math. Anal. Appl., ,