An Inequality for Logarithms and Applications in Information Theory

Transcription

1 PERGAMON Computers ad Mathematics with Applicatios 38 (1999) A Iteratioal Joural computers l mathematics with applicatios A Iequality for Logarithms ad Applicatios i Iformatio Theory S. S. DRAGOMIR School of Commuicatios ad Iformatics Victoria Uiversity of Techology P.O. Box 14428, Melboure City MC, Victoria 8001, Australia sever~matilda, vut. edu. au N. M. DRAGOMIR Departmet of Mathematics Uiversity of Traskei, Private Bag X1, UNITRA Umtata, 5117, South Africa K. PRANESH Departmet of Statistics Uiversity of Traskei, Private Bag X1, UNITRA Umtata, 5117, South Africa (Received May 1998; revised ad accepted March 1999) Abstract--A ew aalytic iequality for logarithms which provides a coverse to arithmetic meageometric mea iequality ad its applicatios i iformatio theory are give Elsevier Sciece Ltd. All rights reserved. Keywords--Jese's iequality, Aalytic iequalities, Etropy mappig, Joit etropy. 1. INTRODUCTION The preset paper cotiues the ivestigatios started i [1], where the mai result is the followig. THEOREM 1.1. Let ~k E (0, c~), pk>0, k = 1,...,with ~-~=1'~ Pk = 1 adb> 1. The 0 <_ log b Pk~k -- E Pk logb (k 1 ~ PkPi < ~ k~l ~ (~ - ~k)2" k=l (1.1) The equality holds i both iequalities simultaeously if ad oly if ~1... ~,~. The authors are grateful to the referee for his valuable suggestios /99/$ - see frot matter Elsevier Sciece Ltd. All rights reserved. Typeset by ~4J~t,~-TEX PII: S (99)

2 - 41b 12 S.S. DRAGOMIR et al. 2. A NEW ANALYTIC INEQUALITY FOR LOGARITHMS We shall start to the followig aalytic iequality for logarithms which provides a differet boud tha the iequality of (1.1). THEOREM 2.1. Let ~k E [1, oo) ad pk > 0 with ~-~=l Pk = 1 ad b > 1. The we have 0 _< log b Pk(k -- Pk logb ~k k=l < PiPJ (~ _ ~j)2. ~,j=l (2.1) The equality holds i both iequalities simultaeously if ad oly/f~l... ~,~. PROOF. We shall use the well-kow Jese's discrete iequality for covex mappigs which states that f pixi <_ pj (x~), (2.2) for all Pi > 0, ~-~i=1pi = 1, f a covex mappig o a give iterval I ad xi E I(i = 1,...,). Now, let cosider the mappig f: [1, ~) --~ R, f(x) = x2/2 + lx. The i=l 1 x2+l f' (x) = x , X X for all x E [1, oo) ad 1 x 2-1 f"(x)=l x2 x2, for allxe[1,oo), i.e., f is a strictly covex mappig o [1, oo). Applyig Jese's discrete iequality for covex mappigs, we have 1 ~i + I Pi(i < "~ Pi(~ + Pi I ~i, 2 - \i=1 / i=1 i=l (2.3) which is equivalet to But 1 2 i P~i - Pi l~i _< ~ Pi~i - P~ i=l ki=l Li=I i=1 = 2 i~ - P~i Li=l ad the the above iequality becomes 1 _ j)2 I Pi~i - Epi I ~i _< m i~1 (2.4) Now, as logb x = (l x/l b), iequality (2.4) is equivalet to the desired iequality (2.1). The case of equality follows by the strict covexity of f ad we omit the details.

3 Iequality for Logarithms 13 REMARK 2.1. Defie 1 ~ p~pj BI:= 21bi,.= ~-~j({i-~j) 2, (as i Theorem 1.1) ad ad compute the differece 1 B2 :- (as i Theorem 2.1) 41b E PiPj (~ - ~j)2, [ 1] 1 EPiPJ (~i - ~j)2 1 B1 - B2 = 2 I b ~j PiPj (~i -- ~j)2 (2 4 I b Z_, ~j Cosequetly, if~i E [1, oc) so that ~i~j -< 2, for all i,j E {1,...,}, the the boud B2 provided by Theorem 2.1 is better tha the boud B1 provided by Theorem 1.1. If ~i E [1,cc) so that ~i~j _> 2, for all i,j E {1,...,}, the Theorem 1.1 provides a better result tha Theorem 2.1. We give ow some applicatios of the above results for arithmetic mea-geometric mea iequality. Recall that for qi > 0 with Q := ~i=lqi, the arithmetic mea of xi with the weights qi, i E {1,...,} is 1 (A) A (~,5) := ~ 4=1 ad the geometric mea of xi with the weights qi, i E {1,...,}, is / \ 1/Q~ It is well kow that the followig iequality so-called arithmetic mea-geometric mea iequality, holds A (~, 5) Z G (~, z) (2.5) with equality if ad oly if xl... x. Now, usig Theorem 1.1, we ca state the followig propositio cotaiig a couterpart of the arithmetic mea-geometric mea iequality (2.5). PROPOSITION 2.2. With the above assumptios for -~ ad 5, we have 1 < A (-~,5) 1 ~ qiqj where expb(x ) = b x, (b > 1). The equa//ty holds i both iequalities simultaeously//:ad oly if Xl... -= X. Also, usig Theorem 2.1, we have aother coverse iequality for (2.5). PROPOSITION 2.3. Let ~ be as above ad 5 E R with x~ >_ 1, i = 1,...,. The we have the iequality 1 < A (~,5) 1 ~ qiqj - G (~, ~ <- expb 4Q b i,~=1 XiXj (xi -- xj)2 ' (2.7) where b > 1. The equality holds i both iequalities simultaeously if ad oly/fxl... REMARK 2.2. As i the previous remark, if 1 _< xixj <_ 2 the boud (2.7) is better tha (2.6). If xixj >_ 2, the (2.6) is better tha (2.7). x.

4 14 S.S. DRAGOMIR et al. 3. APPLICATIONS FOR THE ENTROPY MAPPING Let us cosider ow, the b-etropy mappig of the discrete radom variable X with possible outcomes ad havig the probability distributio p -- (p~), i = {1,..., }, (1) Hb (X) = Zpi log b. We kow (see [1]) that the followig coverse iequality holds: i=l O<--l gb--hb(x) <-- 21b (P~-PJ) (3.1) with equality if ad oly ifpi = 1i, for all i E {1,...,}. The followig similar result also holds. THEOREM 3.1. Let X be as above. The we have 1 ~ (Pi :pj)2. 0 _< logb -- Hb(X) <_ 41b PiPj i,j----1 (3.2) The equality holds if ad oly if p~ = 1i, for ali i E {1,..., }. PROOF. As Pi E (0, 1], the ~i = 1/p~ E [1, oc) ad we ca apply Theorem 2.1 to get 1 ~ PiPj 0 _< logb- Hb(X) <_ 41b _ 1 (pi_pj)2 4 ~ b,~j l.= PiPj (3.3) The equality holds iff ~i = ~j, for all i,j E {1,...,} which is equivalet to Pi = Pj, for all i,j E {1,...,}, i.e., Pi = 1i, for all i C {1,...,}. The followig corollary is importat i applicatios as it provides a sufficiet coditio o the probability p so that log b - Hb(X) is small eough. COROLLARY 3.2. Let X be as above ad ~ > O. If the probabifities Pi, i = 1,...,, satisfy the coditios 2 -pj - 2 for all 1 <_ i < j <_, where the we have the estimatio 4e i b k - ( - 1)' ( > 2), 0 < log b - Hb(X) <_ e. (a.5) PROOF. Observe that 1 ~-~ (pi=_pj) 2 _ 1 ~ (pi-pj) 2 4I b~,j=l p~pj 2 I b l<_i<j<_u PiPj Suppose that (p~ _ pj)2 PiPj <_k, forl<_i<j<.

5 Iequality for Logarithms 15 The p/2 _ (2 + k)p@j + pj 2<0, _ forl<i<j<. Deotig t = p~/pj, the above iequality is equivalet to t 2 - (2 + k)t + 1 N 0, i.e., t E It1, t2], where 2+k-v/~+4) ad t2 2+k+v/k(k+4) tl = If we choose k = (4 lb/ ( - 1)), the by (3.3) we have 0 < log b - Hb(X) <_ I1 b ~ (Pi---pj)2 4 PiPj 1 v" _ p J)2 2 I b A.~ p~pj l<i<j< 1 ( - 1) 4elb -<21---b E k= l~.i<j< 41b (-1) ---~, ad the corollary is proved. Now, cosider the bouds 1 21 b - p;)2, %j=1 (give by (3.1)) ad 1 -~ (p~ _pj)2 (give by (3.3)). M2 := 4lab "~'=, : p-~j ' We give a example for which M1 is less tha M2 ad aother example for which M2 is less tha M1 which will suggest that we ca use both of them to estimate the above differece log b - Hb (X). EXAMPLE 3.1. Cosider the probability distributio Pl = , t)2 = , P3 = , P4 = , P , P6 = I this case, M1 = , M2 = , where _ 2 M1 := -~ (Pi - Pj), ~ (Pi- P j) M2 :-= 4i,~ 1 p-~j, ad = 6. EXAMPLE 3.2. Cosider the probability distributio Pl = , P2 = , P3 = , P , P5 = , P6 = I this case, M1 = , M2 =

6 - 16 S.S. DRAGOMIR et al. 4. BOUNDS FOR JOINT ENTROPY Cosider the joit etropy of two radom variable X ad Y [2, p. 25] Hb(X, Y) := Ep x,y (x, y)log b p (x, y)' where the joit probability p (x, y) = P {X = x, Y = y}. I [3], Dragomir ad Goh have proved the followig result usig Theorem 1.1. THEOREM 4.1. With the above assumptios, we have that 1 0 < log b (rs) - Hb (X, Y) < 2 I b E E (p (x, y) - p (u, v)) 2, x,y ~,V (4.1) where the rage of X cotais r elemets ad the rage of Y cotais s elemets. Equality holds i both iequalities simultaeously if ad oly if p (x, y) = p (u, v), for all (x,y), (u,v). The followig corollary is useful i practice. COROLLARY 4.2. With the above assumptios ad if the we have the estimatio max [p(x,y) p(u,v)[ </~/2~lb -- _ (x,y),(u,v) V rs 0 _< log b (r, s) - Hb (X, Y) <_ e. -, ~ > O~ Now, usig the secod coverse iequality embodied i Theorem 2.1, we are able to prove aother upper boud for the differece log b (rs) - Hb (X, Y). THEOREM 4.3. With the above assumptios, we have 1 (p (x, y) - p (u, v)) 2 0 < log b (rs) - Hb (X, Y) < 4 I b E E p (x, y) p (u, v) ' X~y U~V (4.2) where the rage of X ad Y are as above. Equality holds i both iequalities simultaeously iff p (x, y) -- p (u, v), for all (x, y) ad (u, v). PROOF. Usig Theorem 2.1, we have for pi = p(x, y) ad (i = (1/p (x, y)), 0 _< log b E p (x, y). P (x, 1 y----~ ) _ EP(x,y)log b p (x, 1 y-----~ x,y x,y 1 EEP(x,y)p(u,v ) P(x,Y) p(lv) 2 -< 41b X~y U,~ 1 (p (x, y) - p (u, v)) 2 = p(x,y)p(u,v) ' ) which is clearly equal to the desired result. The case of equality is obvious by Theorem 2.1. The followig corollary is importat i practical applicatios.

7 Iequality for Logarithms 17 COROLLARY 4.4. mip(x, y). If where the we have the boud Let X ad Y be as above ad e > O. Deote P P _< 1+ k + v/k(k +2), P 2e I b k.'~ - - (rs) ~' 0 _< log b (rs) - Hb (X, Y) _< ~. = maxp(x,y) ad p = (4.3) PROOF. At the begiig, let us cosider the iequality (a - b) 2 <_ k, 2ab fora, b>0, adk>_0. This iequality is clearly equivalet to a~-2(l+k)ab+b 2~0 or deotig t :-- a/b, to i.e. t 2-2(1+ k)t+ 1 < O, l+k-v/k(k+2)<t<l+k+v/k(k+2). Now, let suppose that l +k-x/k(k p(x,y) + 2 ) <_-- p(u,v) < 1 +k + ~/k (k + 2), (4.4) for all (x, y) ad (u, v) ad k := (2El b/(rs)2). The by (4.2), we have 1 (p (x, y) - p (u, v)) 2 0 _< log b (rs) - Hb (X, Y) < 4 I b E E p (x, y) p (u, v) 1 (rs) 2 2elb < - -. (rs) 2 k = = ~. - 21b 21b (rs) 2 Now, let observe that iequality (4.4) is equivalet to l + k - ~ < _ - f _ P <P<l+k+v/~(k+2)._ p _ But p/p >_ 1 + k - X/k (k + 2) is equivalet to P 1 _< ; - l+k- v~(k+2) = k x/k (k + 2) ad the corollary is proved. REFERENCES 1. S.S. Dragomir ad C.J. Goh, A couterpart of Jese's discrete iequality for differetiable covex mappig ad applicatio i iformatio theory, Mathl. Comput. Modellig 24 (2), 1-4, (1996). 2. S. Roma, Cedig ad Iformatio Theory, Spriger-Verlag, New York. 3. S.S. Dragomir ad C.J. Goh, Further couterparts of some iequalities i iformatio theory, (submitted).