An Inequality for Logarithms and Applications in Information Theory

Similar documents
S. S. Dragomir and Y. J. Cho. I. Introduction In 1956, J. Aczél has proved the following interesting inequality ([2, p. 57], [3, p.

A GRÜSS TYPE INEQUALITY FOR SEQUENCES OF VECTORS IN NORMED LINEAR SPACES AND APPLICATIONS

A REFINEMENT OF JENSEN S INEQUALITY WITH APPLICATIONS. S. S. Dragomir 1. INTRODUCTION

ON SOME INEQUALITIES IN NORMED LINEAR SPACES

INEQUALITY FOR CONVEX FUNCTIONS. p i. q i

MONOTONICITY OF SEQUENCES INVOLVING GEOMETRIC MEANS OF POSITIVE SEQUENCES WITH LOGARITHMICAL CONVEXITY

The random version of Dvoretzky s theorem in l n

Generalization of Samuelson s inequality and location of eigenvalues

Information Theory and Statistics Lecture 4: Lempel-Ziv code

Bounds for the Extreme Eigenvalues Using the Trace and Determinant

University of Manitoba, Mathletics 2009

Let us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.

Lecture 8: Convergence of transformations and law of large numbers

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

INTRODUCTION ABSTRACT

Journal of Mathematical Analysis and Applications 250, doi: jmaa , available online at http:

Available online at J. Math. Comput. Sci. 2 (2012), No. 3, ISSN:

Lecture 7: October 18, 2017

An Introduction to Randomized Algorithms

Machine Learning Theory Tübingen University, WS 2016/2017 Lecture 12

J. Stat. Appl. Pro. Lett. 2, No. 1, (2015) 15

ABOUT CHAOS AND SENSITIVITY IN TOPOLOGICAL DYNAMICS

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Rademacher Complexity

BETWEEN QUASICONVEX AND CONVEX SET-VALUED MAPPINGS. 1. Introduction. Throughout the paper we denote by X a linear space and by Y a topological linear

Precise Rates in Complete Moment Convergence for Negatively Associated Sequences

ANOTHER WEIGHTED WEIBULL DISTRIBUTION FROM AZZALINI S FAMILY

Lecture 23: Minimal sufficiency

PAijpam.eu ON TENSOR PRODUCT DECOMPOSITION

Differential Entropy

Approximation theorems for localized szász Mirakjan operators

Oscillation and Property B for Third Order Difference Equations with Advanced Arguments

Fixed Point Theorems for Expansive Mappings in G-metric Spaces

Properties of Fuzzy Length on Fuzzy Set

Probability and Statistics

PREDICTION INTERVALS FOR FUTURE SAMPLE MEAN FROM INVERSE GAUSSIAN DISTRIBUTION

Chapter 6 Infinite Series

2.2. Central limit theorem.

Linear chord diagrams with long chords

New Inequalities of Hermite-Hadamard-like Type for Functions whose Second Derivatives in Absolute Value are Convex

FLUID LIMIT FOR CUMULATIVE IDLE TIME IN MULTIPHASE QUEUES. Akademijos 4, LT-08663, Vilnius, LITHUANIA 1,2 Vilnius University

ON POINTWISE BINOMIAL APPROXIMATION

Maximal sets of integers not containing k + 1 pairwise coprimes and having divisors from a specified set of primes

ON SOME DIOPHANTINE EQUATIONS RELATED TO SQUARE TRIANGULAR AND BALANCING NUMBERS

~j=zhax~ 6=0, t,2... ), k=~

The standard deviation of the mean

Pairs of disjoint q-element subsets far from each other

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

On Functions -Starlike with Respect to Symmetric Conjugate Points

CHAPTER 5. Theory and Solution Using Matrix Techniques

Two Topics in Number Theory: Sum of Divisors of the Factorial and a Formula for Primes

sin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =

Bangi 43600, Selangor Darul Ehsan, Malaysia (Received 12 February 2010, accepted 21 April 2010)

Fermat s Little Theorem. mod 13 = 0, = }{{} mod 13 = 0. = a a a }{{} mod 13 = a 12 mod 13 = 1, mod 13 = a 13 mod 13 = a.

DECOMPOSITION METHOD FOR SOLVING A SYSTEM OF THIRD-ORDER BOUNDARY VALUE PROBLEMS. Park Road, Islamabad, Pakistan

VECTOR SEMINORMS, SPACES WITH VECTOR NORM, AND REGULAR OPERATORS

Dominant of Functions Satisfying a Differential Subordination and Applications

4.1 Data processing inequality

Lecture Notes for Analysis Class

Assignment 5: Solutions

Asymptotic distribution of products of sums of independent random variables

Rational Bounds for the Logarithm Function with Applications

Boundaries and the James theorem

n=1 a n is the sequence (s n ) n 1 n=1 a n converges to s. We write a n = s, n=1 n=1 a n

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

Self-normalized deviation inequalities with application to t-statistic

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

Berry-Esseen bounds for self-normalized martingales

Research Article Approximate Riesz Algebra-Valued Derivations

Lecture 2. The Lovász Local Lemma

Advanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology

Journal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula

INEQUALITIES BJORN POONEN

The Boolean Ring of Intervals

A GRÜSS-TYPE INEQUALITY AND ITS APPLICATIONS

ON A SUBCLASS OF HARMONIC UNIVALENT FUNCTIONS DEFINED BY CONVOLUTION. G. Shelake, S. Joshi, S. Halim

lim za n n = z lim a n n.

An elementary proof that almost all real numbers are normal

EE 4TM4: Digital Communications II Probability Theory

Lecture 3 : Random variables and their distributions

Advanced Stochastic Processes.

TRACES OF HADAMARD AND KRONECKER PRODUCTS OF MATRICES. 1. Introduction

5.1 A mutual information bound based on metric entropy

Moment-entropy inequalities for a random vector

A Hadamard-type lower bound for symmetric diagonally dominant positive matrices

Infinite Sequences and Series

A 2nTH ORDER LINEAR DIFFERENCE EQUATION

MAS111 Convergence and Continuity

EECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1

Lecture 7: Properties of Random Samples

62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +

About the use of a result of Professor Alexandru Lupaş to obtain some properties in the theory of the number e 1

Lecture 19: Convergence

Resolution Proofs of Generalized Pigeonhole Principles

Fastest mixing Markov chain on a path

Several properties of new ellipsoids

Stat410 Probability and Statistics II (F16)

Topic 9: Sampling Distributions of Estimators

Confidence Interval for Standard Deviation of Normal Distribution with Known Coefficients of Variation

Power series are analytic

Transcription:

PERGAMON Computers ad Mathematics with Applicatios 38 (1999) 11-17 A Iteratioal Joural computers l mathematics with applicatios A Iequality for Logarithms ad Applicatios i Iformatio Theory S. S. DRAGOMIR School of Commuicatios ad Iformatics Victoria Uiversity of Techology P.O. Box 14428, Melboure City MC, Victoria 8001, Australia sever~matilda, vut. edu. au N. M. DRAGOMIR Departmet of Mathematics Uiversity of Traskei, Private Bag X1, UNITRA Umtata, 5117, South Africa K. PRANESH Departmet of Statistics Uiversity of Traskei, Private Bag X1, UNITRA Umtata, 5117, South Africa (Received May 1998; revised ad accepted March 1999) Abstract--A ew aalytic iequality for logarithms which provides a coverse to arithmetic meageometric mea iequality ad its applicatios i iformatio theory are give. 1999 Elsevier Sciece Ltd. All rights reserved. Keywords--Jese's iequality, Aalytic iequalities, Etropy mappig, Joit etropy. 1. INTRODUCTION The preset paper cotiues the ivestigatios started i [1], where the mai result is the followig. THEOREM 1.1. Let ~k E (0, c~), pk>0, k = 1,...,with ~-~=1'~ Pk = 1 adb> 1. The 0 <_ log b Pk~k -- E Pk logb (k 1 ~ PkPi < ~ k~l ~ (~ - ~k)2" k=l (1.1) The equality holds i both iequalities simultaeously if ad oly if ~1... ~,~. The authors are grateful to the referee for his valuable suggestios. 0898-1221/99/$ - see frot matter. 1999 Elsevier Sciece Ltd. All rights reserved. Typeset by ~4J~t,~-TEX PII: S0898-1221(99)00177-7

- 41b 12 S.S. DRAGOMIR et al. 2. A NEW ANALYTIC INEQUALITY FOR LOGARITHMS We shall start to the followig aalytic iequality for logarithms which provides a differet boud tha the iequality of (1.1). THEOREM 2.1. Let ~k E [1, oo) ad pk > 0 with ~-~=l Pk = 1 ad b > 1. The we have 0 _< log b Pk(k -- Pk logb ~k k=l < PiPJ (~ _ ~j)2. ~,j=l (2.1) The equality holds i both iequalities simultaeously if ad oly/f~l... ~,~. PROOF. We shall use the well-kow Jese's discrete iequality for covex mappigs which states that f pixi <_ pj (x~), (2.2) for all Pi > 0, ~-~i=1pi = 1, f a covex mappig o a give iterval I ad xi E I(i = 1,...,). Now, let cosider the mappig f: [1, ~) --~ R, f(x) = x2/2 + lx. The i=l 1 x2+l f' (x) = x + - -- - -, X X for all x E [1, oo) ad 1 x 2-1 f"(x)=l x2 x2, for allxe[1,oo), i.e., f is a strictly covex mappig o [1, oo). Applyig Jese's discrete iequality for covex mappigs, we have 1 ~i + I Pi(i < "~ Pi(~ + Pi I ~i, 2 - \i=1 / i=1 i=l (2.3) which is equivalet to But 1 2 i P~i - Pi l~i _< ~ Pi~i - P~ i=l ki=l Li=I i=1 = 2 i~ - P~i Li=l ad the the above iequality becomes 1 _ j)2 I Pi~i - Epi I ~i _< m i~1 (2.4) Now, as logb x = (l x/l b), iequality (2.4) is equivalet to the desired iequality (2.1). The case of equality follows by the strict covexity of f ad we omit the details.

Iequality for Logarithms 13 REMARK 2.1. Defie 1 ~ p~pj BI:= 21bi,.= ~-~j({i-~j) 2, (as i Theorem 1.1) ad ad compute the differece 1 B2 :- (as i Theorem 2.1) 41b E PiPj (~ - ~j)2, [ 1] 1 EPiPJ (~i - ~j)2 1 B1 - B2 = 2 I b ~j 1 @, PiPj (~i -- ~j)2 (2 4 I b Z_, ~j Cosequetly, if~i E [1, oc) so that ~i~j -< 2, for all i,j E {1,...,}, the the boud B2 provided by Theorem 2.1 is better tha the boud B1 provided by Theorem 1.1. If ~i E [1,cc) so that ~i~j _> 2, for all i,j E {1,...,}, the Theorem 1.1 provides a better result tha Theorem 2.1. We give ow some applicatios of the above results for arithmetic mea-geometric mea iequality. Recall that for qi > 0 with Q := ~i=lqi, the arithmetic mea of xi with the weights qi, i E {1,...,} is 1 (A) A (~,5) := ~ 4=1 ad the geometric mea of xi with the weights qi, i E {1,...,}, is / \ 1/Q~ It is well kow that the followig iequality so-called arithmetic mea-geometric mea iequality, holds A (~, 5) Z G (~, z) (2.5) with equality if ad oly if xl... x. Now, usig Theorem 1.1, we ca state the followig propositio cotaiig a couterpart of the arithmetic mea-geometric mea iequality (2.5). PROPOSITION 2.2. With the above assumptios for -~ ad 5, we have 1 < A (-~,5) 1 ~ qiqj where expb(x ) = b x, (b > 1). The equa//ty holds i both iequalities simultaeously//:ad oly if Xl... -= X. Also, usig Theorem 2.1, we have aother coverse iequality for (2.5). PROPOSITION 2.3. Let ~ be as above ad 5 E R with x~ >_ 1, i = 1,...,. The we have the iequality 1 < A (~,5) 1 ~ qiqj - G (~,5-------~ <- expb 4Q21------b i,~=1 XiXj (xi -- xj)2 ' (2.7) where b > 1. The equality holds i both iequalities simultaeously if ad oly/fxl... REMARK 2.2. As i the previous remark, if 1 _< xixj <_ 2 the boud (2.7) is better tha (2.6). If xixj >_ 2, the (2.6) is better tha (2.7). x.

14 S.S. DRAGOMIR et al. 3. APPLICATIONS FOR THE ENTROPY MAPPING Let us cosider ow, the b-etropy mappig of the discrete radom variable X with possible outcomes ad havig the probability distributio p -- (p~), i = {1,..., }, (1) Hb (X) = Zpi log b. We kow (see [1]) that the followig coverse iequality holds: i=l O<--l gb--hb(x) <-- 21b (P~-PJ) (3.1) with equality if ad oly ifpi = 1i, for all i E {1,...,}. The followig similar result also holds. THEOREM 3.1. Let X be as above. The we have 1 ~ (Pi :pj)2. 0 _< logb -- Hb(X) <_ 41b PiPj i,j----1 (3.2) The equality holds if ad oly if p~ = 1i, for ali i E {1,..., }. PROOF. As Pi E (0, 1], the ~i = 1/p~ E [1, oc) ad we ca apply Theorem 2.1 to get 1 ~ PiPj 0 _< logb- Hb(X) <_ 41b _ 1 (pi_pj)2 4 ~ b,~j l.= PiPj (3.3) The equality holds iff ~i = ~j, for all i,j E {1,...,} which is equivalet to Pi = Pj, for all i,j E {1,...,}, i.e., Pi = 1i, for all i C {1,...,}. The followig corollary is importat i applicatios as it provides a sufficiet coditio o the probability p so that log b - Hb(X) is small eough. COROLLARY 3.2. Let X be as above ad ~ > O. If the probabifities Pi, i = 1,...,, satisfy the coditios 2 -pj - 2 for all 1 <_ i < j <_, where the we have the estimatio 4e i b k - ( - 1)' ( > 2), 0 < log b - Hb(X) <_ e. (a.5) PROOF. Observe that 1 ~-~ (pi=_pj) 2 _ 1 ~ (pi-pj) 2 4I b~,j=l p~pj 2 I b l<_i<j<_u PiPj Suppose that (p~ _ pj)2 PiPj <_k, forl<_i<j<.

Iequality for Logarithms 15 The p/2 _ (2 + k)p@j + pj 2<0, _ forl<i<j<. Deotig t = p~/pj, the above iequality is equivalet to t 2 - (2 + k)t + 1 N 0, i.e., t E It1, t2], where 2+k-v/~+4) ad t2 2+k+v/k(k+4) tl = -- 2 2 If we choose k = (4 lb/ ( - 1)), the by (3.3) we have 0 < log b - Hb(X) <_ I1 b ~ (Pi---pj)2 4 PiPj 1 v" _ p J)2 2 I b A.~ p~pj l<i<j< 1 ( - 1) 4elb -<21---b E k= l~.i<j< 41b (-1) ---~, ad the corollary is proved. Now, cosider the bouds 1 21 b - p;)2, %j=1 (give by (3.1)) ad 1 -~ (p~ _pj)2 (give by (3.3)). M2 := 4lab "~'=, : p-~j ' We give a example for which M1 is less tha M2 ad aother example for which M2 is less tha M1 which will suggest that we ca use both of them to estimate the above differece log b - Hb (X). EXAMPLE 3.1. Cosider the probability distributio Pl = 0.3475, t)2 = 0.2398, P3 = 0.1654, P4 = 0.1142, P5 -- 0.0788, P6 = 0.0544. I this case, M1 = 6.5119, M2 = 12.1166, where _ 2 M1 := -~ (Pi - Pj), 2 -- 1 ~ (Pi- P j) M2 :-= 4i,~ 1 p-~j, ad = 6. EXAMPLE 3.2. Cosider the probability distributio Pl = 0.2468, P2 = 0.2072, P3 = 0.1740, P4 -- 0.1461, P5 = 0.1227, P6 = 0.1031. I this case, M1 = 5.2095, M2 = 2.3706.

- 16 S.S. DRAGOMIR et al. 4. BOUNDS FOR JOINT ENTROPY Cosider the joit etropy of two radom variable X ad Y [2, p. 25] Hb(X, Y) := Ep x,y (x, y)log b p (x, y)' where the joit probability p (x, y) = P {X = x, Y = y}. I [3], Dragomir ad Goh have proved the followig result usig Theorem 1.1. THEOREM 4.1. With the above assumptios, we have that 1 0 < log b (rs) - Hb (X, Y) < 2 I b E E (p (x, y) - p (u, v)) 2, x,y ~,V (4.1) where the rage of X cotais r elemets ad the rage of Y cotais s elemets. Equality holds i both iequalities simultaeously if ad oly if p (x, y) = p (u, v), for all (x,y), (u,v). The followig corollary is useful i practice. COROLLARY 4.2. With the above assumptios ad if the we have the estimatio max [p(x,y) p(u,v)[ </~/2~lb -- _ (x,y),(u,v) V rs 0 _< log b (r, s) - Hb (X, Y) <_ e. -, ~ > O~ Now, usig the secod coverse iequality embodied i Theorem 2.1, we are able to prove aother upper boud for the differece log b (rs) - Hb (X, Y). THEOREM 4.3. With the above assumptios, we have 1 (p (x, y) - p (u, v)) 2 0 < log b (rs) - Hb (X, Y) < 4 I b E E p (x, y) p (u, v) ' X~y U~V (4.2) where the rage of X ad Y are as above. Equality holds i both iequalities simultaeously iff p (x, y) -- p (u, v), for all (x, y) ad (u, v). PROOF. Usig Theorem 2.1, we have for pi = p(x, y) ad (i = (1/p (x, y)), 0 _< log b E p (x, y). P (x, 1 y----~ ) _ EP(x,y)log b p (x, 1 y-----~ x,y x,y 1 EEP(x,y)p(u,v ) P(x,Y) p(lv) 2 -< 41b X~y U,~ 1 (p (x, y) - p (u, v)) 2 = p(x,y)p(u,v) ' ) which is clearly equal to the desired result. The case of equality is obvious by Theorem 2.1. The followig corollary is importat i practical applicatios.

Iequality for Logarithms 17 COROLLARY 4.4. mip(x, y). If where the we have the boud Let X ad Y be as above ad e > O. Deote P P _< 1+ k + v/k(k +2), P 2e I b k.'~ - - (rs) ~' 0 _< log b (rs) - Hb (X, Y) _< ~. = maxp(x,y) ad p = (4.3) PROOF. At the begiig, let us cosider the iequality (a - b) 2 <_ k, 2ab fora, b>0, adk>_0. This iequality is clearly equivalet to a~-2(l+k)ab+b 2~0 or deotig t :-- a/b, to i.e. t 2-2(1+ k)t+ 1 < O, l+k-v/k(k+2)<t<l+k+v/k(k+2). Now, let suppose that l +k-x/k(k p(x,y) + 2 ) <_-- p(u,v) < 1 +k + ~/k (k + 2), (4.4) for all (x, y) ad (u, v) ad k := (2El b/(rs)2). The by (4.2), we have 1 (p (x, y) - p (u, v)) 2 0 _< log b (rs) - Hb (X, Y) < 4 I b E E p (x, y) p (u, v) 1 (rs) 2 2elb < - -. (rs) 2 k = = ~. - 21b 21b (rs) 2 Now, let observe that iequality (4.4) is equivalet to l + k - ~ < _ - f _ P <P<l+k+v/~(k+2)._ p _ But p/p >_ 1 + k - X/k (k + 2) is equivalet to P 1 _< ; - l+k- v~(k+2) = k + 1 + x/k (k + 2) ad the corollary is proved. REFERENCES 1. S.S. Dragomir ad C.J. Goh, A couterpart of Jese's discrete iequality for differetiable covex mappig ad applicatio i iformatio theory, Mathl. Comput. Modellig 24 (2), 1-4, (1996). 2. S. Roma, Cedig ad Iformatio Theory, Spriger-Verlag, New York. 3. S.S. Dragomir ad C.J. Goh, Further couterparts of some iequalities i iformatio theory, (submitted).

2024 © sciencedocbox.com Privacy Policy | Terms of Service | Feedback