ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS

Transcription

1 Adv. Oper. Theory ISSN: 538-5X electronic ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS SADIA KHALID, DILDA PEČARIĆ, and JOSIP PEČARIĆ3 Communicated by M. S. Moslehian Abstract. In this paper, we present some interesting results related to the bounds of Zipf-Mandelbrot entropy and the 3-convexity of the function. Further, we define linear functionals as the non-negative differences of the obtained inequalities and we present mean value theorems for the linear functionals. Finally, we discuss the n-exponential convexity and the log-convexity of the functions associated with the linear functionals.. Introduction and preliminaries Definition.. The Shannon entropy of a positive probability distribution p = p,..., p n is defined by S p := n = p log p. An important result related to the bounds of the Shannon entropy is given in []. Copyright 09 by the Tusi Mathematical Research Group. Date: Received: Oct. 3, 08; Accepted: Feb., 09. Corresponding author. 00 Mathematics Subject Classification. Primary 6A4; Secondary 6A48, 6A5, 6D5. Key words and phrases. Shannon entropy, Zipf-Mandelbrot entropy, divided difference, n- convex function, Cauchy means and n-exponential convexity and logarithmic convexity.

2 S. KHALID, D. PEČARIĆ. J. PEČARIĆ Theorem.. Let p > 0 n be a probability distribution with Shannon entropy S p and P = i= p i n. Then S p p P log log F p S p p P log P log P,. where F x = x log x x log x x > 0.. Equalities hold in. if p = n. n S. Khalid, J. Pečarić and M. Pralja presented the following generalization of Throrem. in [3, Theorems. and.3]. Theorem.3. Let a > 0 and p > 0 n be real numbers such that P = i= p i n. Let g : [a, b] R be a differentiable function such that g x h g x is convex for all x, x h [a, b], where h 0. i If P, P, s R, we have i= p i a, i= p i a [a, b] for all {,..., n}, then for any a s gp gp a s g i= p i a a s g P g P a s i= g a s i= p i P a g P g P i= p i a p i P a g i= p i a g i= p i. a.3

3 ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS 3 ii If p a, P a, P a, i= p i, i= p i [a, b] for all {,..., n}, then we have g p a gp a gp a p i P a g P a g P a i= g p i i= g p a g P a g P a p i P a g p i g p i..4 i= i= i= If g x h g x is concave for all x, x h [a, b] such that h 0, then the reversed inequalities hold in.3 and.4. Divided difference of a function is defined as follows see [0, p. 4] : Definition.4. The nth-order divided difference of a function f : [a, b] R at mutually distinct points x 0,..., x n [a, b] is defined recursively by [x i ; f] = f x i, i {0,..., n}, [x 0,..., x n ; f] = [x,..., x n ; f] [x 0,..., x n ; f] x n x 0 The value [x 0,..., x n ; f] is independent of the order of the points x 0,..., x n. n-convex functions can be characterized by the nth-order divided difference see [0, p. 5]. Definition.5. A function f : [a, b] R is said to be n-convex n 0 if and only if for all choices of n distinct points x 0,..., x n [a, b], the nth-order divided difference of f satisfies [x 0,..., x n ; f] 0. Remar.6. Note that 0-convex functions are non-negative functions, -convex functions are increasing functions and -convex functions are simply the convex functions. The following interesting result related to the 3-convexity of the function g, is also presented in [3, Corollaries. and.4]. Corollary.7. Let a > 0 and p > 0 n be real numbers such that P = i= p i n and let g : [a, b] R be a differentiable function. i= p i i= p i i Let P, P, a, a [a, b] for all {,..., n}. If g is 3-convex, then.3 holds for any s R.

4 4 S. KHALID, D. PEČARIĆ. J. PEČARIĆ ii Let p a, P a, P a, i= p i, i= p i [a, b] for all {,..., n}. If g is 3-convex, then.4 holds. If g is 3-concave, then the reversed inequalities hold in.3 and.4. The results related to Shannon entropy and Zipf-Mandelbrot law are topic of great interest see for example [], [5] and [6]. Zipf-Mandelbrot law is revisited in the context of linguistics in [8] see also [7]. The paper is organized as follows: in Section, we present some interesting results related to Zipf-Mandelbrot entropy. In Section 3, we define linear functionals as the non-negative differences of the obtained inequalities and present mean value theorems for the linear functionals. In Section 4, we present the properties of functionals, such as n-exponential and logarithmic convexity. Finally, we give an example of the family of functions for which the results can be applied.. Inequalities related to Zipf-Mandelbrot entropy Definition.. Zipf-Mandelbrot law is a discrete probability distribution depending on three parameters n N, r 0 and t > 0, and is defined as f i; n, r, t = i r t, i {,..., n}, where f is nown as the probability mass function and := is the generalized harmonic number. = r t. If we tae p = r t n, r 0, t > 0 and is the same as defined in Definition. in S p, then simple calculations reveal that = r t log r t = t = := Z r, t,, where Z r, t, is nown as Zipf-Mandelbrot entropy. Now we define the cumulative distribution function as follows : log r r t log C,n,r,t := H,r,t, {,..., n}, n N, r 0, and t > 0.. The aim of this section is to present some interesting results related to Zipf- Mandelbrot entropy. The first result of this section states that :

5 ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS 5 Theorem.. Let Z r, t, be the Zipf-Mandelbrot entropy and F and C,n,r,t be the same as defined in. and. respectively. Then Z r, t, C r t,n,r,t F r t Z r, t, C r t,n,r,t log H,r,t log. H,r,t n, r 0, t > 0 in., the result is im- Proof. Tae p = r t mediate. The second main result of this section states that : Theorem.3. Let a > 0 be real numbers and and C,n,r,t be the same as defined in. and. respectively. Let g : [a, b] R be a differentiable function such that g x h g x is convex for all x, x h [a, b], where h 0. ir t, a i= i Let C,n,r,t, C,n,r,t, a i= {,..., n}. Then for any s R, we have ir t [a, b] for all a s g C,n,r,t g C,n,r,t a s i r a C t,n,r,t g C,n,r,t g C,n,r,t i= a s g g a i r t a i= i r t i= a s g C,n,r,t g C,n,r,t a s g a i r a C t,n,r,t i= g i r t a i= i= i r t..3

6 6 S. KHALID, D. PEČARIĆ. J. PEČARIĆ a r t, a C,n,r,t, a C,n,r,t, i= ii Let [a, b] for all {,..., n}. Then ir t, i= ir t a g g a r t C,n,r,t g a C,n,r,t i r t a C,n,r,t g a C,n,r,t g a C,n,r,t i= g i r t i= a g g a r t C,n,r,t g a C,n,r,t i r t a C,n,r,t i= g g..4 i r t i r t i= i= If g x h g x is concave for all x, x h [a, b] such that h 0, then the reversed inequalities hold in.3 and.4. Proof. i By taing p i = ir t in.3, the result is immediate. ii The idea of the proof is the same as discussed in i. Corollary.4. Let a > 0 be real numbers, and C,n,r,t be the same as defined in. and. respectively and let g : [a, b] R be a differentiable function. i Let the condition of Theorem.3 i holds. If g is 3-convex, then.3 holds for any s R. ii Let the condition of Theorem.3 ii holds. If g is 3-convex, then.4 holds. If g is 3-concave, then the reversed inequalities hold in.3 and.4.

7 ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS 7 3. Linear functionals and mean value theorems Consider the inequalities.3 and.4 and define linear functionals Ψ i i = -6 by the non-negative differences of the inequalities.3 and.4 as follows : Ψ g = a s g g a i r t a i= a s g C,n,r,t g C,n,r,t Ψ g = Ψ 3 g = a s g i= i r t a s D g C,n,r,t g C,n,r,t, 3. a i= i r a C t,n,r,t g i r t a i= i= i r t a s D g C,n,r,t g C,n,r,t, 3. a s g C,n,r,t g C,n,r,t a s g a i= a s D g a i r t i= g a g i r t a Ψ 4 g = g i r t g r t H i= n,r,t g a C,n,r,t g a C,n,r,t i= a i r t i= i r t D g a C,n,r,t g a C,n,r,t, 3.3, 3.4

8 8 S. KHALID, D. PEČARIĆ. J. PEČARIĆ Ψ 5 g = i r t a C,n,r,t i= g g i r t H i= n,r,t i r t i= D g a C,n,r,t g a C,n,r,t 3.5 and a Ψ 6 g = g g r t i r t i= g a C,n,r,t g a C,n,r,t D g i= where D = i= a ir t C,n,r,t. i r t g i= i r t, 3.6 If g : [a, b] R is differentiable and 3-convex, then Corollary.4 implies that Ψ i g 0, i {,..., 6}. Now we give mean value theorems for the functionals Ψ i i = -6 as defined in These theorems enable us to define various classes of means that can be expressed in terms of linear functionals. Theorem 3.. Let g : [a, b] R be such that g C 3 [a, b] and let Ψ i i = -6 be linear functionals as defined in Then there exists δ i [a, b] such that Ψ i g = g δ i Ψ i g 0, i {,..., 6}, 6 where g 0 x = x 3. Proof. The proof is analogous to the proof of Theorem.7 in [3]. The following theorem is a new analogue of the classical Cauchy mean value theorem, related to the functionals Ψ i i = -6 and it can be proven by following the proof of Theorem.8 in [3]. Theorem 3.. Let g, h : [a, b] R be such that g, h C 3 [a, b] and let Ψ i i = -6 be linear functionals as defined in Then there exists δ i [a, b] such that Ψ i g Ψ i h = g δ i, i {,..., 6}, 3.7 h δ i provided that the denominators are non-zero.

9 ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS 9 Remar 3.3. i By taing g x = x s and hx = x q in 3.7, where s, q R \ {0,, } are such that s q, we have δ sq i = q q q Ψ i x s, i {,..., 6}. s s s Ψ i x q ii If the inverse of the function g /h exists, then 3.7 implies that g Ψi g δ i =, i {,..., 6}. h Ψ i h 4. n-exponential convexity and log-convexity In this section first we will present some important definitions which will be useful further. In the sequel, let I be an open interval in R. The next four definitions are given in [9]. Definition 4.. A function f : I R is n-exponentially convex in the Jensen sense if xi x j ς i ς j f 0 i,j= holds for every ς i R and x i I i n. Definition 4.. A function f : I R is n-exponentially convex if it is n- exponentially convex in the Jensen sense and continuous on I. Definition 4.3. A function f : I R is exponentially convex in the Jensen sense if it is n-exponentially convex in the Jensen sense for all n N. Definition 4.4. A function f : I R is exponentially convex if it is exponentially convex in the Jensen sense and continuous. A log-convex function is defined as follows see [0, p. 7] : Definition 4.5. A function f : I 0, is said to be log-convex if log f is convex. Equivalently, f is log-convex if for all x, y I and for all λ [0, ], the inequality f λx λ y f λ x f λ y holds. If the inequality reverses, then f is said to be log-concave. Next we study the n-exponential convexity and log-convexity of the functions associated with the linear functionals Ψ i i = -6 as defined in Theorem 4.6. Let Λ = {f s : s I R} be a family of differentiable functions defined on [a, b] such that the function s [z 0, z, z, z 3 ; f s ] is n-exponentially convex in the Jensen sense on I for every four mutually distinct points z 0, z, z, z 3 [a, b]. Let Ψ i i = -6 be the linear functionals as defined in Then the following statements hold :

10 0 S. KHALID, D. PEČARIĆ. J. PEČARIĆ i The function s Ψ i f s is n-exponentially ] convex in the Jensen sense on m I and the matrix [Ψ i is positive semi-definite for all m N, f s j s j,= m n and s,..., s m I. Particularly, ] m det [Ψ i 0, m N, m n. f s j s j,= ii If the function s Ψ i f s is continuous on I, then it is n-exponentially convex on I. Proof. The proof is analogous to the proof of Theorem 3. in [3]. The following corollary is an immediate consequence of Theorem 4.6. Corollary 4.7. Let Λ = {f s : s I R} be a family of differentiable functions defined on [a, b] such that the function s [z 0, z, z, z 3 ; f s ] is exponentially convex in the Jensen sense on I for every four mutually distinct points z 0, z, z, z 3 [a, b]. Let Ψ i i = -6 be the linear functionals as defined in Then the following statements hold : i The function s Ψ i f s is] exponentially convex in the Jensen sense on I m and the matrix [Ψ i is positive semi-definite for all m N, f s j s j,= m n and s,..., s m I. Particularly, ] m det [Ψ i 0, m N, m n. f s j s j,= ii If the function s Ψ i f s is continuous on I, then it is exponentially convex on I. Corollary 4.8. Let Λ = {f s : s I R} be a family of differentiable functions defined on [a, b] such that the function s [z 0, z, z, z 3 ; f s ] is -exponentially convex in the Jensen sense on I for every four mutually distinct points z 0, z, z, z 3 [a, b]. Let Ψ i i = -6 be the linear functionals as defined in Further, assume that Ψ i f s i = -6 is strictly positive for f s Λ. Then the following statements hold : i If the function s Ψ i f s is continuous on I, then it is -exponentially convex on I and so it is log-convex on I and for r, s, t I such that r < s < t, we have [Ψ i f s ] t r [Ψ i f r ] ts [Ψ i f t] s r, i {,..., 6}, 4. nown as Lyapunov s inequality. If r < t < s or s < r < t, then the reversed inequalities hold in 4.. ii If the function s Ψ i f s is differentiable on I, then for every s, q, u, v I such that s u and q v, we have µ s,q Ψ i, Λ µ u,v Ψ i, Λ, i {,..., 6}, 4.

11 where ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS for f s, f q Λ. µ s,q Ψ i, Λ = Ψi f s sq, s q, Ψ i f q d ds exp Ψ i f s, s = q Ψ i f s Proof. The proof is analogous to the proof of the Corollary 3.3 in [3]. 4.3 Remar 4.9. Note that the results from Theorem 4.6, Corollary 4.7 and Corollary 4.8 still hold when two of the points z 0, z, z, z 3 [a, b] coincide, say z = z 0, for a family of differentiable functions f s such that the function s [z 0, z 0, z, z 3 ; f s ] is n-exponentially convex in the Jensen sense exponentially convex in the Jensen sense, log-convex in the Jensen sense on I, when three of the points z 0, z, z, z 3 [a, b] coincide, say z = z = z 0, for a family of differentiable functions f s such that the function s [z 0, z 0, z 0, z 3 ; f s ] is n-exponentially convex in the Jensen sense, when three of the points z 0, z, z, z 3 [a, b] coincide again, say z = z = z 0, for a family of twice differentiable functions f s such that the function s [z 0, z 0, z 0, z 3 ; f s ] is n-exponentially convex in the Jensen sense and furthermore, they still hold when all four points coincide for a family of thrice differentiable functions with the same property. There are several families of functions which fulfil the conditions of Theorem 4.6, Corollaries 4.7 and 4.8, and Remar 4.9 and so the results of these theorem and corollaries can be applied for them. Here we present an example for such a family of functions and for more examples see [3] and [4]. Example 4.0. Consider the family of functions defined by f s x = Λ = {f s : 0, R : s R}, s / {0,, }, log x, s = 0, x log x, s =, x log x, s =. x s sss Here d3 f dx 3 s x = x s3 = e s3 ln x > 0, which shows that f s is 3-convex for x > 0 and s d3 f dx 3 s x is exponentially convex by definition. In order to prove that s [z 0, z, z, z 3 ; f s ] is exponentially convex, it is enough to show that [ ] n j,= ς jς z 0, z, z, z 3 ; f s j s = [ z 0, z, z, z 3 ; ] n j,= ς jς f s j s 0, 4.4 for all n N, ς j, s j R, j {,..., n}. By Definition.5, inequality 4.4 will hold if Γ x := n j,= ς jς f s j s x is 3-convex. Since s d3 f dx 3 s x is

12 S. KHALID, D. PEČARIĆ. J. PEČARIĆ exponentially convex, that is j,= ς j ς f s j s 0, n N, ς j, s j R, j {,..., n}, which implies that Γ is 3-convex, inequality 4.4 is immediate. Now as s [z 0, z, z, z 3 ; f s ] is exponentially convex, s [z 0, z, z, z 3 ; f s ] is exponentially convex in the Jensen sense and by using Corollary 4.7, we have s Ψ i f s i = -6 is exponentially convex in the Jensen sense. Since these mappings are continuous, s Ψ i f s i = -6 is exponentially convex. If r, s, t R are such that r < s < t, then from 4. we have Ψ i f s [Ψ i f r ] ts t r [Ψi f t] s r t r, i {,..., 6}. 4.5 If r < t < s or s < r < t, then the reversed inequality holds in 4.5. Particularly, for i = and r, s, t R \ {0,, } such that r < s < t, we have s s s H s n,r,t i= s i r t a s C s,n,r,t C,n,r,t s sa s D C s r r r H r n,r,t i= i=,n,r,t Cs,n,r,t r i r t a r C r,n,r,t C,n,r,t r ra r D C r,n,r,t C r t t t t H t n,r,t i r t i= i r t i= ] ts t r,n,r,t a t C t,n,r,t C t,n,r,t ta t D C t,n,r,t C t,n,r,t In this case, µ s,q Ψ i, Λ i = -6 defined in 4.3 becomes µ s,q Ψ i, Λ = Ψi f s sq Ψ i f q exp Ψi f sf 0 Ψ i f s 3s 6s sss Ψi f0 exp exp exp 3 Ψ i f 0 Ψi f 0 f Ψ i f Ψi f 0 f 3 Ψ i f i= s i r t i r t ] s r t r, s q, r t, s / {0,, }., s = q / {0,, },, s = q = 0,, s = q =,, s = q =.

13 ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS 3 In particular for i =, we have s 3 Ψ f s = Ψ f 0 = Ψ f = Ψ f = s Hn,r,t s i r t i= a s C s,n,r,t C,n,r,t s s i= i r t s a s D C s,n,r,t,n,r,t Cs, s / {0,, }, C,n,r,t a s i= log C,n,r,t i= a s ir t ir t a s D a r t, C,n,r,t C,n,r,t i= i r t log i= i r t log a i r t i= i r t a i= a s C,n,r,t log C,n,r,t C,n,r,t log C,n,r,t H n,r,t a s C,n,r,t D log a s C,n,r,t i= i r t log i= a s, i r t a log i= a i r t i= i r t C,n,r,t log C,n,r,t C,n,r,t log C,n,r,t a s D C,n,r,t log C,n,r,t C,n,r,t log C,n,r,t a s D r t,

14 4 S. KHALID, D. PEČARIĆ. J. PEČARIĆ 4Ψ f0 = s 3 Ψ f s f 0 = Ψ f 0 f = a s log a i r t i= a s log a i r t i= log C,n,r,t log C,n,r,t a s Hn,r,t s i= a s D log C,n,r,t a s i= i= i r t C,n,r,t i r t s log s log a log C,n,r,t C,n,r,t a i= i= i r t, i r t C s,n,r,t log C,n,r,t C s,n,r,t log C,n,r,t a s D C s,n,r,t s log C,n,r,t a s D C s,n,r,t s log C,n,r,t, s / {0,, }, a s i= i r t log a s i r t log a i= a i r t i= i r t C,n,r,t log C,n,r,t C,n,r,t log C,n,r,t a s D log C,n,r,t log C,n,r,t a s D log C,n,r,t log C,n,r,t

15 and 4Ψ f 0 f = ON ZIPF-MANDELBROT ENTROPY AND 3-CONVEX FUNCTIONS 5 Hn,r,t i= a s a s i r t i= log i r t a log i= a i= i r t i r t C,n,r,t log C,n,r,t C,n,r,t log C,n,r,t a s D C,n,r,t log C,n,r,t log C,n,r,t a s D C,n,r,t log C,n,r,t log C,n,r,t, where s 3 : = s s s is the Pochhammer symbol. If Ψ i i = -6 is positive, then Theorem 3. applied for g = f s Λ and h = f q Λ yields that there exists δ i [a, b] such that i = Ψ i f s, i {,..., 6}. Ψ i f q δ sq Since the function δ i δ sq i min{a, b} i = -6 is invertible for s q, we have sq max{a, b}, i {,..., 6}, Ψi f s Ψ i f q which together with the fact that µ s,q Ψ i, Λ i = -6 is continuous, symmetric and monotonous by 4. shows that µ s,q Ψ i, Λ i = -6 is a mean. Acnowledgments. The research of the 3rd author is supported by the Ministry of Education and Science of the Russian Federation the Agreement number No. 0.a References. J. Jašetić, D. Pečarić, and J. Pečarić, Some properties of Zipf-Mandelbrot law and Hurwitz ξ-function, Math. Inequal. Appl. 08, no., C. Jardas, J. Pečarić, R. Roi, and N. Sarapa, On an inequality of Hardy-Littlewood-Pólya and some applications to entropies, Glas. Mat. Ser. III , no., S. Khalid, J. Pečarić, and M. Pralja, 3-convex functions and generalizations of an inequality of Hardy-Littlewood-Pólya, Glas. Mat. Ser. III , no., S. Khalid, J. Pečarić, and A. Vuelić, Refinements of the majorization theorems via Fin identity and related results, J. Classical. Anal. 7 05, no., M. A. Khan, D. Pečarić, and J. Pečarić, Bounds for Shannon and Zipf-mandelbrot entropies, Math. Methods Appl. Sci , no. 8, N. Latif, D. Pečarić, and J. Pečarić, Majorization, Csiszar divergence and Zipf-Mandelbrot law, J. Inequal. Appl. 07, Paper No. 97, 5 pp.

16 6 S. KHALID, D. PEČARIĆ. J. PEČARIĆ 7. M. G. Mann and C. Tsallis ed., Nonextensive entropy: Interdisciplinary applications, Santa Fe Institute Studies in the Sciences of Complexity. Oxford University Press, New Yor, M. A. Montemurro, Beyond the Zipf-Mandelbrot law in quantitative linguistics, Physica A , J. Pečarić and J. Perić, Improvements of the Giaccardi and the Petrović inequality and related Stolarsy type means, An. Univ. Craiova Ser. Mat. Inform. 39 0, no., J. Pečarić, F. Proschan, and Y. L. Tong, Convex functions, partial orderings, and statistical applications, Mathematics in Science and Engineering, 87. Academic Press, Inc., Boston, MA, 99. Department of Mathematics, COMSATS University Islamabad, Lahore Campus, Paistan. address: saadiahalid76@gmail.com; shalid@cuilahore.edu.p Catholic University of Croatia, Ilica 4, 0000 Zagreb, Croatia. address: gpecaric@yahoo.com 3 Rudn University, Miluho-Malaya str. 6798, Moscow, Russia. address: pecaric@element.hr