Nongeneralizability of Tsallis Entropy by means of Kolmogorov-Nagumo averages under pseudo-additivity arxiv:math-ph/0505078v1 30 May 2005 Ambedkar Dukkipati, 1 M. Narsimha Murty,,2 Shalabh Bhatnagar 3 Department of Computer Science and Automation, Indian Institute of Science, Bangalore-560012, India. Abstract As additivity is a characteristic property of the classical information measure, Shannonentropy, pseudo-additivityoftheformx+ q y = x+y+(1 q)xy isacharacteristic property of Tsallis entropy. Rényi in [1] generalized Shannon entropy by means of Kolmogorov-Nagumo averages, by imposing additivity as a constraint. In this paper we show that there exists no generalization for Tsallis entropy, by means of Kolmogorov-Nagumo averages, which preserves the pseudo-additivity. Key words: Kolmogorov-Nagumo averages, Rényi entropy, Tsallis entropy PACS: 65.40.Gr, 89.70.+c, 02.70.Rr The starting point of the theory of generalized measures of information is due to Alfred Rényi [1,2], now having an extensive literature. Rényi introduced generalized information measure, known as α-entropy or Rényi entropy, which is derived by replacing linear averaging in Shannon entropy with the generalized averages, in particular Kolmogorov-Nagumo averages and by posing the additivity of the information measures. On the other hand, however, Tsallis in [3] proposed a non-logarithmic generalization of entropic measure, known as q-entropy or Tsallis entropy which is considered as a useful measure in describing thermostatistical properties of a certain class of physical systems that entail long-range interactions, long-term memories and multi-fractal systems. corresponding author 1 ambedkar@csa.iisc.ernet.in 2 mnm@csa.iisc.ernet.in (Tel:+91-80-22932779) 3 shalabh@csa.iisc.ernet.in Preprint submitted to Elsevier Science 2 October 2018
Tsallis and Rényi entropy measures are two possible different generalization of the Shannon entropy but are not generalizations of each other. To understand these generalizations, the so called Hartley information measure [4], of a single stochastic event plays a fundamental role. It can be interpreted either as a measure of how unexpected the event is, or as a measure of the information yielded by the event. In a system with the finite configuration space x = {x k } n, Hartley information measure of a single event with probability is defined as H( ) = ln 1, k = 1,...n. (1) Hartley information measure satisfies: (1) H is nonnegative: H( ) 0 (2) H is additive: H(p i p j ) = H(p i )+H(p j ) (3) H is normalized: H( 1 ) = 1. These are 2 both necessary and sufficient [5]. Hartley information measure can be viewed as a random variable and hence we use the notation H = (H 1,...H n ). Shannon entropy is defined as an average Hartley information: [6] n n S(p) = H = H k = ln. (2) The characteristic property of Shannon entropy is additivity, i.e., for two independent probability distributions p and r we have S(pr) = S(p)+S(r), (3) where pr is the joint distribution of p and r. Rényi in [1,2] used a well known idea in mathematics that the linear mean, though most widely used, is not the only possible way of averaging, but one can define the mean with respect to an arbitrary function [7,8] to generalize the Shannon entropy. In the general theory of means, a mean of x = (x 1,x 2,...,x n ) with respect to a probability distribution p = (p 1,p 2,...,p n ) is defined as [7] [ W ] x ψ = ψ 1 ψ(x k ), (4) where ψ is continuous and strictly monotonic (increasing or decreasing) in which case it has an inverse ψ 1 which satisfies the same conditions; ψ is generally called the Kolmogorov-Nagumo function associated with the mean 2
(4) 4. If, in particular, ψ is linear, then (4) reduces to the expression of linear averaging, x = n x k. The mean of form (4) is also referred as quasilinear mean. In the definition of Shannon entropy (2), if the linear average of Hartley information is replaced with the generalized average of the form(4), the information corresponding to the probability distribution p with respect to KN-function ψ will be )] [ S ψ (p) = ψ 1 n ] ψ (ln 1pk = ψ 1 ψ(h k ), (5) where H = (H 1,...H n ) is the Hartley information measure associated with p. If we impose the constraint of additivity in S ψ, then ψ should satisfy [1] x+c ψ +C, (6) for any x = (x 1,...,x n ) and a constant C. Rényi employed the above formalism to define an one-parameter family of measures of information (α-entropies) S α = 1 ( n ) 1 α ln p α k, (7) where the KN-function ψ is chosen in (5) as ψ(x) = e (1 α)x, choice motivated by well known theorem in the theory of means (Theorem 89, [7]) that (6) can hold only for linear and exponential functions. Rényi entropy is an oneparametergeneralizationofshannonentropyinthesensethat,thelimitα 1 in (7) retrieves Shannon entropy. On the other hand, Tsallis entropy is given by [3] S q (p) = 1 n p q k q 1, (8) where q is called nonextensive index (q is positive in order to ensure the concavity of S q ). Tsallis entropy too, like Rényi entropy, is an one-parameter 4 A. N. Kolmogorov [9] and M. Nagumo [10] were the firstto investigate the characteristic properties of general means. They considered only the case of equal weights; the generalization to arbitrary weights and the characterization of means of form (4) are due to B. de Finetti [11], B. Jessen [12], T. Kitagawa [13], J. Aczél [8] and many others 3
generalization of Shannon entropy in the sense that q 1 in (8) retrieves Shannon entropy. The entropic index q characterizes the degree of nonextensivity reflected in the pseudo-additivity property S q (pr) = S q (p)+ q S q (r) = S q (p)+s q (r)+(1 q)s q (p)s q (r), (9) where p and r are independent probability distributions. Though the derivation of Tsallis entropy, when it was proposed in 1988 is slightly different, one can understand this generalization using q-logarithm (see 11) function: where one would first generalize, logarithm in the Hartley information with q-logarithm and define q-hartley information measure H = ( H 1,..., H n ) as H k = H( ) = ln q 1, k = 1,...n, (10) where q-logarithm is defined as ln q (x) = x1 q 1 1 q, (11) which satisfies pseudo-additivity ln q (xy) = ln q x+ q ln q y and in the limit q 1 we have ln q lnx. Tsallis entropy (8) defined as the average of q-hartley information i.e [14]: S q (p) = H 1 = ln q. (12) Now a natural question arises whether one could generalize Tsallis entropy in the similar lines of derivation of Rényi entropy i.e., by replacing linear average in (12) by KN-averages under the pseudo-additivity. The class of information measures that represent the KN-average of q-hartley information measure is written as [ 1 W ( )] [ S ψ (p) = ln q = ψ 1 1 ψ ln q = ψ 1 W ψ ( H ) ] k.(13) ψ By the pseudo-additivity constraint, ψ should satisfy S ψ (pr) = S ψ (p)+ q S ψ (r) (14) or 4
( ) n n ψ 1 1 p i r j ψ ln q i=1 j=1 p i r j ( )] ( ) = ψ 1 1 n p i ψ ln q + q ψ 1 1 r j ψ ln q, (15) i=1 p i j=1 r j where p and r are independent probability distributions and pr denotes the joint probability distribution of p and r. Equivalently, we need ψ 1 n n p i r j ψ ( H j) p i + H r q i=1 j=1 = ψ 1 p i ψ ( H p ) ] n i + q ψ 1 r j ψ ( H j) r, (16) i=1 j=1 where H p and H r represents the q-hartley information of probability distributions p and r respectively. Note that (16) must hold for arbitrary finite discrete probability distributions p i and r j and for arbitrary numbers H and H r k. If we choose H r k = J independently of j then (16) yields that ψ 1 ψ ( H + q J )] [ = ψ 1 n ψ ( H k) ] p + q J (17) In general ψ satisfies (17) only if ψ satisfies x+ q C ψ + q C, (18) for any x = (x 1,...,x n ), which can be rearranged as x+c +(1 q)xc ψ +C +(1 q) x ψ C (19) or (1+(1 q)c)x+c ψ = (1+(1 q)c) x ψ +C. (20) Since q is independent of other quantities, ψ should satisfy the equation of form (By B = (1+(1 q)c)) Bx+C ψ = B x ψ +C. (21) 5
Finally ψ must satisfy x+c ψ +C (22) and Bx ψ = B x ψ, (23) for any x = (x 1,...,x n ) and for any constants B and C, for S ψ to preserve the pseudo-additivity. From the generalized theory of means (22) is satisfied only when ψ is linear or exponential, but the requirement (23) is satisfied only when ψ is linear and it is not satisfied when ψ is exponential. Hence ψ is linear in which case (13) is nothing but Tsallis entropy. This establishes the nongeneralizability of Tsallis entropy by means of KNaverages under pseudo-additivity. References [1] A. Rényi, Some fundamental questions of information theory, MTA III. Oszt. Közl. 10 (1960) 251 282, (reprinted in [15], pp. 256-552). [2] A. Rényi, On measures of entropy and information, in: Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, Berkeley-Los Angeles, 1961, pp. 547 561, (reprinted in [15], pp. 565-580). [3] C. Tsallis, Possible generalization of Boltzmann Gibbs statistics, J. Stat. Phys. 52 (1988) 479. [4] R. V. L. Hartley, Transmission of information, Bell System Technical Journal 7 (1928) 535. [5] J. Aczel, Z. Daroczy, On Measures of Information and Their Characterization, Academic Press, New York, 1975. [6] C. E. Shannon, A mathematical theory of communication, Bell System Technical Journal 27 (1948) 379. [7] G. H. Hardy, J. E. Littlewood, G. Pólya, Inequalities, Cambridge, 1934. [8] J. Aczél, On mean values, Bull. Amer. Math. Soc. 54 (1948) 392 400. 6
[9] A. Kolmogorov, Sur la notion de la moyenne, Atti della R. Accademia Nazionale dei Lincei 12 (1930) 388 391. [10] M.Nagumo,Übereineklassevonmittelwerte, JapaneseJournalofMathematics 7 (1930) 71 79. [11] B. de Finetti, Sul concetto di media, Giornale di Istituto Italiano dei Attuarii 2 (1931) 369 396. [12] B. Jessen, Über die verallgemeinerung des arthmetischen mittels, Acta Sci. Math. 5 (1931) 108 116. [13] T. Kitagawa, On some class of weighted means, Proceedings Physico- Mathematical Society of Japan 16 (1934) 117 126. [14] C. Tsallis, F. Baldovin, R. Cerbino, P. Pierobon, Introduction to nonextensive statistical mechanics and thermodynamics, in: F. Mallamace, H. E. Stanley (Eds.), The Physics of Complex Systems: New Advances and Perspectives, Vol. 155, Enrico Fermi International School of Physics, 2003. [15] P. Turán (Ed.), Selected Papers of Alfréd Rényi, Akademia Kiado, Budapest, 1976. 7