PHYSICAL REVIE E 66, 4634 22 Stability of Tsallis entropy and instabilities of Rényi and normalized Tsallis entropies: A basis for -exponential distributions Sumiyoshi Abe Institute of Physics, University of Tsukuba, Ibaraki 35-857, Japan Received 6 June 22; published 24 October 22 The -exponential distributions, which are generalizations of the Zipf-Mandelbrot power-law distribution, are freuently encountered in complex systems at their stationary states. From the viewpoint of the principle of maximum entropy, they can apparently be derived from three different generalized entropies: the Rényi entropy, the Tsallis entropy, and the normalized Tsallis entropy. Accordingly, mere fittings of observed data by the -exponential distributions do not lead to identification of the correct physical entropy. Here, stabilities of these entropies, i.e., their behaviors under arbitrary small deformation of a distribution, are examined. It is shown that, among the three, the Tsallis entropy is stable and can provide an entropic basis for the -exponential distributions, whereas the others are unstable and cannot represent any experimentally observable uantities. DOI:.3/PhysRevE.66.4634 PACS numbers: 65.4.Gr, 5.2.y, 5.9.m, 2.5.r I. INTRODUCTION It is known 3 that there are a number of complex systems whose statistical properties at the stationary states are well described by the -exponential distributions, which are generalizations of the Zipf-Mandelbrot power-law distribution 4. The -exponential distributions are anomalous distributions from the viewpoint of conventional statistical mechanics characterized by Boltzmann s exponential factor. Since it is so freuently observed in nature, it is important to develop bases for such distributions. In this context, we wish to mention that uite recently the -exponential factor has been obtained for the logistic map at the edge of chaos by the renormalization-group method as well as by the Pesin euality for the generalized Kolmogorov-Sinai entropy and the generalized Lyapunov exponent 5. There, the value of the entropic index has been calculated analytically. The explicit form of the -exponential distribution is the following: p i e Q i i,2,...,, Z the -exponential distribution to the Boltzmann-Gibbs- Jaynes exponential distribution. Following Gibbs procedure, one may also wish to derive the -exponential distribution from the stationarity condition on a certain generalized entropy. Such an entropy is found to be not uniue, however. There exist three known different entropies that are maximized by the -exponential distribution under the constraint on the normalized -expectation value of Q. This can be seen as follows. Consider the functional J p;,s J p p i i i P i Q i Q JR,T,NT. and are the Lagrange multipliers associated with the normalization condition on the basic distribution, p i i,2,...,, and the normalized -expectation value of Q, i P i Q i Q, where P i is the escort distribution 6 defined by 4 Z i e Q i, where is the number of accessible microscopic states of a system under consideration, Q i is the ith value of a physical uantity Q, is a factor related to the Lagrange multiplier, and e (t) is the -exponential function defined by e t t/ t t. is a positive real number termed the entropic index. This distribution has the cutoff at Q i,max /() if, whereas it is euivalent to the Zipf-Mandelbrot-type asymptotic power-law distribution with the exponent /( ) if. In the limit, the -exponential function converges to the ordinary exponential function and so does 2 3 P i j p i p j. The three generalized entropies are listed as follows: S R p ln i p i, S T p p i, i S NT p i p i, 5 6 7 8 63-65X/22/664/46346/$2. 66 4634-22 The American Physical Society
SUMIYOSHI ABE which are the Rényi entropy 7, the Tsallis entropy 8, and the normalized Tsallis entropy 9,, respectively. These are connected to each other in the obvious ways, and all converge to the Boltzmann-Gibbs-Shannon entropy in the limit : lim S R p lim i S T p lims NT psp p i ln p i. For a statistically independent bipartite system, A,B, these entropies satisfy S J A,BS J AS J B J S J AS J B, where R, T, NT. 9 2 3 Thus, the Rényi entropy is additive, whereas the Tsallis and normalized Tsallis entropies are nonadditive. The Rényi entropy is conventionally used for the definition of the generalized dimension in multifractals 6, and the Tsallis entropy plays a central role in nonextensive statistical mechanics 3. Variation of (J) with respect to p i gives rise to the following stationary distribution: PHYSICAL REVIE E 66, 4634 22 c J i p i J. 9 (J) Also, in the above expressions of s, Q stands for the normalized -expectation value of Q with respect to p (J) i in E. 4. Here, it is worth mentioning that, as long as the -exponential distribution is concerned, the expectation value has to be defined in terms of the escort distribution as in E. 4, since only in this case can the principle of maximum generalized entropy be consistent with the principle of eual a priori probability. In the limit, E. 4 tends to the Boltzmann-Gibbs-Jaynes exponential distribution and the power-law distributions of Q cannot be obtained as long as they are based on the principle of maximum entropy with the constraint linear in Q. Thus, in fact, the Rényi, Tsallis, and normalized Tsallis entropies all lead to the -exponential distributions of the same type. In other words, mere fittings of observed data by the -exponential distributions do not tell us anything about which type the underlying physical entropy is. In this respect, it should be noted that, in Ref. 2, Lesche has presented a counterexample showing instability of the Rényi entropy. In this paper, we show that the Rényi and normalized Tsallis entropies are unstable under small deformation of a distribution and therefore cannot represent experimentally observable uantities, whereas the Tsallis entropy is stable and can provide an entropic basis for the -exponential distributions. The discussion is general and is independent of any stationary properties. The paper is organized as follows. In Sec. II, the rigorous definition of stability of a statistical uantity is given. In Sec. III, instability of the normalized Tsallis entropy as well as the Rényi entropy is shown. In Sec. IV, a general proof is established for stability of the Tsallis entropy. Section V is devoted to a conclusion. p J i e J Q i, Z J J Z J J i e J Q i, where s are given by R R Q, T c T T Q, 4 5 6 7 II. OBSERVABILITY AND STABILITY Consider a statistical uantity CCp, which has its maximum value C max Cp is said to be stable if the amount of its change under an arbitrary small deformation of the distribution remains small. Any observable uantities have to be stable, since otherwise their values cannot be experimentally reproducible. Let us measure the size of deformation from p i i,2,..., to p i i,2,..., by the l norm: pp p i p i. i 2 Note that this uantity should be independent of. Then, an observable uantity, Cp, has to possess the following property 2: NT /c NT NT Q, 8 respectively, provided that, in Es. 7 and 8, we have set for arbitrary values of. pp CpCp C max 2 4634-2
STABILITY OF TSALLIS ENTROPY AND... III. INSTABILITIES OF RÉNYI AND NORMALIZED TSALLIS ENTROPIES IN THE THERMODYNAMIC LIMIT In this section, we discuss instabilities of the Rényi and normalized Tsallis entropies by using a counterexample which violates the condition in E. 2. First of all, we recall that Rényi, Tsallis, and normalized Tsallis entropies take their maximum values for the euiprobability p i / (i,2,...,): R S,max ln, T S,max ln, S NT,max ln. Here, ln x stands for the -logarithmic function defined by 22 23 24 ln x x x, 25 which is the inverse function of the -exponential function and converges to the ordinary logarithmic function in the limit. The deformation of a distribution to be examined is given as follows 2. For, For, p i i, p i 2 p i 2. 26 p i i, p i 2 p i 2 i. 27 Clearly this preserves the normalization condition. In both the cases of and, the l norm is seen to be pp. 28 Also, from Es. 26 and 27, it is immediate to obtain the following. For, i For, p i, i i i PHYSICAL REVIE E 66, 4634 22 p i 2 2. 29 p i, p i 2 2. A. Rényi entropy 3 The following discussion about instability of the Rényi entropy can be found in Ref. 2, but we present it here in order to make the discussion self-contained. Using Es. 29 and 3 in E. 6, we find the following. For, For, S R p, S R p R S,max ln 2 2 2, S R ps R p ln 2. 32 ln S R ln, S R p S R ps R pln R S,max 2 ln 2 ln 2 2, 3 33 ln. 34 Therefore, the condition in E. 2 is violated. 4634-3
SUMIYOSHI ABE PHYSICAL REVIE E 66, 4634 22 Using Es. 29 and 3 in E. 7, we find the following. For, S T p, S T p S T ps T p T S,max B. Tsallis entropy 2 2 2 2, 2. 35 36 For, S T pln, S T p S T ps T p T S,max 2 2 2, 2 2. 37 38 Therefore, if is taken to be 2 /, the condition in E. 2 is satisfied. Using Es. 29 and 3 in E. 8, we find the following. For, For, C. Normalized Tsallis entropy S NT p, S NT p S NT ps NT p NT S,max 2 2 2 2 S NT pln, S NT p S NT ps NT p NT S,max, 39. 4 2 2 2 2, 4. 42 Therefore, as in the case of the Rényi entropy, the condition in E. 2 is violated. The results in Es. 32, 34, 4, and 42 mean that the Rényi and normalized Tsallis entropies with overestimate a large number of occupied states even if their overall probability is so small that they are irrelevant and those with overestimate a high peak of probability. Thus, among the three, there is a possibility only for the Tsallis entropy to be observable. 4634-4
STABILITY OF TSALLIS ENTROPY AND... IV. STABILITY OF TSALLIS ENTROPY Stability of the Boltzmann-Gibbs-Shannon entropy has been shown in Ref. 2. Here, we prove stability of the Tsallis entropy by generalizing the discussion in Ref. 2. Let us define the following uantity: A p;t i p i, 43 e t where t is a positive parameter and the symbol (x) means x maxx,. The following will be useful later: x xx, x y xy, 44 45 46 where (x) is the Heaviside unit step function defined by (x) for x and (x) for x. The uantity in E. 43 has several interesting properties. From E. 46, it immediately follows that Using the relation we have i A p;ta p;tpp. p i e t i e t p i, e t A p;t. In particular, if tln, then E. 49 becomes e t A p;t. 47 48 49 5 The same is true for another distribution p i i,2,...,, that is, A p;t e t. Adding Es. 5 and 5, wefind 5 t maxdt A p;t i t maxdt t maxdt e t. p i e t p i e t The second term on the right-hand side gives t maxdt e t. 53 54 On the other hand, noting ln (/p i )t max, the integral in the first term is calculated as follows: i t maxdt i p i e t i p i ln /p i dt p i p i p i i. e t e t Therefore, we obtain t maxdt A p;t S T p, or, conversely, PHYSICAL REVIE E 66, 4634 22 S T p t maxdt A p;t. 55 56 57 Now, using the representation in E. 57 and taking a satisfying ln at max, we have S T ps T p t maxdt A p;ta p;t t maxdta p;ta p;t A p;ta p;t e t tln. 52 aln dta p;ta p;t In the limit t t max with t max (), /() () see E. 3, e (t) diverges and therefore A p;t) tends to unity. So, the integral t max dt A p;t) may converge. This is in fact the case. Using E. 45, this integral is written as t max dta p;ta p;t. aln From E. 47, the first integral is found to satisfy 58 4634-5
SUMIYOSHI ABE PHYSICAL REVIE E 66, 4634 22 aln dta p;ta p;tpp aln. 59 Likewise, from E. 52, the second integral is evaluated as aln ln, pp E. 57 is reexpressed as follows: 62 t max dta p;ta p;t aln t max dt aln e t e aln. 6 S T ps T ppp ln pp pp. 63 Therefore, we have S T ps T ppp aln e aln. 6 This ineuality holds for any values of a satisfying ln at max. Evaluating the minimum of the right-hand side, which is realized when Using the euality ln (y/x)x (ln yln x), we further obtain S T ps T ppp ln pp from which we find ln pp, 64 S T ps T p T S,max pp pp ln pp ln pp pp. 65 Therefore, taking pp / () or p p / (), we see that the condition in E. 2 is satisfied by the Tsallis entropy. The above discussion holds for, and so, as a simple byproduct, stability of the Boltzmann-Gibbs-Shannon entropy 2 corresponding to the limit also of the Rényi and normalized Tsallis entropies is reestablished. V. CONCLUSION e have shown that among the Rényi, Tsallis, and normalized Tsallis entropies, only the Tsallis entropy is stable and can give rise to experimentally observable uantities. Therefore, it is the Tsallis entropy on which the ubiuitous -exponential distributions have their basis. A remaining important and hard uestion is whether or not the Tsallis entropy is the uniue generalized entropy. Regarding this point, we wish to mention that there are some affirmative points: there exists a set of axioms and the uniueness theorem for the Tsallis entropy 3, and the structure of nonadditivity in E. is essential from the viewpoint of the zeroth law of thermodynamics 4,5. Nonextensive Statistical Mechanics and Its Applications, edited by S. Abe and Y. Okamoto, Lecture Notes in Physics Vol. 56 Springer-Verlag, Heidelberg, 2. 2 Special issue of Physica A 35, Nos. /2 22, edited by G. Kaniadakis, M. Lissia, and A. Rapisarda. 3 Nonextensive Entropy Interdisciplinary Applications, edited by M. Gell-Mann and C. Tsallis Oxford University Press, Oxford, in preparation. 4 B. B. Mandelbrot, The Fractal Geometry of Nature Freeman, New York, 983. 5 F. Baldovin and A. Robledo, Phys. Rev. E to be published, e-print cond-mat/2537. 6 C. Beck and F. Schlögl, Thermodynamics of Chaotic Systems: An Introduction Cambridge University Press, Cambridge, 993. 7 A. Rényi, Probability Theory North-Holland, Amsterdam, 97. 8 C. Tsallis, J. Stat. Phys. 52, 479 988. 9 P. T. Landsberg and V. Vedral, Phys. Lett. A 247, 2998. A. K. Rajagopal and S. Abe, Phys. Rev. Lett. 83, 7 999. S. Abe and A. K. Rajagopal, J. Phys. A 33, 8733 2. 2 B. Lesche, J. Stat. Phys. 27, 49 982. 3 S. Abe, Phys. Lett. A 27, 742. 4 S. Abe, Phys. Rev. E 63, 65 2. 5 S. Abe, Physica A 35, 6222. 4634-6