From Multiplicity to Entropy

Size: px

Start display at page:

Download "From Multiplicity to Entropy"

Dylan Noel Carroll
5 years ago
Views:

From Multiplicity to Entropy R Hanel, S Thurner, and M. Gell-Mann http://www.

1 From Multiplicity to Entropy R Hanel, S Thurner, and M. Gell-Mann Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

2 When I first visited CBPF... Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

3 Introduction What is the probability of finding a probability distribution function? In other words: What is the probability to find a histogram? Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

4 What is the probability to find a histogram? Sequence x = (x 1, x 2,, x N ) with x n {1, 2} (coin tosses) 1q 0 1 q0 2 1q 1 1 q0 2 1q 0 1 q1 2 1q 2 1 q0 2 2q 1 1 q1 2 1q 0 1 q2 2 1q 3 1 q0 2 3q 2 1 q1 2 3q 1 1 q2 2 1q 1 1 q3 2 1q 4 1 q0 2 4q 3 1 q1 2 6q 2 1 q2 2 4q 1 1 q3 2 1q 0 1 q4 2 q 1 and q 2... probabilities to toss 1 (heads) or 2 (tails) in a trial x n. k = (k 1, k 2 )... is the histogram of x; i.e. k 1 (k 2 )... number of times 1 (2) appears in x. Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

5 What is the probability to find a histogram? Binomial The probability to find the histogram k is: P(k; q) = N! k 1!k 2! qk 1 1 qk 2 2 N... length of sequence x 2 states x n {1, 2} Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

6 What is the probability to find a histogram? Multinomial The probability to find the histogram k is: P(k; q) = N! k 1!k 2! k W! qk 1 1 qk 2 2 qk W W N... length of sequence x W states x n {1, 2,, W } Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

7 What is the probability to find a histogram? Multinomial The probability to find the histogram k is: P(k; q) = N! k 1!k 2! k W! }{{} multiplicity M(k) q k 1 1 qk 2 2 qk W }{{ W} probability G(k;q) Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

8 What is the probability to find a histogram? Multinomial We make two observations... Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

9 Observation 1: The factorization property Factorization P(k; q) = M(k) G(k; q) Multiplicity M(k) does not depend on q! All dependencies on q are captured by G(k; q) Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

10 Observation 1: The factorization property Take the log on both sides and scale with 1/N... P(k; q) = M(k) G(k; q) 1 log P(k; q) = 1 log M(k) + 1 log G(k; q) N N N Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

11 Observation 2: From factorization property to MEP ( 1 N log M(k) = 1 N log 1 N log ( N! k 1!k 2! k W! ) N N k k 1 1 k k 2 2 k k W W ) Stirling = log N 1 N W i=1 k i log k i = W k i i=1 N log k i N = W i=1 p i log p i frequencies p i = k i N = S Shannon [p] Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

12 Observation 2: From factorization property to MEP ( ) 1 N log G(k; q) = 1 N log q k 1 1 qk 2 2 qk W W = 1 N W i=1 k i log q i = W i=1 p i log q i = S cross [p q] Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

13 Observation 2: From factorization property to MEP The identified terms: 1 log P(k; q) = 1 log M(k) + 1 log G(k; q) N N N = W i=1 p i log(p i ) + W i=1 p i log(q i ) Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

14 The Maximum Entropy Principle Variable transformation... q i = exp( α βε i ) W 1 N log P(k; q) = p i log(p i ) i=1 }{{} constraint independent α W p i β i=1 W p i ε i i=1 } {{ } constraints What is the most likely histogram k? for fixed N i=1 0 = k i log P(k; q) ( ) 0 = W W W p i log(p i ) α p i β p i ε i p i i=1 i=1 Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

15 Principle Principle If a factorization exists then there exists a MEP with P(k; q) = M(k)G(k; q) 1 1 log P(k; q) = log M(k) + 1 log G(k; q) φ(n) φ(n) φ(n) }{{}}{{} generalized entropy constraints Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

16 When does factorization exist? When does factorization exist? There is little one can say in general but... focus on processes out of equilibrium that are aging, non-markovian, path-dependent Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

17 Aging & memory When do factorizations of P exist? process remembers its entire past in terms of the histogram k Consider growing sequences x = (x 1,, x N ) x = (x 1,, x N, x N+1 ) ˆp(x N+1 k; q)... conditional probability to find x N+1 given histogram k of x IF process follows ˆp(x N+1 k; q) and has stable solutions d dn k i(n) = ˆp(i k; q) There exists a recursive definition of how M(k) (similar to the multinomial coefficients) Asymptotically M(k) is given by a deformed multinomial... Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

18 Deformed factorials and deformed multinomials N! u M u,t (k) ( ) i NT ki N! u }{{} Deformed multinomial T monotonic with T (0) = 0, T (1) = 1, N! u deformed multinomials lead to trace-form entropies S[p] = 1 φ(n) log M u,t (k) =... = N u(n) n=1 } {{ } Deformed factorial W i=1 pi 0 dz Λ(z) Λ(z) = 1 a ( [ ] ) N T (z) log u(nt (z)) log λ φ(n) Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

19 General solution for deformed multinomials Asymptotic identity for constants a and λ Λ(z) = 1 ( [ ] N T (z) log u(nt (z)) a φ(n) ) log λ Λ(z) does not depend on N separation of variables with separation constant ν Λ(z) = T (z)t (z) ν T (1) T (1)+νT (1) 2 u(n) = λ(nν ) 1 λ 1...q shifted factorials (ν = 1) φ(n) = N 1+ν Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

20 Happy Birthday The MEP with Tsallis entropy derived for a class of aging processes for the simplest (and probably the generic) case T (z) = z... Λ(z) = T (z)t (z) ν T (1) T (1) + νt (1) 2 = zν 1 ν and S[p] = W i=1 pi where Q 1 + ν and K a constant 0 dzλ(z) = K 1 W i=1 pq i Q 1 Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

21 Remark on Extensive vs MEP Are the extensive (c, d)-entropies of a process equivalent to the (c, d)-entropy of the MEP? No they are NOT but they are related d extensive = 1 c MEP R. Hanel & S. Thurner, Generalized (c,d)-entropy and aging random walks, arxiv: Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

22 Example Consider for example the conditional probability ˆp(x N+1 = i k; q) = 1 Z (k) q i W j=i+1 plug p(i k; q) into the recursive definition of M(k) [ do some not so easy math ]... λ (k ν j ) Use S[p] = 1 φ(n) log M(k) and solve the MEP to get theoretical predictions for the p i for fixed N generate sequences x of length N on the computer (with q i = 1/W ) and produce the histograms k i of the sequences (full lines) Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

23 Example ˆp(i k; q) = 1 Z (k) q i W j=i+1 λ (k ν j ) 10 1 ν=0.25, λ=1.1 α,β 2 1 α β p i simul N=1000 N=5000 N=10000 theory N=1000 N=5000 N= N ε i =i (i) Simulation: generate sequences x on the computer with q i = 1/W and W = 50; (ii) MEP prediction: p i = exp q ( α βε i ) with q = 1 + ν and ε i = i. Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

24 FIN Happy birthday, Constantino! Hanel (CeMSIIS2013) Aging & Entropy Tsallis Birthday / 24

LECTURE 2 Information theory for complex systems

LECTURE 2 Information theory for complex systems Stefan Thurner www.complex-systems.meduniwien.ac.at www.santafe.edu rio summer school oct 28 2015 Lecture 1: What are CS? Lecture 2: Basics in information