Entropy for Department of Physics, Chungbuk National University October 4, 2018
Entropy for A measure for the lack of information (ignorance): s i = log P i = log 1 P i. An average ignorance: S = k B i P is i = k B i P i log P i = k B log P i. We call S as entropy (Shannon s Entropy). k B : Boltzmann s constant.
Entropy for Maximum Entropy The prior probability distribution maximizes entropy (the average ignorance) while respecting macroscopic constraints. A natural constraint of normalization is i P i = 1. Applying the Lagrange multiplier method, [S + λ( i P i 1)] = 0, where stands for the derivative with respect to P. To be specific, [ S + λ( ] [ P i 1) = P i log P i + λ( ] P i 1) P P i i i = [ ] d log P i log P i P i + λ dp i i = i [ log P i 1 + λ] = 0.
Entropy for Maximum Entropy It leads log P i 1 + λ = 0. Finally, we get P i = e λ 1 = const. 1 Ω.
Entropy for Boltzmann s Entropy In an equilibrium state, the probability of i state is given by p i = 1 Ω. Then the entropy can be expressed as S = log P i = log 1 Ω = log Ω. We will see the meaning of entropy later.
Entropy for Boltzmann and Shannon Ludwig Boltzmann and Claude Shannon
Entropy for Fixed energy and the number of particles. We define Ω is the accessible volume in phase space with E H E + E. Ω = dp dq, E H E+ E where P = (p 1, p 2,, p 3N ) and Q = (q 1, q 2,, q 3N ). The probability and the expectation value is then P = 1 Ω, O = 1 Ω E H E+ E O(P, Q)dP dq. Conceptually the microcanonical ensemble approach is extremely simple, in practice it is not so easy.
Entropy for Ideal Gas in a box The Hamiltonian of N-ideal gas molecules in a box with volume V (i.e, helium atoms at high temperatures and low densities): H = 3N i p 2 i 2m + V (x i). (1) where the potential V (x i ) is given by 0 if x V and otherwise V =. The number (or volume) of states is Ω = dp dq = E H E+ E dq dp E H E+ E = Ω Q Ω P = V N Ω(P ), since there is V = 0 in a box.
Entropy for Momentum Space The constant energy surface is a sphere in 3N-dimensional space, 3N i=1 p 2 i = 2mE = R 2, where radius R = 2mE. If we define Σ(E) as the volume of region H E, Ω P = dp E H E+ E = Σ(E + E) Σ(E). Then, the volume Σ can be computed as Σ(E) = dp H E = 3N i=1 p2 i 2mE dp.
Entropy for The volume of n-dimensional sphere with radius R is V N (R) = dx 1 dx 2 dx N x2 i R2 = R N dy 1 dy 2 dy N Ni y2i 1 = R N C N, where C N = N i y2 i 1 dy 1dy 2 dy N. N i
Entropy for [ N N π = e dx] x2 = = dv N (R)e (x2 1 +x2 2 + +x2 N ) dx 1 dx 2 dx N e (x2 1 +x2 2 + +x2 N ) where dv N (R) = dx 1 dx 2 dx N = NR N 1 C N dr. In n-dimensional polar coordinates, N π = NR N 1 C N e R2 dr 0 = C N N 2 0 ( ) X N 2 1 e X N N dx = C N 2 Γ 2 = C N Γ (N/2 + 1) = C N (N/2)!, where X = R 2 and dx = 2RdR. Therefore, C N = πn/2 (N/2)!, V N(R) = R N πn/2 (N/2)!.
Entropy for Momentum Space The volume of region H E where Ω P = Σ(E + E) Σ(E) Σ(E + E) Σ(E) = E E E dσ(e) de, dσ(e) de = d π 3N/2 (2mE) 3N/2 de (3N/2)! = 3N 2 π 3N/2 (2mE) 3N/2 1 (3N/2)! = (2πm)3N/2 E 3N/2 1. (3N/2 1)! (2)
Entropy for Number of States & Entropy The total number (volume) of states is Ω = V N E (2πm)3N/2 E 3N/2 1. (3N/2 1)! (3) The entropy is then k B log Ω, and log Ω = log V N + log E + 3N 2 log(2πm) ( ) 3N + 2 1 log E log(3n/2 1)! N log V + 3N 2 log(2πme) (3N/2) log(3n/2) + (3N/2).
Entropy for Entropy for Ideal Gas The entropy is S Nk B log V + 3Nk B 2 1 T = log(2πme) (3Nk B /2) log(3n/2) + (3Nk B /2). P T = ( ) S E ( ) S V V,N E,N = 3Nk B 2E, = Nk B V. Therefore, we can derive equipartition theorem and the equation of state for the ideal gas, E = 3 2 Nk BT, P V = Nk B T.
Entropy for Gibbs Paradox Compare 2N particles in two boxes with volume V and 2N particles in a box with volume 2V : S 1 = 2k B log V N, S 2 = k B log(2v ) 2N.
Entropy for Gibbs Paradox & Gibbs factor All particles are perfectly identical or indistinsuishable: S 1 = 2k B log V N /N!, S 2 = k B log(2v ) 2N /(2N)!.
Entropy for Planck Constant h has the unit of momentum distance. The uncertainty principle: δxδp h. Applying two factors, we find the Sackur-Tetrode formula: Ω = Ω crude /(N!h 3n ). [ ( V 4πmE S(N, V, E) = Nk B log N 3Nh 2 ) 3/2 ] + Nk B 5 2. (4)