Mathematical Structures of Statistical Mechanics: from equilibrium to nonequilibrium and beyond Hao Ge Beijing International Center for Mathematical Research and Biodynamic Optical Imaging Center Peking University, China
Equilibrium statistical mechanics and maximum entropy principle
Microcanonical ensemble and equal probability priori In statistical physics, the microcanonical ensemble refers to an isolated system, where all the possible macrostates of the system have the same energy; The probability for the system to be in any given microstate is the same; Khinchin was the first mathematician to make this statement rigorous; Equal probability priori could also be viewed as maximum entropy principle for microcanonical ensemble. S = i p i log p i
Canonical ensemble: Boltzmann/Gibbs/Darwin-Fowler Suppose one has a large microcanonical ensemble consisting of N closed, identical small canonical ensembles; Let X i represent the microstate of the i-th canonical ensemble, say (p, q). So the high dimensional microstate vector X=(X 1,X 2,...,X N ); Then assume the function g(x i ) is the energy of the i-th canonical ensemble. ( g x 1 ) + ( g x 2 ) +... + ( g x n ) = E tot
Canonical ensemble: Boltzmann/Gibbs/Darwin-Fowler Boltzmann's approach: Suppose the probability of X is only confined in the subspace {E tot =H} and equally distributed (zero probability outside {E tot =H}), then the marginal distribution of each X i is exponentially dependent on g(x i ) when N tends to infinity; Boltzmann s most probable state method, Gibbs partition function method and Darwin-Fowler s steepest descent method are equivalent; In mathematical language, they all concern about the convergence of marginal distribution or empirical distribution.
Canonical ensemble: Maximum entropy principle approach E. T. Jaynes proposed an alternative approach to statistical mechanics based on maximum entropy principle. He called this framework as subjective thermodynamics ; Canonical distribution (Boltzmann s law) is exactly the posterior distribution from the maximum entropy principle up to the given averaged energy. Can Maximum Entropy really replace Boltzmann/Gibbs's theory as the foundation of statistical mechanics?
What is maximum entropy principle Given that some quantity averaged over a large number of individual random variables shows highly unlikely behavior, what is the conditional distribution of an individual sample? Maximum entropy principle is a special case of the minimum relative entropy principle: {X i i=1,2,,n} i.i.d. with probability density f priori Under the condition g(x 1 ) + g(x 2 ) + + g(x N ) = α = Na What is the asymptotic posterior distribution?
Minimum relative entropy principle X i Now, if N is very large, then a is essentially the expected value of the posterior distribution for each X i! X j subjected to: Minimum relative entropy: f posterior ( x)ln f posterior f priori ( x) ( x) g( x) f posterior ( x) dx = a. dx
Several remarks The mathematical result is a consequence of the theory of large deviations in stochastic analysis, and is called Gibbs conditioning (Level II); It is essentially the content of E.T. Jaynes Principle of Maximum Entropy (MaxEnt), when choosing the prior distribution as uniformly random; Shannon theory s application to communication engineering aside, the information theory as a new paradigm essentially based on this mathematical theorem. It really has nothing to do with real physics.
Markov processes follow from MaxEnt approach in the time domain Markov processes are a natural consequence of the dynamical principle of Maximum Caliber; (1) Assuming general stochastic processes a la A. Kolmogorov; (2) Conditioned on observing frequencies of singlet or pair-wise statistics. H. Ge, S. Pressé, K. Ghosh and K. Dill: (preprint) (2011)
Equivalence of Boltzmann s and MaxEnt arguments for canonical ensemble Boltzmann assumed energy conservation following classical mechanics, and equal probability a priori in microcanonical ensemble; MaxEnt assumes uniform prior distribution in canonical ensemble due to ignorance and constraint on observed energy. They are mathematically equivalent! But for MaxEnt approach, we should assume there is a prior distribution for canonical ensemble when not up to any energy constrain. It is not a big deal if we accept Jaynes subjective point of view.
Grand canonical ensemble and Gibbs paradox In classic textbooks, people just imitate Boltzmann s most probable state approach to express the microstate probability distribution of grand canonical ensemble with energy and particle number; However, this approach could not give the correct distribution of particle numbers (missing the n factorial); And we would not be able to catch this n factorial term either applying the MaxEnt principle; We must appeal to quantum mechanics? 1 1 µ n p = n exp Z( T, µ ) n! kbt
Grand canonical ensemble and Gibbs paradox If we correctly apply Botzmann s equal probability priori of the large microcanonical ensemble, we would definitely see the n factorial term of the particle number statistics comes out, no matter the particles are distinguishable or not; For undistinguishable particles, the n factorial merges due to the calculation of phase space volume; for distinguishable particles, it is due to the partition of particles into grand canonical ensemble; This is known as Poisson statistics for a point process. Minimum relative entropy principle is a purely mathematical theorem, which could really substitute equilibrium statistical mechanics.
Superstructure and possible new ingredients for nonequilibrium statistical mechanics Oono, Y. and Paniconi, M., Prog. Theor. Phys. Suppl. (1998) M. Esposito, U. Harbola and S. Mukamel, Phys. Rev. E. (2007) H. Ge, Phys. Rev. E. (2009) H. Ge and H. Qian, Phys. Rev. E (2010) M. Esposito and C. van den Broeck, Phys. Rev. Lett. (2010)
Master equation model Consider a motor protein with N different conformations R 1,R 2,,R N. k ij is the first-order or pseudo-first-order rate constants for the reaction R i R j. No matter starting from any initial distribution, it will finally approach its stationary distribution satisfying N j= 1 ( ss ss c k c k ) = 0 j ji dci ( t) i = ( c k c Self-assembly or self-organization ij j j ji i k ij ) c k = ss j ji c ss i k ij Detailed balance
de df ds e Superstructure of thermodynamics Relative entropy = ; f d : free energy dissipation rate f d = Q hk h d ; p h d : heat dissipation/work out Q hk : house keeping heat/work in ss ci kij Q t) = ( c ( t) k c ( t) k ) log. = e h p d ; e p : entropy production rate c k ( t) = ; c k i ij ( ci ( t) kij c j ( t) k ji ) log hd ( t) = ( ci ( t) kij c j ( t) k ji ) i> j j hk ji ( i> j i i> j ij j ji c ss j k ji k log k ij ji ;
Two origins of irreversibility f d 0, Q hk 0, e p = f d + Q hk 0. e p characterizes total time irreversibility in a Markov process. When system reaches stationary, f d = 0. When system is closed (i.e., no active energy drive, detailed balaned) Q hk = 0. Boltzmann: f d = e p >0 but Q hk =0; Prigogine (Brussel school, NESS): Q hk =e p > 0 but f d =0. f d 0 in driven systems is self-organization.
Recover the fundamental equation of classical thermodynamics Isothermal system, not entropy increasing, but free energy decreases (Helmholtz); However, the rate of free energy decrease is precisely entropy production. In equilibrium steady state, entropy production = 0, heat dissipation = 0, time is reversible. If the equilibrium distribution is constant, i.e. equal probability a priori, we have ds( t) df( t) = = fd Entropy increases for systems with stationary equal probability, such as microcanonical ensemble 0
Time-inhomogeneous systems dci ( t) = ( c k ( t) j j ji c k i ij ( t)) Entropy in Hatano- Sasa equality. We would like to call it intrinsic entropy, which could be defined at individual level. Q hk ds( t) df( t) de( t) = e p = W = W ( t) h ext ext d ( t) ( t) f ( t) Q d ex ( t) ( t) ( t) = e ( t) f ( t) = h ( t) Q ( t) p d d Dissipative work in Jarzynski equality ex
Two faces of the Second Law ds h d df In detailed-balance case, they are equivalent. ep = fd, Qhk 0 W ext In non-detailed balance case, the new one is stronger than the traditional one. df W ext = du ds + Q ex h d ds Q ex
Phase transition and emergent landscape for singular-perturbed diffusion Ge, H., Qian, H.: Phys. Rev. Lett. (2009) Qian, H.: J. Stat. Phys. (2010) Ge, H., Qian, H.: J. Roy. Soc. Interface (2011) Ge, H., Qian, H.: (in preparation) (2011) ( ss ss ( ) ( ) ( )) N ε p x p x F x = 0, x R. ε ε
Emergent landscape limε log ε 0 p ss ε ( x) = φ ( x) 0. Large deviation rate function dx = F(x) dφ ( x( t)) = φ( x) F( x) = x 2 ( φ( )) 0. Related to Hamiltonian-Jacobi equation as well as Freidlin- Wentzel large deviation theory (action along a path). The corresponding deterministic nonlinear chemical dynamics follow the downhill of the function φ(x) (Lyapunov property).
Emergent dynamic landscape Unstable fixed point of deterministic dynamics Maximum: the barrier φ(x) Local minimum Global minimum dx = F(x) Stable fixed points of deterministic dynamics
Relative stability of stable steady states Many nonlinear dynamical systems have multiple, locally stable steady states. Is one attractor more important than another? p ss ε ( ( x) / ), 0. ( x) exp φ ε ε The most important steady state when ε is small would be the global minimum of dynamic landscape.
Maxwell construction Steady States x* Global minimum(lln) abruptly transferred (discontinuous, phase transition, symmetry breaking). θ * θ dx = F( x, θ ) φ (x,θ ) x Related to non-convexity of the landscape and broken ergodicity of the system
Kramers theory H 2 1 H 1 2 The barrier H here may not be the same as the barrier in the previous landscape φ(x) for multistable cases. The switching time between attractors: T 1 2 e, T e H1 2 / ε H 2 1 / ε 2 1.
Local-global confliction (a) A B B C C A Dynamic on a ring as an example. (b) (c) A B C The emergent Markovian jumping process being nonequilibrium is equivalent to the discontinuity of the local landscapes (time symmetry breaking). Completely nonlinear, nonequilibrium phenomenon. Just cut and glue on the local landscapes (non-derivative point).
Local landscape and Kramers theory in multistable case According to the large deviation theory of Freidlin and Wentzel, the local landscape in each attractor equals logarithm of the probability of the trajectory with the highest probability starting from the stable fixed point or limit cycle in this attractor. In this case, Kramers theory says that the barrier crossing time from the i-th attractor to the j-th attractor is just exponentially dependent on the lowest barrier of the local landscape along the boundary between the two attractors. Within these transition rates in hand, one could build a emergent Markovian jumping process in the time scale of inter-attractorial dynamics, whose state space is just the attractors of the diffusion process. The steady distribution of this emergent process would help to construct the global landscape from the local ones.
Possible future generalization The relation of Boltzmann s approach and maximum entropy approach in continuous case, i.e. in the real high-dimensional phase space; Thermodynamic superstructure for reactiondiffusion process, or even for fluid mechanics; Maxwell construction and local-global landscape confliction of phase transition occur for highdimensional chemical master equation (when V tends to infinity).
Summary Minimum relative entropy principle is a purely mathematical result, which then could be applied beyond statistical mechanics. Thermodynamic superstructure would explicitly distinguish Boltzmann and Prigogine s thesis, which are actually two faces of the Second Law; Maxwell construction of phase transition occurs in singular-perturbed diffusion as well as chemical master equation, and local-global landscape confliction is the origin of nonequilibrium emergent Markovian jumping process.
Acknowledgement Prof. Hong Qian University of Washington Department of Applied Mathematics
Thanks for your attention!