We consider a gas of atoms or molecules at temperature T. In chapter 9 we defined the concept of the thermal wavelength λ T, h 2πmkB T,

Transcription

1 Chapter Quantu statistics. Theral wavelength We consider a gas of atos or olecules at teperature T. In chapter 9 we defined the concept of the theral wavelength λ T, λ T = h πkb T, as the wavelength of a wave packet associated with each gas particle (ato or olecule) of ass. At high teperatures, the gas particles can be described as billiard balls since their size is uch saller than the interparticle distance. As the teperature is lowered, the particles wave packets begin to gain in iportance the quantu nature of the particles cannot be neglected! When the wave packets of the particles begin to overlap with each other, quantu effects ust be taken into account. For a syste to be in the classical regie, λ T should be uch saller than the average interparticle distance r 0 (Fig..), with n being the density of particles. r 0 n /3, The gas is in the classical regie when r 0 >> λ T, n /3 >> λ T (.) λ 3 Tn <<. 3

2 3 CHAPTER. QUANTUM STATISTICS Figure.: Theral wavelength λ T and interparticle distance r 0. Quantu regie: quantu effects becoe iportant when λ 3 Tn. When this condition is fulfilled, the wave functions of different particles begin to overlap and the syste has to be treated according to quantu echanics. The condition nλ 3 T =, ( π h ) 3/ n = k B T defines a line in the T n plane that sets the division between the classical and the quantu regies. We define the degeneracy teperature T 0 as ( π h ) k B T 0 = n /3. T 0 can have very different values, depending on the physical syste under study. Exaples of quantu degeneracy teperatures: H gas n = 0 9 c 3 T 0 = 0.05 K Liquid 4 He n = 0 c 3 T 0 = K Electrons in a etal n = 0 c 3 T 0 = 0000 K

3 .. FUNDAMENTALS OF QUANTUM STATISTICS 33 We observe that, for instance, a gas can be described classically at roo teperature, whereas electrons in a etal are in the extree quantu region. Liquid heliu has a degeneracy teperature in between. At.7 K it akes a transition to a quantu phase that exhibits superfluidity.. Fundaentals of quantu statistics Classically, the coplete description of a physical syste is given by - the phase space (q,p): Γ and - the Hailton equations of otion. The solution of the equations of otion defines a trajectory in the phase space. One has to eploy statistical ethods in order to describe the acroscopic state out of the (incoplete) icroscopic inforation. By saying incoplete, we ean that we don t necessarily know the initial conditions for the trajectory of each particle of the syste. That s why we treat the syste statistically. In quantu echanics, on top of the incoplete knowledge of the icroscopic inforation wehavethequantuechanicalindeterinis: eventhoughthestateofthesysteay be copletely known in soe special cases (pure state), we still cannot predict the result of a easureent since any easureent perturbs the syste. This lack of knowledge anifests itself in the statistical interpretation of the wave function and in the uncertainty relation between q i and p i, which cannot be anyore sharply easured at the sae tie. The concepts of phase space and phase trajectory have no eaning anyore..3 Statistical operator: density atrix In quantu echanics, we classify the states of a syste into pure and ixed states..3. Pure state Consider a coplete set of couting observables. A state prepared through easuring these observables is a pure state and is represented by a vector Ψ in the Hilbert space. Now, if ˆB is an operator associated with one of the observables in this coplete set, with ˆB b n = b n b n, then Ψ = n b n b = δ n, c n b n ; c n = b n Ψ. One can think of the expectation value of ˆB, < ˆB >, as the averaged result of

4 34 CHAPTER. QUANTUM STATISTICS - successive easureents of B done on the sae syste or - easureents of B done siultaneously on copies of the syste. The second interpretation reinds us about the concept of the enseble that we introduced in statistics. < ˆB > = Ψ ˆB Ψ = n = n b n c n. Ψ ˆB b n b n Ψ }{{} c n.3. Mixed state When the inforation about the syste is incoplete, i.e., no easureent through a coplete set of couting observables was possible, then the syste is in a ixed state. A ixed state cannot be represented by a Hilbert-vector. For a ixed state, we only know the probabilities that the syste is in a given pure state, while the inforation about the relative phases is issing. Wehavethatthesystecanbefoundinpurestates Ψ, =,,..., withprobabilities 0 P, so that P =, with Ψ n Ψ = δ n. The expectation value of an operator ˆB is then given by: < ˆB >= P Ψ ˆB Ψ. Calculation of this expectation value includes two distinct operations: ) calculation of quantu echanical expectation values Ψ ˆB Ψ and ) statistical averaging over the states Ψ weighed by P, which is a consequence of the incoplete inforation about the syste. The central operator in quantu statistics which includes both averages is the density atrix operator ˆρ: ˆρ = P Ψ Ψ.

5 .4. PROPERTIES OF ˆρ 35.4 Properties of ˆρ. The average value of an operator ˆB is given as < ˆB >= Tr(ˆρˆB). Proof: < ˆB > = P Ψ ˆB Ψ =,ijp Ψ b i b i ˆB b j b j Ψ ( ) = ij P b j Ψ Ψ b i b i ˆB b j = ij ˆρ ji ˆBij = j (ˆρˆB) jj = Tr(ˆρˆB).. ˆρ is heritian: ˆρ = ˆρ since the projector Ψ Ψ is heritian. 3. Trˆρ =. 4. ˆρ is non-negative. Proof: given a state ϕ, ϕ ˆρ ϕ = P ϕ Ψ Eigenvalues of ˆρ are real. 6. Pure state Ψ can be represented as ˆρ Ψ P(Ψ) = Ψ Ψ. 7. ˆρ = P Ψ Ψ, Trˆρ = P Since 0 P, P P =. 8. Tie dependence: in the Schrödinger picture i h ˆρ = [H, ˆρ], t (QMI) which is the quantu echanical analogue of the Liouville equation.

6 36 CHAPTER. QUANTUM STATISTICS.5 Statistical operator in theral equilibriu In theral equilibriu, the expectation value of a acroscopic observable B, < ˆB >= Tr(ˆρ(t)ˆB), has to be tie independent. This is only possible if ˆρ(t) is tie independent: This iplies that ˆρ is a conserved quantity. ˆρ t = [ˆρ,H] = 0. i h ˆρ = ˆρ(H) (Liouville theore)..6 Correspondence principle In this section, we want to establish the correspondence between classical statistical physics and quantu statistical physics. Statistical enseble: a set of exact copies of a real syste for which there is no coplete knowledge, i.e., of a syste in a ixed state. Each eber of the enseble is in a possible state Ψ in which the real syste could be. The enseble is described by an incoherent set of states as the ebers of the enseble don t interact with each other and their states don t interfere with each other. This definition of the statistical enseble is valid both in classical statistical physics and in quantu statistical physics.

7 .6. CORRESPONDENCE PRINCIPLE 37 Correspondences: Classical QM Phase space function Operator B(q,p) ˆB Probability distribution function Statistical operator ρ(q,p) ˆρ 3 Poisson brackets Coutators [B,G] L = ( B G G B ) [ˆB, Ĝ] = (ˆBĜ ĜˆB) q j j p j q j p j i h i h 4 Phase space integration Trace d 3N q d 3N p... Tr(...) h 3N N! 5 Enseble in equilibriu (stationary state): [ρ,h] L = 0 [ˆρ,H] = 0 6 Statistical average B = h 3N N! of an observable B = h 3N N! d 3N q d 3N p ρ(q,p)b(q,p) d 3N q d 3N p ρ(q,p) < ˆB >= Tr(ˆρˆB) = Trˆρ 7 Equation of otion Liouville equation: ρ t = [H,ρ] L ˆρ ] [Ĥ, t = ī ˆρ h H: Hailton function Ĥ: Hailton operator

8 38 CHAPTER. QUANTUM STATISTICS.7 Microcanonical enseble Consider an isolated syste in therodynaical equilibriu, with N particles and energy between E and E+. Since the corresponding enseble is characterized by a stationary distribution, i.e., [ˆρ,Ĥ] = 0, ˆρ and Ĥ have a coon set of eigenstates: Ĥ E n = E n E n, E n E = δ n, and E n Ĥ E = E δ n E n ˆρ E δ n. For an isolated syste, the principle of a priori probability states that all possible states of the syste have the sae probability. Therefore, we can write: ˆρ icro = P E E, with { const if E < E < E +, P = 0 otherwise, where the constant can be obtained out of the noralization condition Trˆρ =. We define Γ(E) = Tr ( E<E<E+ E E ), (.) which is the quantu echanical analogue of the classical phase-space volue. In the energy representation, eq. (.) can be rewritten as Γ(E) = E<E <E+ = nuber of states with energies between E and E +. Then, P = Γ(E) with E < E < E +, and the expectation value of an operator ˆB in the icrocanonical enseble is given as ( < ˆB >= E<E<E+ ) Γ(E) Tr E E ˆB

9 .8. THIRD LAW OF THERMODYNAMICS 39 Note that is the su over states! The internal energy U is now ( U < Ĥ >= E<E<E+ ) Γ(E) Tr E E Ĥ = E<E <E+ E E Γ(E) and the entropy is S = k B lnγ(e). Note that with the definition of Γ(E) given in equation (.), there is no Gibbs paradox since the correct counting of states is provided by the equation. In analogy with classical statistics, we can also define the phase space volue Φ(E) as Φ(E) = E E (the nuber of eigenstates of the Hailtonian operator with energies sall or equal to E ), with Γ(E) = Φ(E + ) Φ(E), and Ω(E) = δφ δe, For 0, we can define S = k B lnω(e) the density of states..8 Third law of therodynaics The third law of therodynaics is of quantu echanical nature: The entropy of a therodynaical syste at T = 0 is a universal constant that can be chosen to be zero and this choice is independent of the values taken by the other state variables. For a syste with a discrete energy spectru, there is a lowest energy state, the ground state. At T 0, the syste will go into this state. If the groundstate is n-ties degenerate, the entropy of the syste at T = 0 is n: degeneracy ultiplicity. S(T = 0) = k B lnn,

10 40 CHAPTER. QUANTUM STATISTICS For n = (no degeneracy), S = 0. But for n >, S doesn t apparently fulfill the third law of therodynaics. However, there is no paradox: when n >, the groundstate is degenerate due to the existence of internal syetries of the Hailtonian (for instance, spin rotational syetry); at T = 0 the syetry gets broken through a phase transition thatletstheentropygotozero. Inthecaseofspinrotationalsyetry, aphasetransition to a ferroagnetic state breaks the syetry, and the syste goes into one of the any possible degenerate states..9 Exaple in the icrocanonical enseble: Two-level syste and the concept of negative teperature ConsideranisolatedsysteatenergyE ofn particles,eachwithspins = /,s z = ±/. Due to two possible spin orientations, each particle can have two energy states ±ǫ. The nuber of particles in the energy state +ǫ is N +, and the nuber of particles in the energy state ǫ is N, such that N = N + +N and E = (N + N )ǫ. Obviously, and N + = N = ( N + E ) ǫ ( N E ). ǫ The nuber of possible states with energy E and particle nuber N is given by the density of states Ω(E,N): Ω(E,N) = N! N +!N! = N! [ ( )] [ N + E ǫ! ( )] N E ǫ! There are exactly N! possibilities to count the possible icrostates (the particles are indistinguishable!). But all icrostates obtained by interchanging particles in the state +ǫ are equivalent, and the sae holds for the particles in the state ǫ. Therefore, in order to avoid ulticounting, N! has to be divided by N +! and N!. The entropy is then: S(E,N) = k B ln N! N +!N!.

11 .9. EXAMPLE IN THE MICROCANONICAL ENSEMBLE: 4 Now, we ake use of the Stirling forula (N >>,N + >>,N >> ) and get ( S(E,N) = k B N lnn k B k B N + E ( ǫ N E ǫ The teperature of the syste is then obtained as ) { ln ) ln { T = βk B = S { } E = k B N E N ǫ ln ǫ. N + E ǫ ( N + E )} ( ǫ N E )}. ǫ Situations:. If E < 0, we are in the situation where there are ore particles in the lower energy level ǫ than in the higher energy level + ǫ and T > 0.. For T, the particles distribute equally between the two levels: N + = N E = Suppose that through a special echanis particles can be excited to the higher energy level (for instance, by light irradiation as in a Laser). If the particles reain in the excited state for a certain period of tie (etastable state), the situation N + >> N can be realized. This phenoenon is called inversion. Fro E = (N + N ) ǫ > 0 follows that T < 0, i.e., we are in a situation of a negative teperature. The three situations can be presented graphically (Fig..). Figure.: Phase diagra for a two-level syste.. For E S =, all the particles are in the lower energy level. The slope β is Nǫ E infinity and T = 0.

12 4 CHAPTER. QUANTUM STATISTICS. For equal distribution, N + = N, E = 0 and β = 0. Nǫ The teperature T jups fro + to as soon as ore particles are in the higher than the lower level. 3. When all the particles are in the higher level, then E = and T = 0. Nǫ Such systes with an inverted state are described in ters of negative teperatures. Note that the concept of negative teperature akes only sense for systes with an upper bound for the energy..0 Canonical enseble We consider a quantu syste A in theral equilibriu with a reservoir A. Our goal is to obtain the density atrix operator ˆρ for A. The canonical enseble contains copies of the syste A, which are in possible allowed states Ψ for A, with Ĥ Ψ = E Ψ. If we neglect interaction H between the systes A and A, the Hailtonian of the isolated total syste A = A A is just the su of their individual Hailtonians: Ĥ = Ĥ +Ĥ. Then, the total energy of the isolated syste A is and the nuber of states of A is E = E +E n Γ(E) = E Γ (E )Γ (E E ). (.3) If A is in the state E = E, the nuber of possible states of the total syste is Γ (E E ), all with equal probabilities (principle of a priori probability). Then, P Γ (E E ).

13 .0. CANONICAL ENSEMBLE 43 Since A << A and E << E, we can perfor a Taylor expansion of the previous expression: ( ) lnγ (E E ) = lnγ (E) E lnγ (E ) +... E E =E Then, for P we have lnγ (E) E k B T +... P Γ (E E ) e βe and for the density atrix operator ˆρ: ˆρ = c e βe E E = ce βĥ E E = ce βĥ, which after noralization becoes ˆρ = e βĥ Tre βĥ [ˆρ,Ĥ] = 0 We drop fro now on the subscript. We define the canonical partition function as In the energy representation, Z(T) = Tre βĥ, Z(T) = Z(T,V,N) = Z N (T,V). Z(T) = n e βen. Then, the expectation value of an observable ˆB is given as ( ) < ˆB Tr e βĥ ˆB >= Tr(ˆρˆB) =. For the internal energy U =< Ĥ > we thus have Tre βĥ U =< Ĥ >= β lnz N(T,V). One can also show that the energy fluctuations are given by < Ĥ > < Ĥ > CV k B T < Ĥ =. > U N

14 44 CHAPTER. QUANTUM STATISTICS Note the analogy of the derived expressions with the corresponding ones fro classical statistical physics. For a acroscopic syste N, alost all of its copies in the canonical enseble have the sae energy E =< Ĥ >. This eans that for N the canonical enseble is statistically equivalent to the icrocanonical enseble. Additivity of the free energy F, F(T,V,N) = k B T lnz N (T,V). For two systes and in theral equilibriu, such that Ĥ = Ĥ +Ĥ and Ĥ 0, the partition function of the total syste is Z(T) = n n e β(en+en) = Z (T)Z (T), which eans that in this case the free energy is an additive quantity: F(T) = F (T)+F (T).. Exaple in the canonical enseble: N quantu echanical haronic oscillators We consider N localized independent linear haronic oscillators with frequency ω in theral equilibriu at teperature T. Such a syste would describe, for instance, the radiation energy of a black body. The energy of one oscillator is: Ĥ Ψ n = ε n Ψ n, ( ε n = hω n+ ), where n is the occupation nuber. Then, the partition function in the canonical enseble for one oscillator is given by Z(T,V,) = Tre βĥ = e βεn = n ( = e ) β hω e β hω n. n=0 e β hω(n+ ) n=0

15 .. EXAMPLE IN THE CANONICAL ENSEMBLE: 45 This is a geoetrical series: Therefore, For N particles, n= r n = r. β hω Z(T,V,) = e e = β hω e β hω e β hω = sinh ( β hω [ ( )] N β hω Z(T,V,N) = [Z(T,V,)] N = sinh. The above expression is valid only because we assued that the oscillators don t interact with each other and that they are distinguishable. The free energy is { ( )} β hω F(T,V,N) = Nk B T ln sinh = N hω }{{} zero-point energy of N oscillators +Nk B T ln ( e β hω). ( ) F µ = N ( ) F P = V T,V T,N = F N, = 0 (since the oscillators are localized). ). The entropy: S = ( ) F T V,N [ ( ) β hω β hω = Nk B coth ( ln sinh = Nk B [ β hω e β hω ln( e β hω) ]. ( ))] β hω The internal energy: In fact, with [ ] U = N hω +. (.4) e β hω U = N < ǫ n >, [ ] ( ) < ǫ n >= hω + = hω e β hω + < n >,

16 46 CHAPTER. QUANTUM STATISTICS where < n >= e β hω is the average quantu nuber describing the average particle occupation at teperature T. We will see thus expression again when we introduce the Bose gas. Note that for T 0 U N hω gives the zeroth energy of a syste of N quantu oscillators. For T, U k B T, as we obtained for a syste of n classical oscillators.. Grand canonical enseble We consider a syste A that interchanges energy and particles with a reservoir A. A = A A is isolated, with V = V + V. In theral equilibriu the syste has teperature T and cheical potential µ. We consider the case when E << E and N << N. In the grand canonical enseble, the copies of the syste A are in possible eigenstates E (N ) of Ĥ and ˆN, with [Ĥ, ˆN ] = 0: Ĥ E (N ) = E (N ) E (N ), ˆN E (N ) = N E (N ),

17 .. GRAND CANONICAL ENSEMBLE 47 E = E (N )+E (N ), N = N +N. We want to find the density atrix operator for the syste A in the grand canonical enseble: ˆρ = P (N ) E (N ) E (N ). N Since A is an isolated syste, the phase space volue of A is Γ N (E,V) = Γ () N (E (N ),V )Γ () N N.(E E (N ),V ) N If A is in the state E (N ), there are Γ () N N (E E (N ),V ) possible states for A and therefore for A. Since all these states are equally possible, P (N ) Γ () N N (E E (N ),V ). For E << E and N << N, we can perfor a Taylor expansion of Γ () N N : lnγ () N N (E E,V ) k B S (E,N,V ) E (N ) k B N ( S k B N ( ) S E ) E = E N = N E = E N = N, ( ) S E ( ) S N N = N E = E N = N E = E = T, = µ T, P (N ) = ce β(e(n ) µn ). Then, the density atrix operator is ˆρ e β(e(n ) µn ) E (N ) E (N ) N Noralizing ˆρ, we get = e β(ĥ µ ˆN ) E (N ) E (N ) N ˆρ = e β(ĥ µ ˆN) Tre β(ĥ µ ˆN) (we dropped the subindex ).

18 48 CHAPTER. QUANTUM STATISTICS The partition function in the grand canonical enseble is then Z(T,V,µ) = Tre β(ĥ µ ˆN). In the particle nuber representation, Z is given as Z(T,V,µ) = z N Z N (T,V). N=0 The average value of an operator ˆB is < ˆB >= Tr(ˆρˆB) = and in the energy-particle representation: ( ) Tr e β(ĥ µ ˆN) ˆB Tre β(ĥ µ ˆN) < ˆB >= Z e β(e(n) µn) B (N), N=0 with B (N) = E (N) ˆB E (N). If < ˆB > c is the average value of ˆB in the canonical enseble, then < ˆB >= N=0 z N Z N (T,V) < ˆB > c. z N Z N (T,V) N=0 The grand canonical potential is Ω(T,V,µ) = k B T lnz(t,v,µ) and all the properties of the syste are derived analogously as was done in classical statistics..3 Entropy and the density atrix operator We want to show that for all three ensebles, the relation between entropy and density atrix operator ˆρ is S = k B Tr(ˆρln ˆρ) = k B < ln ˆρ >, (.5) i.e., the entropy is given by the expectation value of the logarith of the density atrix operator ˆρ. Note that since the eigenvalues of ˆρ are probabilities, i.e., they are between 0 and, the ln in eq. (.5) will be negative and S will be positive.

19 .3. ENTROPY AND THE DENSITY MATRIX OPERATOR 49 Microcanonical enseble ˆρ icro = E<E <E+ E E, Γ(E) with the eigenvalue equation: ˆρ icro E = Γ(E) E if E < E < E +, 0 otherwise. Tr(ˆρ icro ln ˆρ icro ) = i E i ˆρ icro ln ˆρ icro E i Since S was defined as = E<E i <E+ = lnγ(e). S = k B lnγ(e), S = k B Trˆρ icro ln ˆρ icro. i Γ(E) ln Γ(E) E i E i Canonical enseble ˆρ cano = Z e βĥ, with Then, and Z = Tre βĥ = e βf. ˆρ cano = e βf e βĥ k B < ln ˆρ cano >= k B β(f < Ĥ >) = T (F U) = ( TS) = S. T Grand canonical enseble ˆρ gc = Z e β(ĥ µ ˆN), with Then, and k B < ln ˆρ gc > = T Z = e βω. ˆρ gc = e βω β( Ĥ+µ ˆN) e ( Ω < Ĥ > +µ < ˆN ) > = (F U) = S. T

20 50 CHAPTER. QUANTUM STATISTICS.4 Inforation theory Inthissection, wewanttoprovide ashortintroductiontotheinforation theory, which is analternativewayofinterpretingstatisticalphysics. Inthefollowing, weshallconcentrate on the canonical enseble and present a derivation of this enseble based purely on probabilities and the extreu principle. We consider a syste A with constant ean energy E. Let us suppose that an enseble of this syste is such that a r copies of the syste are in state r. Then, a r = a (.6) and a r a r E r = E. (.7) r The nuber Γ(a,a,...) of distinct possible ways of selecting a total of a distinct systes in such a way that a of the are in state r =, a are in state r =, etc. is given by Γ = a! a!a!a 3!.... The logarith of Γ is then lnγ = lna! r lna r!. (.8) For a >> and a r >>, we can apply the Stirling s approxiation: so that eq. (.8) becoes lna r! = a r lna r a r, lnγ = alna a r a r lna r + r a r, lnγ = alna r a r lna r. Extreu condition: For what set of nubers a,a,a 3,... subject to conditions (.6) and (.7) will Γ be axiu? Or, in other words, for which a r will the equality δlnγ = r (δa r +lna r δa r ) = 0 subject to δa r = 0 (.9) r

21 .4. INFORMATION THEORY 5 and E r δa r = 0 (.0) hold? Fro the above expressions, it follows that one has: lna r δa r = 0 subject to eq. (.9) and (.0). r r In order to handle this extreu condition subject to constraints, we ake use of the Lagrange ultipliers: (lna r +α+βe r )δa r = 0, r where α,β are the Lagrange ultipliers. With a proper choice of α and β, all δa r can be regarded as independent and therefore each coefficient of δa r ust separately vanish. We denote by ā r the value of a r when Γ is axiu. Then, lnā r +α+βe r = 0, ā r = e α e βer. α is deterined by the noralization condition (.6): ( ) e α = a e βe r and β by the condition (.7): e βe r E r e βe r = E P r ār a = e βer r e βer. We obtained the probability distribution in the canonical enseble as corresponding to the distribution of systes in the enseble which akes the nuber Γ of possible configurations a axiu. In ters of P r = ar, Γ can be written as a lnγ = alna r ap r ln(ap r ) Therefore, = alna a P r (lna+lnp r ) r ( ) = alna alna P r a r r lnγ = a P r lnp r. r P r lnp r.

22 5 CHAPTER. QUANTUM STATISTICS Since r P r =, 0 < P r < and lnγ is positive because lnp r < 0. The canonical distribution P r is characterized by the fact that it akes the quantity r P rlnp r a axiu subject to a given value P r E r = E of the ean energy. The entropy is given by S = k B a lnγ = k B P r lnp r. AnincreaseinlnΓreflectsaore rando distribution ofsystesovertheavailablestates. The axiu possible value of Γ gives the entropy of the final equilibriu state. The function - r P rlnp r can be used as a easure of inforation available about the systes in the enseble. This function plays an iportant role as a easure of inforation in probles of counication. For instance, if an event (state) has probability we get no inforation fro the occurrence of the event I()=0. The event is copletely predictable. Shannon s entropy : quantifies the inforation contained in a essage (for instance in bits units). A fair coin has an entropy of one bit log ( ). However, if the coin is not fair, then the uncertainty is lower since if asked to bet on the next outcoe, we would bet preferentially on the ost frequent result, and thus the Shannon entropy is lower. The concept was introduced by Claude E. Shannon in his 948 paper A Matheatical Theory of Counication Bell Syste Technical Journal, vol. 7, pp and , July and October, 948. Intuitively, entropy quantifies the uncertainty involved when encountering a rando variable. (see, for instance, Entropy and inforation theory, Robert M. Gray, Springer-Verlag 008) r