INFORMATION OF EVENTS WITH DISCRETE OUTCOMES: METHODS OF PHYSICS

Transcription

1 CHAPER 6 INFORMAION OF EVENS WIH DISCREE OUCOMES: MEHODS OF PHYSICS ORIENAION It s mportant to understand the manner n whch the concept of nformaton s appled n physcs, because t s closely algned wth the manner n whch we shall be applyng nformaton n the analyss of sensory systems. In a way, t s not proper to speak of the applcatons of nformaton theory n physcs, because the statstcal entropy concept was ntroduced nto physcs by Boltzmann more than half a century before Wener and Shannon s treatses and the conng of the term nformaton theory. Physcst Leon Brlloun wrote two books dealng wth the nformaton concept n scence (1962 and 1964). he earler of the two books s the one more commonly cted, however, the later of the two books I fnd the more useful. hs chapter s desgned to gve the gst of how the nformaton dea s used n physcs. In the pages to follow, the frst and second laws of thermodynamcs wll be presented as they were understood n the nneteenth century. It wll be shown how Carnot engnes operatng under constrant of the second law gave rse to Clausus nequalty, and how ths nequalty can be used to ntroduce the concept of physcal or thermodynamc entropy. Change n physcal entropy wll be shown to be calculable from macroscopc measurements made n the laboratory. Physcal entropy wll then be expressed from the perspectve of statstcal mechancs that s, n terms of the probablstc behavor of molecules and lnked wth nformaton theoretcal entropy. he nterdcton of It can be shown that (Chapter 4, note 1) wll be suspended here as we try to compress a rather lengthy subject nto a few pages. he reader s encouraged to capture the flow of deas rather than the detals of the proofs. I acknowledge my supreme debt n preparng ths chapter to F. W. Sears text on thermodynamcs and statstcal mechancs (1953). It was from Professor Sears book that I frst learned thermodynamcs as a student; and t s n ths book that I search agan and agan for nherent jewels and errant joules. I also recommend the texts by Zemansky (1943) and Zemansky and Dttman (1981). HE FIRS LAW OF HERMODYNAMICS he development of thermodynamcs preceded the establshment of the atomc theory of matter, so that the formulatons of the frst and second laws of thermodynamcs n the nneteenth century were made largely whle matter and energy were regarded as contnua. he frst law of thermodynamcs s an expresson of the prncple of conservaton of energy. A quantty called the nternal energy of a system was formulated, whch we shall represent by U. Any change n U s produced by the dfference between the heat, Q, flowng nto the system and work, :, done by the system. hat s, du = d U Q d U :. (6.1) he prmes ndcate that d U Q and d U : are not exact dfferentals; that s, no meanng can be attached to terms such as the heat n the system. he dfference between the two nexact dfferentals d U Q and d U : s equal to the exact dfferental, du. We say that the nternal energy s a determnant of the state of the system. 58

2 6. Dscrete Outcomes: Methods of Physcs 59 Dvdng Equaton (6.1) by the number of moles n the system, we can wrte du = d U q d U Z. (6.2) Internal energy, then, reflects a balance between heat flow nto, and work done by the system. he frst law s, n effect, an accountng prncple for energy. he frst law of thermodynamcs s certanly adequate, n tself, to account for many physcal processes. For example, suppose that our system conssts of two metal blocks at dfferent temperatures that are n contact wth each other, but nsulated thermally from ther envronment. Snce the system s thermally nsulated, d U Q = 0; and f we permt the system to do no work, d U : = 0. By the frst law, du = 0. It s common experence that heat wll flow from the block at the hgher temperature to the block at the lower temperature untl both blocks reach the same temperature. he frst law smply acknowledges the legtmacy of the redstrbuton of heat between the blocks whle conservng nternal energy. Suppose now that we begn wth the two blocks at the same temperature and n contact wth each other. Is t possble that heat wll flow from one block to the other, leavng the frst block at a lower temperature and the second at a hgher temperature? Snce no heat leaves the system and no work s done, du remans equal to zero, and so the frst law does not prohbt such occurrences. However, experence tells us that ths phenomenon never occurs. From ths example and smlar consderatons, t s recognzed that the frst law alone s not adequate to account for all thermodynamc phenomena. Another state varable lke the nternal energy s requred. HE SECOND LAW OF HERMODYNAMICS Hence, one expresson of the second law of thermodynamcs, due to Clausus, s the followng: No process s possble whose sole result s the removal of heat from a reservor at one temperature (lower) and the absorpton of an equal quantty of heat at a hgher temperature. 1 HE CARNO CYCLE / ENGINE Clausus was aded by the work of Sad Carnot (1824; please see Carnot, 1977), who abstracted the propertes of heat engnes. Carnot understood even before the formulaton of the frst law that just as mechancal work s done by water fallng from a hgher to a lower level, so, too, mechancal work can be done when heat flows from a heat reservor at a hgher temperature to a heat reservor at a lower temperature. He also understood that the heat engne s best analyzed wth reference to a cyclcal process where a gas or some other workng substance s passed through a sequence of compressons and expansons. he heat engne s known as a Carnot engne, and the assocated cycle as the Carnot cycle. Let the heat ssung from the hot reservor be Q 1, that enterng the cold reservor be Q 2, and the work done by the workng substance (let s call t the system) be :. hese processes are llustrated n Fgure 6.1. he Carnot engne can operate n reverse, so that heat Q 2 may be absorbed from the cooler reservor nto the system, coupled wth work, :, done by the system, and heat Q 1 released at the hotter reservor. When operatng n the latter fashon, the system s known as a Carnot refrgerator. he algebrac sgns of Q 1, Q 2 and : are taken wth respect to the system; that s, Q 1 > 0 when heat flows from the hotter reservor to the system, Q 2 < 0 when heat flows from the system to the colder reservor, and : > 0 when the system does postve work on the outsde world. he Carnot engne s a reversble 2 engne. he exchanges of energy depcted n Fgure 6.1 occurred durng a Carnot cycle, after whch the nternal energy of the system was left unchanged. herefore, from the frst law as expressed by Equaton (6.1), the changes n heat mnus the change n work must equal zero, or Q 1 + Q 2 : = 0,

3 6. Dscrete Outcomes: Methods of Physcs 60 Fgure 6.1 Carnot Engne. Heat Q 1 flows from a heat reservor at a hgher temperature to a system (workng substance). Heat Q 2 flows from the system to a heat reservor at a lower temperature. Work : s done by the system. Carnot refrgerator. Heat Q 2 s drawn from the cooler reservor, work : s done on the system. Heat Q 1 s delvered to the warmer reservor. or : = Q 1 + Q 2. (6.3) hese results are ndependent of the workng substance, whch s the substance undergong the Carnot cycle. Lord Kelvn then utlzed the rato of Q 1 to Q 2 to defne the temperature of the reservors: Q Q 2 2 = 0. (6.4) It s possble to show that when the workng substance s an deal gas, Equaton (6.4) can be derved usng the deal gas law, Equaton (1.1), and the frst law of thermodynamcs, Equaton (6.3). he Kelvn temperature s numercally equal to the absolute temperature. We note that the second law of thermodynamcs has not yet been brought to bear on the Carnot engne / refrgerator. HE CLAUSIUS INEQUALIY Armed now wth a verbal statement of the second law, and Equaton (6.4) relatng heat flows from two heat reservors and the respectve temperatures of these reservors, we can apprecate the development of Clausus nequalty. We consder any arbtrarly chosen process n whch a system s carred through a closed cycle (nvolvng expansons, contractons, heat loss, heat gan, postve work done by the system, postve work done on the system), such that ts end state s dentcal wth ts ntal state (pressure, volume and temperature of a fxed quantty of matter returned to ther orgnal values). hen t can be shown wth the ad of Carnot engnes (that s, the frst law of thermodynamcs), usng Equaton (6.4) and usng the second law of thermodynamcs that all reservors Q =1 0, (6.5) where are the temperatures of the reservors and Q are the heat flows as seen by the system (that s,

4 6. Dscrete Outcomes: Methods of Physcs 61 Q s postve when heat proceeds from the heat reservor toward the system). When only nfntesmally small quanttes of heat are exchanged wth nfntely many heat reservors, we may pass to the ntegral form d U Q 0. (6.6) Both (6.5) and (6.6) express the Clausus nequalty, whch s vald whether the cyclc process s reversble or rreversble. 2 he smplest example to llustrate (6.5) s the flow of, say, Q joules from a hgher temperature reservor, hgh, to a lower temperature reservor, low. hen the summaton on the left-hand sde of (6.5) s just Q hgh + Q low = Q 1 hgh 1 low joule / deg, whch must be less than zero as demanded by the rght-hand sde of (6.5). PHYSICAL ENROPY he term entropy was ntroduced by Clausus n 1854 to mean a transformaton (Greek, trepen, to turn). We are now n poston to understand what ths term means and how t may be calculated. Whle the Clausus nequalty was derved for a cyclc system that was ether reversble or rreversble, we now consder just a reversble system. Suppose our system cycles frst forward and then backward returnng to ts startng pont. Let d U Q f be the heat flowng nto the system n ts forward cycle, and d U Q b the heat n ts reverse cycle. hen because of the reversble nature of the system d U Q f = d U Q b (6.7) and the temperature,, of the heat reservor s equal to that of the system. Usng the contour ntegral > to represent ntegraton around a cycle, we have from (6.6) > d U Q f 0 and > d U Q b 0. (6.8) Introducng (6.7) nto (6.8) we see that the only way (6.8) can be true s f the equalty sgn holds, so that > rev d U Q = 0, (6.9) where rev ndcates that the process s reversble. We are followng the argument of Sears (1953) very closely. We can now show that d U Q/ s an exact dfferental. Consder an arbtrary closed path for the reversble cycle, represented schematcally n Fgure 6.2. A and B are any two ponts on the contour; path I leads from A to B and path II from B to A. hen > rev d U Q = A B d U Q + B A d U Q = 0. (6.10) along path I along path II

5 6. Dscrete Outcomes: Methods of Physcs 62 Fgure 6.2 Reversble cycle If path II were traversed n the opposte drecton then, as a consequence of the reversblty of the reacton, B A d U Q = A B d U Q. (6.11) along path II along path II Insertng ths result nto Equaton (6.10), we have A B d U Q = A B d U Q. (6.12) along paht I along paht II hat s, the ntegral between two ponts A and B has the same value along all reversble paths. herefore, d U Q/ s an exact dfferental and may be represented by ds: A B d U Q = A B ds = SB S A. (6.13) hat s, the value of the ntegral depends not on the path, but only on ts end ponts. S s called the physcal or thermodynamc entropy of the system. In ths chapter we must dstngush explctly between physcal and nformatonal entropy. he second law of thermodynamcs for a reversble process, therefore, permts the mathematcal statement ds = d U Q. (6.14) Specfc entropy, s, can be obtaned by dvdng S by the number of moles n the system: ds = d U q/ (6.15) (cf. Equaton (6.2)). he unts of entropy are, of course, joule.deg 1 or joule. deg 1.mole 1. HE PRINCIPLE OF INCREASE OF PHYSICAL ENROPY In Fgure 6.3 s depcted a cyclcal process wheren an solated system proceeds from A to B rreversbly by some natural process, and s then returned from B to A reversbly. he cycle as a whole

6 6. Dscrete Outcomes: Methods of Physcs 63 Fgure 6.3 s rreversble. By the Clausus nequalty (6.6), > d U Q B d = U Q A A d + U Q B < 0. rreversble path reversble path he quantty d U Q/ n the frst ntegral s not an exact dfferental because the process s rreversble, but d U Q n the frst ntegral s equal to zero because the system s solated and does not exchange heat wth ts surroundngs. he frst ntegral s, therefore, equal to zero. he second ntegral does represent a reversble process, so ts value s equal to S A S B. herefore, S A S B < 0 S B > S A. (6.16) he thermodynamc entropy of the system n state B s greater than ts entropy n state A. Snce the process was arbtrary, we learn as a consequence of the second law of thermodynamcs that the thermodynamc entropy of an solated system ncreases n every natural (rreversble) process. HE LABORAORY MEASURE OF PHYSICAL ENROPY CHANGE Change n physcal entropy s a measurable, nearly-palpable quantty, that s measured n joules/degree or joules/degree-mole. For example, a change n phase (sold to lqud or lqud to gas) s carred out at a constant temperature, whle the phases reman n equlbrum wth each other. When the phase change s carred out reversbly, d U Q, the heat absorbed, s the latent heat of fuson or vaporzaton, and the temperature wll be the meltng or bolng pont respectvely. o take a specfc nstance, consder the meltng of ce. he latent heat of fuson of water s equal to cal/g at one atmosphere. he entropy change as ce melts can be found as follows: Heat change q = (79.70 cal/g) (4.18 joule/cal) (18 g/mole) Ice melts at 0 ( C = 273 ( K = joule/mole Entropy change s = q / = 6000 / 273 = 22.0 joule/deg.mole.

7 6. Dscrete Outcomes: Methods of Physcs 64 As another example, f a sold s heated from temperature 1 to 2, dq = C d, where C s specfc heat per mole. hen, calculatng on the bass of a reversble reacton, ds = dq / = C d/ Et cetera. s = 1 2 C d / = C ln( 2 / 1 ) joule/deg.mole. SUMMARY OF SECION ON CLASSICAL HERMODYNAMICS hs s not a treatse on thermodynamcs. he purpose of the precedng pages s to demonstrate () that changes n thermodynamc entropy are calculated from standard laboratory measurements of macroscopc varables made by calormeters and thermometers; and () that thermodynamc entropy was defned before the atomc theory of matter was establshed, and so calculatons of entropy change can be made wthout reference to atoms and molecules. As we proceed, t wll be seen that drect comparson can be made between the thermodynamc varable, S, and the perceptual varable, F, that has been defned n Chapter 2. SAISICAL MECHANICS: µ-space AND PHASE POINS he atomc theory of matter, conceved n the West n the ffth century BCE n Greece by Democrtus and Leukppus, matured some 2300 years later n the nneteenth century. Physcsts and chemsts, employng the prncples of knetc theory, sought to translate the laws of macroscopc physcs and chemstry nto terms of atoms and molecules. Usng a gas as a model, t s not dffcult to see how the macroscopc varable, pressure, can be nterpreted as many molecules colldng wth a surface and changng ther drecton of moton. he rate of change of momentum of the molecules as they strke the surface consttutes a force, and the force per unt area s measured macroscopcally as pressure. emperature can be nterpreted as a measure of the average knetc energy of molecules comprsng the gas. Physcal entropy, however, s somewhat more dffcult to express n terms of the propertes of gas molecules. hs task was acheved by the Austran physcst, Ludwg Boltzmann, usng the methods of what we now call statstcal mechancs. he term was frst coned by Gbbs n Statstcal mechancs concerns tself not wth the motons and collsons of ndvdual molecules, but wth the propertes of large assembles of molecules, whch t treats probablstcally. In order to do so, t s necessary to utlze spaces consstng of more than the usual three dmensons. Let s take a smple example from everyday lfe. Suppose that a helcopter s located 100 metres above a helport, and s proceedng at constant alttude n a northwesterly drecton at 80 klometers per hour. he helcopter s 3 spatal coordnates are gven by () ts alttude and () the longtude and () lattude of the helport; and ts 3 coordnates n velocty space are gven by () ts vertcal speed (0) and ts horzontal speed n the drectons () north and () west. herefore, we see that even f the orentaton of the helcopter (whch drecton t s facng and ts nclnaton to the horzontal) s gnored, t stll requres sx coordnates, three n ordnary space and three n velocty space, to defne ts poston and moton completely. hs sx-dmensonal space s needed to defne the poston and velocty of a molecule, rrespectve of ts orentaton. hree coordnates, x, y, z, defne a molecule s poston s confguraton space, and three coordnates, v x, v y, v z, defne ts poston n velocty space. One speaks of the hyperspace needed to defne the poston and moton of a molecule as a phase space, and the partcular sx-dmensonal space descrbed above as a type of µ-space ( µ Greek mu, for molecule). 3 One also speaks of the coordnates of a representatve pont (x, y, z, v x, v y, v z ) of a type of molecule n µ-space. 4 Such a representatve pont s called a phase pont.

8 6. Dscrete Outcomes: Methods of Physcs 65 We can now subdvde µ-space nto small, 6-dmensonal elements of volume, δv µ = δx δy δzδv x δv y δv z, whch we call cells n µ-space. We can number the cells 1, 2, 3,...,,..., and let N be the number of phase ponts n the th cell. he densty of ponts n the th cell s, then, N / δv µ. MACROSAES AND MICROSAES We magne our gas confned to a rgd contaner whose walls do not permt the exchange of ether molecules or energy. We deal wth the equlbrum stuaton. Specfcaton of the whch cell s occuped by each molecule n our system would completely defne a mcrostate of the system. For example, molecule 1 s n cell 1507, molecule 2 s n cell 2,345,678,..., would defne a specfc mcrostate. However, clearly the macroscopc or observable propertes of the gas do not permt the specfcaton of a unque mcrostate. For example, the densty of the gas wll depend only on the total number of molecules contaned n one cubc meter of gas, and wll tell us lttle about whch ndvdual molecule occupes whch cell n µ-space. he macroscopc propertes of the gas wll, n general, depend only on the numbers, N, the number of phase ponts that le n each cell n µ-space. Complete specfcaton of the N wll defne a macrostate of the system. Clearly, many mcrostates make up one macrostate. For example, we could take one molecule from N 1 and place t n N 2, and one from N 2 and place t n N 1, producng a new mcrostate but leavng the macrostate unchanged. he number of mcrostates that make up a gven macrostate s called the thermodynamc probablty of that macrostate, and s commonly represented by W. It s not dffcult to compute W gven the values of N. o take a smple example, suppose that there are only 5 cells n µ-space: N 1 = 2, N 2 = 1, N 3 = 1, N 4 = 0, N 5 = 0. hat s, the frst cell contans two phase ponts, the second and thrd cells each contan a sngle pont, whle the fourth and ffth cells contan no ponts at all. In ths example we shall have only four phase ponts whch can be desgnated a, b, c, and d. he only possble dstrbutons of the 4 phase ponts n the fve cells are shown n able 6.1. From combnatoral theory, the value of W s gven by W = 4! 2!1!1!0!0!, where 0! can be taken equal to one (Refer to a standard text on probablty such as Freund and Walpole). hat s, W = 12 mcrostates per macrostate. 5 Laboratory observaton cannot dstngush one mcrostate from another. We can say that the uncertanty pertanng to a macrostate s a functon of the number of mcrostates assocated wth t. In general, f there are N representatve ponts, the number of mcrostates s gven by where W = N! N 1!N 2!N 3!... N = N. (6.17) (6.18) able 6.1 Mcrostates Correspondng to a Sngle Macrostate Mcrostates Cell N 1 ab ac ad bc bd cd ab ac ad bc bd cd N 2 c b b a a a d d c d c b N 3 d d c d c b c b b a a a N 4 N 5 here are 4 phase pont: a, b, c, d. A total of 12 mcrostates correspond to the sngle macrostate where 2 partcles are contaned wthn phase cell N 1, 1 partcle wthn phase cells N 2 and N 3, and 0 partcles wthn the two remanng phase cells N 4 and N 5.

9 6. Dscrete Outcomes: Methods of Physcs 66 Recall that we have, by assumpton, a system wth constant total energy. Moreover, we shall confne the system to have a constant volume, and to be n a state of equlbrum. he totalty of mcrostates whose total energy s so constraned comprse what Gbbs has termed a mcrocanoncal ensemble. Desloge (1966) offers a partcularly clear descrpton of mcrostates and ensembles. HE BASIC ASSUMPION OF SAISICAL MECHANICS All mcrostates of a system that have the same energy are equally probable. hs statement has been called (for example, Jackson) the basc assumpton of statstcal mechancs. Wth reference, then, to our mcrocanoncal ensemble, all mcrostates expressed by Equaton (6.17) are equally probable. herefore (Equaton (4.4)), the nformaton theoretcal entropy of the macrostate s gven by H I = ln W. (6.19) H I encodes the uncertanty of the human percever about whch partcular mcrostate gves rse the a specfed macrostate. Boltzmann then postulated that physcal entropy, S, wll be proportonal to (what we now call) nformaton theoretcal entropy, H I. hat s, 6 or expressed alternatvely, S = k B H I, S = k B lnw. (6.20) (6.21) hs remarkable equaton, that equates the macroscopc, observable varable, S, to lnw, the uncertanty about whch molecular mcrostate makes up a macrostate, s nscrbed on Boltzmann s gravestone, although he apparently never wrote t n ths form. he k B s known as Boltzmann s constant. Not surprsngly, not all physcsts are happy about the nterpretaton of lnw as a human uncertanty, because t seems to be njectng a subjectve quantty nto Equaton (6.21), whch s an equaton of physcs and, therefore, an objectve entty completely free from human nfluence. o avod such quas-subjectvty, lnw s smply taken to be the natural log of the thermodynamc probablty, W. Perod. We have seen before how a macroscopc varable can be expressed as a measure of uncertanty, or nformaton theoretcal entropy, F = kh I. (2.6) We are now begnnng to fll n the hstory of Equaton (2.6). Let us contnue wth the development of statstcal mechancs n order to apprecate the utlty of Equatons (6.20) and (6.21). We shall need to use Strlng s approxmaton for evaluatng lnx!, where x s a large, postve nteger. Stated here wthout proof, ln x! = x lnx x. (6.22) he proof n not dffcult, and the reader s referred to the texts. akng logarthms of both sdes of Equaton (6.17), lnw = ln N! lnn! Applyng Strlng s approxmaton, usng Equaton (6.18). lnw = (N lnn N) ( = N lnn N lnn N lnn N ) (6.23)

10 6. Dscrete Outcomes: Methods of Physcs 67 We recall that the system s n a state of equlbrum, whch we mght desgnate by a maxmum value of the thermodynamc probablty, W. hat s, the change n W at equlbrum produced by the aggregate of all the small changes n the occupances N resultng from motons of the phase ponts n µ-space wll equal zero. Expressng ths varaton mathematcally usng Equaton (6.23), hat s, But δlnw = δ(n lnn) δ N lnn = 0. 0 N δlnn = lnn δn N δlnn = 0. (N / N ) δn = δn = 0, snce the total number of partcles does not change. herefore (droppng the subscrpt under the summaton sgn for smplcty), lnn δn = 0. (6.24) hs equaton, then, expresses the maxmum value of W at equlbrum. We ntroduce, now, the constrants mposed by the mcrocanoncal ensemble. Frst, the total number of molecules, N, remans constant. akng the varaton of N, δn = δn = 0. (6.25) Second, the total energy of the system remans constant. If the energy of the th cell s desgnated by w, the total nternal energy wll be gven by U = w N. (6.26) he varaton of the nternal energy s also equal to zero, so that δu = w δn = 0. (6.27) Introducng three Lagrangan multplers of undetermned value, we combne Equatons (6.24), (6.25) and (6.27) to gve (lnn + lna + β w ) δn = 0. Snce the δn are ndependent, each coeffcent s ndependent, so that (6.28) or lnn + lna + β w = 0 N = (1 / A) exp( β w ), (6.29) whch s the celebrated Maxwell-Boltzmann energy dstrbuton. hat s, N = N = (1 / A) exp( β w ) = Z / A. Z = exp( β w ) (6.30) where Z, whch s known as the partton functon, stands for the German word Zustandssumme, usually translated as sum of states. Snce 1/A = N / Z,

11 6. Dscrete Outcomes: Methods of Physcs 68 able 6.2 Illustratng that Dfferent Macrostates Are Compatble wth the Same otal Energy. Confguraton (macrostate) Cell Energy Confguraton A Confguraton B N N N N N otal energy = 7 7 Number of mcrostates = 4! 2!1!1!0!0! = 12 4! 3!0!0!1!0! = 4 Note. otal energy for confguraton A s obtaned from (2 phase ponts)(energy = 1) + (1 phase pont)(energy = 2)... = 7 unts of energy. Smlarly, the total energy for confguraton B s 7 unts. Although both confguratons of 4 phase ponts have total energy equal to 7, confguraton A s the more spread out or dsordered and, therefore, has the greater number of mcrostates or ( thermodynamc probablty ). (6.29) becomes N = N Z exp( β w ) (6.31) gvng the number of phase ponts n the th cell n the state of maxmum thermodynamc probablty. We are almost home. Substtutng the value of N from (6.31) nto (6.23), Now so that Snce lnw = N lnn N (ln N lnz β w ) = N lnn lnn N + lnz N + β w N. N = N and w N = U, lnw = N lnn N lnn + N ln Z + βu. S = k B lnw, S = k B N lnz + kβ U. (6.21) (6.32) Usng smple thermodynamc arguments, t can be shown that kβ s equal to the recprocal of temperature, so that S = Nk B ln Z + U/. (6.33) Usng the partton functon, we have been able to dentfy the thermodynamc entropy, S, wth a statstcal functon of molecular behavor n a gas. Agan, usng Equatons (6.26) wth (6.30) and (6.31), the nternal energy can also be dentfed wth statstcal propertes of molecules: U = Nk B 2 d(lnz) d. (6.34)

12 6. Dscrete Outcomes: Methods of Physcs 69 he relatonshp S = k B ln W has been the key that permtted the mappng of macroscopc thermodynamcs onto the mcroscopc functon of (nvsble) molecules n a gas. he value of W s the maxmum value consstent wth constant energy, volume and number of partcles. o contnue the example of able 6.1, suppose that cell N 1 contaned phase ponts wth energy w 1 = 1, N 2 wth energy w 2 = 2, N 3 wth energy w 3 = 3, N 4 wth energy 4, and N 5 wth energy 5. hen the mcrostate wth two phase ponts n cell 1, one pont n cell 2, one n cell 3, and none n cells 4 and 5, has a total energy of (2)(1) + (1)(2) + (1)(3) + (0)(4) + (0)(5) = 7. W for ths confguraton was shown to be equal to 12. However another way of obtanng a macrostate wth total energy 7 would be N 1 = 3, N 2 = 0, N 3 = 0, N 4 = 1, and N 5 = 0 (see able 6.2). W for ths new confguraton would be equal to 4! 3!0!0!1!0! = 4. hat s, the confguraton (2, 1, 1, 0, 0) has a greater value of W than the confguraton (3, 0, 0, 1, 0), and would, therefore, be closer to the maxmum value of W for the gven total energy of 7 unts. he reader mght lke to check ths confguraton for maxmum W approxmately, usng N -values from Equaton (6.29) wth β = 1. PHYSICAL ENROPY AND DISORDER he quantty W permts us to relate physcal entropy to dsorder, whch s ts usual nterpretaton n the lay lterature. he more phase cells occuped by the gas n confguraton space (x-y-z-space), the greater s the physcal entropy, S. Dssemnaton of phase ponts throughout a larger volume sgnfes larger W, more dsorder, and greater physcal entropy. We can llustrate ths dea by calculatng the change n thermodynamc entropy n the sothermal, reversble expanson of one mole of an deal gas from volume V 1 to volume V 2. If we allow that the nternal energy of an deal gas s ndependent of ts volume, then from the frst law of thermodynamcs, Equaton (6.1), d U Q = d U : = p dv. From the deal gas law, Equaton (1.1), pdv = (R/V) dv. From the second law of thermodynamcs, so that ds = d U Q = pdv = R V dv, V 2 S = R dv = R ln V 2. V1 V V 1 (6.35) Now let us calculate the same quantty, S, from statstcal mechancs. Suppose that V 1 comprses n 1 cells of volume δv so that V 1 = n 1 δv. Smlarly, V 2 = n 2 δv. here are n 1 ways of puttng one N molecule nto V 1, n 12 ways of puttng two molecules n V 1,..., n o 1 ways of puttng Avogadro s number, N N o, molecules nto V 1. herefore, W 1 = n o N 1. Smlarly, W 2 = n o 2. Snce S = k B ln W, S = k B ln W = k B lnn 2 N o k B lnn 1 N o = k B N o ln(n 2 / n 1 ) = R ln(n 2 / n 1 ) snce R = k B N o (where R s the gas constant, k B s Boltzmann s constant, and N o s Avogadro s number). hat s, S = R ln V 2 / δv V 1 / δv

13 6. Dscrete Outcomes: Methods of Physcs 70 or S = R ln V 2 V 1 (6.36) exactly as before (6.35). hus, we can see clearly how the crtcal equaton, S = k B ln W, permts the mappng of macroscopc thermodynamcs onto the propertes of molecules. APPLICAIONS OF ENROPY CHANGE IN CHEMISRY We recall from nequalty (6.16) that the physcal entropy of an solated system ncreases n every natural (rreversble) process. We now understand wth reference to the molecular representaton of entropy that the ncrease n thermodynamc entropy can be nterpreted as a progresson from a more-ordered to a less-ordered state. In chemstry, ths prncple of ncreasng physcal entropy s appled to determne whether a system wll change spontaneously. If a change, such as that produced by chemcal reacton, wll result n an ncrease n physcal entropy, the reacton s lkely to go forward (see, for example, Hargreaves and Socrates (1973)). HE PURPOSE OF HIS CHAPER In ths chapter we have made a survey of the concept of physcal entropy as t was born nto classcal thermodynamcs, and as t was translated, largely through the work of Boltzmann, nto statstcal mechancs. Physcal entropy was seen, through the S = k B ln W = k B H I relatonshp, to be proportonal to the nformaton theoretcal entropy, n the equlbrum state. hrough the use of S H I, classcal macroscopc thermodynamcs became nterpretable completely n terms of the propertes of molecules. he mappng process was not completed durng Boltzmann s lfetme. We learned from Chapter 5 that the nformaton concept, as t has been appled n communcatons scence and psychology and bochemstry, s non-unque and extrnsc to the system. By contrast, the use of the nformaton measure n physcs s both unque and ntrnsc. he measure H I = lnw (6.19) s a unque measure of uncertanty. No other measure s acceptable. No other measure wll permt the mappng of thermodynamcs onto statstcal mechancs. hat s, f H I = lnw s correct, our calculatons of physcal events wll be correct; but f H I = lnw were ncorrect, we would get wrong answers. Our calculatons of the thermodynamc entropy, nternal energy, free energy, etc., would be wrong. H I = lnw s ntrnsc n the sense that ths equaton enters physcs as a natural law. It s an ntegral part of physcs. here s yet another way of vewng the two statements,, (X Y) = H(X ) H(X, Y ) (4.21) and H I = ln W. (6.19) Both statements, the frst about, (X Y) and the second about H I, mght be called metastatements (Greek, meta: after). hat s, Equaton (4.21) arose n the case of a channel transmttng a message, or makng a statement. he statement s arbtrary; for example, he weather s fne today. hen Equaton (4.21) s a metastatement because t talks about the statement. It tells us how much nformaton s transmtted by the statement wthout actually tellng us what the statement s. Smlarly, H I = lnw s a metastatement, gvng us the uncertanty that any partcular mcrostate s assocated wth a macrostate. he correspondng statement would be We have mcrostate bc, a, d, -, - (see able 6.1). However, the statement s hdden from us. Instead, embedded as an ntegral part of our physcal law s

14 6. Dscrete Outcomes: Methods of Physcs 71 the metastatement he uncertanty about whch partcular mcrostate s present s the maxmum value of W consstent wth constrants... he contrasts and comparsons made above are mportant, because when I propose Equaton (2.6) 7 F = kh I, I suggest ts use n the physcst s sense rather than n the communcaton scentst s sense. hat s, I propose H I as a unque measure, ntrnsc to the physologcal functon of an organsm. Moreover, by ts use I propose that sensory messages that are relayed neuronally to the bran are, n realty, metastatements about the so-called external world. hat s, the bran wll receve only messages that express the state of uncertanty of the perpheral sensory receptors about the world external, but wll never receve messages that drectly descrbe the state of ths world. For example, the conscous bran wll not receve a message about the brghtness of a lght, but only about how uncertan the eye s about how brght the lght s (Norwch, 1977). hese deas now set the stage for development of the perceptual theory. here s, however, one more topc we must study by way of background before developng the perceptual model: the nformatonal entropy of contnuous dstrbutons. NOES 1. Clausus statement of the second law of thermodynamcs can be shown to be equvalent to Kelvn s statement of the second law (Sears, 1953) No process s possble whose sole result s the abstracton of heat from a sngle reservor and the performance of an equvalent amount of work. hat s, heat cannot be totally converted nto work. 2. A reversble process s one carred out wth the system effectvely n equlbrum wth ts envronment: a successon of equlbrum states. Pressure, temperature and densty of each porton of the system reman unform. 3. µ-space may have more than 6 dmensons f one allows for the nternal confguraton of the molecule. 4. Often, the coordnates are x, y, z, and p x, p y, p z, where the latter 3 coordnates are the momenta of the partcle, mv x, mv y, mv z. 5. Notce that we do not consder permutatons wthn a cell; representatve ponts ab and ba wthn a phase cell are consdered dentcal. 6. Equaton (6.20), as we have wrtten t, s, of course, an anachronsm. H I was not born, so to speak, untl the tme of Wener and Shannon. Boltzmann dd, ndeed, have an H-functon, but we shall not ntroduce t at ths tme. 7. he k n ths equaton s a constant, but not Boltzmann s constant. REFERENCES Brlloun, L Scence and Informaton heory. 2nd edton. Academc Press, New York. Brlloun, L Scentfc Uncertanty and Informaton. Academc Press, New York. Carnot, S Reflectons on the Motve Power of Fre (E. Medonza, ed.). Peter Smth, Gloucester, M.A. Desloge, E. A Statstcal Physcs. Holt, Rnehart and Wnston, New York. Freund, J.E. and Walpole, R.E Mathematcal Statstcs. 3rd edton. Prentce-Hall, Englewood Clffs, N.J. Hargreaves, G., and Socrates, G Elementary Chemcal hermodynamcs. Butterworths, London. Jackson, E.A Equlbrum Statstcal Mechancs. Prentce-Hall, Englewood Clffs N.J. Norwch, K.H On the nformaton receved by sensory receptors. Bulletn of Mathematcal Bology, 39, Sears, F.W An Introducton to hermodynamcs, the Knetc heory of Gases, and Statstcal Mechancs. 2nd ed. Addson-Wesley, Readng, MA. Zemansky, M.W Heat and hermodynamcs. McGraw-Hll, New York. Zemansky, M.W. and Dttman Heat and hermodynamcs: An Intermedate extbook, 6th edton. McGraw-Hll, N.Y.