In-class exercises. Day 1, parts d, e

Transcription

1 Physics 4488/6562: Statistical Mechanics Material for Week 5 Exercises due Mon Feb 26 Last correction at February 4, 2018, 11:15 am c 2017, James Sethna, all rights reserved Pre-class Preparation All exercises are from Version 2.0 of the text: sethna/ StatMech/v2EntropyOrderParametersComplexity.pdf Friday Read: Chapter 5, Sec (Information entropy) Pre-class question: 5.22: Gravity and entropy (Submit electronically by 9:30 Thursday evening.) Monday Read: Chapter 6, Sec. 6.1 (Canonical Ensemble), 6.2 (Uncoupled Systems), and 6.3 (Grand canonical ensemble) Pre-class question: 6.15: Rubber band free energy. (Submit electronically by 9:30 Sunday evening.) Exercises Those in 4488 may choose two of the four exercises. 5.3: Reversible computatation. 5.7: Does entropy increase?. 5.15: Shannon entropy. 5.17: Deriving entropy.

2 In-class exercises Day 1, parts d, e 5.12 Rubber band. (Condensed matter) i Figure 1 shows a one-dimensional model for rubber. Rubber is formed from long polymeric molecules, which undergo random walks in the undeformed material. When we stretch the rubber, the molecules respond by rearranging their random walk to elongate in the direction of the external stretch. In our model, the molecule is represented by a set of N links of length d, which with equal energy point either parallel or antiparallel to the previous link. Let the total change in position to the right, from the beginning of the polymer to the end, be L. As the molecule extent L increases, the entropy of our rubber molecule decreases. (a) Find an exact formula for the entropy of this system in terms of d, N, and L. (Hint: How many ways can one divide N links into M right-pointing links and N M left-pointing links, so that the total length is L?) d L Fig. 1 Rubber band. Simple model of a rubber band with N = 100 segments. The beginning of the polymer is at the top; the end is at the bottom; the vertical displacements are added for visualization. The external world, in equilibrium at temperature T, exerts a force pulling the end of the molecule to the right. The molecule must exert an equal and opposite entropic force F. (b) Find an expression for the force F exerted by the molecule on the bath in terms of the bath entropy. (Hint: The bath temperature 1/T = S bath / E, and force times distance is energy.) Using the fact that the length L must maximize the entropy of the Universe, write a general expression for F in terms of the internal entropy S of the molecule. (c) Take our model of the molecule from part (a), the general law of part (b), and Stirling s formula log(n!) n log n n, write the force law F (L) for our molecule for large lengths N. What is the spring constant K in Hooke s law F = KL for our molecule, for small L?

3 Our model has no internal energy; this force is entirely entropic. Note how magical this is we never considered the mechanism of how the segments would generate a force. Statistical mechanics tells us that the force generated by our segmented chain is independent of the mechanism. The joint angles in the chain may jiggle from thermal motion, or the constituent polymer monomers may execute thermal motion so long as the configuration space is segment orientations and the effective potential energy is zero the force will be given by our calculation. For the same reason, the pressure due to compressing an ideal gas is independent of the mechanism. The kinetic energy of particle collisions for real dilute gases gives the same pressure as the complex watersolvent interactions give for osmotic pressure (Section 5.2.2). (d) If we increase the temperature of our rubber band while it is under tension, will it expand or contract? Why? In a more realistic model of a rubber band, the entropy consists primarily of our configurational random-walk entropy plus a vibrational entropy of the molecules. If we stretch the rubber band without allowing heat to flow in or out of the rubber, the total entropy should stay approximately constant. (Rubber is designed to bounce well; little irreversible entropy is generated in a cycle of stretching and compression, so long as the deformation is not too abrupt.) (e) True or false? (T) (F) When we stretch the rubber band, it will cool; the configurational entropy of the random walk will decrease, causing the entropy in the vibrations to decrease, causing the temperature to decrease. (T) (F) When we stretch the rubber band, it will cool; the configurational entropy of the random walk will decrease, causing the entropy in the vibrations to increase, causing the temperature to decrease. (T) (F) When we let the rubber band relax, it will cool; the configurational entropy of the random walk will increase, causing the entropy in the vibrations to decrease, causing the temperature to decrease. (T) (F) When we let the rubber band relax, there must be no temperature change, since the entropy is constant. This more realistic model is much like the ideal gas, which also had no configurational energy. (T) (F) Like the ideal gas, the temperature changes because of the net work done on the system. (T) (F) Unlike the ideal gas, the work done on the rubber band is positive when the rubber band expands. You should check your conclusions experimentally; find a rubber band (thick and stretchy is best), touch it to your lips (which are very sensitive to temperature), and stretch and relax it.

4 5.13 How many shuffles? (Mathematics) i Day 1 For this exercise, you will need a deck of cards, either a new box or sorted into a known order (conventionally with an ace at the top and a king at the bottom). 1 How many shuffles does it take to randomize a deck of 52 cards? The answer is a bit controversial; it depends on how one measures the information left in the cards. Some suggest that seven shuffles are needed; others say that six are enough. 2 We will follow reference [2], and measure the growing randomness using the information entropy. We imagine the deck starts out in a known order (say, A, 2,..., K ). (a) What is the information entropy of the deck before it is shuffled? After it is completely randomized? (b) Take a sorted deck of cards. Pay attention to the order; (in particular, note the top and bottom cards). Riffle it exactly once, separating it into two roughly equal portions and interleaving the cards in the two portions.. Examine the card sequence, paying particular attention to the top few and bottom few cards. Can you tell which cards came from the top portion and which came from the bottom? The mathematical definition of a riffle shuffle is easiest to express if we look at it backward. 3 Consider the deck after a riffle; each card in the deck either came from the top portion or the bottom portion of the original deck. A riffle shuffle makes each of the 2 52 patterns tbbtbttb... (denoting which card came from which portion) equally likely. It is clear that the pattern tbbtbttb... determines the final card order: the number of t s tells us how many cards were in the top portion, and then the cards are deposited into the final pile according to the pattern in order bottom to top. Let us first pretend the reverse is also true: that every pattern corresponds one-to-one with a unique final card ordering. (c) Ignoring the possibility that two different riffles could yield the same final sequence of cards, what is the information entropy after one riffle? 1 Experience shows that those who do not do the experiment in part (b) find it challenging to solve part (c) by pure thought. 2 More substantively, as the number of cards N, some measures of information show an abrupt transition near 3 / 2 log 2 N, while by other measures the information vanishes smoothly and most of it is gone by log 2 N shuffles. 3 In the forward definition of a riffle shuffle, one first cuts the deck into two portions, according to a binomial distribution: the probability that n cards are chosen for the top portion is 2 52( ) 52 n. (This mimics an expert riffler, who will split the deck roughly, but not precisely, into two equal portions.) We then drop cards in sequence from the two portions into a pile, with the probability of a card being dropped proportional to the number of cards remaining in its portion. You can check that this makes each of the 2 52 choices in the backward definition equally likely.

5 You can convince yourself that the only way two riffles can yield the same sequence is if all the cards in the bottom portion are dropped first, followed by all the cards in the top portion. (d) How many riffles drop the entire bottom portion and then the entire top portion, leaving the card ordering unchanged? What fraction of the 2 52 riffles does this correspond to? (Hint: 2 10 = Indeed, this approximation underlies measures of computing resources: a gigabyte is not 10 9 bytes, but (1024) 3 = 2 30 bytes.) Hence, what is the actual information entropy after one riffle shuffle? We can put a lower bound on the number of riffles needed to destroy all information by assuming the entropy increase stays constant for future shuffles. (e) Continuing to ignore the possibility that two different sets of m riffles could yield the same final sequence of cards, how many riffles would it take for the entropy to pass that of a completely randomized deck? Day Aging, entropy, and DNA. (Biology) 3 Is human aging inevitable? Does the fact that entropy must increase mean that our cells must run down? In particular, as we get older the DNA in our cells gradually builds up damage thought to be a significant contribution to the aging process (and a key cause of cancer). Can we measure DNA damage using entropy, to quantify the challenge of keeping ourselves young? There are roughly thirty trillion ( ) cells in the human body, and about three billion ( ) nucleotides in the DNA of each cell. Each nucleotide comes in four types (C, T, A, and G). The damaged DNA of each cell will be different. The repair job for fixing all of our DNA cannot be worse than changing totally random sequences back into exact copies of the correct sequence. (a) How many bits of information is it possible to store in the nucleotide sequence of the DNA in an entire human body? How much entropy, in joules/kelvin, is associated with a completely randomized sequence in every cell? (Boltzmann s constant is J/K.) Life exists by consuming low-entropy sources and turning them into higher-entropy biproducts. A small cookie has 100 Calories. (Be warned: a Calorie is 1000 calories. Food calories are measured in kilocalories, and then the kilo is conventionally dropped in favor of a capital C.) Body temperature is about 310K. (b) Calculate the minimum free energy needed to repair a human s DNA if it starts in a completely scrambled state. How many cookies would one need to consume? (There are joules per calorie, and 1000 calories per Calorie.) Entropy does not discriminate between important and trivial information. Knowing that air molecules are confined in a balloon is much less useful than knowing that your

6 kid s toys are put away neatly (or knowing the contents of the Library of Congress), and there are a lot of air molecules... Day Entropy increases: diffusion. (Mathematics) i We saw that entropy technically does not increase for a closed system, for any Hamiltonian, either classical or quantum. However, we can show that entropy increases for most of the coarse-grained effective theories that we use in practice; when we integrate out degrees of freedom, we provide a means for the information about the initial condition to be destroyed. Here you will show that entropy increases for the diffusion equation. Let ρ(x, t) obey the one-dimensional diffusion equation ρ/ t = D 2 ρ/ x 2. Assume that the density ρ and all its gradients die away rapidly at x = ±. 4 Derive a formula for the time derivative of the entropy S = k B ρ(x) log ρ(x) dx and show that it strictly increases in time. (Hint: Integrate by parts. You should get an integral of a positive definite quantity.) Day 3, part c 5.4 Black hole thermodynamics. (Astrophysics) 3 Astrophysicists have long studied black holes: the end state of massive stars which are too heavy to support themselves under gravity (see Exercise 7.16). As the matter continues to fall into the center, eventually the escape velocity reaches the speed of light. After this point, the in-falling matter cannot ever communicate information back to the outside. A black hole of mass M has radius 5 R s = G 2M c 2, (1) where G = cm 3 /g s 2 is the gravitational constant, and c = cm/s is the speed of light. Hawking, by combining methods from quantum mechanics and general relativity, calculated the emission of radiation from a black hole. 6 He found a wonderful result: black holes emit perfect black-body radiation at a temperature T bh = c 3 8πGMk B. (2) According to Einstein s theory, the energy of the black hole is E = Mc 2. 4 Also, you may assume n ρ/ x n log ρ goes to zero at x = ±, even though log ρ goes to. 5 This is the Schwarzschild radius of the event horizon for a black hole with no angular momentum or charge. 6 Nothing can leave a black hole; the radiation comes from vacuum fluctuations just outside the black hole that emit particles.

7 (a) Calculate the specific heat of the black hole. The specific heat of a black hole is negative. That is, it gets cooler as you add energy to it. In a bulk material, this would lead to an instability; the cold regions would suck in more heat and get colder. Indeed, a population of black holes is unstable; the larger ones will eat the smaller ones. 7 (b) Calculate the entropy of the black hole, by using the definition of temperature 1/T = S/ E and assuming the entropy is zero at mass M = 0. Express your result in terms of the surface area A = 4πR 2 s, measured in units of the Planck length L = G/c 3 squared. As it happens, Bekenstein had deduced this formula for the entropy somewhat earlier, by thinking about analogies between thermodynamics, information theory, and statistical mechanics. On the one hand, when black holes interact or change charge and angular momentum, one can prove in classical general relativity that the area can only increase. So it made sense to assume that the entropy was somehow proportional to the area. He then recognized that if you had some waste material of high entropy to dispose of, you could ship it into a black hole and never worry about it again. Indeed, given that the entropy represents your lack of knowledge about a system, once matter goes into a black hole one can say that our knowledge about it completely vanishes. 8 (More specifically, the entropy of a black hole represents the inaccessibility of all information about what it was built out of.) By carefully dropping various physical systems into a black hole (theoretically) and measuring the area increase compared to the entropy increase, he was able to deduce these formulæ purely from statistical mechanics. We can use these results to provide a fundamental bound on memory storage. (c) Calculate the maximum number of bits that can be stored in a sphere of radius one centimeter. Finally, in perhaps string theory s first physical prediction, your formula for the entropy (part (b)) was derived microscopically for a certain type of black hole. Day Information entropy. (Computer science, Mathematics, Complexity) i Entropy is a measure of your ignorance about a system; it is a measure of the lack of information. It has important implications in communication technologies: messages passed across a network communicate information, reducing the information entropy for the receiver. 7 A thermally insulated glass of ice water also has a negative specific heat. The surface tension at the curved ice surface will decrease the coexistence temperature a slight amount (see Section 11.3); the more heat one adds, the smaller the ice cube, the larger the curvature, and the lower the resulting temperature [1]. 8 Except for the mass, angular momentum, and charge. This suggests that baryon number, for example, is not conserved in quantum gravity. It has been commented that when the baryons all disappear, it will be hard for Dyson to build his progeny out of electrons and neutrinos (Exercise 5.1).

8 Your grandparent has sent you an message. From the header of the message, you know it contains 1000 characters. You know each character is made of 8 bits, which allows 2 8 = 256 different letters or symbols per character. (a) Assuming all possible messages from your grandparent are equally likely (a typical message would then look like G*me!8V[beep]... ), how many different messages N could there be? What is the corresponding upper bound S max for the information entropy k S log N? Your grandparent writes rather dull messages; they all fall into the same pattern. They have a total of 16 equally likely messages. 9 After you read the message, you forget the details of the wording anyhow, and only remember these key points of information. (b) What is the actual information entropy change S Shannon you undergo when reading the message? If your grandparent writes one message per month, what is the minimum number of 8-bit characters per year that it would take to send your grandparent s messages? (You may lump multiple messages into a single character.) (Hints: S Shannon is your change in entropy from before you read the message to after you read which of 16 messages it was. The length of 1000 is not important for this part.) This is an extreme form of lossy data compression, like that used in jpeg images, mpeg animations, and mp3 audio files. We are asking for the number of characters per year for an optimally compressed signal. References [1] Nielsen, O. H., Sethna, J. P., Stoltze, P., Jacobsen, K. W., and Nørskov, J. K. (1994). Melting a copper cluster: Critical-droplet theory. Europhysics Letters, 26, [2] Trefethen, L. N. and Trefethen, L. M. (2000). How many shuffles to randomize a deck of cards? Proceedings of the Royal Society of London A, 456, Each message mentions whether they won their bridge hand last week (a fifty-fifty chance), mentions that they wish you would write more often (every time), and speculates who will win the women s college basketball tournament in their region (picking at random one of the eight teams in the league).