Physics Sep The Binomial Distribution

Size: px

Start display at page:

Download "Physics Sep The Binomial Distribution"

Anne Hunt
6 years ago
Views:

1 Physics 30 3-Sep The Biomial Distributio As a example of workig with probabilities, we cosider the biomial distributio. We have N trials or N copies of similar systems. Each trial or system has two possible outcomes or states. We ca call these heads or tails (if the experimet is tossig a coi), spi up or spi dow (for spi / systems), etc. We suppose that each trial or system is idepedet ad we suppose the probability of heads i oe trial or spi up i oe system is p ad the probability of tails or spi dow is p = q. (Let s just call these up ad dow, I m gettig tired of all these words!) To completely specify the state of the system, we would have to say which of the N systems are up ad which are dow. Sice there are states for each of the N systems, the total umber of states is N. The probability that a particular state occurs depeds o the umber of ups ad dows i that state. I particular, the probability of a particular state with up spis ad N dow spis is Prob(sigle state with up spis) = p q N. Usually, we are ot iterested i a sigle state with up spis, but we are iterested i all the states that have up spis. We eed to kow how may there are. There is state with o up spis. There are N differet ways we have exactly oe of the N spis up ad N dow.therearen(n )/ ways to have two spis up. I geeral, there are ( ) N differet states with up spis. These states are distict, so the probability of gettig ay state with up spis is just the sum of the probabilities of the idividual states. So Prob(ay state with up spis) = p q N. Note that =(p + q) N = =0 ad the probabilities are properly ormalized. p q N, To illustrate a trick for computig average values, suppose that whe there are up spis, a measuremet of the variable y produces. What are the mea ad variace of y? To calculate the mea, we wat to perform the sum, Cosider the biomial expasio y = p q N. =0 (p + q) N = =0 p q N, Copyright c 999, Edward J. Groth

2 Physics 30 3-Sep ad observe that if we treat (for the momet) p ad q as idepedet mathematical variables ad we differetiate both sides of this expressio with respect to p (keepig q fixed), we get N(p + q) N = p q N. =0 The RHS is almost what we wat it s missig oe power of p. No problem, just multiply by p, Np(p + q) N = p q N. =0 This is true for ay (positive) values of p ad q. Now specialize to the case where p+ q =. The Np = p q N = y. A similar calculatio gives =0 var(y) =Npq. The fractioal spread about the mea is proportioal to N /. This is typical; as the umber of particles grows, the fractioal deviatios from the mea of physical quatities decreases i proportio to N /.Sowith N 0 umbers of particles, fractioal fluctuatios i physical quatities are 0. This is extremely small. Eve though the macroscopic parameters i statistical mechaics are radom variables, their fluctuatios are so small that they ca usually be igored. We speak of the eergy of a system ad write dow a sigle value, eve though the eergy of a system i thermal cotact with a heat bath is properly a radom variable which fluctuates cotiuously. Copyright c 999, Edward J. Groth

3 Physics 30 3-Sep Example A Spi System I the previous sectio, we discussed the biomial distributio. Now, I would like to add a little physical cotet by cosiderig a spi system. Actually this will be a model for a paramagetic material. We ll cosider a system with a large umber, N, of idetical spi / systems. As you kow, if you pick a axis, ad measure the compoet of agular mometum of a spi / system alog that axis, you ca get oly two aswers: + h/ ad h/. If there s charge ivolved, the there s a magetic momet, m, parallel or atiparallel to the agular mometum. If there s a magetic field, B, the this defies a axis ad the eergy m B of the spi system i the magetic field ca be either mb if the magetic momet is parallel to the field or +mb if the magetic momet is ati-parallel to the field. To save some writig, let E = mb > 0 so the eergy of a idividual system is ±E. I this model, we are cosiderig oly the eergies of the magetic dipoles i a exteral magetic field. We are igorig all other iteractios ad sources of eergy. For example, we are igorig magetic iteractios betwee the idividual systems, which meas we are dealig with a paramagetic material, ot a ferromagetic material. Also, we are igorig diamagetic effects effects caused by iduced magetic momets whe the field is established. Geerally, if there is a permaet dipole momet m, paramagetic effects domiate diamagetic effects. Of course, there must be some iteractios of our magets with each other or with the outside world or there would be o way for them to chage their eergies ad come to equilibrium. What we re assumig is that these iteractios are there, but just so small that we do t eed to cout them whe we add up the eergy. (Of course the smaller they are, the loger it will take for equilibrium to be established...) Our goal here is to work out expressios for the eergy, etropy, temperature, i terms of the umber of parallel ad atiparallel magetic momets. If there is o magetic field, the there is othig to pick out ay directio, ad we expect that ay give magetic momet or spi is equally likely to be parallel or atiparallel to ay directio we pick. So the probability of parallel should be the same as the probability of atiparallel should be /: p = p = q =/. If we tur o the magetic field, we expect that more magets will lie up parallel to the field tha atiparallel (p >q) so that the etire system has a lower total eergy tha it would have with equal umbers of magets parallel ad atiparallel. If we did t kow aythig about thermal effects, we d say that all the magets should alig with the field i order to get the lowest total eergy. But we do kow somethig about thermal effects. What we kow is that these magets are exchagig eergy with each other ad the rest of the world, so a maget that is parallel to the field, havig eergy Copyright c 999, Edward J. Groth

4 Physics 30 3-Sep E, might receive eergy +E ad alig atiparallel to the field with eergy +E. It will stay atiparallel util it ca give up the eergy E to a differet maget or to the outside world. The stregths of the iteractios determie how rapidly equilibrium is approached (a subject we will skip for the time beig), but the temperature sets a eergy scale ad determies how likely it is that chuks of eergy of size E are available. So suppose that of the magets are parallel to the field ad N are atiparallel. K&K defie the spi excess, as the umber parallel mius the umber atiparallel, s = (N ) = N or = s + N/. The eergy of the etire system is the U() = E +(N )E = ( N)E = se. The etropy is the log of the umber of ways our system ca have this amout of eergy ad this is just the biomial coefficiet. N! σ() =log =log (N/+s)! (N/ s)!. To put this i the cotext of our previous discussio of etropy ad eergy, ote that we talked about determiig the etropy as a fuctio of eergy, volume, ad umber of particles. I this case, the volume does t eter ad we re ot chagig the umber of particles (or systems) N. At the momet, we are ot writig the etropy as a explicit fuctio of the eergy. Istead, the two equatios above are parametric equatios for the etropy ad eergy. To fid the temperature, we eed σ/ U. I our formulatio, the etropy ad eergy are fuctios of a discrete variable, ot a cotiuous variable. No problem! We ll just sed oe maget from parallel to ati-parallel. This will make a chage i eergy, U, ada chage i etropy, σ ad we simply take the ratio as the approximatio to the partial derivative. So, U = U( ) U() =E, σ = σ( ) σ() ( ) ( ) N N =log log [ ] N!!(N )! =log ( )! (N +)! N! =log N + =log ca t matter if N N 0 N =log N/+s N/ s, Copyright c 999, Edward J. Groth

5 Physics 30 3-Sep where the last lie expresses the result i terms of the spi excess. Throwig away the is OK, provided we are ot at zero temperature where = N. The temperature is the τ = U σ = E log(n/+s)/(n/ s). At this poit it s coveiet to solve for s. Wehave N/+s N/ s = ee/τ, ad with a little algebra s N = tah E τ. The plot shows this fuctio fractioal spi excess versus E/τ. To the left, thermal eergy domiates magetic eergy ad the et aligmet is small. To the right, magetic eergy domiates thermal eergy ad the aligmet is large. Just what we expected! Suppose the situatio is such that E/τ is large. The the magets are all aliged. Now tur off the magetic field, leavig the magets aliged. What happes? The system is o loger i equilibrium. It absorbs eergy ad etropy from its surroudigs, coolig the surroudigs. This techique is actually used i low temperature experimets. It s called adiabatic demagetizatio. Demagetizatio refers to removig the exteral magetic field ad adiabatic refers to doig it getly eough to leave the magets aliged. Copyright c 999, Edward J. Groth

6 Physics 30 3-Sep The Boltzma Factor A additioal commet o probabilities: Whe the spi excess is s, the probabilities of parallel or atiparallel aligmet are: The ratio of the probabilities is p = + s N, q = s N. q p = s/n +s/n = e E/τ. This is a geeral result. The relative probability that a system is i two states with a eergy differece E is just Probability of high eergy state Probability of low eergy state = e E/τ = e E/kT. This is called the Boltzma factor. As we ve already metioed, this says that eergies < kt are easy to come by, while eergies >kt are hard to come by! The temperature sets the scale of the relevat eergies. The Gaussia Distributio We ve discussed two discrete probability distributios, the biomial distributio ad (i the homework) the Poisso distributio. As a example of a cotiuous distributio, we ll cosider the Gaussia (or ormal) distributio. It is a fuctio of oe cotiuous variable ad occurs throughout the scieces. The reaso the Gaussia distributio is so prevalet is that uder very geeral coditios, the distributio of a radom variable which is the sum of a large umber of idepedet, idetically distributed radom variables, approaches the Gaussia distributio as the umber of radom variables i the sum goes to ifiity. This result is called the cetral limit theorem ad is prove i probability courses. The distributio depeds o two parameters, the mea, µ, (ot the chemical potetial!) ad the stadard deviatio, σ (ot the etropy!). The probability desity is You should be able to show that p(x) = πσ µ) e (xσ. πσ + µ) (x e σ dx =, Copyright c 999, Edward J. Groth

7 Physics 30 3-Sep x = var(x) = πσ πσ + + µ) (x xe σ dx = µ, µ) (x µ) (x e σ dx = σ. Appedix A of K&K might be useful if you have trouble with these itegrals. Oe ca always receter so that x is measured from µ ad rescale so that x is measured i uits of σ. The the desity takes the dimesioless form, p(x) = π e x /. Sometimes you might eed itegrate this desity over a fiite (rather tha ifiite) rage. Two related fuctios are of iterest, the error fuctio erf(z) = π z ad the complemetary error fuctio erfc(z) = π 0 z e t dt = π z 0 e x / dx, e t dt = π z e x / dx, where the first expressio (ivolvig t) is the typical defiitio, ad the secod (obtaied by chagig variables t = x/ rewrites the defiitio i terms of the Gaussia probability desity. Note that erf(0) = 0, erf( ) =, ad erf(z) + erfc(z) =. The Gaussia desity is just the bell curve, peaked i the middle, with small tails. The error fuctio gives the probability associated with a rage i x at the middle of the curve, while the complemetary error fuctio gives probabilities associated with the tails of the distributio. I geeral, you have to look these up i tables, or have a facy calculator that ca geerate them. As a example, you might hear someoe at a research talk say, I ve obtaied a margial two-sigma result. What this meas is that the sigal that was detected was oly σ larger tha o sigal. A oise effect this large or larger will happe with probability π e x / dx = erfc =0.03. That is, more tha percet of the time, oise will give a σ result just by chace. This is why σ is margial. We re strayig a bit from thermal physics, so let s get back o track. Oe of the reasos for brigig up a Gaussia distributio is that may other distributios approach Copyright c 999, Edward J. Groth

8 Physics 30 3-Sep a Gaussia distributio whe large umbers are ivolved. (The cetral limit theorem might have somethig to do with this!) For example, the biomial distributio. Whe the umbers are large, we ca replace the discrete distributio i with a cotiuous distributio. The advatage is that it is ofte easier to work with a cotiuous fuctio. I particular, the probability of a spi excess, s, is p s = N! (N/+s)! (N/ s)! p N +s q N s. We eed to do somethig with the factorials. I K&K, Appedix A, Stirlig s approximatio is derived. For large N, N! πnn N e N. With this, we have πn N N p s π(n/+s)π(n/ s) (N/+s) (N/+s) (N/ s) (N/ s) pn/+s q N/ s p N/+s q N/ s = πn (/+s/n)(/ s/n) (/+s/n) (N/+s) (/ s/n) (N/ s) pq p N/+s q N/ s = πn (/+s/n)(/ s/n) pq (/+s/n) (N/+s) (/ s/n) (N/ s) ( ) (N/+s+/) ( ) (N/ s+/) p q =. πnpq /+s/n / s/n Recall that the variace of the biomial distributio is Npq, so thigs are startig to look promisig. Also, we are workig uder the assumptio that we are dealig with large umbers. This meas that s caot be close to ±N/. If it were, the we would have a small umber of aliged, or a small umber of ati-aliged magets. So, i the expoets i the last lie, N/ ± s is a large umber ad we ca igore the /. The p s = ( p πnpq /+s/n ) (N/+s) ( q / s/n ) (N/ s). This is a sharply peaked fuctio. We expect the peak to be cetered at s = s 0 = s = N/ =Np N/ =N(p /). We wat to expad this fuctio about its maximum. Actually, it will be easier to locate the peak ad expad the fuctio, if we work with its logarithm. log p s = A + ( )[ ( N + s log p log + s )] ( )[ ( N + N s log q log s )] N, Copyright c 999, Edward J. Groth

9 Physics 30 3-Sep where A = ( ) log. πnpq To locate the maximum of this fuctio, we take the derivative ad set it to 0 d log p s ds =logp log ( + s N ) log q +log ( s N ) +. We ote that this expressio is 0 whe s/n = p /, just as we expected. So this is the poit about which we ll expad the logarithm. The ext term i a Taylor expasio requires the secod derivative d log p s ds = N/+s N/ s = Np Nq = Npq, where, i the last lie, we substituted the value of s at the maximum. We ca expad the logarithm as log p s = A Npq (s s 0) + where s 0 = N(p /) is the value of s at the maximum. Fially, we let σ = Npq, expoetiate the logarithm, ad obtai, p(s) πσ e (s s 0) /σ, where the otatio has bee chaged to idicate a cotiuous variable rather tha a discrete variable. You might worry about this last step. I particular, we have a discrete probability that we just coverted ito a probability desity. I fact, p(s) ds is the probability that that the variable is i the rage s s + ds. I the discrete case, the spacig betwee values of s is uity, so we require, p(s) ( (s +) s ) = p s, which leads to p(s) =p s. Had there bee a differet spacig there would be a differet factor relatig the discrete ad cotiuous expressios. All this was a lot of work to demostrate i some detail that for large N (ad ot too large s), the biomial distributio describig our paramagetic system goes over to the Gaussia distributio. Of course, expadig the logarithm to secod order guaratees a Gaussia! I practice, you would ot go to all this trouble to do the coversio. The way you would actually do the coversio is to otice that large umbers are ivolved, so the Copyright c 999, Edward J. Groth

10 Physics 30 3-Sep distributio must be Gaussia. The all you eed to kow are the mea ad variace which you calculate from the biomial distributio or however you ca. The you just write dow the Gaussia distributio with the correct mea ad variace. Returig to our paramagetic system, we foud earlier that the mea value of the spi excess is s 0 = N tah E τ. We ca use the Gaussia approximatio provided s is ot too large compared to N/ which meas E < τ. I this case, a little algebra shows that the variace is σ = Npq = N ( ) seche. τ For give E/τ, the actual s fluctuates about the mea s 0 with a spread proportioal to N ad a fractioal spread proportioal to / N.AtypicalsystemhasN N0,sothe fractioal spread is of order 0 ad the actual s is always very close to s 0. While we re at it, it s also iterestig to apply Stirlig s approximatio to calculate the etropy of our paramagetic system. Recallig Stirlig s approximatio for large N, Takig the logarithm, we have N! πnn N e N. log N! log π + log N + N log N N. The first two terms ca be igored i compariso with the last two, so log N! N log N N. Suppose our spi system has s 0 0. The the etropy is N! σ log (N/)! (N/)! N log N N ( (N/) log(n/) (N/) ) = N log N N log(n/) = N log = (fudametal uits) = erg K (covetioal uits), where the last two lies assume oe mole of magets. Copyright c 999, Edward J. Groth

Physics Sep Example A Spin System

Physics Sep Example A Spin System Physics 30 7-Sep-004 4- Example A Spin System In the last lecture, we discussed the binomial distribution. Now, I would like to add a little physical content by considering a spin system. Actually this