8333: Statstcal Mechancs I Problem Set # 3 Solutons Fall 3 Characterstc Functons: Probablty Theory The characterstc functon s defned by fk ep k = ep kpd The nth coeffcent of the Taylor seres of fk epanded around k = gves the nth moment of as k n fk = n n! n= a A unform probablty dstrbuton for a < < a p = a otherwse for whch there est many eamples gves fk = a ep kd = a a a k ep k a a Therefore = ak snka = m akm m +! m= m = = and m = = 3 a b The Laplace PDF p = a ep a for eample descrbng lght absorpton through a turbd medum gves fk = d ep k a a = a = a d ep k /a + a k + /a = k /a = ak + ak 4 + ak d ep k + /a
Therefore m = = and m = = a c The Cauchy or Lorentz PDF descrbes the spectrum of lght scattered by dffusve modes and s gven by For ths dstrbuton fk = = π p = a π + a a ep k π + a d ep k a + a d The easest method for evaluatng the above ntegrals s to close the ntegraton contours n the comple plane and evaluate the resdue The vanshng of the ntegrand at nfnty determnes whether the contour has to be closed n the upper or lower half of the comple plane and leads to π fk = π C B ep k d = ep ka for k + a = ep ka ep k d = epka for k < a Note that fk s not an analytc functon n ths case and hence does not have a Taylor epanson The moments have to be determned by another method eg by drect evaluaton as m = = and m = = d π a + a The frst moment vanshes by symmetry whle the second and hgher moments dverge eplanng the non-analytc nature of fk d The Raylegh dstrbuton p = a ep a for can be used for the length of a random walk n two dmensons Its characterstc functon s fk = = ep k a ep d a cosk snk a ep a d
The ntegrals are not smple but can be evaluated as and resultng n cosk a ep d = a snk a ep d = a fk = n= n n! n! The moments can also be calculated drectly from m = = m = = a ep a 3 a ep = n= π ka ep n n! n! a k n snk a ep a k a a k n π ka ep k a a d = = a y ep ydy = a a ep d = a a ep a d π d = a d a e It s dffcult to calculate the characterstc functon for the Mawell dstrbuton p = π a 3 ep a say descrbng the speed of a gas partcle However we can drectly evaluate the mean and varance as and m = = π = π a = π a m = = 3 d a 3 ep a a ep a d a y ep ydy = π a 4 π o a 3 ep a d = 3a 3 a
At each step a drected random walk can move along angles θ and φ wth probablty pθ = θ π cos and pφ = π where the sold angle factor of sn θ s already ncluded n the defnton of pθ; pθdθ = π π cos θ π dθ = cos θ + dθ = π a From symmetry arguments = y = whle along the z-as z = z = Nz = Na cos θ = Na The last equalty follows from cos θ = = π π pθ cos θdθ = cos θ cos θ + dθ π π cos θ + dθ = The second moment of z s gven by z = j z z j = z z j + j z z = j z z j + = NN z + N z Notng that z a = π π π cos θcos θ + dθ = π cos θ + dθ = we fnd z a a = NN + N = NN + a 4 The second moments n the and y drectons are equal and gven by = j = N j = j j + 4
Usng the result we obtan a = sn θ cos φ = π π π dφ cos φ dθ sn θcos θ + = 4 = y = Na 4 Whle the varables y and z are not ndependent because of the constrant of unt length smple symmetry consderatons suffce to show that the three covarances are n fact zero e y = z = yz = b From the Central lmt theorem the probablty densty should be Gaussan However for correlated random varable we may epect cross terms that descrbe ther covarance Snce we showed above that the covarances between y and z are all zero we can treat them as three ndependent Gaussan varables and wrte y y p y z ep σ σy z z σz There wll be correlatons between y and z appearng n hgher cumulants but all such cumulants become rrelevant n the N lmt Usng the moments = y = and z = N a we obtan and p y z = σ = = N a 4 = σ y σ z = z z = NN + a 4 Na = N a 4 3/ ep + y + z Na/ πna Na / 3 Tchebycheff s Inequalty: By defnton for a system wth a PDF p and average λ the varance s σ = λ pd 5
Let us break the ntegral nto two parts as σ = λ pd + λ nσ λ <nσ λ pd resultng n Now snce we obtan and σ λ <nσ λ pd = λ pd λ nσ nσ pd σ λ nσ λ nσ λ nσ λ nσ λ <nσ pd n λ pd nσ pd λ pd σ 4 Optmal Selectons: a The probablty that the mamum of n random numbers falls between and + d s equal to the probablty that one outcome s n ths nterval whle all the others are smaller than e n p n = pr = r < r 3 < r n < where the second factor corresponds to the number of ways of choosng whch r α = As these events are ndependent n p n = pr = pr < pr 3 < pr n < = pr = pr < n n The probablty of r < s just a cumulatve probablty functon and n p n = n p prdr 6
b If each r α s unformly dstrbuted between and pr = prdr = dr = Wth ths PDF we fnd p n = n p and the mean s now gven by = The second moment of the mamum s resultng n a varance σ = = n n prdr = n dr = n n p n d = n n d = = n n+ d = n n + n n n + = n + n n + n n + n + Note that for large n the mean approaches the lmtng value of unty whle the varance vanshes as /n There s too lttle space at the top of the dstrbuton for a wde varance 5 Informaton: a For an unbased probablty estmaton we need to mamze entropy subject to the two constrants of normalzaton and of gven average speed v = c Usng Lagrange multplers α and β to mpose these constrants we need to mamze S = ln p = pv ln pvdv + α pvdv + β c pv v dv Etremzng the above epresson yelds S pv = ln pv α β v = whch s solved for or ln pv = α β v pv = Ce β v wth C = e α 7
The constrants can now be used to f the parameters C and β: = pvdv = Ce β v dv = C e βv dv = C β e βv = C β whch mples C = β From the second constrant we have c = Ce β v v dv = β e βv vdv whch when ntegrated by parts yelds or c = β β ve βv + β e βv dv = β e βv β = c = β The unbased PDF s then gven by pv = Ce β v = c ep v c b When the second constrant s on the average knetc energy mv / = mc / we have S = pv ln pvdv + α The correspondng etremzaton mc pvdv + β pv mv dv S pv = ln pv α β mv = results n The normalzaton constrant mples = pv = C ep βmv pvdv = C 8 e βmv / = C π/βm
or C = βm/π The second constrant mc = = m pv mv dv = m βm π π 3/ βm βm π = β ep βmv v dv gves for a full PDF of β = mc pv = C ep βmv = ep v πc c c The entropy of the frst PDF s gven by S = ln p = = lnc c ep v c v c ep c dv + c lnc v c ep v v c c dv = lnc ep v/c + c c ep v/c = lnc + = + ln + ln c For the second dstrbuton we obtan S = ln p = πc = ln πc πc = ln πc + dv ep v c ln πc v c ep v /c dv + πc πc c c πc = ln πc + = + lnπ + ln c For a dscrete probablty the nformaton content s I α = ln M S α / ln 9 v c ep v /c dv
where M denotes the number of possble outcomes Whle M and also the proper measure of probablty are not well defned for a contnuous PDF the ambgutes dsappear when we consder the dfference I I = S + S / ln = S S / ln ln π ln = ln 3956 Hence the constrant of constant energy provdes 3956 more bts of nformaton Ths s partly due to the larger varance of the dstrbuton wth constant speed 6 Benford s Law: Let us consder the observaton that the probablty dstrbuton for frst ntegers s unchanged under multplcaton by any e a random number Presumably we can repeat such multplcatons many tmes and t s thus suggestve that we should consder the propertes of the product of random numbers Why ths should be a good model for stock prces s not entrely clear but t seems to be as good an eplanaton as anythng else! Consder the = N = r where r are postve random varables taken from some reasonably well behaved probablty dstrbuton The random varable l ln = N = ln r s the sum of many random contrbutons and accordng to the central lmt theorem should have a Gaussan dstrbuton n the lmt of large N e lm pl = ep N l Nl Nσ πnσ where l and σ are the mean and varance of ln r respectvely The product s dstrbuted accordng to the log-normal dstrbuton p = pl dl d = ln Nl ep Nσ πnσ The probablty that the frst nteger of n a decmal representaton s s now obtaned appromately as follows: p = q q + q dp
The ntegral consders cases n whch s a number of magntude q e has q + dgts before the decmal pont Snce the number s qute wdely dstrbuted we then have to sum over possble magntudes q The range of the sum actually need not be specfed! The net stage s to change varables from to l = ln leadng to p = q q+ln+ q+ln dlpl = q q+ln+ q+ln dl ep l Nl Nσ πnσ We shall now make the appromaton that over the range of ntegraton q + ln to q + ln + the ntegrand s appromately constant The appromaton works best for q Nl where the ntegral s largest Ths leads to p q ep q Nl Nσ ln + ln ln + πnσ where we have gnored the constants of proportonalty whch come from the sum over q We thus fnd that the dstrbuton of the frst dgt s not unform and the properly normalzed proportons of ln+/ ndeed reproduce the probabltes p p 9 of 3 76 5 97 79 67 58 5 46 accordng to Benford s law For further nformaton check http://wwwtreasure-trovescom/math/benfordslawhtml