1 Introduction: within and beyond the normal approximation

Size: px

Start display at page:

Download "1 Introduction: within and beyond the normal approximation"

Randolph Montgomery
5 years ago
Views:

1 Tel Aviv Uiversity, 205 Large ad moderate deviatios Itroductio: withi ad beyod the ormal approximatio a Mathematical prelude b Physical prelude a Mathematical prelude Tossig fair cois we get a radom variable S distributed biomially, P S = k ) ) = 2! = for k = 0,,..., ; k 2 k! k)! it is asymptotically ormal: for all x R, 2S P ) x x 2π e u2 /2 du } {{ } Φx) as ; ad o the other had, Φx) = x e x2 /2 + o) ) as. 2π }{{} ϕx)=φ x) Does it mea that P 2S x ) ϕx) for large ad x? Yes ad o. x Yes if meas a small absolute error; but this is trivial: both sides are 0. No if it meas a small relative error; ideed, for x > the biomial probability is 0, while its ormal approximatio is ot. Well, what happes for x =? The biomial probability is 2, while its ormal approximatio is roughly e /2 ; quite bad: 2 e. The Stirlig formula leads to P S = k ) = 2 a 2! = e 2π + O ) ) 2π e γa) +O a ) ) where a = 2k

2 Tel Aviv Uiversity, 205 Large ad moderate deviatios 2 for all ad k = 0,,..., ; here a) γa) = 2 + a) l + a) + a) l a) for a, ), 2 γ ) = γ+) = l 2. γ Takig ito accout that P S = k + ) P S = k ) = k k + a + a, P S = k + 2 ) P S = k ) a ) 2,... + a we get for k > /2 P S k ) P S = k ) + a 2a, thus, e γa) as a approximatio to P 2S a ) is rather crude, but still much better tha the ormal approximatio i the limit, a = cost > 0. Some umeric data for = 50: k P MD/P LD/P Here P = P S k ), MD = Φ ) 2k, LD = exp γ 2k )). We see that the ormal approximatio is better for k 40 moderate deviatios ) ad worse for k 46 large deviatios ). Much better approximatios are available, the so-called strog moderate deviatios ad strog large deviatios: 2 x 2 x ) ) smd = MD exp 2 γ where x = 2k ; There is o official defiitio of moderate. I the cotext of sums of i.i.d. radom variables with expoetial momets) moderate meas a = o). For more geeral thoughts, see: Iglot et al. 992, A. Prob. 20:2, ). 2 The word strog is overloaded. Here I use it, followig Chagaty ad Sethurama 993, A. Prob. 2:3, But sometimes it meas that covergece of distributios results from covergece of radom variables as i: Iglot et al. 992).

3 Tel Aviv Uiversity, 205 Large ad moderate deviatios 3 sld = LD Now, for = 50 agai, + a 2π a a where a = 2k k P smd/p sld/p ad for = 000, k P smd/p sld/p A woder: smd looks better tha sld i all cases. Also a woder: we ca compute easily such probability as However, what is it really good for? Does it matter that P S ) is rather tha ? Moreover, does it matter that it is ot 0? Tossig 000 fair cois we may be pretty sure that heads will ot appear 950 times. Not eve oce i ay feasible umber of trials. Is it reasoable to say that, for all practical purposes, a) ? b) ? My aswers: b) sometimes it is, but ot always; a) I am ot sure; maybe, always. The reaso is related to statistical physics. b Physical prelude To uderstad why rare evets are importat at all, oe oly has to thik of a lottery to be coviced that rare evets such as hittig the jackpot) ca have a eormous impact. Amir Dembo ad Ofer Zeitoui 2 The umbers that arise i statistical mechaics ca defeat your calculator. A googol is 0 00 oe with a hudred zeros after it). A googolplex is 0 googol. James P. Setha 3 Really, I do ot kow, why. A feature of the symmetric) biomial case? Or a maifestatio of a more geeral pheomeo? 2 See page i the book Large deviatios techiques ad applicatios, Joes ad Bartlett Publ., See page 54 i the book Statistical mechaics: etropy, order parameters, ad complexity, Oxford,

4 Tel Aviv Uiversity, 205 Large ad moderate deviatios 4 Small probabilities, such as 0 6, are importat for lotteries, reliability etc., which caot be said about much smaller probabilities, such as However, these mosters do appear i statistical physics as e c where the umber of particles like = 0 20 is quite usual). a physical questio A system of so-called spi-/2 particles is described by the cofiguratio space {, }. Each cofiguratio x,..., x ) {, } has its eergy ) x + + x H x,..., x ) = f, where f : [, ] R is a give smooth fuctio ot depedig o ). If the system is i thermal equilibrium with a heat bath at temperature T, the each cofiguratio x,..., x ) appears with the probability cost exp ) k B T H x,..., x ), where k B = J/K) is the so-called Boltzma costat. For large, up to small fluctuatios, the eergy per particle f x + +x ) is a fuctio of the temperature. Fid this fuctio. a solutio The distributio P β of the umber k of spis +, correspodig to the so-called iverse temperature β =, is k B T 2k ) ) P β k) = P 0 k) cost β, exp βf, where P 0 is the fair coi biomial distributio treated i Sect. a). We ote that 2k ) )) P 0 k) = exp γ + o) Assumig that the spis iteract oly with the same magetic field gx + +x )/) that depeds o the mea field x + + x )/ via a fuctio g describig geerally, oliear) magetic properties of the eviromet. Thus, fs) = sgs). See also Sect. 9 i: R.S. Ellis, The theory of large deviatios ad applicatios to statistical mechaics, 2006, rsellis/pdf-files/dresde-lectures.pdf; ad Sect i: D. Yoshioka, Statistical physics, Spriger, 2007.

5 Tel Aviv Uiversity, 205 Large ad moderate deviatios 5 where o) as ) is uiform over all k such that 2k is bouded away from. Thus, 2k ) 2k ) )) P β k) = cost β, exp γ + βf + o). Assumig that the fuctio γ + βf has a sigle miimum a β, ) we see that P β cocetrates for large ) o k such that 2k a β. The eergy per particle is therefore fa β ) + o). Cosider, for example, the simple case fa) = a a exteral magetic field oly). We have γ + βf) a β ) = 0, that is, γ a β ) = β; geerally γ a) = +a l ; thus, +a β 2 a a β = e 2β ; a β = eβ e β = tah β. e β + e β Note that a β as β, ad o woder; at low temperature the eergy is roughly miimal. Note that P β is cocetrated o a set of k) of very small probability P 0 ; ideed, expoetially small i ). Takig ito accout that = 0 20 is usual, we observe the probability about exp 0 20 ). Surely a umber that ca defeat a calculator! Why does such a tiy probability matter? Because of the iterplay betwee differet probability measures related through expoetially small or large umbers. This is why we caot replace a small probability with zero. O the other had, a rough approximatio, of the form exp γ... ) + o) )), is all we eed. It meas that, for istace, 0 27 is a reasoably good approximatio for , sice their logarithms are relatively close. Likewise, exp 0 20 ) is a good approximatio for exp0 5 ) exp 0 20 ) i this framework. What about the distictio betwee ad ? Ca it matter i aother framework? I priciple, why ot; but I did ot face such situatios. Still, a eighborhood of ± does ot harm; for ow I do ot explai, why.

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite