5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY

Size: px

Start display at page:

Download "5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY"

Dorothy Lynne Jefferson
5 years ago
Views:

1 IA Probability Let Term 5 INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY 51 Iequalities Suppose that X 0 is a radom variable takig o-egative values ad that c > 0 is a costat The P X c E X, c is Markov s iequality It follows because P X c = E X I X c E c I X c E X = E X c c From this result we may deduce Chebyshev s iequality: for ay radom variable X ad costat c > 0, This follows by observig that P X c E X 2 c 2 P X c = P X 2 c 2, ad applyig Markov s iequality for the radom variable Y = X 2 ad costat c 2 We should ote the followig poits about Chebyshev s iequality: 1 The iequality is distributio free ; it holds for all radom variables irrespective of the distributio of the radom variable 2 If E X 2 c 2, the the iequality provides o boud o the probability 3 If E X 2 < c 2, the iequality is the best possible i the sese that give c there is a radom variable X for which the iequality holds with equality To see this suppose that d < c 2 ad let X be the radom variable X = c c with probability with probability d 2c 2, d 2c 2, 0 with probability 1 d c 2 85

2 The E X 2 = d ad P X c = d/c 2 = E X 2 /c 2 Suppose that φ : [0, [0, is a o-decreasig fuctio, with φx > 0 for x > 0, the we obtai the geeralized Chebyshev s iequality: for ay radom variable X ad c > 0, P X c E φ X φc This follows i the same way by observig that P X c P φ X φc, ad usig Markov s iequality with Y = φ X As a example, take φx = x 4 ad we obtai P X c E X 4 /c 4 The ext iequality ivolvig radom variables that we will cosider is based o the cocept of covexity A fuctio f : R R is covex if for all x 1, x 2 R ad λ 1 0, λ 2 0 with λ 1 + λ 2 = 1, we have fx fλ 1 x 1 + λ 2 x 2 λ 1 fx 1 + λ 2 fx 2 x 1 x 2 Thus a fuctio is covex if the chord joiig ay two poits x 1, fx 1 ad x 2, fx 2 o the graph of the fuctio lies above the fuctio betwee the poits It is easy to see that if fx is a covex fuctio the for x 1 < x 2 < x 3, the slope of the chord joiig the poits x 1, fx 1 ad x 2, fx 2 is less tha or equal to the slope of the chord joiig the poits x 2, fx 2 ad x 3, fx 3 ; that is this follows from the defiitio of covexity, sice fx 2 fx 1 x 2 x 1 fx 3 fx 2 x 3 x 2 ; 51 x3 x 2 x 2 = x 3 x 1 x 1 + x2 x 1 x 3 x 1 x 3, so that x3 x 2 fx 2 x 3 x 1 fx 1 + x2 x 1 x 3 x 1 fx 3, 52 86

3 ad rearragig 52 gives 51 Furthermore, from 51, it is immediate that for poits x 1 < x 2 < x 3 < x 4, we have sice we may apply 51 twice to obtai fx 2 fx 1 x 2 x 1 fx 4 fx 3 x 4 x 3, 53 fx 2 fx 1 x 2 x 1 fx 3 fx 2 x 3 x 2 fx 4 fx 3 x 4 x 3 54 It is the case that a fuctio f is covex if ad oly if 51 holds for all choices of x 1 < x 2 < x 3 Moreover, whe f is differetiable the f beig covex is equivalet to the derivative f x beig o-decreasig or f x 0; this may be see by lettig x 2 x 1 ad x 4 x 3 i 53 With a similar argumet, we may see that, whe f is covex, the fy fx y xf x, for all x, y 55 Now if f is a covex fuctio, for each 2, ad for ay poits x 1,, x R ad ay λ i 0, 1 i, with λ λ = 1, we have f λ 1 x λ x λ 1 fx λ fx 56 The proof of the iequality 56 is by iductio o The case = 2 is just the defiitio of covexity So assume that 56 holds for ay poits x 1,, x R ad ay λ i 0, 1 i, with λ λ = 1, ad suppose that we are give x 1,, x +1 R ad λ i 0, 1 i + 1, with λ λ +1 = 1 We may assume that λ i > 0 each i, otherwise the result follows by the iductive step immediately The, by first usig the case = 2 ad the the iductive hypothesis, we have +1 λ i f λ i x i = f 1 λ +1 x i + λ +1 x +1 1 λ +1 λ i 1 λ +1 f x i + λ +1 fx +1, 1 λ λ i 1 λ +1 fx i + λ +1 fx +1 = λ i fx i, 1 λ +1 87

4 completig the iductio Jese s Iequality For a radom variable X ad a covex fuctio f, fe X E fx The proof of Jese s Iequality i the case whe X takes o just fiitely may values x 1,, x, with probabilities p i = PX = x i, 1 i, with p i 0 ad p i = 1, is just a restatemet of 56, sice fe X = f p i x i p i fx i = E fx I geeral, for ay radom variable, we may use 55, to see that fx fe X X E Xf E X, ad takig the expectatio of both sides we see that E fx fe X E X E X f E X = 0, which gives the result Example 57 Arithmetic-Geometric Mea Iequality For positive real umbers x 1, x, 1/ x i 1 x i This follows by Jese s iequality applied to the covex fuctio fx = log x, ad the radom variable X which takes the value x i with probability 1, so that 1 1 log x i = E logx loge X = log x i from which we see that log 1/ x i 1 log x i, which gives the result, sice log is a icreasig fuctio 88

5 52 Weak Law of Large Numbers Theorem 58 Weak Law of Large Numbers Let X 1, X 2, be idepedet, idetically distributed radom variables with E X 1 = µ ad Var X 1 < For ay costat ɛ > 0, X X P µ > ɛ 0, as Proof By Chebyshev s iequality we have X X P µ > ɛ 1 ɛ 2 E X1 + + X 2 µ = 1 2 ɛ 2 E X X µ 2 = 1 2 ɛ 2 Var X X = = 1 ɛ 2 Var X 1 0, 2 ɛ 2 Var X 1 as required Notes 1 The statemet i Theorem 58 is ormally referred to by sayig that the radom variable X X / coverges i probability to µ, writte X X P µ, as 2 This result should be distiguished from the Strog Law of Large Numbers which states that X1 + + X P µ, as = 1 As the ame implies, the Strog Law of Large Numbers implies the Weak Law The mode of covergece i the Strog Law is referred to as covergece with probability oe or almost sure covergece 3 Notice that the requiremet that the radom variables i the Weak Law be idepedet is ot oe that we may dispese with For example, suppose that Ω = {ω 1, ω 2 } has just two poits ad let X ω 1 = 1 ad X ω 2 = 0 for each, so that the radom 89

6 variables are idetically distributed, but ot of course idepedet Let p = P {ω 1 } = 1 P {ω 2 }, where 0 < p < 1, the E X 1 = p, ad we have X 1 ω X ω 1 = 1 ad X 1 ω X ω 2 = 0, for all, so that the coclusio of Theorem 58 caot hold 4 By givig a more refied argumet it is possible to dispese with the requiremet i the statemet of the Theorem that Var X 1 < The coclusio still holds provided E X 1 < 5 It should be oticed that the Weak Law of Large Numbers is distributio free i that the particular distributio of the summads {X i } oly iflueces the result through the mea, µ, ad, i the form we have stated it, through the fact that the variace is fiite but otherwise the uderlyig distributio does ot eter the coclusio of the Theorem 6 The Weak Law of Large Numbers uderlies the frequetist iterpretatio of probability Suppose that we have idepedet repetitios of a experimet, ad we set X i = 1 if a particular outcome occurs o the ith repetitio eg, Heads, ad X i = 0, otherwise eg, Tails The E X i = p, say, where p = P X i = 1 is the probability of the outcome The X X / is the average umber of occurreces of the outcome i repetitios ad this coverges i the above sese to E X i = p; thus the probability p is the log-ru proportio of times that the outcome occurs 53 Cetral Limit Theorem Theorem 59 Cetral Limit Theorem Let X 1, X 2, be idepedet, idetically distributed radom variables with E X 1 = µ ad σ 2 = Var X 1, where 0 < σ 2 < For ay x, < x <, lim P X1 + + X µ σ x = 1 x e y2 /2 dy = Φx, 2π which is the distributio fuctio of the stadard N0, 1 distributio Sketch of Proof: We will illustrate why the result of the Theorem holds by usig momet geeratig fuctios i the case whe the momet geeratig fuctio of the {X i }, mθ = 90

7 E e θx 1, satisfies mθ < for a o-trivial rage of values of θ that is, a ope iterval which ecessarily cotais the poit θ = 0 It should be oted that this coditio o the momet geeratig fuctio is ot ecessary for the result to hold Let Y = X X µ σ, ad m θ = E e θy, the we will show that as, m θ e θ2 /2, which is the momet geeratig fuctio of the N0, 1 distributio This is sufficiet to establish the coclusio of the Theorem, although we will ot prove that i this course We ow have m θ = E e θx 1+ +X µ/σ = e θµ /σ E e θx 1+ +X /σ ad sice the {X i } are idepedet [ ad idetically distributed, this = e θµ /σ E e θx 1/σ ] [ ] θ = e θµ/σ m σ Expad the two terms usig Taylor s Theorem to see that m θ equals [ 1 θµ σ + θ2 µ 2 1 2σ 2 + O 1 + θ 3/2 σ m 0 + ] θ2 1 2σ 2 m 0 + O ; 3/2 ow, usig the fact that m 0 = E X 1 = µ, ad m 0 = E X 2 1 = σ 2 + µ 2, this shows that m θ = [1 θ2 µ 2 σ 2 + θ2 σ 2 + µ 2 2σ 2 + θ2 µ 2 ] 1 2σ 2 + O 3/2 ] = [1 + θ O e θ2 /2, as required 3/2 Notes 1 The mode of covergece described i Theorem 59 for the radom variables Y = X X µ /σ D is kow as covergece i distributio ad the coclusio is writte as Y Z, where Z N0, 1 2 Note that, like the Weak Law of Large Numbers, the Cetral Limit Theorem is distributio free, i that the uderlyig distributio of the {X i } iflueces the form of the result oly through the mea µ = E X 1 ad variace σ 2 = Var X 1 91

8 3 Note that i the Cetral Limit Theorem, by subtractig off at each stage the mea of the sum of the radom variables X X, that is µ, ad dividig by its stadard deviatio, σ, we are esurig that the radom variable Y has E Y = 0 ad Var Y = 1, for each Example 510 Normal approximatio to the biomial distributio If the radom variable Y Bi, p, we may thik of the distributio of Y as beig the same as that of the sum of iid radom variables each of which has the Beroulli distributio; thus the radom variable Y p/ p1 p has approximately the N0, 1 distributio for large Note that here p is beig held fixed ad, ulike the situatio we described i the Poisso approximatio to the biomial where ad p 0 i such a way that the product p λ > 0 Example 511 Normal approximatio to the Poisso distributio Whe the radom variable Y Poiss, where 1 is a iteger, we may thik of Y as havig the same distributio as that of the sum of iid radom variables each with the Poiss1 distributio Thus Y / has approximately the N0, 1 distributio for large The same coclusio is true for Y Poissλ for o-iteger λ; that is, Y λ/ λ is approximately N0, 1 for λ large Example 512 Opiio polls Suppose that the proportio of voters i the populatio who vote Labour is p, where p is ukow A radom sample of voters is take ad it is foud that S voters i the sample vote Labour ad we estimate p by S/ We wat to esure that S/ p < ɛ, for some small give ɛ, with high probability, 0 95, say How large must be? Note that S Bi, p, so that E S = p ad Var S = p1 p The by the Cetral Limit Theorem, we require S P p < ɛ = P ɛ Φ ɛ = 2Φ ɛ < ɛ p1 p p1 p < S p Φ ɛ p1 p , p1 p 92 p1 p p1 p sice Φx = 1 Φ x,

9 so that we eed ɛ /p1 p 1 96, that is p1 p/ɛ 2 We do ot kow the value of p, but it is always the case that p1 p 1 4, with equality occurrig whe p = 1 2, so to esure that we have the required boud we eed 1 96/2ɛ2 For example, if we take ɛ = 0 02, so that the estimate of the percetage of Labour voters is accurate to withi 2 percetage poits with 95% probability we would eed to take a sample with 2401 The typical sample size i opiio polls is 1000 which correspods to a error ɛ Geometric probability Buffo s Needle Cosider a eedle of legth r which is throw at radom o to a plae surface o which there are parallel straight lies at distace d > r apart What is the probability that the eedle itersects oe of the lies? Thik of the parallel lies ruig West-East ad let X be the distace from the poit represetig the Souther ed of the eedle to the earest lie North of that poit; if the eedle is parallel to the lies, take the right-had ed poit Let Θ be the agle that the eedle makes with the West-East lies The we will assume that X is uiformly distributed o [0, d ad Θ is uiformly distributed o [0, π], ad that X ad Θ are idepedet X Θ θ x 0 d r π r si θ The joit probability desity of X, Θ is fx, θ = 1 πd, for 0 x < d ad 0 θ π, 0 otherwise 93

10 The if A is the shaded area i the x θ plae illustrated, the probability that the eedle itersects a lie is P X r si Θ = A fx, θdxdθ = π r si θ πd dxdθ = r π si θdθ = 2r πd 0 πd This probability was derived i 1777 by the Frech mathematicia ad aturalist Georges Louis Leclerc, Comte de Buffo, who suggested a method of approximatig the value of π by repeatedly droppig a eedle ad estimatig the probability that a lie is itersect ad hece the value of π by recordig the proportio of times that the eedle crosses a lie First ote that if X Nµ, σ 2, where σ 2 is small, ad g : R R the gx = gµ + X µg µ + N gµ, g µ 2 σ 2, where the symbol may be read as approximately distributed as Now, if S = X X deotes the total umber of times that the eedle itersects a lie i drops of the eedle, where X i is the idicator of the evet that a lie is itersected o the ith drop, the S Bi, p where p = 2r/πd By the Cetral Limit Theorem we have that S / Np, p1 p/ Let gx = 2r/xd, so that gp = π ad g p = π 2 d/2r We see that a estimate of π is give by ˆπ = gs /, where ˆπ N π, π2 πd 2r 2r Oe small difficulty that arises whe oe tries to replicate Buffo s procedure for estimatig π o a computer is how to do the simulatio without usig the value of π to take radom samples of Θ from the uiform distributio o 0, π] Oe way aroud this is to geerate a sample X, Y which has the uiform distributio over a quadrat of the circle cetre the origi ad of uit radius as follows: Step 1 geerate idepedet X ad Y each with the uiform distributio o [0, 1]; Step 2 if X 2 + Y 2 > 1 repeat Step 1, otherwise take X, Y Now set Θ = 2 ta 1 Y/X, which will be uiform o 0, π 94

11 Bertrad s Paradox This results from the followig questio posed by Bertrad i 1889: What is the probability that a chord chose at radom joiig two poits of a circle of radius r has legth r? The difficulty with the questio is that there is ot a uique iterpretatio of what it meas for a chord to be chose at radom ; there are differet ways to do this ad they lead to differet probabilities for the legth, C, of the chord beig less tha r We will cosider two approaches Approach 1 Let X be a radom variable havig the uiform distributio o 0, r ad let Θ be a radom variable, idepedet of X, with the uiform distributio o 0, 2π The legth of the chord is C = 2 r 2 X 2 Θ X Costruct the chord by takig a referece lie the x-axis, say ad drawig the radius at agle Θ with the lie The take the chord at right agles to this radius at distace X from the cetre of the circle We the have P C r = P 4r 2 X 2 r 2 = P 3r/2 X = 1 3/ Approach 2 Let Θ 1 ad Θ 2 be idepedet radom variables each with the uiform distributio o 0, 2π Take the ed poits of the chord as the poits r cos Θ i, r si Θ i, i = 1, 2, o the circumferece of the circle, where the agles are measured from some referece lie The legth of the chord is C = 2r si Θ1 Θ 2 2 Θ 1 Θ 2 Θ 1 Θ 2 2π 2π π 3 π 3 5π 3 5π 3 95

12 The probability is the Θ1 Θ 2 P C r = P si = P Θ 1 Θ 2 π 3 or Θ 1 Θ 2 5π 3 this probability is the area of the shaded regio i the square divided by 2π 2 which gives the probability to be This probability is the same as whe you take oe ed of the chord as fixed say o the referece lie ad take the other ed at a poit at agle Θ uiformly distributed o 0, 2π It should be oted that both probabilities may arise as the outcomes of physical experimets which are choosig the chord at radom For example, the probability i Approach 1 would be foud if a circular disc of radius r is throw radomly oto a table o which parallel lies at distace 2r are draw; the chord would be determied by the uique lie itersectig the circle ad the distributio of the distace of ceter of the circle to the earest lie would be uiform o 0, r By cotrast, the probability i Approach 2 is obtaied if the disc is pivoted o a poit o its circumferece, which is o a give straight lie ad the disc is spu aroud that poit, the the chord would be determied by the itersectio of the give lie ad the circumferece of the disc ; Jauary 2010

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite