Math 6070 A Primer on Probability Theory
|
|
- Virgil Golden
- 6 years ago
- Views:
Transcription
1 Math 67 A Primer o Probability Theory Davar Khoshevisa Uiversity of Utah Sprig 214 Cotets 1 Probabilities 1 2 Distributio Fuctios Discrete Raom Variables Cotiuous Raom Variables Expectatios Momets A (Very) Partial List of Discrete Distributios A (Very) Partial List of Cotiuous Distributios Raom Vectors Distributio Fuctios Expectatios Multivariate Normals Iepeece 9 6 Covergece Criteria Covergece i Distributio Covergece i Probability Momet Geeratig Fuctios Some Examples Properties Characteristic Fuctios Some Examples
2 9 Classical Limit Theorems The Cetral Limit Theorem (Weak) Law of Large Numbers Variace Stabilizatio Refiemets to the CLT Coitioal Expectatios Coitioal Probabilities a Desities Coitioal Expectatios A Ituitive Iterpretatio Probabilities Let F be a collectio of sets. A probability P is a fuctio, o F, that has the followig properties: 1. P( ) = a P(Ω) = 1; 2. If A B the P(A) P(B); 3. (Fiite aitivity). If A a B are isjoit the P(A B) = P(A)+P (B); 4. For all A, B F, P(A B) = P(A) + P(B) P(A B); 5. (Coutable Aitivity). If A 1, A 2,... F are isjoit, the P( i=1 A i) = i=1 P(A i). 2 Distributio Fuctios Let X eote a raom variable. It istributio fuctio is the fuctio F (x) = P{X x}, (1) efie for all real umbers x. It has the followig properties: 1. lim x F (x) = ; 2. lim x F (x) = 1; 3. F is right-cotiuous; i.e., lim x y F (x) = F (y), for all real y; 4. F has left-limits; i.e., F (y ) := lim x y F (x) exists for all real y. I fact, F (y ) = P{X < y}; 5. F is o-ecreasig; i.e., F (x) F (y) wheever x y. It is possible to prove that (1) (5) are always vali for all what raom variables X. There is also a coverse. If F is a fuctio that satisfies (1) (5), the there exists a raom variable X whose istributio fuctio is F. 2
3 2.1 Discrete Raom Variables We will mostly stuy two classes of raom variables: iscrete, a cotiuous. We say that X is a iscrete raom variable if its possible values form a coutable or fiite set. I other wors, X is iscrete if a oly if there exist x 1, x 2,... such that: P{X = x i for some i 1} = 1. I this case, we are itereste i the mass fuctio of X, efie as the fuctio p such that p(x i ) = P{X = x i } (i 1). (2) Implicitly, this meas that p(x) = if x x i for some i. By coutable aitivity, i=1 p(x i) = x p(x) = 1. By coutable aitivity, the istributio fuctio of F ca be compute via the followig: For all x, F (x) = y x p(y). (3) Occasioally, there are several raom variables arou a we ietify the mass fuctio of X by p X to make the structure clear. 2.2 Cotiuous Raom Variables A raom variable is sai to be (absolutely) cotiuous if there exists a oegative fuctio f such that P{X A} = f(x) x for all A. The fuctio A f is sai to be the esity fuctio of X, a has the properties that: 1. f(x) for all x; 2. f(x) x = 1. The istributio fuctio of F ca be compute via the followig: For all x, F (x) = x By the fuametal theorem of calculus, F x f(y) y. (4) = f. (5) Occasioally, there are several raom variables arou a we ietify the esity fuctio of X by f X to make the structure clear. Cotiuous raom variables have the peculiar property that P{X = x} = for all x. Equivaletly, F (x) = F (x ), so that F is cotiuous (ot just rightcotiuous with left-limits). 3
4 3 Expectatios The (mathematical) expectatio of a iscrete raom variable X is efie as EX = x xp(x), (6) where p is the mass fuctio. Of course, this is well efie oly if x x p(x) <. I this case, we say that X is itegrable. Occasioally, EX is also calle the momet, first momet, or the mea of X. Propositio 1. For all fuctios g, Eg(X) = x g(x)p(x), (7) provie that g(x) is itegrable, a/or x g(x) p(x) <. This is ot a trivial result if you rea thigs carefully, which you shoul. Iee, the efiitio of expectatio implies that Eg(X) = y yp{g(x) = y} = y yp g(x) (y). (8) as The (mathematical) expectatio of a cotiuous raom variable X is efie EX = xf(x) x, (9) where f is the esity fuctio. This is well efie whe x f(x) x is fiite. I this case, we say that X is itegrable. Some times, we write E[X] a/or E{X} a/or E(X) i place of EX. Propositio 2. For all fuctios g, Eg(X) = g(x)f(x) x, (1) provie that g(x) is itegrable, a/or g(x) f(x) x <. As was the case i the iscrete settig, this is ot a trivial result if you rea thigs carefully. Iee, the efiitio of expectatio implies that Eg(X) = yf g(x) (y) y. (11) Here is a result that is sometimes useful, a ot so well-kow to stuets of probability: Propositio 3. Let X be a o-egative itegrable raom variable with istributio fuctio F. The, EX = (1 F (x)) x. (12) 4
5 Proof. Let us prove it for cotiuous raom variables. The iscrete case is prove similarly. We have ( ) (1 F (x)) x = P{X > x} x = f(y) y x. (13) Chage the orer of itegratio to fi that ( y ) (1 F (x)) x = x f(y) y = x yf(y) y. (14) Because f(y) = for all y <, this proves the result. It is possible to prove that for all itegrable raom variables X a Y, a for all reals a a b, E[aX + by ] = aex + bey. (15) This justifies the buzz-phrase, expectatio is a liear operatio. 3.1 Momets Note that ay raom variable X is itegrable if a oly if E X <. For all r >, the rth momet of X is E{X r }, provie that the rth absolute momet E{ X r } is fiite. I the iscrete case, E[X r ] = x r p(x), (16) x a i the cotiuous case, E[X r ] = x r f(x) x. (17) Whe it makes sese, we ca cosier egative momets as well. For istace, if X, the E[X r ] makes sese for r < as well, but it may be ifiite. Propositio 4. If r > a X is a o-egative raom variable with E[X r ] <, the E[X r ] = r x r 1 (1 F (x)) x. (18) Proof. Whe r = 1 this is Propositio 3. The proof works similarly. For istace, whe X is cotiuous, ( x ) E[X r ] = x r f(x) x = r y r 1 y f(x) x ( ) (19) = r y r 1 f(x) x y = r y r 1 P{X > y} y. y 5
6 This verifies the propositio i the cotiuous case. A quatity of iterest to us is the variace of X. If is efie as [ VarX = E (X EX) 2], (2) a is equal to VarX = E[X 2 ] (EX) 2. (21) Variace is fiite if a oly if X has two fiite momets. 3.2 A (Very) Partial List of Discrete Distributios You are expecte to be familar with the followig iscrete istributios: 1. Biomial (, p). Here, < p < 1 a = 1, 2,... are fixe, a the mass fuctio is ( ) p(x) = p x (1 p) x if x =,...,. (22) x EX = p a VarX = p(1 p). The biomial (1, p) istributio is also kow as Beroulli (p). 2. Poisso (λ). Here, λ > is fixe, a the mass fuctio is: p(x) = e λ λ x x! EX = λ a VarX = λ. x =, 1, 2,.... (23) 3. Negative biomial (, p). Here, < p < 1 a = 1, 2,... are fixe, a the mass fuctio is: ( ) x 1 p(x) = p (1 p) x x =, + 1,.... (24) 1 EX = /p a VarX = (1 p)/p A (Very) Partial List of Cotiuous Distributios You are expecte to be familar with the followig cotiuous istributios: 1. Uiform (a, b). Here, < a < b < are fixe, a the esity fuctio is f(x) = 1 if a x b. (25) b a EX = (a + b)/2 a VarX = (b a) 2 /12. 6
7 2. Gamma (α, β). Here, α, β > are fixe, a the esity fuctio is f(x) = βα Γ(α) xα 1 e βx < x <. (26) Here, Γ(α) = t α 1 e t t is the (Euler) gamma fuctio. It is efie for all α >, a has the property that Γ(1+α) = αγ(α). Also, Γ(1+) =! for all itegers, whereas Γ(1/2) = π. EX = α/β a VarX = α/β 2. Gamma (1, β) is also kow as Exp (β). [The Expoetial istributio.] Whe 1 is a iteger, Gamma (/2, 1/2) is also kow as χ 2 (). [The chi-square istributio with egrees of freeom.] 3. N(µ, σ 2 ). [The ormal istributio] Here, < µ < a σ > are fixe, a the esity fuctio is: f(x) = 1 σ 2 2π e (x µ) /(2σ 2 ) < x <. (27) EX = µ a VarX = σ 2. N(, 1) is calle the staar ormal istributio. We have the istributioal ietity, µ+σn(, 1) = N(µ, σ 2 ). Equivaletly, N(µ, σ 2 ) µ = N(, 1). (28) σ The istributio fuctio of a N(, 1) is a importat object, a is always eote by Φ. That is, for all < a <, 4 Raom Vectors Φ(a) := 1 2π a e x2 /2 x. (29) Let X 1,..., X be raom variables. The, X := (X 1,..., X ) is a raom vector. 4.1 Distributio Fuctios Let X = (X 1,..., X ) be a N-imesioal raom vector. Its istributio fuctio is efie by F (x 1,..., x ) = P {X 1 x 1,..., X x }, (3) vali for all real umbers x 1,..., x. 7
8 If X 1,..., X are all iscrete, the we say that X is iscrete. O the other ha, we say that X is (absolutely) cotiuous whe there exists a o-egative fuctio f, of variables, such that for all -imesioal sets A, P{X A} = f(x 1,..., x ) x 1... x. (31) A The fuctio f is calle the esity fuctio of X. It is also calle the joit esity fuctio of X 1,..., X. Note, i particular, that x1 x F (x 1,..., x ) = f(u 1,..., u ) u u 1. (32) By the fuametal theorem of calculus, 4.2 Expectatios If g is a real-value fuctio of variables, the Eg(X 1,..., X ) = F x 1 x 2... x = f. (33) g(x 1,..., x )f(x 1,..., x ) x 1... x. (34) A importat special case is whe = 2 a g(x 1, x 2 ) = x 1 x 2. I this case, we obtai E[X 1 X 2 ] = The covariace betwee X 1 a X 2 is efie as It turs out that u 1 u 2 f(u 1, u 2 ) u 1 u 2. (35) Cov(X 1, X 2 ) := E [(X 1 EX 1 ) (X 2 EX 2 )]. (36) Cov(X 1, X 2 ) = E[X 1 X 2 ] E[X 1 ]E[X 2 ]. (37) This is well efie if both X 1 a X 2 have two fiite momets. I this case, the correlatio betwee X 1 a X 2 is ρ(x 1, X 2 ) := Cov(X 1, X 2 ) VarX1 VarX 2, (38) provie that < VarX 1, VarX 2 <. The expectatio of X = (X 1,..., X ) is efie as the vector EX whose jth cooriate is EX j. Give a raom vector X = (X 1,..., X ), its covariace matrix is efie as C = (C ij ) 1 i,j, where C ij := Cov(X i X j ). This makes sese provie that the X i s have two fiite momets. Lemma 5. Every covariace matrix C is positive semi-efiite. That is, x Cx for all x R. Coversely, every positive semi-efiite ( ) matrix is the covariace matrix of some raom vector. 8
9 4.3 Multivariate Normals Let µ = (µ 1,..., µ ) be a -imesioal vector, a C a ( )-imesioal matrix that is positive efiite. The latter meas that x Cx > for all o-zero vectors x = (x 1,..., x ). This implies, for istace, that C is ivertible, a the iverse is also positive efiite. We say that X = (X 1,..., X ) has the multivariate ormal istributio N (µ, C) if the esity fuctio of X is f(x 1,..., x ) = for all x = (x 1,..., x ) R. EX = µ a Cov(X) = C. 1 (2π) /2 et C e 1 2 (x µ) C 1 (x µ), (39) X N (µ, C) if a oly if there exists a positive efiite matrix A, a i.i.. staar ormals Z 1,..., Z such that X = µ + AZ. I aitio, AA = C. Whe = 2, a multivariate ormal is calle a bivariate ormal. Warig. Suppose X a Y are each ormally istribute. The it is ot true i geeral that (X, Y ) is bivariate ormal. A similar caveat hols for the -imesioal case. 5 Iepeece Raom variables X 1,..., X are (statistically) iepeet if P {X 1 A 1,..., X A } = P {X 1 A 1 } P {X A }, (4) for all oe-imesioal sets A 1,..., A. It ca be show that X 1,..., X are iepeet if a oly if for all real umbers x 1,..., x, P {X 1 x 1,..., X x } = P {X 1 x 1 } P {X x }. (41) That is, the cooriates of X = (X 1,..., X ) are iepeet if a oly if F X (x 1,..., x ) = F X1 (x 1 ) F X (x ). Aother equivalet formulatio of iepeece is this: For all fuctios g 1,..., g such that g i (X i ) is itegrable, E [g(x 1 )... g(x )] = E[g 1 (X 1 )] E[g (X )]. (42) A reay cosequece is this: If X 1 a X 2 are iepeet, the they are ucorrelate provie that their correlatio exists. Ucorrelate meas that ρ(x 1, X 2 ) =. This is equivalet to Cov(X 1, X 2 ) =. If X 1,..., X are (pairwise) ucorrelate with two fiite momets, the Var(X X ) = VarX VarX. (43) 9
10 Sigificatly, this is true whe the X i s are iepeet. I geeral, the formula is messier: ( ) Var X i = VarX i + 2 Cov(X i, X j ). (44) i=1 i=1 1 i<j I geeral, ucorrelate raom variables are ot iepeet. A exceptio is mae for multivariate ormals. Theorem 6. Suppose (X, Y ) N +k (µ, C), where X a Y are respectively -imesioal a k-imesioal raom vectors. The: 1. X is multivariate ormal. 2. Y is multivariate ormal. 3. If EX i Y j = for all i, j, the X a Y are iepeet. For example, suppose (X, Y ) is bivariate ormal. The, X a Y are ormally istribute. If, i aitio, Cov(X, Y ) = the X a Y are iepeet. 6 Covergece Criteria Let X 1, X 2,... be a coutably-ifiite sequece of raom variables. There are several ways to make sese of the statemet that X X for a raom variable X. We ee a few of these criteria. 6.1 Covergece i Distributio We say that X coverges to X i istributio if F X (x) F X (x), (45) for all x R at which F X is cotiuous. We write this as X X. Very ofte, F X is cotiuous. I such cases, X X if a oly if FX (x) F X (x) for all x. Note that if X X a X has a cotiuous istributio the also P{a X b} P{a X b}, (46) for all a < b. Similarly, we say that the raom vectors X 1, X 2,... coverge i istributio to the raom vector X whe F X (a) F X (a) for all a at which F X is cotiuous. This covergece is also eote by X X. 1
11 6.2 Covergece i Probability We say that X coverges to X i probability if for all ɛ >, P { X X > ɛ}. (47) We eote this by X P X. It is the case that if X P X the X X, but the coverse is patetly false. There is oe exceptio to this rule. Lemma 7. Suppose X c where c is a o-raom costat. The, X P c. Proof. Fix ɛ >. The, P{ X c ɛ} P{c ɛ < X c + ɛ} = F X (c + ɛ) F X (c ɛ). (48) But F c (x) = if x < c, a F c (x) = 1 if x c. Therefore, F c is cotiuous at c ± ɛ, whece we have F X (c + ɛ) F X (c ɛ) F c (c + ɛ) F c (c ɛ) = 1. This proves that P{ X c ɛ} 1, which is aother way to write the lemma. Similar cosieratios lea us to the followig. Theorem 8 (Slutsky s theorem). Suppose X X a Y c for a costat c. If g is a cotiuous fuctio of two variables, the g(x, Y ) g(x, c). [For istace, try g(x, y) = ax + by, g(x, y) = xye x, etc.] Whe c is a raom variable this is o loger vali i geeral. 7 Momet Geeratig Fuctios We say that X has a momet geeratig fuctio if there exists t > such that M(t) := M X (t) = E[e tx ] is fiite for all t [ t, t ]. (49) If this coitio is met, the M is the momet geeratig fuctio of X. If a whe it exists, the momet geeratig fuctio of X etermies its etire istributio. Here is a more precise statemet. Theorem 9 (Uiqueess). Suppose X a Y have momet geeratig fuctios, a M X (t) = M Y (t) for all t sufficietly close to. The, X a Y have the same istributio. 7.1 Some Examples 1. Biomial (, p). The, M(t) exists for all < t <, a M(t) = ( 1 p + pe t). (5) 11
12 2. Poisso (λ). The, M(t) exists for all < t <, a M(t) = e λ(et 1). (51) 3. Negative Biomial (, p). The, M(t) exists if a oly if < t < log(1 p). I that case, we have also that ( pe t ) M(t) = 1 (1 p)e t. (52) 4. Uiform (a, b). The, M(t) exists for all < t <, a M(t) = etb e ta t(b a). (53) 5. Gamma (α, β). The, M(t) exists if a oly if < t < β. I that case, we have also that ( ) α β M(t) =. (54) β t Set α = 1 to fi the momet geeratig fuctio of a expoetial (β). Set α = /2 a β = 1/2 for a positive iteger to obtai the momet geeratig fuctio of a chi-square (). 6. N(µ, σ 2 ). The momet geeratig fuctio exists for all < t <. Moreover, M(t) = exp (µt + σ2 t 2 ). (55) Properties Besie the uiqueess theorem, momet geeratig fuctios have two more properties that are of iterest i mathematical statistics. Theorem 1 (Covergece Theorem). Suppose X 1, X 2,... is a sequece of raom variables whose momet geeratig fuctios all exists i a iterval [ t, t ] arou the origi. Suppose also that for all t [ t, t ], M X (t) M X (t) as, where M is the momet geeratig fuctio of a raom variable X. The, X X. Example 11 (Law of Rare Evets). Let X have the Bi(, λ/) istributio, where λ > is iepeet of. The, for all < t <, ( M X (t) = 1 λ + λ ) et. (56) 12
13 We claim that for all real umbers c, ( 1 + c ) e c as. (57) Let us take this for grate for the time beig. The, it follows at oce that That is, M X (t) exp ( λ + λe t) = e λ(et 1). (58) Bi (, λ/) Poisso (λ). (59) This is Poisso s law of rare evets (also kow as the law of small umbers ). Now we wrap up this example by verifyig (57). Let f(x) = (1 + x), a Taylor-expa it to fi that f(x) = 1 + x ( 1)x2 +. Replace x by c/, a compute to fi that ( 1 + c ) ( 1)c 2 = 1 + c j= c j j!, (6) a this is the Taylor-series expasio of e c. [There is a little bit more oe has to o to justify the limitig proceure.] The seco property of momet geeratig fuctios is that if a whe it exists for a raom variable X, the all momets of X exist, a ca be compute from M X. Theorem 12 (Momet-Geeratig Property). Suppose X has a fiite momet geeratig fuctio i a eighborhoo of the origi. The, E( X ) exists for all, a M () () = E[X ], where f () (x) eotes the th erivative of fuctio f at x. Example 13. Let X be a N(µ, 1) raom variable. The we kow that M(t) = exp(µt t2 ). Cosequetly, M (t) = (µ + t)e µt+(t2 /2), a M (t) = [ 1 + (µ + t) 2] e µt+(t2 /2) (61) Set t = to fi that EX = M () = µ a E[X 2 ] = M () = 1 + µ 2, so that VarX = E[X 2 ] (EX) 2 = 1. 8 Characteristic Fuctios The characteristic fuctio of a raom variable X is the fuctio φ(t) := E [ e itx] < t <. (62) Here, the i refers to the complex uit, i = 1. We may write φ as φ X, for example, whe there are several raom variables arou. 13
14 I practice, you ofte treat e itx as if it were a real expoetial. However, the correct way to thik of this efiitio is via the Euler formula, e iθ = cos θ+i si θ for all real umbers θ. Thus, φ(t) = E[cos(tX)] + ie[si(tx)]. (63) If X has a momet geeratig fuctio M, the it ca be show that M(it) = φ(t). [This uses the techique of aalytic cotiuatio from complex aalysis.] I other wors, the aive replacemet of t by it oes what oe may guess it woul. However, oe avatage of workig with φ is that it is always wellefie. The reaso is that cos(tx) 1 a si(tx) 1, so that the expectatios i (63) exist. I aitio to havig this avatage, φ shares most of the properties of M as well! For example, Theorem 14. The followig hol: 1. (Uiqueess Theorem) Suppose there exists t > such that for all t ( t, t ), φ X (t) = φ Y (t). The X a Y have the same istributio. 2. (Covergece Theorem) If φ X (t) φ X (t) for all t ( t, t ), the X X. Coversely, if X X, the φx (t) φ X (t) for all t. 8.1 Some Examples 1. Biomial (, p). The, φ(t) = M(it) = ( 1 p + pe it). (64) 2. Poisso (λ). The, φ(t) = M(it) = e λ(eit 1). (65) 3. Negative Biomial (, p). The, ( pe it ) φ(t) = M(it) = 1 (1 p)e it. (66) 4. Uiform (a, b). The, 5. Gamma (α, β). The, φ(t) = M(it) = eitb e ita t(b a). (67) φ(t) = M(it) = ( ) α β. (68) β it 6. N(µ, σ 2 ). The, because (it) 2 = t 2, φ(t) = M(it) = exp (iµt σ2 t 2 ). (69) 2 14
15 9 Classical Limit Theorems 9.1 The Cetral Limit Theorem Theorem 15 (The CLT). Let X 1, X 2,... be i.i.. raom variables with two fiite momets. Let µ := EX 1 a σ 2 = VarX 1. The, j=1 X j µ σ N(, 1). (7) Elemetary probability texts prove this by appealig to the covergece theorem for momet geeratig fuctios. This approach oes ot work if we kow oly that X 1 has two fiite momets, however. However, by usig characteristic fuctios, we ca relax the assumptios to the fiite mea a variace case, as state. Proof of the CLT. Defie j=1 T := X j µ σ. (71) The, φ T (t) = E = j=1 E j=1 ( exp it [ exp ( it ( )) Xj µ σ ( ))] Xj µ σ, (72) thaks to iepeece; see (42) o page 8. Let Y j := (X j µ)/σ eote the staarizatio of X j. The, it follows that φ T (t) = ( ) [ ( )] φ Yj t/ = φy1 t/, (73) j=1 because the Y j s are i.i.. Recall the Taylor expasio, e ix = 1 + ix 1 2 x2 +, a write φ Y1 (s) as E[e ity1 ] = 1 + itey t2 E[Y 2 1 ] + = t2 +. Thus, φ T (t) = ] [1 t2 2 + e t2 /2. (74) See (57) o page 12. Because e t2 /2 is the characteristic fuctio of N(, 1), this a the covergece theorem (Theorem 15 o page 14) together prove the CLT. The CLT has a multiimesioal couterpart as well. Here is the statemet. 15
16 Theorem 16. Let X 1, X 2,... be i.i.. k-imesioal raom vectors with mea vector µ := EX 1 a covariace matrix Q := CovX. If Q is o-sigular, the j=1 X j µ Nk (, Q). (75) 9.2 (Weak) Law of Large Numbers Theorem 17 (Law of Large Numbers). Suppose X 1, X 2,... are i.i.. a have a fiite first momet. Let µ := EX 1. The, j=1 X j P µ. (76) Proof. We will prove this i case there is also a fiite variace. The geeral case is beyo the scope of these otes. Thaks to the CLT (Theorem 15, page 14), (X X )/ coverges i istributio to µ. Slutsky s theorem (Theorem 8, page 1) proves that covergece hols also i probability. 9.3 Variace Stabilizatio Let X 1, X 2,... be i.i.. with µ = EX 1 a σ 2 = VarX 1 both efie a fiite. Defie the partial sums, S := X X. (77) We kow that: (i) S µ i probability; a (ii) (S µ) N(, σ 2 ). Now use Taylor expasios: For ay smooth fuctio h, ( ) S h(s /) h(µ) + µ h (µ), (78) i probability. By the CLT, (S /) µ N(, σ 2 /). Therefore, Slutsky s theorem (Theorem 8, page 1) proves that [ ( ) ] S h h(µ) N (, σ 2 h (µ) 2). (79) [Techical coitios: h shoul be cotiuously-ifferetiable i a eighborhoo of µ.] 9.4 Refiemets to the CLT There are may refiemets to the CLT. Here are 2 particularly well-kow oes. The first gives a escriptio of the farthest the istributio fuctio of ormalize sums is from the ormal. 16
17 Theorem 18 (Berry Essee). If ρ := E{ X 1 3 } <, the { max <a< P i=1 X } i µ σ a Φ(a) 3ρ σ 3. (8) The seco is a oe-term example of a family is results that are calle Egeworth expasios. Theorem 19 (Egeworth). Suppose E exp(itx 1) t < a E( X 1 ρ ) < for some ρ > 3, the we ca write where: { i=1 P X } i µ σ a = Φ(a) + κ 1(1 a 2 ) 6 φ(a) + R (a), 1. φ(a) := (2π) 1/2 exp( a 2 /2) eotes the staar ormal esity; 2. κ 1 := σ 3 E[(X 1 µ) 3 ] eotes the skewess of the istributio of X 1 ; 3. max <a< R (a) cost 1. Remark 2. The coitio E exp(itx 1) t < hols roughly whe X 1 has a ice pf. Remark 21. Uer further restrictios, oe ca i fact write a asymptotic expasio of the form { i=1 P X } i µ σ a = Φ(a) + r j=1 κ j H j (a) φ(a) + R j/2,r (a), for every [fixe] positive iteger r, where κ j s are fiite costats, each H j is a certai polyomial of egree j [Hermite polyomials], a the remaier is very small i the sese that max <a< R,r (a) cost (r+1)/2. 1 Coitioal Expectatios Let us begi by recallig some basic otios of coitioig from elemetary probability. Throughout this sectio, X eotes a raom variable a Y := (Y 1,..., Y ) a -imesioal raom vector. 1.1 Coitioal Probabilities a Desities If X, Y 1,..., Y are all iscrete raom variables, the the coitioal mass fuctio of X, give that Y = y, is p X Y (x y) := P{X = x, Y 1 = y 1,..., Y = y }, (81) P{Y 1 = y 1,..., Y = y } 17
18 provie that P{Y = y} >. This is a boa fie mass fuctio [as a fuctio of the variable x] for every fixe choice of y. [It oes t make sese to worry about its behavior i the variables y 1,..., y.] Similarly, if the istributio of (X, Y 1,..., Y ) is absolutely cotiuous, the the coitioal esity fuctio of X, give that Y = y, is f X Y (x y) := f X,Y (x, y 1,..., y ), (82) f Y (y 1,..., y ) provie that the observe value y is such that the joit esity f X,Y raom vector (X, Y ) satisfies of the f Y (y 1,..., y ) >. (83) Note that (83) is etirely possible, though P{Y = y} = simply because Y has a absolutely cotiuous istributio. Coitio (83) is quite atural i the followig sese: Let B eote the collectio of all -imesioal vectors y such that f Y (y 1,..., y ) =. The, P{Y B} = f Y (y 1,..., y ) y 1 y =. (84) B I other wors, we o ot have to worry about efiig f X Y (x y) whe y is ot i B. 1.2 Coitioal Expectatios If we have observe that Y = y, for a kow vector y = (y 1,..., y ), the the best liear preictor of X is the [classical] coitioal expectatio E(X Y = y) := { x xp{x = x Y = y} if (X, Y ) is iscrete, xf X Y (x y) x if (X, Y ) has a joit pf. (85) The preceig assumes tacitly that the sum/itegral coverges absolutely. More geerally, we have for ay ice fuctio ϕ, E(ϕ(X) Y = y) := { x ϕ(x)p{x = x Y = y} if iscrete, ϕ(x)f X Y (x y) x if joit pf exists, (86) provie that the sum/itegral coverges absolutely. The preceig is i fact a theorem, but a careful statemet requires writig too may techical etails from itegratio theory. 1.3 A Ituitive Iterpretatio The basic use of coitioal expectatios is this: If we observe that Y = y, the we preict X, base oly o our observatio that Y = y, as E(X Y = y). 18
19 Example 22. We perform 1 iepeet Beroulli trials [p := probability of success per trial]. Let X eote the total umber of successes. We kow that X has a Bi(1, p) istributio. If Y := the total umber of successes i the first 5 trials, the you shoul check that E(X Y = ) = 5p. More geerally, E(X Y = y) = y + 5p for all y {,..., 5}. The previous example shows you that it is frequetly more coveiet to use a slightly ifferet form of coitioal expectatios: We write E(X Y ) for the raom variable whose value is E(X Y = y) whe we observe that Y = y. I the previous example, this efiitio traslates to the followig computatio: E(X Y ) = Y + 5p. This ought to make very goo sese to you, before you rea o! The classical Bayes formula for coitioal probabilities has a aalogue for coitioal expectatios. Suppose (X, Y ) has a joit esity fuctio f X,Y. The, E(X) = xf X (x) x ( ) = x f X,Y (x, y 1,..., y ) y x R ( ) = x f X Y (x y)f Y (y) y x R ( ) = xf X Y (x y) x f Y (y) y R = E(X Y = y)f Y (y) y R = E {E(X Y )}. (87) This is always true. That is, we always have provie that E X <. E(X) = E {E(X Y )}, (88) 19
Chapter 2 Transformations and Expectations
Chapter Trasformatios a Epectatios Chapter Distributios of Fuctios of a Raom Variable Problem: Let be a raom variable with cf F ( ) If we efie ay fuctio of, say g( ) g( ) is also a raom variable whose
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More information(average number of points per unit length). Note that Equation (9B1) does not depend on the
EE603 Class Notes 9/25/203 Joh Stesby Appeix 9-B: Raom Poisso Poits As iscusse i Chapter, let (t,t 2 ) eote the umber of Poisso raom poits i the iterval (t, t 2 ]. The quatity (t, t 2 ) is a o-egative-iteger-value
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationInhomogeneous Poisson process
Chapter 22 Ihomogeeous Poisso process We coclue our stuy of Poisso processes with the case of o-statioary rates. Let us cosier a arrival rate, λ(t), that with time, but oe that is still Markovia. That
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS Cotets 1. A few useful discrete radom variables 2. Joit, margial, ad
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationSome Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationLecture 6 Testing Nonlinear Restrictions 1. The previous lectures prepare us for the tests of nonlinear restrictions of the form:
Eco 75 Lecture 6 Testig Noliear Restrictios The previous lectures prepare us for the tests of oliear restrictios of the form: H 0 : h( 0 ) = 0 versus H : h( 0 ) 6= 0: () I this lecture, we cosier Wal,
More information3. Calculus with distributions
6 RODICA D. COSTIN 3.1. Limits of istributios. 3. Calculus with istributios Defiitio 4. A sequece of istributios {u } coverges to the istributio u (all efie o the same space of test fuctios) if (φ, u )
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More informationMIT Spring 2016
MIT 18.655 Dr. Kempthore Sprig 2016 1 MIT 18.655 Outlie 1 2 MIT 18.655 Beroulli s Weak Law of Large Numbers X 1, X 2,... iid Beroulli(θ). S i=1 = X i Biomial(, θ). S P θ. Proof: Apply Chebychev s Iequality,
More informationDefinition 2 (Eigenvalue Expansion). We say a d-regular graph is a λ eigenvalue expander if
Expaer Graphs Graph Theory (Fall 011) Rutgers Uiversity Swastik Kopparty Throughout these otes G is a -regular graph 1 The Spectrum Let A G be the ajacecy matrix of G Let λ 1 λ λ be the eigevalues of A
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More information1 Review and Overview
CS229T/STATS231: Statistical Learig Theory Lecturer: Tegyu Ma Lecture #12 Scribe: Garrett Thomas, Pega Liu October 31, 2018 1 Review a Overview Recall the GAN setup: we have iepeet samples x 1,..., x raw
More informationProbability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
More informationCommonly Used Distributions and Parameter Estimation
Commoly Use Distributios a Parameter stimatio Berli Che Departmet of Computer Sciece & Iformatio gieerig Natioal Taiwa Normal Uiversity Referece:. W. Navii. Statistics for gieerig a Scietists. Chapter
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationIIT JAM Mathematical Statistics (MS) 2006 SECTION A
IIT JAM Mathematical Statistics (MS) 6 SECTION A. If a > for ad lim a / L >, the which of the followig series is ot coverget? (a) (b) (c) (d) (d) = = a = a = a a + / a lim a a / + = lim a / a / + = lim
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationClassical Electrodynamics
A First Look at Quatum Physics Classical Electroyamics Chapter Itrouctio a Survey Classical Electroyamics Prof. Y. F. Che Cotets A First Look at Quatum Physics. Coulomb s law a electric fiel. Electric
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More information6.3.3 Parameter Estimation
130 CHAPTER 6. ARMA MODELS 6.3.3 Parameter Estimatio I this sectio we will iscuss methos of parameter estimatio for ARMAp,q assumig that the orers p a q are kow. Metho of Momets I this metho we equate
More informationk=1 s k (x) (3) and that the corresponding infinite series may also converge; moreover, if it converges, then it defines a function S through its sum
0. L Hôpital s rule You alreay kow from Lecture 0 that ay sequece {s k } iuces a sequece of fiite sums {S } through S = s k, a that if s k 0 as k the {S } may coverge to the it k= S = s s s 3 s 4 = s k.
More informationChi-squared tests Math 6070, Spring 2014
Chi-squared tests Math 6070, Sprig 204 Davar Khoshevisa Uiversity of Utah March, 204 Cotets MLE for goodess-of fit 2 2 The Multivariate ormal distributio 3 3 Cetral limit theorems 5 4 Applicatio to goodess-of-fit
More informationExponential function and its derivative revisited
Expoetial fuctio a its erivative revisite Weg Ki Ho, Foo Him Ho Natioal Istitute of Eucatio, Sigapore {wegki,foohim}.ho@ie.eu.sg Tuo Yeog Lee NUS High School of Math & Sciece hsleety@us.eu.sg February
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationThe Chi Squared Distribution Page 1
The Chi Square Distributio Page Cosier the istributio of the square of a score take from N(, The probability that z woul have a value less tha is give by z / g ( ( e z if > F π, if < z where ( e g e z
More informationNOTES ON DISTRIBUTIONS
NOTES ON DISTRIBUTIONS MICHAEL N KATEHAKIS Radom Variables Radom variables represet outcomes from radom pheomea They are specified by two objects The rage R of possible values ad the frequecy fx with which
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationSTATISTICAL METHODS FOR BUSINESS
STATISTICAL METHODS FOR BUSINESS UNIT 5. Joit aalysis ad limit theorems. 5.1.- -dimesio distributios. Margial ad coditioal distributios 5.2.- Sequeces of idepedet radom variables. Properties 5.3.- Sums
More informationSummary. Recap. Last Lecture. Let W n = W n (X 1,, X n ) = W n (X) be a sequence of estimators for
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 17 Asymptotic Evaluatio of oit Estimators Hyu Mi Kag March 19th, 2013 What is a Bayes Risk? What is the Bayes rule Estimator miimizig square
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationThe structure of Fourier series
The structure of Fourier series Valery P Dmitriyev Lomoosov Uiversity, Russia Date: February 3, 2011) Fourier series is costructe basig o the iea to moel the elemetary oscillatio 1, +1) by the expoetial
More informationAP Calculus BC Review Chapter 12 (Sequences and Series), Part Two. n n th derivative of f at x = 5 is given by = x = approximates ( 6)
AP Calculus BC Review Chapter (Sequeces a Series), Part Two Thigs to Kow a Be Able to Do Uersta the meaig of a power series cetere at either or a arbitrary a Uersta raii a itervals of covergece, a kow
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More informationSOME RESULTS RELATED TO DISTRIBUTION FUNCTIONS OF CHI-SQUARE TYPE RANDOM VARIABLES WITH RANDOM DEGREES OF FREEDOM
Bull Korea Math Soc 45 (2008), No 3, pp 509 522 SOME RESULTS RELATED TO DISTRIBUTION FUNCTIONS OF CHI-SQUARE TYPE RANDOM VARIABLES WITH RANDOM DEGREES OF FREEDOM Tra Loc Hug, Tra Thie Thah, a Bui Quag
More informationChi-Squared Tests Math 6070, Spring 2006
Chi-Squared Tests Math 6070, Sprig 2006 Davar Khoshevisa Uiversity of Utah February XXX, 2006 Cotets MLE for Goodess-of Fit 2 2 The Multiomial Distributio 3 3 Applicatio to Goodess-of-Fit 6 3 Testig for
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationProbability in Medical Imaging
Chapter P Probability i Meical Imagig Cotets Itrouctio P1 Probability a isotropic emissios P2 Raioactive ecay statistics P4 Biomial coutig process P4 Half-life P5 Poisso process P6 Determiig activity of
More informationWe are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n
Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationIntroduction to Probability I: Expectations, Bayes Theorem, Gaussians, and the Poisson Distribution. 1
Itroductio to Probability I: Expectatios, Bayes Theorem, Gaussias, ad the Poisso Distributio. 1 Pakaj Mehta February 25, 2019 1 Read: This will itroduce some elemetary ideas i probability theory that we
More information6. Sufficient, Complete, and Ancillary Statistics
Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary
More informationA Note on the Weak Law of Large Numbers for Free Random Variables
A Note o the Weak Law of Large Numbers for Free Raom Variables Raluca Bala Uiversity of Ottawa George Stoica Uiversity of New Bruswick July 28, 2006 Abstract I this article we prove that if {X k } k are
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationAdvanced Analysis. Min Yan Department of Mathematics Hong Kong University of Science and Technology
Advaced Aalysis Mi Ya Departmet of Mathematics Hog Kog Uiversity of Sciece ad Techology September 3, 009 Cotets Limit ad Cotiuity 7 Limit of Sequece 8 Defiitio 8 Property 3 3 Ifiity ad Ifiitesimal 8 4
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationDiscrete Mathematics for CS Spring 2005 Clancy/Wagner Notes 21. Some Important Distributions
CS 70 Discrete Mathematics for CS Sprig 2005 Clacy/Wager Notes 21 Some Importat Distributios Questio: A biased coi with Heads probability p is tossed repeatedly util the first Head appears. What is the
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationB Supplemental Notes 2 Hypergeometric, Binomial, Poisson and Multinomial Random Variables and Borel Sets
B671-672 Supplemetal otes 2 Hypergeometric, Biomial, Poisso ad Multiomial Radom Variables ad Borel Sets 1 Biomial Approximatio to the Hypergeometric Recall that the Hypergeometric istributio is fx = x
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More information6.451 Principles of Digital Communication II Wednesday, March 9, 2005 MIT, Spring 2005 Handout #12. Problem Set 5 Solutions
6.51 Priciples of Digital Commuicatio II Weesay, March 9, 2005 MIT, Sprig 2005 Haout #12 Problem Set 5 Solutios Problem 5.1 (Eucliea ivisio algorithm). (a) For the set F[x] of polyomials over ay fiel F,
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More information= p x (1 p) 1 x. Var (X) =p(1 p) M X (t) =1+p(e t 1).
Prob. fuctio:, =1 () = 1, =0 = (1 ) 1 E(X) = Var (X) =(1 ) M X (t) =1+(e t 1). 1.1.2 Biomial distributio Parameter: 0 1; >0; MGF: M X (t) ={1+(e t 1)}. Cosider a sequece of ideedet Ber() trials. If X =
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 19
CS 70 Discrete Mathematics ad Probability Theory Sprig 2016 Rao ad Walrad Note 19 Some Importat Distributios Recall our basic probabilistic experimet of tossig a biased coi times. This is a very simple
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationCS166 Handout 02 Spring 2018 April 3, 2018 Mathematical Terms and Identities
CS166 Hadout 02 Sprig 2018 April 3, 2018 Mathematical Terms ad Idetities Thaks to Ady Nguye ad Julie Tibshirai for their advice o this hadout. This hadout covers mathematical otatio ad idetities that may
More informationLecture 3: MLE and Regression
STAT/Q SCI 403: Itroductio to Resamplig Methods Sprig 207 Istructor: Ye-Chi Che Lecture 3: MLE ad Regressio 3. Parameters ad Distributios Some distributios are idexed by their uderlyig parameters. Thus,
More informationJoint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { }
UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig
More informationSparsification using Regular and Weighted. Graphs
Sparsificatio usig Regular a Weighte 1 Graphs Aly El Gamal ECE Departmet a Cooriate Sciece Laboratory Uiversity of Illiois at Urbaa-Champaig Abstract We review the state of the art results o spectral approximatio
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationSTAT Homework 2 - Solutions
STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.
More informationNANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS
NANYANG TECHNOLOGICAL UNIVERSITY SYLLABUS FOR ENTRANCE EXAMINATION FOR INTERNATIONAL STUDENTS AO-LEVEL MATHEMATICS STRUCTURE OF EXAMINATION PAPER. There will be oe 2-hour paper cosistig of 4 questios.
More informationVariance function estimation in multivariate nonparametric regression with fixed design
Joural of Multivariate Aalysis 00 009 6 36 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Variace fuctio estimatio i multivariate oparametric
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More information