Lecture 20: Multivariate convergence and the Central Limit Theorem
|
|
- Alisha Simon
- 5 years ago
- Views:
Transcription
1 Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece i distributio of Z to Z by lim F Z (x) = F Z (x), for every x R k. But this is ot good eough if F Z is ot always cotiuous. We ca adopt the followig defiitio. Defiitio. Let Z,Z 1,Z 2,... be radom vectors o R k. If, for every c R k, c Z coverges i distributio to c Z, the we say that Z coverges i distributio to Z. For ay costat vector c R k, c Z is a liear combiatio of compoets of Z. Note that i this defiitio, the covergece of c Z has to be true for every c (ot just some c). UW-Madiso (Statistics) Stat 609 Lecture / 16
2 If Z coverges i distributio to Z, the every compoet of Z coverges i distributio to the correspodig compoet of Z. The coverse is ot true: if every compoet of Z coverges i distributio to the correspodig compoet of Z, Z does ot ecessarily coverge i distributio to Z, because each compoet correspods to a particular c oly. (A couter-example is give below.) This is differet from covergece i probability ad covergece almost surely. Theorems ad ca be exteded to the case of radom vectors. A couter-example Let Z = (X,Y ) have the joit pdf f (x,y) = 1 2π e (x 2 +y 2 )/2 [1 + g(x,y)], (x,y) R 2 { xy 1 < x < 1, 1 < y < 1 g(x,y) = 0 otherwise UW-Madiso (Statistics) Stat 609 Lecture / 16
3 I Chapter 4 we showed that X N(0,1) ad Y N(0,1), but the joit distributio of Z is ot ormal. Let Z = (X,Y ), where for each, the radom variables X ad Y are idepedet, X N(0,1), ad Y N(0,1). Sice X s have the same N(0,1) distributio for all, obviously that X coverges" i distributio to X N(0,1); similarly, Y coverges i distributio to Y N(0,1). But the joit distributio of Z = (X,Y ) for every is f Z (x,y) = 1 2π e (x 2 +y 2 )/2, (x,y) R 2 sice X ad Y are idepedet. Sice Z s have the same distributio for every, Z coverges i distributio to Z 1, which has a differet distributio from Z. Thus, Z does ot coverge i distributio to Z although each compoet of Z coverges i distributio to the correspodig compoet of Z. UW-Madiso (Statistics) Stat 609 Lecture / 16
4 The covergece i distributio is equivalet to the covergece i chf (Theorem C7 or M3(ii)) so that the covergece i chf is a tool to study covergece i distributio. We ca also use Theorem or M3(i) to establish covergece i distributio by showig the covergece i mgf, but we have to kow the existece of mgf s. Example. Let X 1,...,X be iid radom variables. We wat to show that there does ot exist a sequece of real umbers {c } such that lim i=1 (X i c i ) exists almost surely, uless P(X 1 = c) = 1 for a costat c. Suppose that Y = lim i=1 (X i c i ) exists almost surely. For ay, the chf of Y = i=1 (X i c i ) is φ Y (t) = i=1 φ X1 (t)e ıtc i = [φ X1 (t)] e ıt(c 1+ +c ) If Y exists almost surely, the Y coverges i distributio to Y ad UW-Madiso (Statistics) Stat 609 Lecture / 16
5 lim [φ X 1 (t)] e ıt(c 1+ +c ) = lim φ X 1 (t) = φ Y (t). However, lim φ X1 (t) is either 0 or 1, depedig o whether φ X1 (t) < 1 or = 1, which meas that φ Y (t) must be either 0 or 1. Sice φ Y (t) is cotiuous ad φ Y (0) = 1, φ Y (t) = 1 for all t ad hece φ X1 (t) = 1 for all t. This proves that P(X 1 = c) = 1 for a costat c. The Cetral Limit Theorem (CLT) is oe of the most importat theorems i probability ad statistics. It derives the limitig distributio of a sequece of ormalized radom variables/vectors. Theorem (Cetral Limit Theorem) Let X 1,X 2,... be iid radom variables with E(X 1 ) = µ ad Var(X i ) = σ 2 <. The, for ay x R, lim P( ( X µ)/σ x) = Φ(x) = x 1 2π e t2 /2 dt That is, ( X µ)/σ coverges i distributio to Z N(0,1). UW-Madiso (Statistics) Stat 609 Lecture / 16
6 Proof. Normality comes from sums of iid radom variables without distributioal assumptio except the fiiteess of the variace. The assumptio of fiite variace is essetially ecessary for covergece of a sume of iid radom variables to ormality. We apply Theorem C7, i.e., we try to show that the chf of ( X µ) Z = = 1 X σ i µ σ i=1 coverges to the chf of N(0,1), which is e t2 /2. Let φ be the chf of (X i µ)/σ (ot depedig o i sice X i s are iid). From the properties of chf, the chf of Z is [ ( )] t φ Z (t) = φ. It remais to show that lim φ Z (t) = e t2 /2 for t R. Sice E((X i µ)/σ) = 0 ad Var((X i µ)/σ) = 1, by Theorem C1 ad Taylor s expasio, UW-Madiso (Statistics) Stat 609 Lecture / 16
7 where The, for ay t R, φ(s) = 1 s2 2 + R(s) as s 0 lim s 0 R(s)/s2 = 0 ( ) t lim R = 0 ad hece [ ( )] t lim φ Z (t) = lim φ = lim [1 t2 2 + R [ = lim 1 1 { t R [ = lim 1 1 ( )] t 2 2 = e t2 /2 ( t )] ( t )}] UW-Madiso (Statistics) Stat 609 Lecture / 16
8 The multivariate CLT Let X 1,X 2,... be iid radom vectors o R k with E(X 1 ) = µ ad fiite covariace matrix Σ. The ( X µ) coverges i distributio to a radom vector X N(0, Σ), the k-dimesioal ormal distributio with mea 0 ad covariace matrix Σ. Proof. By the defiitio of covergece i distributio for radom vectors, we eed to show that for ay costat vector c R k, c ( X µ) coverges i distributio to c X. From the result discussed earlier, we kow that c X N(0,c Σc). By the CLT, (c X E(c X)) Var(c X) = (c X c µ) c Σc coverges i distributio to Z N(0,1). Note that c ΣcZ N(0,c Σc). The proof is the completed because we ca show from the defiitio that, if Y coverges i distributio to Y, the ay coverges i distributio to ay. UW-Madiso (Statistics) Stat 609 Lecture / 16
9 Example Let X 1,...,X be a radom sample from a egative-biomial(r,p) distributio. Recall that r(1 p) µ = E(X 1 ) =, σ 2 r(1 p) = Var(X 1 ) = p p 2 The, the CLT tells us that the sequece of ormalized radom variables ( X µ) = σ coverges i distributio to Z N(0,1). ( X r(1 p)/p), = 1,2,... r(1 p)/p 2 This provides a way to approximately calculate probabilities that are very difficult to compute exactly. For example, if r = 10, p = 1/2 ad = 30, usig the fact that 30 i=1 X i egative-biomial(r,p), we ca exactly compute UW-Madiso (Statistics) Stat 609 Lecture / 16
10 ( ) 30 P( X 11) = P X i 330 i=1 = 330 x=0 The CLT gives us the approximatio ( P( X 30( X 10) 11) = P 20 ( x 1 x ) (0.5) 300+x = ) 30(11 20) 20 Φ ( 30(11 20) 20 ) = Φ(1.2247) = where Φ(x) is the cdf of N(0,1). Normal approximatio to biomial Let Z biomial(,p), = 1,2,... To apply the CLT, ote that each Z is a sum of idepedet Beroulli radom variables X 1,...,X, i.e., Z = X X, = 1,2,... X 1,X 2,... are iid with E(X i ) = p, Var(X i ) = p(1 p) UW-Madiso (Statistics) Stat 609 Lecture / 16
11 The, X = Z / ad Z p ( X p) = coverges i distributio to Z N(0,1) p(1 p) p(1 p) Cosequetly, for ay m, ( P(Z m) = P Z p p(1 p) ) ( m p Φ p(1 p) ) m p p(1 p) From the previous two examples, to approximate a probability of the form P(Z x), we always first ormalize Z to [Z E(Z )]/ Var(Z ) ad the approximate the probability by Φ([x E(Z )]/ Var(Z )). I fact, a little bit more ca be doe usig the followig result. Pólya s theorem If a sequece of radom vectors Z coverges i distributio to Z ad Z has a cotiuous cdf F Z o R k, the lim F Z (x) F Z (x) = 0. x R k UW-Madiso (Statistics) Stat 609 Lecture / 16
12 If the limitig cdf F Z is cotiuous, the the covergece of F Z to F Z is ot oly for every x, but also uiformly for all x R k. This result implies the followig useful result: If Z coverges i distributio to Z with a cotiuous F Z ad c R k with c c, the F Z (c ) F Z (c). Asymptotic ormality If Y, = 1,2,..., is a sequece of radom variables ad µ ad σ > 0, = 1,2,..., are costats (typically µ = E(Y ) ad σ 2 = Var(Y ) if they are fiite) such that (Y µ )/σ coverges i distributio to N(0,1), the for ay sequece {c } of costats, ( ) lim P(Y c µ c ) Φ = 0, i.e., P(Y c ) ca be approximated by Φ ( c µ ) σ, regardless of whether c, µ, or σ has a limit. Sice Φ ( t µ ) σ is the cdf of N(µ,σ 2 ), Y is said to be asymptotically distributed as N(µ,σ 2 ) or simply asymptotically ormal. σ UW-Madiso (Statistics) Stat 609 Lecture / 16
13 Other forms of CLT There are other forms of CLT for o-iid sequeces of radom variables/vectors. We itroduce without proof the followig three. Lideberg s CLT Let X, = 1,2,..., be idepedet radom variables with ) 0 < σ 2 = Var( X j <, = 1,2,... If lim 1 σ 2 [ ] E (X j E(X j )) 2 I { Xj E(X j ) >εσ } = 0 for ay ε > 0, the 1 σ (X j E(X j )) coverges i distributio to N(0,1). UW-Madiso (Statistics) Stat 609 Lecture / 16
14 Liapouov s CLT Let X, = 1,2,..., be idepedet radom variables with lim 1 σ 2+δ E X j EX j 2+δ = 0 for some δ > 0. The the same coclusio as i Lideberg s CLT holds. The coditio i Liapouov s CLT is stroger tha the coditio i Lideberg s CLT, Liapouov s coditio is easier to verify. The CLT for a liear combiatio of iid radom variables Let X, = 1,2,..., be iid radom variables with E(X 1 ) = µ ad Var(X 1 ) = σ 2 <. Let c R, = 1,2,..., be costats such that ( ) The i=1 c i (X i µ) σ lim / ci 2 i=1 max 1 i c2 i / ci 2 i=1 = 0. coverges i distributio to N(0,1). UW-Madiso (Statistics) Stat 609 Lecture / 16
15 Example. We apply Liapouov s CLT to idepedet radom variables X 1,X 2,... satisfyig P(X j = ±j a ) = P(X j = 0) = 1/3, where a > 0, j = 1,2,... Note that E(X j ) = 0 ad for all j ) σ 2 = Var( X j = j ) = Var(X 2 3 We eed to show that, for some δ > 0, For ay δ > 0, Sice lim 1 σ 2+δ E X j EX j 2+δ = 0 E X j E(X j ) 2+δ = j t 1 j+1 x t dx j j (2+δ)a. j t j=2 j 2a. UW-Madiso (Statistics) Stat 609 Lecture / 16
16 ad we coclude that for ay t > 0. The lim 1 σ 2+δ 1 j+1 x t dx = x t dx = t+1 1 j 1 t + 1, lim 1 t+1 E X j EX j 2+δ = lim = lim ( 3 = 2 = 0. j t = 1 t j(2+δ)a ( 2 3 j2a ) 1+δ/2 ( ) 3 δ/2 (2a + 1) 1+δ/2 2 (2 + δ)a + 1 ) δ/2 (2a + 1) 1+δ/2 (2 + δ)a + 1 lim (2+δ)a+1 (2a+1)(1+δ/2) 1 δ/2 UW-Madiso (Statistics) Stat 609 Lecture / 16
Lecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationLecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationLecture 15: Density estimation
Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationSTATISTICAL METHODS FOR BUSINESS
STATISTICAL METHODS FOR BUSINESS UNIT 5. Joit aalysis ad limit theorems. 5.1.- -dimesio distributios. Margial ad coditioal distributios 5.2.- Sequeces of idepedet radom variables. Properties 5.3.- Sums
More informationElements of Statistical Methods Lots of Data or Large Samples (Ch 8)
Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationExercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1).
Assigmet 7 Exercise 4.3 Use the Cotiuity Theorem to prove the Cramér-Wold Theorem, Theorem 4.12. Hit: a X d a X implies that φ a X (1) φ a X(1). Sketch of solutio: As we poited out i class, the oly tricky
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationMA Advanced Econometrics: Properties of Least Squares Estimators
MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationLecture 24: Variable selection in linear models
Lecture 24: Variable selectio i liear models Cosider liear model X = Z β + ε, β R p ad Varε = σ 2 I. Like the LSE, the ridge regressio estimator does ot give 0 estimate to a compoet of β eve if that compoet
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More information5. Limit Theorems, Part II: Central Limit Theorem. ECE 302 Fall 2009 TR 3 4:15pm Purdue University, School of ECE Prof.
5. Limit Theorems, Part II: Cetral Limit Theorem ECE 302 Fall 2009 TR 3 4:15pm Purdue Uiversity, School of ECE Prof. Ilya Pollak WLLN ad CLT X 1,, X i.i.d. with fiite mea μ ad variace σ 2 WLLN ad CLT X
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationVariance of Discrete Random Variables Class 5, Jeremy Orloff and Jonathan Bloom
Variace of Discrete Radom Variables Class 5, 18.05 Jeremy Orloff ad Joatha Bloom 1 Learig Goals 1. Be able to compute the variace ad stadard deviatio of a radom variable.. Uderstad that stadard deviatio
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationLecture 16: UMVUE: conditioning on sufficient and complete statistics
Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. Comments:
Recall: STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Commets:. So far we have estimates of the parameters! 0 ad!, but have o idea how good these estimates are. Assumptio: E(Y x)! 0 +! x (liear coditioal
More informationRegression with an Evaporating Logarithmic Trend
Regressio with a Evaporatig Logarithmic Tred Peter C. B. Phillips Cowles Foudatio, Yale Uiversity, Uiversity of Aucklad & Uiversity of York ad Yixiao Su Departmet of Ecoomics Yale Uiversity October 5,
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationSTAT Homework 2 - Solutions
STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationBHW #13 1/ Cooper. ENGR 323 Probabilistic Analysis Beautiful Homework # 13
BHW # /5 ENGR Probabilistic Aalysis Beautiful Homework # Three differet roads feed ito a particular freeway etrace. Suppose that durig a fixed time period, the umber of cars comig from each road oto the
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationMIT Spring 2016
MIT 18.655 Dr. Kempthore Sprig 2016 1 MIT 18.655 Outlie 1 2 MIT 18.655 Beroulli s Weak Law of Large Numbers X 1, X 2,... iid Beroulli(θ). S i=1 = X i Biomial(, θ). S P θ. Proof: Apply Chebychev s Iequality,
More informationEstimation of the Mean and the ACVF
Chapter 5 Estimatio of the Mea ad the ACVF A statioary process {X t } is characterized by its mea ad its autocovariace fuctio γ ), ad so by the autocorrelatio fuctio ρ ) I this chapter we preset the estimators
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationAre the following series absolutely convergent? n=1. n 3. n=1 n. ( 1) n. n=1 n=1
Absolute covergece Defiitio A series P a is called absolutely coverget if the series of absolute values P a is coverget. If the terms of the series a are positive, absolute covergece is the same as covergece.
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationThe central limit theorem for Student s distribution. Problem Karim M. Abadir and Jan R. Magnus. Econometric Theory, 19, 1195 (2003)
The cetral limit theorem for Studet s distributio Problem 03.6.1 Karim M. Abadir ad Ja R. Magus Ecoometric Theory, 19, 1195 (003) Z Ecoometric Theory, 19, 003, 1195 1198+ Prited i the Uited States of America+
More information11 THE GMM ESTIMATION
Cotets THE GMM ESTIMATION 2. Cosistecy ad Asymptotic Normality..................... 3.2 Regularity Coditios ad Idetificatio..................... 4.3 The GMM Iterpretatio of the OLS Estimatio.................
More informationCentral Limit Theorem using Characteristic functions
Cetral Limit Theorem usig Characteristic fuctios RogXi Guo MAT 477 Jauary 20, 2014 RogXi Guo (2014 Cetral Limit Theorem usig Characteristic fuctios Jauary 20, 2014 1 / 15 Itroductio study a radom variable
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationLecture 4. Random variable and distribution of probability
Itroductio to theory of probability ad statistics Lecture. Radom variable ad distributio of probability dr hab.iż. Katarzya Zarzewsa, prof.agh Katedra Eletroii, AGH e-mail: za@agh.edu.pl http://home.agh.edu.pl/~za
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More informationECE 901 Lecture 14: Maximum Likelihood Estimation and Complexity Regularization
ECE 90 Lecture 4: Maximum Likelihood Estimatio ad Complexity Regularizatio R Nowak 5/7/009 Review : Maximum Likelihood Estimatio We have iid observatios draw from a ukow distributio Y i iid p θ, i,, where
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationProblems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:
Math 224 Fall 2017 Homework 4 Drew Armstrog Problems from 9th editio of Probability ad Statistical Iferece by Hogg, Tais ad Zimmerma: Sectio 2.3, Exercises 16(a,d),18. Sectio 2.4, Exercises 13, 14. Sectio
More information4. Basic probability theory
Cotets Basic cocepts Discrete radom variables Discrete distributios (br distributios) Cotiuous radom variables Cotiuous distributios (time distributios) Other radom variables Lect04.ppt S-38.45 - Itroductio
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationStatistical Analysis on Uncertainty for Autocorrelated Measurements and its Applications to Key Comparisons
Statistical Aalysis o Ucertaity for Autocorrelated Measuremets ad its Applicatios to Key Comparisos Nie Fa Zhag Natioal Istitute of Stadards ad Techology Gaithersburg, MD 0899, USA Outlies. Itroductio.
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More information