Lecture 8: Convergence of transformations and law of large numbers
|
|
- Susanna Barnett
- 5 years ago
- Views:
Transcription
1 Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges to g(x) i the same sese. The cotiuous mappig theorem provides a aswer to the questio i may problems. Theorem Cotiuous mappig theorem Let X,X 1,X 2,... be radom k-vectors defied o a probability space ad g be a measurable fuctio from (R k,b k ) to (R l,b l ). Suppose that g is cotiuous a.s. P X. The (i) X a.s. X implies g(x ) a.s. g(x); (ii) X p X implies g(x ) p g(x); (iii) X d X implies g(x ) d g(x). UW-Madiso (Statistics) Stat 709 Lecture / 15
2 Proof (i) ca be established usig a result i calculus. (iii) follows from Theorem 1.9(i): for ay bouded ad cotiuous h, E[h(g(X ))] E[h(g(X))], sice h g is bouded ad cotiuous. We prove (ii) for the special case of X = c (a costat). From the cotiuity of g, for ay ε > 0, there is a δ ε > 0 such that Hece, ad g(x) g(c) < ε wheever x c < δ ε. {ω : g(x (ω)) g(c) < ε} {ω : X (ω) c < δ ε } P( g(x ) g(c) ε) P( X c δ ε ). Hece g(x ) p g(c) follows from X p c. Is the previous arguemet still valid whe c is replaced by the radom vector X i the geeral case? If ot, how do we fix the proof? UW-Madiso (Statistics) Stat 709 Lecture / 15
3 Example (i) Let X 1,X 2,... be radom variables. If X d X, where X has the N(0,1) distributio, the X 2 d Y, where Y has the chi-square distributio χ 2 1. (ii) Let (X,Y ) be radom 2-vectors satisfyig (X,Y ) d (X,Y ), where X ad Y are idepedet radom variables havig the N(0, 1) distributio. The X /Y d X/Y, which has the Cauchy distributio C(0,1). (iii) Uder the coditios i part (ii), max{x,y } d max{x,y }, which has the c.d.f. [Φ(x)] 2 (Φ(x) is the c.d.f. of N(0,1)). I Example 1.30(ii) ad (iii), the coditio that (X,Y ) d (X,Y ) caot be relaxed to X d X ad Y d Y (exercise); i.e., we eed the covergece of the joit c.d.f. of (X,Y ). This is differet whe d is replaced by p or a.s.. The ext result, which plays a importat role i statistics, establishes the covergece i distributio of X + Y or X Y whe o iformatio regardig the joit c.d.f. of (X,Y ) is provided. UW-Madiso (Statistics) Stat 709 Lecture / 15
4 Theorem 1.11 (Slutsky s theorem) Let X,X 1,X 2,..., Y 1,Y 2,... be radom variables o a probability space. Suppose that X d X ad Y p c, where c is a costat. The (i) X + Y d X + c; (ii) Y X d cx; (iii) X /Y d X/c if c 0. Proof We prove (i) oly. The proofs of (ii) ad (iii) are left as exercises. Let t R ad ε > 0 be fixed costats. The F X +Y (t) = P(X + Y t) P({X + Y t} { Y c < ε}) + P( Y c ε) P(X t c + ε) + P( Y c ε) UW-Madiso (Statistics) Stat 709 Lecture / 15
5 Proof (cotiued) Similarly, F X +Y (t) P(X t c ε) P( Y c ε). If t c, t c + ε, ad t c ε are cotiuity poits of F X, the it follows from the previous two iequalities ad the hypotheses of the theorem that F X (t c ε) limif Sice ε ca be arbitrary (why?), F X +Y (t) limsupf X +Y (t) F X (t c + ε). lim F X +Y (t) = F X (t c). The result follows from F X+c (t) = F X (t c). A applicatio of Theorem 1.11 is give i the proof of the followig importat result. UW-Madiso (Statistics) Stat 709 Lecture / 15
6 Theorem 1.12 Let X 1,X 2,... ad Y = (Y 1,...,Y k ) be radom k-vectors satisfyig a (X c) d Y, where c R k ad {a } is a sequece of positive umbers with lim a =. Let g be a fuctio from R k to R. (i) If g is differetiable at c, the a [g(x ) g(c)] d [ g(c)] τ Y, where g(x) deotes the k-vector of partial derivatives of g at x. (ii) Suppose that g has cotiuous partial derivatives of order m > 1 i a eighborhood of c, with all the partial derivatives of order j, 1 j m 1, vaishig at c, but with the mth-order partial derivatives ot all vaishig at c. The a m 1 [g(x ) g(c)] d m! k i 1 =1 k i m =1 m g x i1 x Y i1 Y im im x=c UW-Madiso (Statistics) Stat 709 Lecture / 15
7 Proof We prove (i) oly. Let Z = a [g(x ) g(c)] a [ g(c)] τ (X c). If we ca show that Z = o p (1), the by a (X c) d Y, Theorem 1.9(iii), ad Theorem 1.11(i), result (i) holds. The differetiability of g at c implies that for ay ε > 0, there is a δ ε > 0 such that g(x) g(c) [ g(c)] τ (x c) ε x c wheever x c < δ ε. The for a fixed η > 0, P( Z η) P( X c δ ε ) + P(a X c η/ε). Sice a, a (X c) d Y ad Theorem 1.11(ii) imply X p c. By Theorem 1.10(iii), a (X c) d Y implies a X c d Y. Without loss of geerality, we ca assume that η/ε is a cotiuity poit of F Y. UW-Madiso (Statistics) Stat 709 Lecture / 15
8 Proof (cotiued) The lim supp( Z η) lim P( X c δ ε ) + lim P(a X c η/ε) = P( Y η/ε). Z p 0 follows sice ε ca be arbitrary. Remarks I statistics, we ofte eed a odegeerated limitig distributio of a [g(x ) g(c)] so that probabilities ivolvig a [g(x ) g(c)] ca be approximated by the c.d.f. of [ g(c)] τ Y, uder Theorem 1.12(i). Whe g(c) = 0, Theorem 1.12(i) idicates that the limitig distributio of a [g(x ) g(c)] is degeerated. I such cases the result i Theorem 1.12(ii) may be useful. UW-Madiso (Statistics) Stat 709 Lecture / 15
9 Corollary 1.1 (the δ-method) Assume the coditios of Theorem If Y has the N k (0,Σ) distributio, the Example 1.31 a [g(x ) g(c)] d N (0,[ g(c)] τ Σ g(c)). Let {X } be a sequece of radom variables satisfyig (X c) d N(0,1). Cosider the fuctio g(x) = x 2. If c 0, the a applicatio of Corollary 1.1 gives that (X 2 c 2 ) d N(0,4c 2 ). If c = 0, g (c) = 0 but g (c) = 2. Hece, a applicatio of Theorem 1.12(ii) gives that X 2 d [N(0,1)] 2, which has the chi-square distributio χ1 2 (Example 1.14). The last result ca also be obtaied by applyig Theorem 1.10(iii). UW-Madiso (Statistics) Stat 709 Lecture / 15
10 Example: Ratio estimator (X 1,Y 1 ),...,(X,Y ) are iid bivariate radom vectors with fiite 2d order momemts X = 1 i X i, Ȳ = 1 i Y i µ x = E(X 1 ), µ y = E(Y 1 ) 0, σ 2 x =Var(X 1 ), σ 2 y =Var(Y 1 ), σ xy =Cov(X 1,Y 1 ) By the CLT, (( X Ȳ ) ( µx µ y )) ( ( σ 2 d N 2 0, x σ xy σ xy σy 2 )) By the δ-method, g(x,y) = x/y, g/ x = y 1, g/ y = xy 2 ( X µ x Ȳ ( σ 2 = (µ y 1, µ x µ y 2 σ 2 ) x σ xy σ xy σy 2 µ y ) d N(0,σ 2 ) )( µ 1 y µ x µ 2 y ) = σ 2 x µ 2 y µ xσ xy µ 3 y + µ2 x σ 2 y µ y 4 UW-Madiso (Statistics) Stat 709 Lecture / 15
11 The law of large umbers The law of large umbers cocers the limitig behavior of a sum of radom variables. The weak law of large umbers (WLLN) refers to covergece i probability. The strog law of large umbers (SLLN) refers to a.s. covergece. The WLLN ad SLLN play importat roles i establishig cosistecy of estimators i large sample statistical iferece. Lemma 1.6. (Kroecker s lemma) Let x R, a R, 0 < a a +1, = 1,2,..., ad a. If the series =1 x /a coverges, the a 1 i=1 x i 0. Our first result gives the WLLN ad SLLN for a sequece of idepedet ad idetically distributed (i.i.d.) radom variables. UW-Madiso (Statistics) Stat 709 Lecture / 15
12 Theorem 1.13 Let X 1,X 2,... be i.i.d. radom variables. (i) (The WLLN). A ecessary ad sufficiet coditio for the existece of a sequece of real umbers {a } for which 1 X i a p 0 i=1 is that P( X 1 > ) 0, i which case we may take a = E(X 1 I { X1 }). (ii) (The SLLN). A ecessary ad sufficiet coditio for the existece of a costat c for which 1 X i a.s. c i=1 is that E X 1 <, i which case c = EX 1 ad 1 c i (X i EX 1 ) a.s. 0 i=1 for ay bouded sequece of real umbers {c i }. UW-Madiso (Statistics) Stat 709 Lecture / 15
13 Proof of Theorem 1.13(i) We prove the sufficiecy. The proof of ecessity ca be foud i Petrov (1975). Cosider a sequece of radom variables obtaied by trucatig X j s at : Y j = X j I { Xj }. Let T = X X, Z = Y Y. The P(T Z ) j=1 P(Y j X j ) = P( X 1 > ) 0. For ay ε > 0, it follows from Chebyshev s iequality that ( ) Z EZ P > ε var(z ) ε 2 2 = var(y 1) ε 2 EY 1 2 ε 2, where the last equality follows from the fact that Y j, j = 1,...,, are i.i.d. UW-Madiso (Statistics) Stat 709 Lecture / 15
14 Proof of Theorem 1.13(i) (cotiued) From itegratio by parts, we obtai that EY1 2 = 1 x 2 df X1 (x) = 2 xp( X 1 > x)dx P( X 1 > ), [0,] 0 which coverges to 0 sice P( X 1 > ) 0 (why?). This proves that (Z EZ )/ p 0, which together with P(T Z ) 0 ad the fact that EY j = E(X 1 I { X1 }) imply the result. Proof of Theorem 1.13(ii) The proof for sufficiecy is give i the textbook. We prove the ecessity. Suppose that T / a.s. holds for some c R, T = X X. The X = T c 1 ( T 1 1 c ) + c a.s. 0. UW-Madiso (Statistics) Stat 709 Lecture / 15
15 Proof of Theorem 1.13 (ii) From Exercise 114, X / a.s. 0 ad the i.i.d. assumptio o X s imply P( X ) = P( X 1 ) <, =1 =1 which implies E X 1 < (Exercise 54). From the proved sufficiecy, c = EX 1. Remarks If E X 1 <, the a = E(X 1 I { X1 }) EX 1 ad result the WLLN is actually established i Example 1.28 i a much simpler way. O the other had, if E X 1 <, the a stroger result, the SLLN, ca be obtaied. Some results for the case of E X 1 = ca be foud i Exercise 148 ad Theorem i Chug (1974). The ext result is for sequeces of idepedet but ot ecessarily idetically distributed radom variables. UW-Madiso (Statistics) Stat 709 Lecture / 15
ST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationLecture 23: Minimal sufficiency
Lecture 23: Miimal sufficiecy Maximal reductio without loss of iformatio There are may sufficiet statistics for a give problem. I fact, X (the whole data set) is sufficiet. If T is a sufficiet statistic
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationLecture 21: Convergence of transformations and generating a random variable
Lecture 21: Convergence of transformations and generating a random variable If Z n converges to Z in some sense, we often need to check whether h(z n ) converges to h(z ) in the same sense. Continuous
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationMath 341 Lecture #31 6.5: Power Series
Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationInformation Theory and Statistics Lecture 4: Lempel-Ziv code
Iformatio Theory ad Statistics Lecture 4: Lempel-Ziv code Łukasz Dębowski ldebowsk@ipipa.waw.pl Ph. D. Programme 203/204 Etropy rate is the limitig compressio rate Theorem For a statioary process (X i)
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More informationArkansas Tech University MATH 2924: Calculus II Dr. Marcel B. Finan
Arkasas Tech Uiversity MATH 94: Calculus II Dr Marcel B Fia 85 Power Series Let {a } =0 be a sequece of umbers The a power series about x = a is a series of the form a (x a) = a 0 + a (x a) + a (x a) +
More informationLecture 15: Density estimation
Lecture 15: Desity estimatio Why do we estimate a desity? Suppose that X 1,...,X are i.i.d. radom variables from F ad that F is ukow but has a Lebesgue p.d.f. f. Estimatio of F ca be doe by estimatig f.
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationLecture 16: UMVUE: conditioning on sufficient and complete statistics
Lecture 16: UMVUE: coditioig o sufficiet ad complete statistics The 2d method of derivig a UMVUE whe a sufficiet ad complete statistic is available Fid a ubiased estimator of ϑ, say U(X. Coditioig o a
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More information62. Power series Definition 16. (Power series) Given a sequence {c n }, the series. c n x n = c 0 + c 1 x + c 2 x 2 + c 3 x 3 +
62. Power series Defiitio 16. (Power series) Give a sequece {c }, the series c x = c 0 + c 1 x + c 2 x 2 + c 3 x 3 + is called a power series i the variable x. The umbers c are called the coefficiets of
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationWe are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n
Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationLecture 4. We also define the set of possible values for the random walk as the set of all x R d such that P(S n = x) > 0 for some n.
Radom Walks ad Browia Motio Tel Aviv Uiversity Sprig 20 Lecture date: Mar 2, 20 Lecture 4 Istructor: Ro Peled Scribe: Lira Rotem This lecture deals primarily with recurrece for geeral radom walks. We preset
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationEntropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP
Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.
More informationGeneralized Semi- Markov Processes (GSMP)
Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationChapter 7 Isoperimetric problem
Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationMAS111 Convergence and Continuity
MAS Covergece ad Cotiuity Key Objectives At the ed of the course, studets should kow the followig topics ad be able to apply the basic priciples ad theorems therei to solvig various problems cocerig covergece
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationMATH301 Real Analysis (2008 Fall) Tutorial Note #7. k=1 f k (x) converges pointwise to S(x) on E if and
MATH01 Real Aalysis (2008 Fall) Tutorial Note #7 Sequece ad Series of fuctio 1: Poitwise Covergece ad Uiform Covergece Part I: Poitwise Covergece Defiitio of poitwise covergece: A sequece of fuctios f
More informationChapter 8. Uniform Convergence and Differentiation.
Chapter 8 Uiform Covergece ad Differetiatio This chapter cotiues the study of the cosequece of uiform covergece of a series of fuctio I Chapter 7 we have observed that the uiform limit of a sequece of
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationApproximation theorems for localized szász Mirakjan operators
Joural of Approximatio Theory 152 (2008) 125 134 www.elsevier.com/locate/jat Approximatio theorems for localized szász Miraja operators Lise Xie a,,1, Tigfa Xie b a Departmet of Mathematics, Lishui Uiversity,
More informationAsymptotic Results for the Linear Regression Model
Asymptotic Results for the Liear Regressio Model C. Fli November 29, 2000 1. Asymptotic Results uder Classical Assumptios The followig results apply to the liear regressio model y = Xβ + ε, where X is
More informationMAXIMAL INEQUALITIES AND STRONG LAW OF LARGE NUMBERS FOR AANA SEQUENCES
Commu Korea Math Soc 26 20, No, pp 5 6 DOI 0434/CKMS20265 MAXIMAL INEQUALITIES AND STRONG LAW OF LARGE NUMBERS FOR AANA SEQUENCES Wag Xueju, Hu Shuhe, Li Xiaoqi, ad Yag Wezhi Abstract Let {X, } be a sequece
More informationECE534, Spring 2018: Solutions for Problem Set #2
ECE534, Srig 08: s for roblem Set #. Rademacher Radom Variables ad Symmetrizatio a) Let X be a Rademacher radom variable, i.e., X = ±) = /. Show that E e λx e λ /. E e λx = e λ + e λ = + k= k=0 λ k k k!
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationElements of Statistical Methods Lots of Data or Large Samples (Ch 8)
Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x
More informationMathematical Methods for Physics and Engineering
Mathematical Methods for Physics ad Egieerig Lecture otes Sergei V. Shabaov Departmet of Mathematics, Uiversity of Florida, Gaiesville, FL 326 USA CHAPTER The theory of covergece. Numerical sequeces..
More informationAnalytic Continuation
Aalytic Cotiuatio The stadard example of this is give by Example Let h (z) = 1 + z + z 2 + z 3 +... kow to coverge oly for z < 1. I fact h (z) = 1/ (1 z) for such z. Yet H (z) = 1/ (1 z) is defied for
More informationn=1 a n is the sequence (s n ) n 1 n=1 a n converges to s. We write a n = s, n=1 n=1 a n
Series. Defiitios ad first properties A series is a ifiite sum a + a + a +..., deoted i short by a. The sequece of partial sums of the series a is the sequece s ) defied by s = a k = a +... + a,. k= Defiitio
More informationMi-Hwa Ko and Tae-Sung Kim
J. Korea Math. Soc. 42 2005), No. 5, pp. 949 957 ALMOST SURE CONVERGENCE FOR WEIGHTED SUMS OF NEGATIVELY ORTHANT DEPENDENT RANDOM VARIABLES Mi-Hwa Ko ad Tae-Sug Kim Abstract. For weighted sum of a sequece
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More informationA Proof of Birkhoff s Ergodic Theorem
A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed
More informationMa 530 Infinite Series I
Ma 50 Ifiite Series I Please ote that i additio to the material below this lecture icorporated material from the Visual Calculus web site. The material o sequeces is at Visual Sequeces. (To use this li
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationChapter 8. Euler s Gamma function
Chapter 8 Euler s Gamma fuctio The Gamma fuctio plays a importat role i the fuctioal equatio for ζ(s) that we will derive i the ext chapter. I the preset chapter we have collected some properties of the
More information