THE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES
|
|
- Gregory Ferguson
- 5 years ago
- Views:
Transcription
1 THE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES Debdeep Pati Idia Statistical Istitute, Kolkata Jue 26, 2006 Abstract The traditioal proof of the strog law of large umbers usig idepedet ad idetically distributed radom variables 3, was developed by Kolmogorov i the 1930 s, ad explais well what happes ad much more, at this level of geerality. I this short article, I have tried to give a simple proof of the strog law of large umbers with the assumptio of oly statioarity of the give sequece of radom variables. The proof is essetially alog the lies of the proof of the ergodic theorem i 1. 1 Itroductio The law of large umbers, ot really a law but a mathematical theorem, is at the same time a justificatio for applicatio of statistics ad a essetial tool for the mathematical theory of probability. Jakob Beroulli, i the cocludig chapter of his Ars cojectadi 2 proposed ad proved the first versio of the law of large umbers ad qualified the problem he was facig as oe of exceedig difficulty ad usefuless whose solutio would icrease importace ad meaig of all of the theory. First, I would try to motivate the essece of the basic law of large umbers by cosiderig, very simply, a ifiite sequece x 0, x 1, x 2,... each of whose elemets is either 0 or 1. Perhaps it will help to thik of x as the result of the th trial of a ucertai experimet with x = 1 with some 1
2 probability p desigig success ad x idicatig failure. Let = 0 with the remaiig probability x = x 0 + x 1 + x x 1 deote the average umber of success upto time. mathematically that for some sequece x, lim x ( 1) It is very easy to see exists, while for other sequeces x this is ot the case. But othig impedes the averages x from oscillatig more ad more slowly as grows. I our day to day life the society makes seemigly uderstadable statemets about the percetage of dyig smokers, the probability of rai tomorrow or a idustrial average yield. I the ext sectio, we formally state ad prove the strog law of large umbers for a statioary sequece. 2 The Strog Law of large umbers I this sectio, I will prove three theorems, the first of which shows that the lim x exists assumig oly statioarity of the give sequece ad the ext theorem helps us to idetify the limit oly i the case of i.i.d sequeces havig fiite secod momet. The limit ca be idetifed i the most geeral case ad ca be foud i advaced probability texts. The proof essetially follows the same lies as that of Kolmogorov s classical strog law of large umbers ad hece ot discussed here. The last theorem proves the L 1 covergece of x. Theorem 1 {X i } i 1 be a statioary sequece of radom variables such that E X 1 <. Let The lim X exists, i.e., X = X 1 + X 2 + X X ( 1) lim sup X = lim if X Proof: Fix α > 0, β > 0. Let X = lim sup X ad X = lim if X. We will show that X = X (2.1) I order to prove 2.1, we will first show E X 1 E X (2.2) 2
3 Let Xα = mi{x, α} ad X β k = max{x k, β} k 1. We will later prove that 2.2 is true if we ca show for ay α > 0, β > 0 E X β 1 E Xα (2.3) We have Xα X ad X β k X k k 1. For ay give ɛ > 0, defie for each k 1, { } N β k = mi r 1 1 r X β k+i 1 r X α ɛ k 1 Next cosider the followig two lemmas. { } Lemma 1 N β k as defied above are idepedet ad idetically distributed fiite radom variables. Proof of the Lemma: For each k 1, ɛ > 0, sice Xα X ad X β k X k k 1, the set { } r 1 1 r X β k+i 1 r X α ɛ is o-empty by the defiitio of lim sup. Hece { } N β k k 1 are fiite radom variables. Now sice X β k = max{x k, β} k 1, {X β k } k 1 is also a statioary sequece sice a cotiuous fuctio of a statioary sequece is also statioary. The i.i.d property follows simply from the statioarity of the sequece {X β k } k 1. Lemma 2 Suppose for each ɛ > 0, there exists M 1 such that k 1, N β k M a.s., the we have E X β 1 E Xα Proof of the Lemma: Obtai a partitio of {1, 2, 3,..., } as: A 1 = {1, 2, 3,..., N β 1 }, A 2 = {N β 1 +1, N β 1 +2,..., N β 2 },..., A l = {N β l 1 +1, N β 1 + 2,..., N β l }, A l+1 = {N β l + 1,..., } such that N β l+1 >. Clearly A l+1 < M so that A 1 A 2 A l > M. Hece we have X β k = X β k + X β k ( M)(Xα ɛ) Mβ (2.4) k=1 k A 1 A l k A l+1 Takig expectatio of 2.4 ad dividig by we get ( ) M E X β 1 (E X α ɛ) Mβ 3
4 Now lettig ad ɛ 0, we have E { } Sice N β k X β 1 E Xα as defied above are i.i.d radom variables ad fiite w.p 1, we k 1 { } are certaily ot goig to get such bouds for N β k. Still fiiteess of the k 1 aforesaid i.i.d radom variables esures) that give ɛ > 0, we are able to get a M large eough such that P (N βk > M < ɛ k 1. Defie k 1, X β k = Xβ k 1 N β k M + max{α, Xβ k }1 N β k >M (2.5) Therefore, X β k Xβ k β k 1. For ay give ɛ > 0, defie for each k 1, { } N β k = mi r 1 1 r X β k+i 1 r X α ɛ Sice, X β k Xβ k, Ñ β k N β k, k 1. Note that if N β k > M, Ñ β k = 1 sice i that case X β k α X α. Therefore Ñ β k M k 1. Proceedig as i lemma 2 we get E But from 2.5 we get, X β 1 = X β 1 + Xβ 1 ( M ( max{α, X β 1 } X β 1 ) (E X α ɛ) Mβ (2.6) ) 1 N β 1 >M Xβ 1 + (α + β) 1 N β 1 >M (2.7) Takig expectatio of 2.7 ad by our choice of M, we get from 2.6, ( ) M E X β 1 E Xβ 1 (α + β) ɛ (E X α ɛ) Mβ (α + β) ɛ Now lettig ad ɛ 0, we have E X β 1 E Xα (2.8) Also we have X β 1 X 1 as β ad X β 1 X 1 ad E X 1 <. Hece by DCT, E X β 1 E X 1 as β. Agai we have, X α = X α1 X α <0 + X α1 X α 0 = X 1 X <0 + X α1 X 0 Now X α1 X 0 X 1 X 0 as α. So by MCT, E X α1 X 0 E X 1 X 0 Now X k X k 4
5 X 1 k=1 X k lim if X lim if 1 k=1 X k E X lim if E 1 k=1 X k E X E X 1 < E X > E X 1 X <0 >. Therefore we have Hece as α E X α = E X 1 X <0 + E X α1 X 0 So as α ad β we fially have E X α E X (2.9) E X 1 E X (2.10) Now replacig X k by X k i the above(ote that all the assumptios of statioarity ad fiiteess of expectatio of the modulus of the radom variables remai valid) we have E X 1 E X (2.11) Combiig 2.10 ad 2.11 we have E X = E X = E X 1 (2.12) Agai sice X X 0 ad E X X = 0, we observe that X = X = S(say) which implies that X a.e S thus provig the theorem. Theorem 2 {X i } i 1 be a i.i.d sequece of radom variables such that E X 2 1 <. The lim sup X = lim if X = E X 1 Proof: The first equality i.e., lim sup X = lim if X ca be proved exactly i the same maer as i the case of statioary sequeces. Let lim sup X = lim if X = lim X = S. We will show that S = E X 1 = µ(say) (2.13) Let V ar X 1 = σ 2 < (by the assumptio). Observe that E X µ 2 = σ 2 5
6 Therefore, lim E X µ 2 = 0 (2.14) We have already proved i Theorem 1 that E S = E X 1 < (2.15) Now E S 2 = E 2 lim X = E lim 2 2 X = E lim if X lim if E 2 X = lim if 1 2 E X2 i + i j E X ix j = lim if 1 2 ((µ 2 + σ 2 ) + ( 1)µ 2 ) = µ 2 <. Therefore, E S 2 < (2.16) Sice X a.e. S, give ɛ > 0, we ca fid a 0 such that 0, we have X (S ɛ, S + ɛ). Therefore ( X µ ) 2 max { (S ɛ µ) 2, (S + ɛ µ) 2} 0. From 2.15 ad 2.16 we get, E max { (S ɛ µ) 2, (S + ɛ µ) 2} <. Also we have ( X µ ) 2 a.e. (S µ) 2 (2.17) Thus from 2.14 ad 2.17 ad applyig DCT, we get lim E X µ 2 = E S µ 2 = 0 S = µ a.s. Theorem 3 {X i } i 1 be a statioary sequece of radom variables such that E X 1 <. Let X = X 1 + X 2 + X X ad lim sup X = lim if X = S(say).The ( 1) L X 1 S Proof: The proof ecessarily rests upo the Egorov s theorem, which we are goig to state as a lemma the proof of which ca be foud i stadard probability texts. a.s Lemma 3 {X } 1 ad X are fiite valued radom variables ad X X, the for every ɛ > 0 a set E with P ( E ) < ɛ ad X X uiformly o E i.e., sup X (ω) X (ω) 0 as ω E 6
7 We cotiue with the proof of the theorem. Let ɛ > 0 be give. Sice sup ω E X (ω) X (ω) 0 as, 0 such that 0, Note that 0, sup X (ω) X (ω) < ɛ ω E E X S = E X S 1 E + E X S 1 E ɛp (E) + E X S 1 E Now, E X S 1 E E 1 X i S 1 E 1 Hece from 2.18 ad 2.19 we get, E X S ɛp (E) + E X 1 S 1 E X1 S λ + E (2.18) E X i S 1 E = E X 1 S 1 E (2.19) X 1 S 1 E X1 S >λ Therefore, E X S ɛp (E) + λp ( E ) + E X 1 S 1 X1 S >λ (2.20) Now ote that E X 1 S E X 1 + E S ad E S = E lim X = E lim X = E lim if X lim if E X lim if 1 E X i = lim if E X 1 = E X 1 <. Therefore we have E X 1 S <. Also ad Hece by DCT, we have X 1 S 1 X1 S >λ 0 as λ X 1 S 1 X1 S >λ X 1 S with E X 1 S < E X 1 S 1 X1 S >λ 0 as λ From 2.20, we get E X S ɛ (P (E) + λ) + E X 1 S 1 X1 S >λ Now let ɛ 0 ad λ to get 0, E X S 0 which shows L X 1 S cocludig the proof. 7
8 Refereces 1 Y.Katzelso, B.Weiss: A simple proof of some ergodic theorems. Isr.J.Math. 42(1982), J.Beroulli: Ars cojectadi. Basel, 1713 (published posthumously). 3 A.N.Kolmogorov: Grudbegriffe der Wahrscheilichkeitsrechug. Spriger Verlag, Berli
BIRKHOFF ERGODIC THEOREM
BIRKHOFF ERGODIC THEOREM Abstract. We will give a proof of the poitwise ergodic theorem, which was first proved by Birkhoff. May improvemets have bee made sice Birkhoff s orgial proof. The versio we give
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationAssignment 5: Solutions
McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationSolutions to home assignments (sketches)
Matematiska Istitutioe Peter Kumli 26th May 2004 TMA401 Fuctioal Aalysis MAN670 Applied Fuctioal Aalysis 4th quarter 2003/2004 All documet cocerig the course ca be foud o the course home page: http://www.math.chalmers.se/math/grudutb/cth/tma401/
More informationMi-Hwa Ko and Tae-Sung Kim
J. Korea Math. Soc. 42 2005), No. 5, pp. 949 957 ALMOST SURE CONVERGENCE FOR WEIGHTED SUMS OF NEGATIVELY ORTHANT DEPENDENT RANDOM VARIABLES Mi-Hwa Ko ad Tae-Sug Kim Abstract. For weighted sum of a sequece
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationON THE DELOCALIZED PHASE OF THE RANDOM PINNING MODEL
O THE DELOCALIZED PHASE OF THE RADOM PIIG MODEL JEA-CHRISTOPHE MOURRAT Abstract. We cosider the model of a directed polymer pied to a lie of i.i.d. radom charges, ad focus o the iterior of the delocalized
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationChapter 7 Isoperimetric problem
Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationMA131 - Analysis 1. Workbook 2 Sequences I
MA3 - Aalysis Workbook 2 Sequeces I Autum 203 Cotets 2 Sequeces I 2. Itroductio.............................. 2.2 Icreasig ad Decreasig Sequeces................ 2 2.3 Bouded Sequeces..........................
More informationDirichlet s Theorem on Arithmetic Progressions
Dirichlet s Theorem o Arithmetic Progressios Athoy Várilly Harvard Uiversity, Cambridge, MA 0238 Itroductio Dirichlet s theorem o arithmetic progressios is a gem of umber theory. A great part of its beauty
More informationMATH301 Real Analysis (2008 Fall) Tutorial Note #7. k=1 f k (x) converges pointwise to S(x) on E if and
MATH01 Real Aalysis (2008 Fall) Tutorial Note #7 Sequece ad Series of fuctio 1: Poitwise Covergece ad Uiform Covergece Part I: Poitwise Covergece Defiitio of poitwise covergece: A sequece of fuctios f
More informationThe Boolean Ring of Intervals
MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,
More informationSequences I. Chapter Introduction
Chapter 2 Sequeces I 2. Itroductio A sequece is a list of umbers i a defiite order so that we kow which umber is i the first place, which umber is i the secod place ad, for ay atural umber, we kow which
More informationMATH 312 Midterm I(Spring 2015)
MATH 3 Midterm I(Sprig 05) Istructor: Xiaowei Wag Feb 3rd, :30pm-3:50pm, 05 Problem (0 poits). Test for covergece:.. 3.. p, p 0. (coverges for p < ad diverges for p by ratio test.). ( coverges, sice (log
More informationSOME TRIBONACCI IDENTITIES
Mathematics Today Vol.7(Dec-011) 1-9 ISSN 0976-38 Abstract: SOME TRIBONACCI IDENTITIES Shah Devbhadra V. Sir P.T.Sarvajaik College of Sciece, Athwalies, Surat 395001. e-mail : drdvshah@yahoo.com The sequece
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationRademacher Complexity
EECS 598: Statistical Learig Theory, Witer 204 Topic 0 Rademacher Complexity Lecturer: Clayto Scott Scribe: Ya Deg, Kevi Moo Disclaimer: These otes have ot bee subjected to the usual scrutiy reserved for
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationA Note on the Kolmogorov-Feller Weak Law of Large Numbers
Joural of Mathematical Research with Applicatios Mar., 015, Vol. 35, No., pp. 3 8 DOI:10.3770/j.iss:095-651.015.0.013 Http://jmre.dlut.edu.c A Note o the Kolmogorov-Feller Weak Law of Large Numbers Yachu
More informationy X F n (y), To see this, let y Y and apply property (ii) to find a sequence {y n } X such that y n y and lim sup F n (y n ) F (y).
Modica Mortola Fuctioal 2 Γ-Covergece Let X, d) be a metric space ad cosider a sequece {F } of fuctioals F : X [, ]. We say that {F } Γ-coverges to a fuctioal F : X [, ] if the followig properties hold:
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationON MEAN ERGODIC CONVERGENCE IN THE CALKIN ALGEBRAS
PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 00, Number 0, Pages 000 000 S 0002-9939(XX0000-0 ON MEAN ERGODIC CONVERGENCE IN THE CALKIN ALGEBRAS MARCH T. BOEDIHARDJO AND WILLIAM B. JOHNSON 2
More informationMath F215: Induction April 7, 2013
Math F25: Iductio April 7, 203 Iductio is used to prove that a collectio of statemets P(k) depedig o k N are all true. A statemet is simply a mathematical phrase that must be either true or false. Here
More informationOn forward improvement iteration for stopping problems
O forward improvemet iteratio for stoppig problems Mathematical Istitute, Uiversity of Kiel, Ludewig-Mey-Str. 4, D-24098 Kiel, Germay irle@math.ui-iel.de Albrecht Irle Abstract. We cosider the optimal
More informationRandom Models. Tusheng Zhang. February 14, 2013
Radom Models Tusheg Zhag February 14, 013 1 Radom Walks Let me describe the model. Radom walks are used to describe the motio of a movig particle (object). Suppose that a particle (object) moves alog the
More information2.4 Sequences, Sequences of Sets
72 CHAPTER 2. IMPORTANT PROPERTIES OF R 2.4 Sequeces, Sequeces of Sets 2.4.1 Sequeces Defiitio 2.4.1 (sequece Let S R. 1. A sequece i S is a fuctio f : K S where K = { N : 0 for some 0 N}. 2. For each
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationTopics. Homework Problems. MATH 301 Introduction to Analysis Chapter Four Sequences. 1. Definition of convergence of sequences.
MATH 301 Itroductio to Aalysis Chapter Four Sequeces Topics 1. Defiitio of covergece of sequeces. 2. Fidig ad provig the limit of sequeces. 3. Bouded covergece theorem: Theorem 4.1.8. 4. Theorems 4.1.13
More informationCommutativity in Permutation Groups
Commutativity i Permutatio Groups Richard Wito, PhD Abstract I the group Sym(S) of permutatios o a oempty set S, fixed poits ad trasiet poits are defied Prelimiary results o fixed ad trasiet poits are
More informationThe Wasserstein distances
The Wasserstei distaces March 20, 2011 This documet presets the proof of the mai results we proved o Wasserstei distaces themselves (ad ot o curves i the Wasserstei space). I particular, triagle iequality
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More informationAdditional Notes on Power Series
Additioal Notes o Power Series Mauela Girotti MATH 37-0 Advaced Calculus of oe variable Cotets Quick recall 2 Abel s Theorem 2 3 Differetiatio ad Itegratio of Power series 4 Quick recall We recall here
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam will cover.-.9. This sheet has three sectios. The first sectio will remid you about techiques ad formulas that you should kow. The secod gives a umber of practice questios for you
More informationA central limit theorem for moving average process with negatively associated innovation
Iteratioal Mathematical Forum, 1, 2006, o. 10, 495-502 A cetral limit theorem for movig average process with egatively associated iovatio MI-HWA KO Statistical Research Ceter for Complex Systems Seoul
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More informationSequences. Notation. Convergence of a Sequence
Sequeces A sequece is essetially just a list. Defiitio (Sequece of Real Numbers). A sequece of real umbers is a fuctio Z (, ) R for some real umber. Do t let the descriptio of the domai cofuse you; it
More informationEntropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP
Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationHomework 4. x n x X = f(x n x) +
Homework 4 1. Let X ad Y be ormed spaces, T B(X, Y ) ad {x } a sequece i X. If x x weakly, show that T x T x weakly. Solutio: We eed to show that g(t x) g(t x) g Y. It suffices to do this whe g Y = 1.
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationA Proof of Birkhoff s Ergodic Theorem
A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationMA131 - Analysis 1. Workbook 3 Sequences II
MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................
More informationFUNDAMENTALS OF REAL ANALYSIS by. V.1. Product measures
FUNDAMENTALS OF REAL ANALSIS by Doğa Çömez V. PRODUCT MEASURE SPACES V.1. Product measures Let (, A, µ) ad (, B, ν) be two measure spaces. I this sectio we will costruct a product measure µ ν o that coicides
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationMath 25 Solutions to practice problems
Math 5: Advaced Calculus UC Davis, Sprig 0 Math 5 Solutios to practice problems Questio For = 0,,, 3,... ad 0 k defie umbers C k C k =! k!( k)! (for k = 0 ad k = we defie C 0 = C = ). by = ( )... ( k +
More information1.3 Convergence Theorems of Fourier Series. k k k k. N N k 1. With this in mind, we state (without proof) the convergence of Fourier series.
.3 Covergece Theorems of Fourier Series I this sectio, we preset the covergece of Fourier series. A ifiite sum is, by defiitio, a limit of partial sums, that is, a cos( kx) b si( kx) lim a cos( kx) b si(
More informationRates of Convergence by Moduli of Continuity
Rates of Covergece by Moduli of Cotiuity Joh Duchi: Notes for Statistics 300b March, 017 1 Itroductio I this ote, we give a presetatio showig the importace, ad relatioship betwee, the modulis of cotiuity
More informationInformation Theory and Statistics Lecture 4: Lempel-Ziv code
Iformatio Theory ad Statistics Lecture 4: Lempel-Ziv code Łukasz Dębowski ldebowsk@ipipa.waw.pl Ph. D. Programme 203/204 Etropy rate is the limitig compressio rate Theorem For a statioary process (X i)
More informationSEMIGROUPS. D. Pfeifer. Communicated by Jerome A. Goldstein Dedicated to E.S. Lyapin on his 70th Birthday
A NOTE ON ~ROBABILISTIC REPRESENTATIONS OF OPERATOR SEMIGROUPS D. Pfeifer Commuicated by Jerome A. Goldstei Dedicated to E.S. Lyapi o his 70th Birthday I the theory of strogly cotiuous semigroups of bouded
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More information2 Banach spaces and Hilbert spaces
2 Baach spaces ad Hilbert spaces Tryig to do aalysis i the ratioal umbers is difficult for example cosider the set {x Q : x 2 2}. This set is o-empty ad bouded above but does ot have a least upper boud
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationRandomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)
Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationHoggatt and King [lo] defined a complete sequence of natural numbers
REPRESENTATIONS OF N AS A SUM OF DISTINCT ELEMENTS FROM SPECIAL SEQUENCES DAVID A. KLARNER, Uiversity of Alberta, Edmoto, Caada 1. INTRODUCTION Let a, I deote a sequece of atural umbers which satisfies
More information