Advanced Stochastic Processes.

Size: px
Start display at page:

Download "Advanced Stochastic Processes."

Transcription

1 Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios. Extesio Theorem. Borel-Catelli Lemma ad SLLN 1.1. Radom variables ad measurable fuctios Defiitio 1.1. Give two pairs (Ω 1, F 1 ), (Ω 2, F 2 ) of a sample space ad a σ-field, a fuctio X : Ω 1 Ω 2 is defied to be measurable if for every A F 2 we must have X 1 (A) F 1. Whe Ω 2 is the set of all reals R ad F 2 is the Borel σ-field, the fuctio X is called a radom variable. d This defiitio aturally exteds to the case whe Ω 2 = R. I this case we call X a radom vector. Also sice the set of itegers is a subset of R, the defiitio of a radom variable icludes the case of iteger valued radom variables. Exercise 1. Suppose a fuctio X : Ω R is such that X 1 (, x) F for every real value x. Prove that X is a measurable fuctio. Note, that we do ot have to have a probability measure P o Ω 1 or Ω 2 i order to defie measurable fuctios. But probability measure is eeded whe we discuss probability distributios below. Examples. (a) It is easy to give a example of a fuctio which is ot measurable. Suppose, for example Ω 1 = Ω 2 ad both cosist of exactly 3 elemets ω 1, ω 2, ω 3. Say F 1 is a trivial σ-field (which cosists of oly ad Ω) ad F 2 is a full σ-field cosistig of all 8 subsets of Ω. The the idetical trasformatio X : ω ω is ot measurable: take ay o-empty 1

2 2 D. GAMARNIK, set A Ω, A = Ω. We have A is measurable with respect to F 2, but X 1 (A) = A is ot measurable with respect to F 1. (b) (Figure.) Say Ω = [0, 1] 2 ad X : Ω R is defied by X(ω) = ω 1 + ω 2. We claim that X is a radom variable whe Ω is equipped with Borel σ-field. Here is the proof. For every real value x cosider the set A = {ω = (ω 1, ω 2 ) : ω 1 + ω 2 > x}. We will prove that A is measurable (belogs to the Borel σ-field of [0, 1] 2 ). The we will take a complemet of A ad this will prove that X is radom variable. Cosider the coutable set of pairs of ratioals (r 1, r 2 ) such that r 1 + r 2 > x. For each of them fid = (r 1, r 2 ) the smallest iteger which is large eough so that the recagle 1 1 B(r 1, r 2 ; 1/) = {(ω 1, ω 2 ) [0, 1] 2 : ω 1 r 1, ω 2 r 2 } lies etirely i A (this is possible by strict iequality r 1 + r 2 > x). Observe that every pair (ω 1, ω 2 ) satisfyig ω 1 + ω 2 > x lies i oe of these rectagles. Thus A is the uio 1 r 1,r 2 B(r 1, r 2, (r1,r 2 ) of the coutable collectio of such rectagles ad therefore belogs ) to the Borel σ-field of [0, 1] 2. (c) Say Ω = C[0, ) equipped with the Borel σ-field, ad X : Ω R is the fuctio which maps every cotiuous fuctio f(t) ito max 0 t 1 f(t). The X is a radom variable o Ω. Ideed, for every x, we have X 1 (x) is the set of all fuctios f such that max 0 t 1 f(t) x. But this is exactly the set B(0, x, 1) used i Defiitio 1.5 of Lecture 1. The sets of this type geerate the Borel σ-field, ad i particular, belog to it. Thus X is measurable. The cocept of radom variables aturally leads to the cocept of probability distributio Defiitio 1.2. (Figure.) Give a probability space (Ω, F, P) ad a radom variables X : Ω R, the associated probability distributio is defied to be the fuctio F : R [0, 1] give by F (x) = P({ω Ω : X(ω) x}). Whe F (x) is a differetiable fuctio of x, its derivative f(x) = F (x) is called the desity fuctio. I other words F (x) is the probability give to the set of all elemetary outcomes ω which are mapped by X ito value at most x. It is the probability distributios which are usually discussed i elemetary probability classes. There, oe usually defies probability distributio as a fuctio satisfyig certai properties (like it should be o-decreasig ad should coverge to uity as x ). Here these properties ca be derived from the give defiitio of a probability distributio. Propositio 1. Prove that F (x) is o-decreasig, o-egative ad lim x F (x) = 0, lim x F (x) = 1. Proof. HW The cocept of probability distributios allows oe to perform the probability related calculatios without alludig to more abstract otios of probability measures. This is ot possible, however, whe we discuss probability spaces like C[0, ).

3 LECTURE 2. PROBABILITY BASICS CONTINUED 3 Havig defied radom variables ad associated probability distributios, we ca defie further expected values, momets, momet geeratig fuctios, etc., i a more formal way the is doe i elemetary probability classes. We do this oly heuristically, highlightig the mai ideas. Defiitio 1.3. A radom variable X : Ω R is called simple if it takes oly fiitely may values x 1, x 2,..., x m. The expected value of a simple radom variable X is defied to be the quatity E[X] = x i P{ω Ω : X(ω) = x i }. 1 i m What if X is ot simple? How do we defie its expected value? The idea is to approximate X by a sequece of simple radom variables. For simplicity assume that X takes oly values i the iterval [0, A] for some A > 0. That is X : Ω [0, A]. Now cosider X (ω) = k if k X(ω) ( k 1, ]. The X is a simple radom variable. It ca be show that the sequece of the correspodig expected values E[X ] coverges. Its limit is called the expected value E[X] of X. It is also sometimes writte as X(ω)dP(ω). This defiitio of expected value satisfies all the properties of expected values oe studies i elemetary probability courses, for example the fact E[X 2 ] (E[X]) 2, Markov iequality, Chebyshev iequality, Jese s iequality, etc What s i.i.d. sequece of radom variables? Now we ca give a formal defiitio of a stochastic process the priciple otio for this course. Defiitio 1.4. Let T be the set of all o-egative reals R + or itegers Z +. A stochastic process {X t } t T is a family of radom variables X t : Ω R parametrized by T. Remark. Note that a sample outcome ω correspodig to a stochastic process is a fuctio X(ω) : T R, ad the sample space of the correspodig stochastic process is the space of fuctios from T ito R. But ofte we cosider restrictios. For example, whe T = [0, ) we might cosider oly cotiuous fuctios from [0, ) ito R: Example. Set Ω = C[0, ) equipped with Borel σ-field. Defie X t (ω) = ω(t) for every sample ω C[0, ). The {Xt} t [0, ) is a stochastic process. This is true because each fuctio X t : C[0, ) R is a radom variable. (we will prove this later i the course). Remark. The defiitio aturally exteds to the case whe observatios are fuctios T ito d-dimesioal Euclidia space. Oe of the simplest (to aalyze, but ot to defie) examples of a stochastic process is a i.i.d. (idepedet, idetically distributed) stochastic process. What is a i.i.d. stochastic process? I probability courses it was commo to say X 1, X 2,..., is a i.i.d. sequece of Beroulli radom variables with parameter 0 < p < 1, or Z 1, Z 2,... is a i.i.d. sequece of Normal (Gaussia) radom variables with expected value µ ad variace σ 2. What do we mea by this? How does it fit with (Ω, F, P)? We are almost equipped to aswer this questio, but eed little more techicalities. Probability space defiitio icludes defiig a fuctio P : F [0, 1]. How ca we defie this fuctio o the etire σ-field, whe we caot sometimes eve describe the σ-field explicitly? The help comes from Extesio Theorem ET. A rough idea is that if the σ-field is geerated by R d

4 4 D. GAMARNIK, some collectio of sets A ad we ca defie P o A oly, the there is a uique extesio of the fuctio P oto etire σ-field, provided some restrictios are satisfied Extesio Theorem Theorem 1.5 ( Extesio Theorem). Give a sample space Ω ad a collectio A of subsets of Ω such that for every A A its complemet Ω \ A is also i Ω ad for every fiite sequece A 1,..., A m its uio 1 j m A j is also i A. Suppose P : A [0, 1] is such that (a) P(Ω) = 1, (b) P( j=1a j ) j=1 P(A j ), wheever j=1a j A. (c) P( A j ) = j=1 j=1 P(A j ), wheever j=1 A j A ad A i, i = 1, 2,... are mutually exclu- sive. The the fuctio P uiquely exteds to a probability measure P : F(A) [0, 1] defied o the σ-field geerated by A. Remark. Note, that the requiremet from A is to be a collectio of sets with properties very similar to that of a σ-field. The oly differece is that we do ot require every ifiite uio of sets to be i A as well Examples ad applicatios Uiform probability measure. Cosider Ω = [0, 1] ad let A be the set of fiite uios of ope or closed o-itersectig itervals: [a 1, b 1 ) [a 2, b 2 ] (am, b m ). It is easy to check that A satisfies the coditios of the ET. Cosider the fuctio P : A [0, 1] which maps every such set of itervals to the value 1 i m b i a i (that is the total legth of these itervals). It ca be checked that this also satisfies the coditios of the ET (we skip the proof). Thus, by ET, there exists a uique extesio of fuctio P to a probability measure o etire Borel σ-field, sice this σ-field is geerated by itervals. This probability measure is called uiform probability measure o [0, 1]. Other types of cotiuous distributios. What about other distributios like Normal, Expoetial, etc.? The proper defiitio of these probability measures is itroduced similarly. For example the stadard Normal distributio is defied as probability space (R, B, P), where b 1 t 2 2π e 2 B is the Borel σ-field o R ad P assigs to each iterval [a, b] value dt. The each a o-itersectig collectio of itervals [a i, b i ], 1 i m is assiged value which is the sum of the correspodig itegrals. Agai the set of fiite collectios of o-itersectig itervals satisfies the coditios of ET, ad applyig ET we obtaied that the probability measure P is defied o the etire Borel σ-field B i.i.d. sequeces i.i.d coi tosses. Let Ω = {0, 1}. Recall that the product σ-field is the field geerated by cylider type sets A(ω). Let A be the set of fiite uios of such sets 1 j k A(ω j ). Agai, it ca be checked that that A satisfies the coditios of ET. For every fiite sequece ω = ω 1,..., ω m 1 ad the correspodig set A(ω) we set P(A(ω)) simply to be 2 m (the probability of a particular 1 sequece of 0/1 observatios i the first m coi tosses is 2 m ). For example, the probability of 1 first four zeros is 2 4. The, for every uio of o-itersectig sets 1 j k A(ω j ) we set their

5 LECTURE 2. PROBABILITY BASICS CONTINUED 5 k correspodig value to 2 m. The coditios of ET agai ca be checked, but we skip the proof. The, by ET there is a uique extesio of P to the etire product σ-field of Ω. This is what we call a sequece of i.i.d. ubiased coi tosses also kow as a sequece of i.i.d. Beroulli radom variables with parameter 1/2. The phrase i.i.d., i proper probabilistic terms, meas (Ω, F, P) the probability space costructed above. Geeral i.i.d. type distributios. We have defied formally i.i.d. Beroulli sequece. What about geeral i.i.d. sequeces? They are defied similarly by cosiderig ifiite products ad cylider type sets. First we set Ω = R. O it we cosider the product σ-field F. Defie A to be the set of fiite uios of cylider type sets. Recall that a cylider set A is the set of the form A = [a 1, b 1 ] (a [a m, b m ) R 2, b 2 ) product of closed or ope or half-closed halfope itervals. Recall also that cylider sets geerate, by defiitio, the product σ-field F. Suppose we have a probability space (R, B, P) defied o R ad its Borel σ-field B (for example P correspods to stadard Normal distributio). The for every cylider set A we defie P(A) = 1 j m P([a j, b j ]). Agai we check that A ad P satisfy the coditios of ET (we skip the proof). Thus there is a uique extesio of P to the etire product σ-field F of R, sice A geerates this σ-field. The we defie X m (ω) = ω m for every ω R. We ote that X m is a radom variable as it is a measurable fuctio from R ito R. The sequece X 1, X 2,... is a stochastic process which we call a i.i.d. sequece of radom variables. Essetially we have embedded a sequece of radom variables {X m } ito a sigle probability space (R, F, P). Is this defiitio cosistet with elemetary defiitio of i.i.d. Recall that elemetary defiitio of i.i.d. sequece is whe P(X 1 x 1,..., X m x m ) = 1 j m P(X j x j ). Is this true i our case? Note P(X 1 x 1,..., X x m ) = P{ω R : ω 1 (, x 1 ],..., ω m (, x m ]} = P{ω (, x 1 ] (, x m] R } = P((, xj ]), 1 j m where the last equality follows from how we defied P o cylider sets. But the product of these probabilities is exactly 1 j P(X j x j ). Thus the idetity checks Borel-Catelli Lemma ad Strog Law of Large Numbers (SLLN) The Strog Law of Large Numbers (SLLN) (like the Cetral Limit Theorem) is oe of the most fudametal theorems i probability theory. Yet properly statig it, let aloe provig it is ot as straightforward as is, for example the Weak Law of Large Numbers (WLLN). We ow use the (Ω, F, P) framework to properly state ad prove SLLN. We begi with a very useful tool, the Borel-Catelli Lemma. Give a sample space Ω, a σ-field F ad a ifiite sequece A 1, A 2,..., A m,... F defie A i.o. (i.o. stads for ifiitely ofte) to be the set of all ω Ω which belog to ifiitely may A m -s. Oe ca write A i.o. as (check

6 6 D. GAMARNIK, the validity of this idetity) A i.o. = m 1 j m A j. Lemma 1.6 (Borel-Catelli Lemma). Give a probability space (Ω, F, P) ad a ifiite sequece of evets A m, m 1 suppose m P(A m ) <. The P(A i.o. ) = 0. I words we say: the probability that A m happe ifiitely ofte is equal to zero. Proof. Defie B m = j m A j. The A i.o. = m B m. The B 1 B 2 B 3. Usig Propositio 4 part (b) (applied to complemet sets) we obtai P(A i.o. ) = lim m P(B m ). But sice m P(A m) < the the tail parts of the sum satisfy lim m j m P(A j ) = 0. But P(B m ) = P( j m A j ) j m P(A j ). Therefore, moreover lim m B m = 0. We coclude P(A i.o. ) = 0. Theorem 1.7 (SLLN). Cosider a i.i.d sequece of radom variables X, = 1, 2,... correspodig to some probability measure (R, B, P). Suppose E[X 1 ] <. The almost surely Formally, defie The P(A) = 1. 1 i X i lim = E[X 1 ]. 1 i X i (ω) A = {ω R : lim = E[X 1 ]}. Proof. The proof of this fudametal result i probability theory is complicated (see for example [2]). Here, for simplicity, we cosider a special case whe the radom variable X 1 has a fiite fourth momet. Namely, E[ X 4 1 ] = X 1 (ω) 4 dp(ω) <. Let us ceter the radom variables X i i the followig way: Y i = X i E[X i ]. The Y i have zero expected value. Sice 1 i Y i 1 i = X i E [X i ], P Y i it suffices to prove that 1 i coverges almost surely to zero. Fix ɛ > 0 ad defie the evet A (ɛ) as P 1 i Y i > ɛ. Formally Applyig Markov iequality A (ɛ) = {ω R : P( 1 i Y i > ɛ) = P(( 1 i Y i (ω) > ɛ}. 1 i Y i E[( 1 i Y i) 4 ] 4 ɛ 4 ) 4 > ɛ 4 ) Whe we expad E[( 1 i Y i) 4 ] we ote that oly the terms of the form E[Y 4 ] ad E[Y 2 Y 2 ] are i i j o-zero, sice the expected value of Y i is zero ad the sequece is i.i.d. Also by idepedece

7 LECTURE 2. PROBABILITY BASICS CONTINUED 7 E[Y 2 Y j 2 ] = (E[Y 1 2 ]) 2. We obtai a boud i 2 E[Y1 4 ] + ( 1)(E[Y 2 1 ]) 2 [E[Y1 4 ] + (E[Y 2 1 ]) 2 ] E[Y1 4 ] + (E[Y 2 1 ]) 2 = 4 ɛ 4 4 ɛ 4 2 ɛ 4 This expressio is fiite by our assumptio of fiiteess of fourth momet. Sice the sum E[Y 4 1 ]+(E[Y 2 1 ]) 2 1 <, the applyig the Borel-Catelli Lemma we coclude that probability that A (ɛ) happes for ifiitely may is zero. I other words, for almost every ω R 2 ɛ 4 P 1 i Y i (ω) there exists 0 = 0 (ɛ, ω) such that for all > 0 we must have ɛ. This meas that for almost every ω, we have 1 i Y i (ω) lim = 0. This cocludes the proof Readig assigmets Notes Modelig experimets, pages 1.4,1.5,2.2. Grimmett ad Stirzaker [2], Chapters 1 ad 2. Chapter 7, Sectios Durrett [1] Chapter 1, Sectios 1-7.

8 BIBLIOGRAPHY 1. R. Durrett, Probability: theory ad examples, Duxbury Press, secod editio, G. R. Grimmett ad D. R. Stirzaker, Probability ad radom processes, Oxford Sciece Publicatios,

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Convergence of random variables. (telegram style notes) P.J.C. Spreij

Convergence of random variables. (telegram style notes) P.J.C. Spreij Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space

More information

Distribution of Random Samples & Limit theorems

Distribution of Random Samples & Limit theorems STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to

More information

1 Convergence in Probability and the Weak Law of Large Numbers

1 Convergence in Probability and the Weak Law of Large Numbers 36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec

More information

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence

Chapter 3. Strong convergence. 3.1 Definition of almost sure convergence Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties

More information

7.1 Convergence of sequences of random variables

7.1 Convergence of sequences of random variables Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite

More information

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014. Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak

More information

Lecture 19: Convergence

Lecture 19: Convergence Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may

More information

Introduction to Probability. Ariel Yadin. Lecture 7

Introduction to Probability. Ariel Yadin. Lecture 7 Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,

More information

This section is optional.

This section is optional. 4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore

More information

An Introduction to Randomized Algorithms

An Introduction to Randomized Algorithms A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis

More information

It is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.

It is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function. MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied

More information

Axioms of Measure Theory

Axioms of Measure Theory MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.

More information

Introduction to Probability. Ariel Yadin

Introduction to Probability. Ariel Yadin Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways

More information

STAT Homework 1 - Solutions

STAT Homework 1 - Solutions STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet

More information

Notes 27 : Brownian motion: path properties

Notes 27 : Brownian motion: path properties Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X

More information

Sequences and Series of Functions

Sequences and Series of Functions Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges

More information

The Borel hierarchy classifies subsets of the reals by their topological complexity. Another approach is to classify them by size.

The Borel hierarchy classifies subsets of the reals by their topological complexity. Another approach is to classify them by size. Lecture 7: Measure ad Category The Borel hierarchy classifies subsets of the reals by their topological complexity. Aother approach is to classify them by size. Filters ad Ideals The most commo measure

More information

Solution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1

Solution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1 Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity

More information

The Boolean Ring of Intervals

The Boolean Ring of Intervals MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,

More information

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number

Integrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios

More information

Lecture 2. The Lovász Local Lemma

Lecture 2. The Lovász Local Lemma Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio

More information

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?

If a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero? 2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a

More information

LECTURE 8: ASYMPTOTICS I

LECTURE 8: ASYMPTOTICS I LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece

More information

4. Partial Sums and the Central Limit Theorem

4. Partial Sums and the Central Limit Theorem 1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems

More information

ST5215: Advanced Statistical Theory

ST5215: Advanced Statistical Theory ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The

More information

Mathematical Statistics - MS

Mathematical Statistics - MS Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios

More information

Math 61CM - Solutions to homework 3

Math 61CM - Solutions to homework 3 Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig

More information

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002

ECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002 ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom

More information

Math 525: Lecture 5. January 18, 2018

Math 525: Lecture 5. January 18, 2018 Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the

More information

Math 341 Lecture #31 6.5: Power Series

Math 341 Lecture #31 6.5: Power Series Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series

More information

Lecture 12: November 13, 2018

Lecture 12: November 13, 2018 Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,

More information

Lecture Notes for Analysis Class

Lecture Notes for Analysis Class Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios

More information

Lecture 3 : Random variables and their distributions

Lecture 3 : Random variables and their distributions Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}

More information

AMS570 Lecture Notes #2

AMS570 Lecture Notes #2 AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)

More information

Measure and Measurable Functions

Measure and Measurable Functions 3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies

More information

Chapter 0. Review of set theory. 0.1 Sets

Chapter 0. Review of set theory. 0.1 Sets Chapter 0 Review of set theory Set theory plays a cetral role i the theory of probability. Thus, we will ope this course with a quick review of those otios of set theory which will be used repeatedly.

More information

Probability for mathematicians INDEPENDENCE TAU

Probability for mathematicians INDEPENDENCE TAU Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet

More information

Learning Theory: Lecture Notes

Learning Theory: Lecture Notes Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic

More information

As stated by Laplace, Probability is common sense reduced to calculation.

As stated by Laplace, Probability is common sense reduced to calculation. Note: Hadouts DO NOT replace the book. I most cases, they oly provide a guidelie o topics ad a ituitive feel. The math details will be covered i class, so it is importat to atted class ad also you MUST

More information

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.

This exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam. Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the

More information

Probability and Random Processes

Probability and Random Processes Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems

More information

Lecture 8: Convergence of transformations and law of large numbers

Lecture 8: Convergence of transformations and law of large numbers Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges

More information

Probability Theory. Muhammad Waliji. August 11, 2006

Probability Theory. Muhammad Waliji. August 11, 2006 Probability Theory Muhammad Waliji August 11, 2006 Abstract This paper itroduces some elemetary otios i Measure-Theoretic Probability Theory. Several probabalistic otios of the covergece of a sequece of

More information

B Supplemental Notes 2 Hypergeometric, Binomial, Poisson and Multinomial Random Variables and Borel Sets

B Supplemental Notes 2 Hypergeometric, Binomial, Poisson and Multinomial Random Variables and Borel Sets B671-672 Supplemetal otes 2 Hypergeometric, Biomial, Poisso ad Multiomial Radom Variables ad Borel Sets 1 Biomial Approximatio to the Hypergeometric Recall that the Hypergeometric istributio is fx = x

More information

Mathematics 170B Selected HW Solutions.

Mathematics 170B Selected HW Solutions. Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli

More information

lim za n n = z lim a n n.

lim za n n = z lim a n n. Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget

More information

Lecture 3: August 31

Lecture 3: August 31 36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,

More information

Introduction to Probability. Ariel Yadin. Lecture 2

Introduction to Probability. Ariel Yadin. Lecture 2 Itroductio to Probability Ariel Yadi Lecture 2 1. Discrete Probability Spaces Discrete probability spaces are those for which the sample space is coutable. We have already see that i this case we ca take

More information

Probability and Statistics

Probability and Statistics ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,

More information

Basics of Probability Theory (for Theory of Computation courses)

Basics of Probability Theory (for Theory of Computation courses) Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.

More information

Infinite Sequences and Series

Infinite Sequences and Series Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet

More information

The Central Limit Theorem

The Central Limit Theorem Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx

More information

Recitation 4: Lagrange Multipliers and Integration

Recitation 4: Lagrange Multipliers and Integration Math 1c TA: Padraic Bartlett Recitatio 4: Lagrage Multipliers ad Itegratio Week 4 Caltech 211 1 Radom Questio Hey! So, this radom questio is pretty tightly tied to today s lecture ad the cocept of cotet

More information

Lecture 7: Properties of Random Samples

Lecture 7: Properties of Random Samples Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ

More information

Random Variables, Sampling and Estimation

Random Variables, Sampling and Estimation Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig

More information

Chapter 6 Principles of Data Reduction

Chapter 6 Principles of Data Reduction Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a

More information

Lecture 3 The Lebesgue Integral

Lecture 3 The Lebesgue Integral Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified

More information

The Random Walk For Dummies

The Random Walk For Dummies The Radom Walk For Dummies Richard A Mote Abstract We look at the priciples goverig the oe-dimesioal discrete radom walk First we review five basic cocepts of probability theory The we cosider the Beroulli

More information

EE 4TM4: Digital Communications II Probability Theory

EE 4TM4: Digital Communications II Probability Theory 1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair

More information

Lecture 2: April 3, 2013

Lecture 2: April 3, 2013 TTIC/CMSC 350 Mathematical Toolkit Sprig 203 Madhur Tulsiai Lecture 2: April 3, 203 Scribe: Shubhedu Trivedi Coi tosses cotiued We retur to the coi tossig example from the last lecture agai: Example. Give,

More information

Here are some examples of algebras: F α = A(G). Also, if A, B A(G) then A, B F α. F α = A(G). In other words, A(G)

Here are some examples of algebras: F α = A(G). Also, if A, B A(G) then A, B F α. F α = A(G). In other words, A(G) MATH 529 Probability Axioms Here we shall use the geeral axioms of a probability measure to derive several importat results ivolvig probabilities of uios ad itersectios. Some more advaced results will

More information

Singular Continuous Measures by Michael Pejic 5/14/10

Singular Continuous Measures by Michael Pejic 5/14/10 Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable

More information

5 Birkhoff s Ergodic Theorem

5 Birkhoff s Ergodic Theorem 5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the

More information

Notes 5 : More on the a.s. convergence of sums

Notes 5 : More on the a.s. convergence of sums Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series

More information

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15

17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15 17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig

More information

Discrete Mathematics for CS Spring 2008 David Wagner Note 22

Discrete Mathematics for CS Spring 2008 David Wagner Note 22 CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig

More information

It is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.

It is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial. Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable

More information

Lecture 12: September 27

Lecture 12: September 27 36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.

More information

(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3

(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3 MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special

More information

Problems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:

Problems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman: Math 224 Fall 2017 Homework 4 Drew Armstrog Problems from 9th editio of Probability ad Statistical Iferece by Hogg, Tais ad Zimmerma: Sectio 2.3, Exercises 16(a,d),18. Sectio 2.4, Exercises 13, 14. Sectio

More information

Law of the sum of Bernoulli random variables

Law of the sum of Bernoulli random variables Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible

More information

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables

Lecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze

More information

Sieve Estimators: Consistency and Rates of Convergence

Sieve Estimators: Consistency and Rates of Convergence EECS 598: Statistical Learig Theory, Witer 2014 Topic 6 Sieve Estimators: Cosistecy ad Rates of Covergece Lecturer: Clayto Scott Scribe: Julia Katz-Samuels, Brado Oselio, Pi-Yu Che Disclaimer: These otes

More information

Discrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22

Discrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22 CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first

More information

Introductory Ergodic Theory and the Birkhoff Ergodic Theorem

Introductory Ergodic Theory and the Birkhoff Ergodic Theorem Itroductory Ergodic Theory ad the Birkhoff Ergodic Theorem James Pikerto Jauary 14, 2014 I this expositio we ll cover a itroductio to ergodic theory. Specifically, the Birkhoff Mea Theorem. Ergodic theory

More information

Math 155 (Lecture 3)

Math 155 (Lecture 3) Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,

More information

Solutions to HW Assignment 1

Solutions to HW Assignment 1 Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.

More information

6.3 Testing Series With Positive Terms

6.3 Testing Series With Positive Terms 6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial

More information

6. Sufficient, Complete, and Ancillary Statistics

6. Sufficient, Complete, and Ancillary Statistics Sufficiet, Complete ad Acillary Statistics http://www.math.uah.edu/stat/poit/sufficiet.xhtml 1 of 7 7/16/2009 6:13 AM Virtual Laboratories > 7. Poit Estimatio > 1 2 3 4 5 6 6. Sufficiet, Complete, ad Acillary

More information

Notes 19 : Martingale CLT

Notes 19 : Martingale CLT Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall

More information

Chapter IV Integration Theory

Chapter IV Integration Theory Chapter IV Itegratio Theory Lectures 32-33 1. Costructio of the itegral I this sectio we costruct the abstract itegral. As a matter of termiology, we defie a measure space as beig a triple (, A, µ), where

More information

Approximations and more PMFs and PDFs

Approximations and more PMFs and PDFs Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.

More information

Math 113, Calculus II Winter 2007 Final Exam Solutions

Math 113, Calculus II Winter 2007 Final Exam Solutions Math, Calculus II Witer 7 Fial Exam Solutios (5 poits) Use the limit defiitio of the defiite itegral ad the sum formulas to compute x x + dx The check your aswer usig the Evaluatio Theorem Solutio: I this

More information

MATH 413 FINAL EXAM. f(x) f(y) M x y. x + 1 n

MATH 413 FINAL EXAM. f(x) f(y) M x y. x + 1 n MATH 43 FINAL EXAM Math 43 fial exam, 3 May 28. The exam starts at 9: am ad you have 5 miutes. No textbooks or calculators may be used durig the exam. This exam is prited o both sides of the paper. Good

More information

6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.

6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer. 6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio

More information

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS

January 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we

More information

Chapter 6 Infinite Series

Chapter 6 Infinite Series Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat

More information

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n. Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator

More information

Seunghee Ye Ma 8: Week 5 Oct 28

Seunghee Ye Ma 8: Week 5 Oct 28 Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value

More information

Lecture 14: Graph Entropy

Lecture 14: Graph Entropy 15-859: Iformatio Theory ad Applicatios i TCS Sprig 2013 Lecture 14: Graph Etropy March 19, 2013 Lecturer: Mahdi Cheraghchi Scribe: Euiwoog Lee 1 Recap Bergma s boud o the permaet Shearer s Lemma Number

More information

Exercises 1 Sets and functions

Exercises 1 Sets and functions Exercises 1 Sets ad fuctios HU Wei September 6, 018 1 Basics Set theory ca be made much more rigorous ad built upo a set of Axioms. But we will cover oly some heuristic ideas. For those iterested studets,

More information

1 The Haar functions and the Brownian motion

1 The Haar functions and the Brownian motion 1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,

More information

Sequences. Notation. Convergence of a Sequence

Sequences. Notation. Convergence of a Sequence Sequeces A sequece is essetially just a list. Defiitio (Sequece of Real Numbers). A sequece of real umbers is a fuctio Z (, ) R for some real umber. Do t let the descriptio of the domai cofuse you; it

More information

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound

Lecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee

More information