Probability Theory. Muhammad Waliji. August 11, 2006
|
|
- Delphia Haynes
- 6 years ago
- Views:
Transcription
1 Probability Theory Muhammad Waliji August 11, 2006 Abstract This paper itroduces some elemetary otios i Measure-Theoretic Probability Theory. Several probabalistic otios of the covergece of a sequece of radom variables are discussed. The theory is the used to prove the Law of Large Numbers. Fially, the otios of coditioal expectatio ad coditioal probability are itroduced. 1 Heuristic Itroductio Probability theory is cocered with the outcome of experimets that are radom i ature, that is, experimets whose outcomes caot be predicted i advace. The set of possible outcomes, ω, of a experimet is called the sample space, deoted by Ω. For istace, if our experimet cosists of rollig a dice, we will have Ω = {1, 2, 3, 4, 5, 6}. A subset, A, of Ω is called a evet. For istace A = {1, 3, 5} correspods to the evet a odd umber is rolled. I elemetary probability theory, oe is ormally cocered with sample spaces that are either fiite or coutable. I this case, oe ofte assigs a probability to every sigle outcome. That is, we have probability fuctio P : Ω [0, 1], where P (ω) is the probability that ω occurs. Here, we issist that P (ω) = 1. ω Ω However, if the sample space is ucoutable, the this coditio becomes osesible. Two elemetary types of problems come ito this category ad hece caot be dealt with by elemetary probability theory: a ifiite umber of repeated coi tosses (or dice rolls), ad a umber draw at radom from [0, 1]. This illustrates the importace of ucoutable sample spaces. The solutio to this problem is to use the theory of measures. Istead of assigig probabilities to outcomes i the sample space, oe ca restrict himself to a certai class of evets that form a structure kow as a σ-field, ad assig probabilities to these special kids of evets. 1
2 2 σ-fields, Probability Measures, ad Distributio Fuctios Defiitio 2.1. A class of subsets of Ω, F, is a σ-field if the followig hold: (i) F ad Ω F (ii) A F = A c F (iii) A 1, A 2,... F = A F Note that this implies that σ-fields are also closed uder coutable iteresectios also. Defiitio 2.2. The σ-field geerated by a class of sets, A, is the smallest σ-field cotaiig A. It is deoted σ(a). Defiitio 2.3. Let F be a σ-field. A fuctio P : F [0, 1] is a probability measure if P ( ) = 0, P (Ω) = 1, ad wheever (A ) N is a disjoit collectio of sets i F, we have ( ) P A = P (A ). =1 Throughout this paper, uless otherwise oted, the words icreasig, decreasig, ad mootoe are always meat i their weak sese. Suppose {A } is a sequece of sets. We say that {A } is a icreasig sequece if A 1 A 2. We say that {A } is a decreasig sequece if A 1 A 2. I both of these cases, the sequece {A } is said to be mootoe. If A is icreasig, the set lim A := A. If A is decreasig, the set lim A := A. The followig properties follow immediately from the defiitios. Lemma 2.4. Let F be a σ-field, ad let P be a probability measure o it. (i) P (A c ) = 1 P (A) (ii) If A B, the P (A) P (B). (iii) P ( i=1 A i) i=1 P (A i). (iv) If {A } is a mootoe sequece i F, the lim P (A ) = P (lim A ). Defiitio 2.5. Suppose Ω is a set, F is a field o Ω, ad P is a probability measure o F. The, the ordered pair (Ω, F) is called a measurable space. The triple (Ω, F, P ) is called a probability space. The a probability space is fiitely additive or coutably additive depedig o whether P is fiitely or coutably additive. Defiitio 2.6. Let (X, τ) be a topological space. The σ-field, B(X, τ) geerated by τ is called the Borel σ-field. I particular, B(X, τ) is the smallest σ-field cotaiig all ope ad closed sets of X. The sets of B(X, τ) are called Borel sets. =1 2
3 Whe the topology, τ, or eve the space X are obvious from the cotext, B(X, τ) will ofte be abbreviated B(X) or eve just B. A particularly importat situatio i probability theory is whe Ω = ad F are the Borel sets i. Defiitio 2.7. A distributio fuctio is a icreasig right-cotiuous fuctio F : [0, 1] such that lim F (x) = 0 ad lim F (x) = 1. x x We ca associate probability fuctios o (, B) with distributio fuctios. Namely, the distributio fuctio associated with P is F (x) := P ((, x]). Coversely, each distributio fuctio defies a probability fuctio o the reals. 3 adom Variables, Trasformatios, ad Expectatio We ow have stated the basic objects that we will be studyig ad discussed their elemetary properties. We ow itroduce the cocept of a adom Variable. Let Ω be the set of all possible drawigs of lottery umbers. The fuctio X :Ω which idicates the payoff X(ω) to a player associated with a drawig ω is a example of a radom variable. The expectatio of a radom variable is the average or expected value of X. Defiitio 3.1. Let (Ω 1, F 1 ) ad (Ω 2, F 2 ) be measurable spaces. A fuctio T : Ω 1 Ω 2 is a measurable trasformatio if the preimage of ay measurable set is a measurable set. That is, T is a measurable trasformatio if ( A F 2 )(T 1 (A) F 1 ). Lemma 3.2. It is sufficiet to check the coditio i Defiitio 3.1 for those A i a class that geerates F 2. More precisely, suppose that A geerates F 2. The, if ( A A)(T 1 (A) F 1 ), the T is a measurable trasformatio. Proof. Let C := {A 2 Ω2 : T 1 (A) F 1 }. The, C is a σ-field, ad A C. But the, σ(a) = F 2 C, which is exactly what we wated. Defiitio 3.3. Let (Ω, F) be a measurable space. A measurable fuctio or a radom variable is a measurable trasformatio from (Ω, F) ito (, B). Lemma 3.4. If f : is a cotiuous fuctio, the f is a measurable trasformatio from (, B) to (, B). Defiitio 3.5. Give a set A, the idicator fuctio for A is the fuctio { 1 if ω A I A (ω) := 0 if ω / A 3
4 If A F, the I A is a measurable fuctio. Note that may elemetary operatios, icludig compositio, arithmetic, max, mi, ad others, whe performed upo measurable fuctios, agai yield measurable fuctios. Let (Ω 1, F 1, P ) be a probability space ad (Ω 2, F 2 ) a measurable space. A measurable trasformatio T :Ω 1 Ω 2 aturally iduces a probability measure P T 1 o (Ω 2, F 2 ). I the case of a radom variable, X, the iduced measure o will geerally be deoted α. The distributio fuctio associated with α will be deoted F X. α will sometimes be called a probability distributio. Now that we have a otio of measure ad of measurable fuctios, we ca develop a otio of the itegral of a fuctio. The itegral will have the probabalistic iterpretatio of beig a expected (or average) value. For the precise defiitio of the Lebesgue itegral, see ay textbook o Measure Theory. Defiitio 3.6. Suppose X is a radom variable. The the expectatio of X is EX := Ω X(ω)dP. We coclude this sectio with a useful chage of variables formula for itegrals. Propositio 3.7. Let (Ω 1, F 1, P ) be a probability space ad let (Ω 2, F 2 ) be a measurable space. Suppose T :Ω 1 Ω 2 is a measurable trasformatio. Suppose f : Ω 2 is a measurable fuctio. The, P T 1 is a probability measure o (Ω 2, F 2 ) ad ft :Ω 1 is a measurable fuctio. Furthermore, f is itegrable iff ft is itegrable, ad ft (ω 1 )dp = f(ω 2 )dp T 1. Ω 1 Ω 2 4 Notios of Covergece We will ow itroduce some otios of the covergece of radom variables. Note that we will ofte ot explicitly state the depedece of a fuctio X(ω) o ω. Hece, sets of the form {ω : X(ω) > 0} will ofte be abbreviated {X > 0}. For the remaider of this sectio, let X be a sequece of radom variables. Defiitio 4.1. The sequece X coverges almost surely (almost everywhere) to a radom variable X if X (ω) X(ω) for all ω outside of a set of probability 0. Defiitio 4.2. The sequece X coverges i probability (i measure) to a fuctio X if, for every ɛ > 0, This is deoted X P X. lim P {ω : X (ω) X(ω) ɛ} = 0. 4
5 Propositio 4.3. If X coverges almost surely to X, the X coverges i probability to X. Proof. We have {ω : X (ω) X(ω)} N, P (N) = 0. That is, ɛ > 0 Therefore, give ɛ > 0, we have =1 m= lim P { X X ɛ} lim P thereby completig the proof. = P { X m X ɛ} N. m= =1 m= { X m X ɛ} { X m X ɛ} P (N) = 0 Note, however, that the coverse is ot true. Let Ω = [0, 1] with Lebesgue measure. Cosider the sequece of sets A 1 = [0, 1 2 ], A 2 = [ 1 2, 1], A 3 = [0, 1 3 ], A 4 = [ 1 3, 2 3 ], ad so o. The, the idicator fuctios, I A, coverge i probability to 0. However, I A (ω) does ot coverge for ay ω, ad i particular the sequece does ot coverge almost surely. However, the followig holds as a sort of coverse: Propositio 4.4. Suppose f coverges i probability to f. The, there is a subsequece f k of f such that f k coverges almost surely to f. Proof. Let B ɛ := {ω : f (ω) f(ω) ɛ}. The, f i f almost surely iff P ( B ɛ j ) = 0. i j>i We kow that for ay ɛ, Now, otice that P ( m lim P (Bɛ ) = 0. Bm) ɛ if P ( Bm) ɛ if m Furthermore, ɛ 1 < ɛ 2 B ɛ1 B ɛ2 P (Bm) ɛ = lim m= P (B ɛ1 ) P (B ɛ2 ). m= P (Bm). ɛ Let δ i := 1/2 i. Now, ote that ( i)( ɛ i )( ɛ i )(P (Bɛ ) < δ i ). Let i := δi i. Choose ɛ 0. Note, ( m)(δ m < ɛ). Hece, P ( i j i B ɛ j ) lim i which is what we wated. j=i P (B ɛ j ) lim i j=i P (B δj j ) = lim i δ j = 0 j=i 5
6 Defiitio 4.5. A sequece of probability measures {α } o coverges weakly to α if wheever α(a) = α(b) = 0, for a < b, we have lim α [a, b] = α[a, b]. A sequece of radom variable {X } coverges weakly to X if the iduced probability measures {α } coverge weakly to α. This is deoted α α or X X. Lemma 4.6. Suppose α ad α are probability measures o with associated distributio fuctios F ad F. The, α α iff F (x) F (x) for each cotiuity poit x of F. Proof. First, ote that x is a cotiuity poit of F iff α(x) = 0. Let a < b be cotiuity poits of F. Suppose F (x) F (x) for each cotiuity poit x of F. The, lim α [a, b] = lim F (b) F (a) = F (b) F (a) = α[a, b]. For the coverse, suppose α α. The, lim F (b) F (a) = lim α [a, b] = α[a, b]. Now, we ca let a i such a way that a is always a cotiuity poit of F. The, we get, lim F (b) = α(, b]. The ext result shows that weak covergece is actually weak : Propositio 4.7. Suppose X coverges i probability to X. The, X coverges weakly to X. Proof. Let F, F be the distributio fuctios of X, X respectively. suppose x is a cotiuity poit of F. Note that ad {X x ɛ} { X X ɛ} {X x} {X x} = {X x ad X x + ɛ} {X x ad X > x + ɛ} Therefore, {X x + ɛ} { X X ɛ} P {X x ɛ} P { X X ɛ} P {X x} P {X x + ɛ} + P { X X ɛ} Sice for each ɛ > 0, lim P { X X ɛ} = 0, whe we let, we have F (x ɛ) lim if F (x) lim sup F (x) F (x + ɛ). 6
7 Fially, sice F is cotiuous at x, lettig ɛ 0, we have so that X X. lim F (x) = F (x) The coverse is ot true i geeral. However, if X is a degeerate distributio (takes a sigle value with probability oe), the the coverse is true. Propositio 4.8. Suppose X X, ad X is a degeerate distributio such that P {X = a} = 1. The, X P X. Proof. Let α ad α be the distributios o iduced by X ad X respectively. Give ɛ > 0, we have Hece, ad so lim α [a ɛ, a + ɛ] = α[a ɛ, a + ɛ] = 1. lim P { X X ɛ} = 1, lim P { X X > ɛ} = 0 5 Product Measures ad Idepedece Suppose (Ω 1, F 1 ) ad (Ω 2, F 2 ) are two measurable spaces. We wat to costruct a product measurable space with sample space Ω 1 Ω 2. Defiitio 5.1. Let A = {A B : A F 1, B F 2 }. Let F 1 F 2 be the σ-field geerated by A. F 1 F 2 is called the product σ-field of F 1 ad F 2. If P 1 ad P 2 are probability measures o the measurable spaces above, the P 1 P 2 (A B) := P 1 (A)P 2 (B) gives a probability measure o A. This ca be exteded i a caoical way to the σ-field F 1 F 2. Defiitio 5.2. P 1 P 2 is called the product probability measure of P 1 ad P 2. Let Ω := Ω 1 Ω 2, F := F 1 F 2, ad P := P 1 P 2. Note that whe calculatig itegrals with respect to a product probability measure, we ca ormally perform a iterated itegral i ay order with respect to the compoet probability measures. This result is kow as Fubii s Theorem. Before we defie a otio of idepedece, we will give some heuristic cosideratios. Two evets A ad B should be idepedet if A occurrig has othig to do with B occurrig. If we deote by P A (X), the probability that X occurs give that A has occurred, the we see that P A (X) = P (A X) P (A). Now, suppose that A ad B are ideed idepedet. This meas that P A (B) = P (B). But the, P (B) = P (A B) P (A), so that P (A B) = P (A)P (B). This leads us to defie, 7
8 Defiitio 5.3. Let (Ω, F, P ) be a probability space. Let A i F for every i. Let X i be a radom variable for every i. (i) A 1,..., A are idepedet if P (A 1 A ) = P (A 1 ) P (A ). (ii) A collectio of evets {A i } i I is idepedet if every fiite subcollectio is idepedet. (iii) X 1,..., X are idepedet if for ay sets A 1,..., A B(), the evets {X i A i } i=1 are idepedet. (iv) A collectio of radom variables {X i } i I subcollectio is idepedet. is idepedet if every fiite Lemma 5.4. Suppose X, Y are radom variables o (Ω, F, P ), with iduced distributios α, β o respectively. The, X ad Y are idepedet if ad oly if the distributio iduced o 2 by (X, Y ) is α β. Lemma 5.5. Suppose X, Y are idepedet radom variables, ad suppose that f, g are measurable fuctios. The, f(x) ad g(y ) are also idepedet radom variables. Propositio 5.6. Let X, Y be idepedet radom variables, ad let f, g be measurable fuctios. Suppose that E f(x) ad E g(y ) are both fiite. The, E[f(X)g(Y )] = E[f(X)]E[g(Y )]. Proof. Let α be the distributio o iduced by f(x), ad let β be the distributio iduced by g(y ). The, the distributio o 2 iduced by (f(x), g(y )) is α β. So, E[f(X)g(Y )] = f(x(ω))g(y (ω)) dp = uv dα dβ Ω = 6 Characteristic Fuctios u dα v dβ = E[f(X)]E[g(Y )] The iverse Fourier trasform of a probability distributio plays a cetral role i probability theory. Defiitio 6.1. Let α be a probability measure o. The, the characteristic fuctio of α is φ α (t) = e ıtx dα If X is a radom variable, the characteristic fuctio of the distributio o iduced by X will sometimes be deoted φ X. These results demostrate the importace of the characteristic fuctio i probability. 8
9 Propositio 6.2. Suppose α ad β are probability measures o with characteristic fuctios φ ad ψ respectively. Suppose further that for each t, φ(t) = ψ(t). The, α = β. Theorem 6.3. Let α, α be probability measures o with distributio fuctios F ad F ad characteristic fuctios φ ad φ. The, the followig are equivalet (i) α α. (ii) for ay bouded cotiuous fuctio f :, f(x)dα = f(x)dα. (iii) for every t, lim lim φ (t) = φ(t). Theorem 6.4. Suppose α is a sequece of probability measures o, with characteristic fuctios φ. Suppose that for each t, lim φ (t) =: φ(t) exists ad φ is cotiuous at 0. The, there is a probability distriubtio α such that φ is the characteristic fuctio of α. Furthermore, α α. Next, we show how to recover the momets of a radom variable from its characteristic fuctio. Defiitio 6.5. Suppose X is a radom variable. The, the kth momet of X is EX k. The kth absolute momet of X is E X k. Propositio 6.6. Let X be a radom variable. Suppose that the kth momet of X exists. The, the characteristic fuctio φ of X is k times cotiuously differetiable, ad φ (k) (0) = ı k EX k. Now, a result o affie trasforms of a radom variable: Propositio 6.7. Suppose X is a radom variable, ad Y = ax + b. Let φ X ad φ Y be the characteristic fuctios of X ad Y. The, φ Y (t) = e ıtb φ X (at). We will ofte be iterested i the sums of idepedet radom variables. Suppose that X ad Y are idepedet radom variables with iduced distributios α ad β o respectively. The, the iduced distributio of (X, Y ) o 2 is α β. Cosider the map f : 2 give by f(x, y) = x + y. The, the distributio o iduced by α β is deoted α β, ad is called the covolutio of α ad β. α β is the distributio of the sum of X ad Y. Propositio 6.8. Suppose X ad Y are idepedet radom variables with distributios α ad β respectively. The, φ X+Y (t) = φ X (t)φ Y (t). 9
10 Proof. φ α β (t) = e ıtz dα β = e ıt(x+y) dα dβ = e ıtx e ıty dα dβ = e ıtx dα e ıty dβ = φ α (t)φ β (t) 7 Useful Bouds ad Iequalities Here, we will prove some useful bouds regardig radom variables ad their momets. Defiitio 7.1. Let X be a radom variable. Var(X) := E[(X EX) 2 ] = EX 2 (EX) 2. The, the variace of X is The variace is a measure of how far spread X is o average from its mea. It exists if X has a fiite secod momet. It is ofte deoted σ 2. Lemma 7.2. Suppose X, Y are idepedet radom variables. The, Var(X + Y ) = Var(X) + Var(Y ) Propositio 7.3 (Markov s Iequality). Let ɛ > 0. Suppose X is a radom variable with fiite kth absolute momet. The, P { X ɛ} 1 ɛ k E X k. Proof. P { X ɛ} = { X ɛ} dp 1 ɛ k { X ɛ} X k dp 1 ɛ k Ω X k dp = 1 ɛ k E X k Corollary 7.4 (Chebyshev s Iequality). Suppose X is a radom variable with fiite 2d momet. The, The followig is also a useful fact: P { X EX ɛ} 1 ɛ 2 Var(X). Lemma 7.5. Suppose X is a oegative radom variable. The, P {X m} EX Proof. E X = = m=1 P { X < + 1} = =1 P {X m} EX m=1 m=1 =m P { X < + 1} 10
11 8 The Borel-Catelli Lemma First, let us itroduce some termiology. Let A 1, A 2,... be sets. The, lim sup A := =1 m= A m. lim sup A cosists of those ω that appear i A ifiitely ofte (i.o.). Also, lim if A := =1 m= A. lim if A cosists of those ω that appear i all but fiitely may A. Theorem 8.1 (Borel-Catelli Lemma). Let A 1, A 2,... F. If =1 P (A ) <, the P (lim sup A ) = 0. Furthermore, suppose that the A i are idepedet. The, if =1 P (A ) =, the P (lim sup A ) = 1. Proof. Suppose =1 P (A ) <. The, ( ) ( ) P m = A m = lim P A m lim =1 =1 m= m= For the coverse, it is eough to show that ( ) P = 0, ad so it is also eough to show that ( P m= A c m A c m ) = 0 m= for all. By idepedece, ad sice 1 x e x, we have ( ) ( +k ) { +k P P = (1 P (A m )) exp m= A c m m= A c m m= Sice the last sum diverges, takig the limit as k, we get ( ) P = 0 m= A c m P (A m ) = 0. +k m= P (A m ) } 11
12 9 The Law of Large Numbers Let X 1, X 2,... be radom variables that are idepedet ad idetically distributed (iid). Let S := X 1 + +X. We will be iterested i the asymptotic behavior of the average S. If X i has a fiite expectatio, the we would thik that S would settle dow to EX i. This is kow as the Law of Large Numbers. There are two varieties of this law: the Weak Law of Large Numbers ad the Strog Law of Large Numbers. The weak law states that the average coverges i probability to EX i. The strog law, however states that the average coverges almost surely to EX i. However, the strog law is sigificatly harder to prove, ad requires a bit of additioal machiery. For the rest of this sectio, fix a probability space (Ω, F, P ). Theorem 9.1 (The Weak Law of Large Numbers). Suppose X 1, X 2,... are iid radom variables with mea EX i = m <. The, S P m. Proof. Let φ be the characteristic fuctio of X i. The, the characteristic fuctio of S is [φ(t)]. The, by 6.7, the characteristic fuctio of S is ψ (t) = [φ( t )]. Furthermore, by 6.6, φ is differetiable, ad φ (0) = im. Therefore, we ca form the Taylor expasio, ( ) t φ = 1 + ımt ( ) 1 + o, ad so ψ (t) = Takig the limit as, we get [ 1 + ımt ( )] 1 + o. lim ψ (t) = e ımt which is the characteristic fuctio for the distributio degeerate at m. Therefore, by Propositio 4.8, S coverges i probability to m. Theorem 9.2 (The Strog Law of Large Numbers). Suppose X 1, X 2,... are iid radom variables with EX i = m <. Let S = X X. The, S coverges to m almost surely. Proof. We ca decompose a arbitrary radom variable X i ito its positive ad egative parts: X + i := X i I {Xi 0} ad X i := X i I {Xi<0}, so that X i = X + i X i. The, we have S = X X+ (X1 + + X ) =: S + S. Hece, it is eough to prove the Theorem for oegative X i. Now, Let Y i := X i I {Xi i}. Let S := Y Y. Furthermore, let α > 1, ad set u := α. We shall first establish the iequality =1 P { S u ES u u } ɛ < ɛ > 0 (9.1) 12
13 Sice the X i are idepedet, we have Var(S ) = Var(Y k ) k=1 k=1 EY 2 k E[Xi 2 I {Xi k}] E[Xi 2 I {Xi }] k=1 By Chebyshev s iequality, we have =1 P { S u ES u u } ɛ =1 1 ɛ 2 Var(S u ) ɛ 2 u 2 =1 = 1 ɛ 2 E [X 2 i E[X 2 i I {X i u }] u =1 1 u I {Xi u } ] (9.2) Now, let K := 2α α 1. Let x > 0, ad let N := if{ : u x}. The, α N x. Also, ote that α 2u, ad so u 2α. The, ad hece, u x u α = 2α N N =1 =0 ( ) 1 = Kα N Kx 1, α 1 u I {Xi u } KX 1 1 if X 1 > 0 ad so, puttig this ito (9.2), we get [ ] 1 ɛ 2 E Xi 2 1 I {Xi u u } 1 ɛ 2 E [ Xi 2 KX 1 ] K i = ɛ 2 EX i < =1 thereby establishig iequality (9.1). Therefore, by the Borel-Catelli Lemma, we have ( { Su P lim sup ES }) u ɛ = 0 ɛ > 0. Takig a itersectio over all ratioal ɛ, we get that u S u ES u u 0 almost surely. However, 1 ES = 1 k=1 EY k, ad sice EY k EX i, takig the limit as, we have that 1 ES EX i. Therefore, we have that S u u EX i almost surely. (9.3) 13
14 Now, otice that by Lemma 7.5, P {X Y } = P {X i > } EX i < =1 =1 Agai, by the Borel-Catelli Lemma, we have ( ) P lim sup{x Y } = 0. Therefore, S S 0 almost surely, ad so by (9.3), S u u EX i almost surely. (9.4) Now, to get that the etire sequece S EX i almost surely, ote that S m is a icreasig sequece. Suppose u k u +1. The, ad so, u u +1 S u u S k k u +1 S u+1 u u +1 1 α EX S k i lim if k k lim sup k S k k αex i almost surely. Takig α 1, we get by (9.4) S k lim k k = EX i almost surely 10 Coditioal Expectatio ad Probability Before defiig codtioal expectatio ad probability, we will make a few observatios about the probabalistic iterpretatio of σ-fields. Cosider a process where a radom umber betwee zero ad oe is chose. More precisely, a outcome ω is chose accordig to some probability law from the set of all possible outcomes, Ω = [0, 1). We may be able to observe this umber ω to some amout of precisio, say up to oe digit. The σ-field that represets this amout of precisio is F 1 := σ{[0,.1), [.1,.2),..., [.9, 1)}. The σ-field F 1 represets all the iformatio that we ca kow about ω by observig it to oe digit of precisio. That is, a observer who ca observe the umber ω to oe digit will be able to determie exactly which sets A F 1 that ω belogs to, but he will ot be able to give ay iformatio more precise tha that. Similarly, if we ca observe ω up to digits of precisio, the σ-field which correspods to this is: F := σ {[ ) i 10, } i+1 10 : 0 i < 10. This example illustrates a geeral cocept: The σ-field that is used represets the amout of iformatio that a observer has about the radom process. 14
15 Defiitio If F is a σ-field, a F-observer is a observer who ca determie precisely which sets A F that a radom outcome ω belogs to but has o more iformatio about ω. Therefore, a 2 Ω -observer has complete iformatio about the outcome ω, whereas a F-observer has less iformatio. Similarly, if Σ F, the a F- observer has more iformatio tha a Σ-observer. Suppose that a radom variable X is F-measurable. This meas that the preimage of ay Borel set uder X i F. Therefore, a F-observer will have complete iformatio about X, or ay other F-measurable radom variable. Note that if Σ F, a Σ-measurable fuctio is also F-measurable. Suppose that X is a F-measurable radom variable, ad that you are a Σ- observer. You do ot have complete iformatio about X. However, give your iformatio Σ, you would like to make a buest guess about the value of X. That is, you wat to create aother radom variable, Y, that is Σ-measurable, but which approximates X. Y is called the coditioal expectatio of X wrt Σ, ad is deoted E[X Σ]. We will require that X(ω)dP = E[X Σ](ω)dP for all A Σ (10.1) A A Lemma Let (Ω, F, P ) be a probability space, ad let Σ be a sub-σ-field of F. Let P Σ deote the restrictio of P to Σ. Suppose f is a Σ-measurable fuctio ad A Σ. The, f(ω)dp Σ = f(ω)dp. A Justified by the previous lemma, we will ofte be sloppy ad ot explicitly say which σ-field a particular itegral is take over. I order to prove that a fuctio satisfyig (10.1) exists, we will have to discuss the ado-nikodym Theorem. First, a defiitio. Defiitio A siged measure λ o a measurable space (Ω, F) is a fuctio λ : F such that wheever A 1, A 2,... is a fiite or coutable sequece of disjoit sets i F, we have ( ) λ A i = λ(a i ) i i I particular, we have for a siged measure, λ( ) = 0. All probability measures are also siged measures. Note that λ is permitted to take o egative values. However, it is ot permitted to take o the values + or. Defiitio a siged measure λ o (Ω, F) is absolutely cotiuous with respect to a probability measure P if, wheever P (A) = 0, we have also λ(a) = 0. This is deoted λ P. A 15
16 For example, if f is a itegrable fuctio wrt P, the λ(a) = A f(ω)dp is a siged measure that is absolutely cotiuous with respect to P. I fact, all absolutely cotiuous siged measures arise i this way. Theorem 10.5 (ado-nikodym). Suppose λ P. The, there is a itegrable fuctio f such that λ(a) = f(ω)dp. (10.2) A Furthermore, if f is aother fuctio satisfyig (10.2), the f = f P -almosteverywhere. Defiitio The fuctio f i Theorem 10.5 is called the ado-nikodym derivative of λ with respect to P. It is deoted dλ dp. Note that the adom-nikodym derivative is oly defied up to equality almost everywhere. We ca use the ado-nikodym derivative to defie the coditioal expectatio satisfyig (10.1). Defiitio Let (Ω, F, P ) be a probability space. Let Σ be a sub-σ-field of F. Let X be a F-itegrable radom variable. Let λ be the siged measure defied by λ(a) = X(ω)dP. The coditioal expectatio of X wrt Σ is A E[X Σ] := dλ Σ dp Σ. We ow state some of the elemetary properties of coditioal expectatio. Lemma Let X ad X i be radom variables o (Ω, F, P ). Let Σ be a sub-σ-field of F. (i) E[E[X Σ]] = E[X] (ii) If X is oegative, the E[X Σ] is oegative almost surely. (iii) Suppose a 1, a 2. The E[a 1 X 1 + a 2 X 2 Σ] = a 1 E[X 1 Σ] + a 2 E[X 2 Σ] almost surely. (iv) E[X Σ] dp X dp. (v) If Y is bouded ad Σ-measurable, the E[XY Σ] = Y E[X Σ] almost surely. (vi) If Σ 2 Σ 1 F are sub-σ-fields, the E[X Σ 2 ] = E[E[X Σ 1 ] Σ 2 ] almost surely. As a special case of coditioal expectatio, we have coditioal probability. 16
17 Defiitio Let (Ω, F, P ) be a probability space, ad let Σ be a subσ-field of F. The, the coditioal probability of a evet A F give Σ is P [A Σ] := E[I A Σ]. P [A Σ](ω) is also sometimes writte P (ω, A). elemetary properties of coditioal probability. We ow state some of the Lemma The followig hold almost surely: (i) P (ω, Ω) = 1 ad P (ω, ). (ii) For ay A F, 0 P (ω, A) 1. (iii) If A 1, A 2... is a fiite or coutable sequece of disjoit sets i F, the ( P ω, ) A i = P (ω, A i ). i i (iv) If A Σ, the P (ω, A) = I A (ω). Lemma i particular implies that give ω Ω, P (ω, ) is a probability measure o (Ω, F). efereces [1] P. Billigsley, Probability ad measure, Joh Wiley & Sos, Ic., [2] S..S. Varadha, Probability theory, Courat lecture otes, 7. America Mathematical Society,
Convergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationMeasure and Measurable Functions
3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationChapter 0. Review of set theory. 0.1 Sets
Chapter 0 Review of set theory Set theory plays a cetral role i the theory of probability. Thus, we will ope this course with a quick review of those otios of set theory which will be used repeatedly.
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationIntroduction to Probability. Ariel Yadin. Lecture 7
Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationf n (x) f m (x) < ɛ/3 for all x A. By continuity of f n and f m we can find δ > 0 such that d(x, x 0 ) < δ implies that
Lecture 15 We have see that a sequece of cotiuous fuctios which is uiformly coverget produces a limit fuctio which is also cotiuous. We shall stregthe this result ow. Theorem 1 Let f : X R or (C) be a
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More information6 Infinite random sequences
Tel Aviv Uiversity, 2006 Probability theory 55 6 Ifiite radom sequeces 6a Itroductory remarks; almost certaity There are two mai reasos for eterig cotiuous probability: ifiitely high resolutio; edless
More informationProbability: Limit Theorems I. Charles Newman, Transcribed by Ian Tobasco
Probability: Limit Theorems I Charles Newma, Trascribed by Ia Tobasco Abstract. This is part oe of a two semester course o measure theoretic probability. The course was offered i Fall 2011 at the Courat
More informationIt is always the case that unions, intersections, complements, and set differences are preserved by the inverse image of a function.
MATH 532 Measurable Fuctios Dr. Neal, WKU Throughout, let ( X, F, µ) be a measure space ad let (!, F, P ) deote the special case of a probability space. We shall ow begi to study real-valued fuctios defied
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationThe Borel hierarchy classifies subsets of the reals by their topological complexity. Another approach is to classify them by size.
Lecture 7: Measure ad Category The Borel hierarchy classifies subsets of the reals by their topological complexity. Aother approach is to classify them by size. Filters ad Ideals The most commo measure
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationThe Boolean Ring of Intervals
MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationLecture 12: November 13, 2018
Mathematical Toolkit Autum 2018 Lecturer: Madhur Tulsiai Lecture 12: November 13, 2018 1 Radomized polyomial idetity testig We will use our kowledge of coditioal probability to prove the followig lemma,
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationA Proof of Birkhoff s Ergodic Theorem
A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed
More information1 Introduction. 1.1 Notation and Terminology
1 Itroductio You have already leared some cocepts of calculus such as limit of a sequece, limit, cotiuity, derivative, ad itegral of a fuctio etc. Real Aalysis studies them more rigorously usig a laguage
More informationM17 MAT25-21 HOMEWORK 5 SOLUTIONS
M17 MAT5-1 HOMEWORK 5 SOLUTIONS 1. To Had I Cauchy Codesatio Test. Exercise 1: Applicatio of the Cauchy Codesatio Test Use the Cauchy Codesatio Test to prove that 1 diverges. Solutio 1. Give the series
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationNotes on Snell Envelops and Examples
Notes o Sell Evelops ad Examples Example (Secretary Problem): Coside a pool of N cadidates whose qualificatios are represeted by ukow umbers {a > a 2 > > a N } from best to last. They are iterviewed sequetially
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More information6 Integers Modulo n. integer k can be written as k = qn + r, with q,r, 0 r b. So any integer.
6 Itegers Modulo I Example 2.3(e), we have defied the cogruece of two itegers a,b with respect to a modulus. Let us recall that a b (mod ) meas a b. We have proved that cogruece is a equivalece relatio
More informationIntroduction to Probability. Ariel Yadin. Lecture 2
Itroductio to Probability Ariel Yadi Lecture 2 1. Discrete Probability Spaces Discrete probability spaces are those for which the sample space is coutable. We have already see that i this case we ca take
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationOn Random Line Segments in the Unit Square
O Radom Lie Segmets i the Uit Square Thomas A. Courtade Departmet of Electrical Egieerig Uiversity of Califoria Los Ageles, Califoria 90095 Email: tacourta@ee.ucla.edu I. INTRODUCTION Let Q = [0, 1] [0,
More informationMA131 - Analysis 1. Workbook 3 Sequences II
MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationSets and Probabilistic Models
ets ad Probabilistic Models Berli Che Departmet of Computer ciece & Iformatio Egieerig Natioal Taiwa Normal iversity Referece: - D. P. Bertsekas, J. N. Tsitsiklis, Itroductio to Probability, ectios 1.1-1.2
More information1 of 7 7/16/2009 6:06 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 6. Order Statistics Defiitios Suppose agai that we have a basic radom experimet, ad that X is a real-valued radom variable
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationf(1), and so, if f is continuous, f(x) = f(1)x.
2.2.35: Let f be a additive fuctio. i Clearly fx = fx ad therefore f x = fx for all Z+ ad x R. Hece, for ay, Z +, f = f, ad so, if f is cotiuous, fx = fx. ii Suppose that f is bouded o soe o-epty ope set.
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationSets and Probabilistic Models
ets ad Probabilistic Models Berli Che Departmet of Computer ciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Referece: - D. P. Bertsekas, J. N. Tsitsiklis, Itroductio to Probability, ectios 1.1-1.2
More information5 Many points of continuity
Tel Aviv Uiversity, 2013 Measure ad category 40 5 May poits of cotiuity 5a Discotiuous derivatives.............. 40 5b Baire class 1 (classical)............... 42 5c Baire class 1 (moder)...............
More informationFUNDAMENTALS OF REAL ANALYSIS by
FUNDAMENTALS OF REAL ANALYSIS by Doğa Çömez Backgroud: All of Math 450/1 material. Namely: basic set theory, relatios ad PMI, structure of N, Z, Q ad R, basic properties of (cotiuous ad differetiable)
More informationFUNDAMENTALS OF REAL ANALYSIS by. V.1. Product measures
FUNDAMENTALS OF REAL ANALSIS by Doğa Çömez V. PRODUCT MEASURE SPACES V.1. Product measures Let (, A, µ) ad (, B, ν) be two measure spaces. I this sectio we will costruct a product measure µ ν o that coicides
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationPart II Probability and Measure
Part II Probability ad Measure Based o lectures by J. Miller Notes take by Dexter Chua Michaelmas 2016 These otes are ot edorsed by the lecturers, ad I have modified them (ofte sigificatly) after lectures.
More informationHere are some examples of algebras: F α = A(G). Also, if A, B A(G) then A, B F α. F α = A(G). In other words, A(G)
MATH 529 Probability Axioms Here we shall use the geeral axioms of a probability measure to derive several importat results ivolvig probabilities of uios ad itersectios. Some more advaced results will
More information1 Lecture 2: Sequence, Series and power series (8/14/2012)
Summer Jump-Start Program for Aalysis, 202 Sog-Yig Li Lecture 2: Sequece, Series ad power series (8/4/202). More o sequeces Example.. Let {x } ad {y } be two bouded sequeces. Show lim sup (x + y ) lim
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationSequences and Series
Sequeces ad Series Sequeces of real umbers. Real umber system We are familiar with atural umbers ad to some extet the ratioal umbers. While fidig roots of algebraic equatios we see that ratioal umbers
More informationRandomized Algorithms I, Spring 2018, Department of Computer Science, University of Helsinki Homework 1: Solutions (Discussed January 25, 2018)
Radomized Algorithms I, Sprig 08, Departmet of Computer Sciece, Uiversity of Helsiki Homework : Solutios Discussed Jauary 5, 08). Exercise.: Cosider the followig balls-ad-bi game. We start with oe black
More informationMAS111 Convergence and Continuity
MAS Covergece ad Cotiuity Key Objectives At the ed of the course, studets should kow the followig topics ad be able to apply the basic priciples ad theorems therei to solvig various problems cocerig covergece
More information