Probability and Measure
|
|
- Monica Juliet Barnett
- 5 years ago
- Views:
Transcription
1 Probability and Measure Robert L. Wolpert Institute of Statistics and Decision Sciences Duke University, Durham, NC, USA Convergence of Random Variables 1. Convergence Concepts 1.1. Convergence of Real Numbers A sequence of real numbers a n converges to a limit a if and only if, for each ɛ > 0, the sequence a n eventually lies within a ball of radius ɛ centered at a. It s okay if the first few (or few million) terms lie outside that ball and the number of terms that do lie outside the ball may depend on how big ɛ is (if ɛ is small enough it will take millions of terms before the remaining sequence lies inside the ball). This can be made mathematically precise by introducing a letter (say, N ɛ ) for how many initial terms we have to throw away, so that a n a if and only if there is an N ɛ < so that, for each n N ɛ, a n a < ɛ: only finitely many a n can be farther than ɛ from a. The same notion of convergence really works in any (complete) metric space, where we require that some measure of the distance d(a n, a) from a n to a tend to zero in the sense that it exceeds each number ɛ > 0 for at most some finite number N ɛ of terms. Points a n in d-dimensional Euclidean space will converge to a limit a R d if and only if each of their coordinates converges; and, since there are only finitely many of them, if they all converge then they do so uniformly (i.e., for each ɛ we can take the same N ɛ for all d of the coordinate sequences). 2. Convergence of Random Variables For random variables X n the idea of convergence to a limiting random variable X is more delicate, since each X n is a function of ω Ω and usually 1
2 there are infinitely many points ω Ω. What should we mean in asking about the convergence of a sequence X n of random variables to a limit X? Should we mean that X n (ω) converges to X(ω) for each fixed ω? Or that these sequences converge uniformly in ω Ω? Or that some notion of the distance d(x n, X) between X n and the limit X decreases to zero? Should the probability measure P be involved in some way? Here are a few different choices of what we might mean by the statement that X n converges to X, for a sequence of random variables X n and a random variable X, all defined on the same probability space (Ω, F, P): pw: The sequence of real numbers X n (ω) X(ω) for every ω Ω (pointwise convergence): ( ɛ > 0) ( ω Ω) ( N ɛ,ω < ) ( n N ɛ,ω ) X n (ω) X(ω) < ɛ. uni: The sequences of real numbers X n (ω) X(ω) uniformly for ω Ω: ( ɛ > 0) ( N ɛ < ) ( ω Ω) ( n N ɛ ) X n (ω) X(ω) < ɛ. a.s..: Outside some null event N F, each sequence of real numbers X n (ω) X(ω) (Almost-Sure convergence, or almost everywhere (a.e.)): for some N F with P[N] = 0, ( ɛ > 0) ( ω / N) ( N ɛ,ω < ) ( n N ɛ,ω ) X n (ω) X(ω) < ɛ, i.e., P { ɛ>0 N< n N X n (ω) X(ω) ɛ } = 0. L : Outside some null event N F, the sequences of real numbers X n (ω) X(ω) converge uniformly ( almost-uniform or L convergence): for some N F with P[N] = 0, ( ɛ > 0) ( N ɛ < ) ( ω / N) ( n N ɛ ) X n (ω) X(ω) < ɛ. i.p.: For each ɛ > 0, the probabilities P[ X n X > ɛ] 0 (convergence in probability, or in measure ): ( ɛ > 0) ( η > 0) ( N ɛ,η < ) ( n N ɛ,η ) P[ X n X > ɛ] < η. L 1 : The expectation E[ X n X ] converges to zero (convergence in L 1 ): ( ɛ > 0) ( N ɛ < ) ( n N ɛ ) E[ X n X ] < ɛ. 2
3 L p : For some fixed number p > 0, the expectation of the p th power E[ X n X p ] converges to zero (convergence in L p, sometimes called in the p th mean ): ( ɛ > 0) ( N ɛ < ) ( n N ɛ ) E[ X n X p ] < ɛ. i.d.: The distributions of X n converge to the distribution of X, i.e., the measures P Xn 1 converge in some way to P X 1 ( vague or weak convergence, or convergence in distribution, sometimes written X n X): ( ɛ > 0) ( φ C b (R)) ( N ɛ,φ < ) ( n N ɛ,φ ) E[ φ(x n ) φ(x) ] < ɛ. Which of these eight notions of convergence is right for random variables? The answer is that all of them are useful in probability theory for one purpose or another. You will want to know which ones imply which other ones, under what conditions. All but the first two (pointwise, uniform) notions depend upon the measure P; it is possible for a sequence X n to converge to X in any of these senses for one probability measure P, but to fail to converge for another P. Most of them can be phrased as metric convergence for some notion of distance between random variables: i.p.: X n X in probability if and only if d 0 (X, X n ) 0 as real numbers, where: ( ) X Y d 0 (X, Y ) E 1 + X Y L 1 : X n X in L 1 if and only if d 1 (X, X n ) = X X n 1 0 as real numbers, where: X Y 1 E X Y L p : X n X in L p if and only if d p (X, X n ) = X X n p 0 as real numbers, where: X Y p (E X Y p ) 1/p L : X n X almost uniformly if and only if d (X, X n ) = X Y 0 as real numbers, where: X Y = l.u.b.{r < : P[ X Y > r] > 0} As the notation suggests, convergence in probability and in L are in some sense limits of convergence in L p as p 0 and p, respectively. Almostsure convergence is an exception: there is no metric notion of distance d(x, Y ) for which X n X almost surely if and only if d(x, X n ) 0. 3
4 2.1. Almost-Sure Convergence Let {X n } and X be a collection of RV s on some (Ω, F, P). The set of points ω for which X n (ω) does converge to X(ω) is just ɛ>0 m=1 n=m [ω : X n (ω) X(ω) ɛ], the points which, for all ɛ > 0, have X n (ω) X(ω) less than ɛ for all but finitely-many n. The sequence X n is said to converge almost everywhere (a.e.) to X, or to converge to X almost surely (a.s..), if this set of ω has probability one, or (conversely) if its complement is a null set: [ P ɛ>0 m=1 n=m ] [ω : X n (ω) X(ω) > ɛ] = 0. The union over ɛ > 0 is only a countable one, since we need include only rational ɛ (or, for that matter, any sequence ɛ k tending to zero, such as ɛ k = 1/k). Thus X n X a.e. if and only if, for each ɛ > 0, [ P m=1 n=m ] [ω : X n (ω) X(ω) > ɛ] = 0. (a.e.) This combination of intersection and union occurs frequently in probability, and has a name; for any sequence E n of events, [ m=1 n=m E n] is called the lim sup of the {E n }, and is sometimes described more colorfully as [E n i.o.], the set of points in E n infinitely often. Its complement is the lim inf of the sets F n = En c, [ m=1 n=m F n]: the set of points in all but finitely many of the F n. Since P is countably additive, and since the intersection in the definition of lim sup is decreasing and the union in the definition of lim inf is increasing, always we have P[ n=m E n] P[ m=1 m. Thus, n=m E n] and P[ n=m F n] P[ m=1 n=m F n] as Theorem 1 X n X P-a.s.. if and only if for every ɛ > 0, lim P[ X n X > ɛ for some n m] = 0. m In particular, X n X P-a.s.. if P[ X n X > ɛ] < for each ɛ > 0 (why?). 4
5 2.2. Convergence In Probability The sequence X n is said to converge to X in probability (i.p.) if, for each ɛ > 0, P[ω : X n (ω) X(ω) > ɛ] 0. (i.p.) If we denote by E n the event [ω : X n (ω) X(ω) > ɛ] we see that convergence almost surely requires that P[ n m E n] 0 as m, while convergence in probability requires only that P[E n ] 0. Thus: Theorem 2 If X n X a.e. then X n X i.p. Here is a partial converse: Theorem 3 If X n X i.p., then there is a subsequence n k such that X nk X a.e. Proof. Set n 0 = 0 and, for each integer k 1, set { [ n k = inf n > n k 1 : P ω : X n (ω) X(ω) > 1 ] } 2 k. k For any ɛ > 0 we have 1 k < ɛ eventually (namely, for k > k 0 = 1 ɛ ) and for each m > k 0, [ P k=m ] [ω : X nk (ω) X(ω) > ɛ] [ P k=m [ω : X nk (ω) X(ω) > 1 k ] ] P[ω : X nk (ω) X(ω) > 1 k ] k=m k=m 2 k = 2 1 m 0 as m. 5
6 A Counter-Example If X n X a.e. implies X n X i.p., and if the converse holds at least along subsequences, are the two notions really identical? Or is it possible for RV s X n to converge to X i.p., but not a.e.? The answer is that the two notions are different, and that a.e. convergence is strictly stronger than convergence i.p. Here s an example: Let (Ω, F, P) be the unit interval with Borel sets and Lebesgue measure. Define a sequence of random variables X n : Ω R by X n (ω) = { i 1 if < ω i+1 2 j 2 j 0 otherwise where n = i + 2 j, 0 i < 2 j. Each X n is one on an interval of length 2 j, where j = log 2 (n) ; since 1 n 1 2 j < 2 n, P[ X n > ɛ] = 2 j < 2 n 0 for each 0 < ɛ < 1 and X n 0 i.p. On the other hand, for every j > 0 we have Ω = 2 j 1 i=0 ( i 2 j, i + 1 ] 2 j = 2 j+1 1 n=2 j [ ω : Xn (ω) = 1 ] so [ω : X n (ω) 0] is empty, not a set of probability one! Obviously X n does not converge a.e. This example is a building-block for several examples to come, so getting to know it well is worth while. Try to verify that X n 0 in probability and in L p but not almost surely. What is X n p? Why doesn t X n 0 a.s.? What would happen if we multiplied X n by n? By n 2? What about the subsequence Y n = X 2 n? Does X n converge in L? 3. Cauchy Convergence Sometimes we wish to consider a sequence X n that converges to some limit X, perhaps without knowing X in advance; the concept of Cauchy Convergence is ideal for this. For any of the distance measures d p above, with 0 p, say X n is a Cauchy sequence in L p if ( ɛ > 0)( N < )( n m N) d p (X m, X n ) < ɛ. 6
7 The spaces L p for 0 p are all complete in the sense that if X n is Cauchy for d p then there exists X L p for which d p (X n, X) 0. To see this, take an increasing subsequence N k along which d p (X m, X n ) < 2 k for n m N k, and set X 0 = 0 and N 0 = 0; set Y k X Nk X Nk 1. Check to confirm that k=1 Y k converges a.s., to some limit X L p with d p (X n, X) Uniform Integrability Let Y 0 be integrable on some probability space (Ω, F, P), E[Y ] = Y dp < ; it follows (from DCT or MCT, for example) that lim E[Y 1 [Y >t]] = Y dp = 0 t Ω [ω:y (ω)>t] and, consequently, that for any sequence of random variables X n dominated by Y in the sense that X n Y a.s.., lim E[X n 1 t [Xn>t]] = X n dp [ω:x n(ω)>t] Y dp [ω:y (ω)>t] = 0, uniformly in n. Call the sequence X n uniformly integrable (or simply UI) if E[X n 1 [Xn>t]] 0 uniformly in n, even if it is not dominated by a single integrable random variable Y. The big result is: Theorem 4 If X n X i.p. and if X n is UI then X n X in L 1. Proof. Without loss of generality take X 0. Fix any ɛ > 0; find (by UI) t ɛ > 0 such that E[ X n 1 [Xn>t ɛ]] ɛ for all n. Now find (by X n X i.p.) N ɛ N such that, for n N ɛ, P[ X n > ɛ] < ɛ/t ɛ ; then: E[ X n ] = X n dp + X n dp + X n dp [ X n ɛ] [ɛ< X n t ɛ] ɛ dp + t ɛ dp + [ X n ɛ] [ɛ< X n t ɛ] ɛ + (t ɛ )P[ X n > ɛ] + ɛ 3ɛ. 7 [t ɛ< X n ] [t ɛ< X n ] X n dp
8 Similarly, for any p > 0, X n X (i.p.) and X n p UI (for example, X n Y L p ) gives X n X (L p ). In the special case of X n Y L p this is just Lebesgue s Dominated Convergence Theorem (DCT). We have seen that {X n } is UI whenever X n Y L 1, but UI is more general than that. Here are two more criteria: Theorem 5 If {X n } is uniformly bounded in L p for some p > 1 then {X n } is UI. Proof. Let c R + be an upper bound for E X n p. First recall that, by Fubini s Theorem, any random variable X satisfies for any q > 0 E X q = 0 q x q 1 P[ X > x] dx. We apply this for q = 1 and q = p to the random variables X n and, for t > 0, to X n 1 Xn >t. Fix any t > 0; then E [ ] X n 1 Xn >t = = 0 t 0 P[ X n 1 Xn >t > x] dx P[ X n > t] dx + t P[ X n p > t p ] + t E X n p t p + 1 p t p 1 t t = t 1 p (1 + p 1 )E X n p c t 1 p (1 + p 1 ) 0 as t, uniformly in n. 0 P[ X n > x] dx p x p 1 p t p 1 P[ X n > x] dx p x p 1 P[ X n > x] dx Theorem 6 If {X n } is UI, then ( ɛ > 0)( δ > 0)( A F w/ P(A) < δ) E[ X n 1 A ] < ɛ. Conversely, if {X n } is uniformly bounded in L 1 and if ( ɛ > 0)( δ > 0) such that E[ X n 1 A ] < ɛ whenever P[A] < δ, then {X n } is UI. Proof. Straightforward. The condition {X n } is uniformly bounded in L 1 is unnecessary if (Ω, F, P) is non-atomic. 8
9 5. Summary: Uniform Integrability and Convergence Concepts I. Uniform Integrability (UI) A. X n < Y L r, r > 0, implies [ X n >t] X n r dp [Y >t] Y r dp 0 as t, uniformly in n. This is the definition of UI. B. If (Ω, F, P) nonatomic, X n UI iff ɛ δ Λ X n dp < ɛ for P[Λ] δ (take δ = ɛ/2t) [If (Ω, F, P) has atoms must also require E X n B] C. E X n p c p < implies X n r UI for each r < p... δ = (ɛ/c) q, 1 p + 1 q = 1, q = p p Remark: not for r = p (counterexample: X n = n1 (0,1/n] ) D. Main result (Thm 4.5.4, p97): If X n X i.p. then X n r UI iff X n X in L r iff E X n r E X r. II. Vague Convergence A. X n X i.p. iff n k n ki X nki X a.e. (by contradiction) B. X n X a.s.. and φ(x) continuous implies φ(x n ) φ(x) a.s.. C. X n X i.p. and φ(x) continuous implies φ(x n ) φ(x) i.p. (use A) D. Definition: X n X if Eφ(X n ) Eφ(X) φ C b (R) 1. Prop: X n X i.p. implies X n X (use II.C) 2. Prop: X n X implies F n (r) F (r) wherever F (r) = F (r ). a. Remark: Even if X n X, F n (r) may not converge where F (r) jumps; b. Remark: Even if X n X, f n (r) = F n (r) may not converge to f(r) = F (r); in fact, either may fail to exist. III. Implications among these notions: a.e., i.p., L r, L p, L, i.d. (0<r<p< ): A. a.e. = i.p. (by Easy Borel-Cantelli) 1. i.p. = a.e. along subsequences 2. i.p. a.e. (counterexample: X n (ω) = 1 (i/2 j,(i+1)/2 j ](ω), n = i + 2 j ) B. L p = i.p. (by Chebychev s inequality) 1. i.p. = L p under Uniform Integrability 2. i.p. L p (counterexample: X n = n 1/p 1 (0,1/n] ) C. L p = L r (by Jensen s inequality) 9
10 1. L r L p (counterexample: X n = n 1/p 1 (0,1/n] ) D. L = L p (simple estimate) 1. L p L (counterexample: X n = n 1/2p 1 (0,1/n] ) E. L = a.e. (uniform cgce implies pointwise cgce) F. i.p. = i.d. (II.D.1 above) 1. i.d. i.p. (counterexample: X n, X on different spaces) 2. i.d. = a.s.. ( (Ω, F, P), X n, X X n X a.e...) 10
11 6. Infinite Coin-Toss and the Laws of Large Numbers The traditional interpretation of the probability of an event E is its asymptotic frequency: the limit as n of the fraction of n repeated, similar, and independent trials in which E occurs. Similarly the expectation of a random variable X is taken to be its asymptotic average, the limit as n of the average of n repeated, similar, and independent replications of X. As statisticians trying to make inference about the underlying probability distribution f(x θ) governing observed random variables X i, this suggests that we should be interested in the probability distribution for large n of quantities like the average of the RV s, 1 n n i=1 X i. Three of the most celebrated theorems of probability theory concern this sum. For independent random variables X i, all with the same probability distribution satisfying E X i 3 <, set µ = EX i, σ 2 = E X i µ 2, and S n = n i=1 X i. The three main results are: Laws of Large Numbers: Central Limit Theorem: S n nµ σn S n nµ σ n 0 (i.p. and a.s..) = N(0, 1) (i.d.) Law of the Iterated Logarithm: lim sup ± S n nµ σ 2n log log n = 1.0 (a.s..) Together these three give a clear picture of how quickly and in what sense 1 n S n tends to µ. We begin with the Law of Large Numbers (LLN), in its weak form (asserting convergence i.p.) and in its strong form (convergence a.s..). There are several versions of both theorems. The simplest requires the X i to be IID and L 2 ; stronger results allow us to weaken (but not eliminate) the independence requirement, permit non-identical distributions, and consider what happens if the RV s are only L 1 (or worse!) instead of L 2. The text covers these things well; to complement it I am going to: (1) Prove the simplest version, and with it the Borel-Cantelli theorems; and (2) Show what happens with Cauchy random variables, which don t satisfy the requirements (the LLN fails). 11
12 I. Weak version, non-iid, L 2 : µ i = EX i, σ ij = E[X i µ i ][X j µ j ] A. Y n = (S n Σµ i )/n satisfies EY n = 0, EY 2 n = 1 n 2 Σ i σ ii + 2 n 2 Σ i<j σ ij ; 1. If σ ii M and σ ij 0, Chebychev = Y n 0, i.p. 2. (pairwise) IID L 2 is OK II. Strong version, non-iid, L 2 : EX i = 0, EX 2 i M, EX i X j 0. A. P[ S n > nɛ] < Mn n 2 ɛ 2 = M nɛ 2 1. P[ S n 2 > n 2 ɛ] < M, Σ n 2 ɛ 2 n P[ S n 2 > n 2 ɛ] < Mπ2 6ɛ 2 2. Borel-Cantelli: P[ S n 2 > n 2 1 ɛ i.o.] = 0, S n 2 n 2 0 a.s.. 3. D n = max n 2 k<(n+1) 2 S k S n 2, EDn 2 2nE S (n+1) 2 S n 2 4n 2 M 4. Chebychev: P[D n > n 2 ɛ] < 4n2 M n 4 ɛ, D 2 n 0 a.s.. B. S k /k S n 2 +D n 0 a.s.., QED n 2 1. Bernoulli RV s, normal number theorem, Monte Carlo. III. Weak version, pairwise-iid, L 1 A. Equivalent sequences: n P[X n Y n ] < 1. n [X n Y n ] < a.s.. 2. n i=1 [X i], a n n i=1 [X i] converge iff n i=1 [Y i], a n n i=1 [Y i] do 3. Y n = X n 1 [ Xn n] IV. Counterexamples: Cauchy, A. X i dx π[1+x 2 ] = P[ S n /n ɛ] 2 π tan 1 (ɛ) 1, WLLN fails. B. P[X i = n] = ±c, n 1; X n 2 i / L 1, and S n /n 0 i.p. or a.s.. C. P[X i = n] = ±c n 2 log n, n 3; X i / L 1, but S n /n 0 i.p. and not a.s.. D. Medians: for ANY RV s X n X i.p., then m n m if m is unique. Let X i be iid standard Cauchy RV s, with P[X 1 t] = and characteristic function E e iλx 1 = t dx π[1 + x 2 ] = π arctan(t) e iλx so S n /n has characteristic function E e iλsn/n = E e i λ n [X 1+ +X n] = dx π[1 + x 2 ] = e λ, ( ) E e i λ n n X 1 = (e λ n ) n = e λ 12
13 and S n /n also has the standard Cauchy distribution with P[S n /n t] = π arctan(t); in particular, S n/n does not converge almost surely, or even in probability. 13
7 Convergence in R d and in Metric Spaces
STA 711: Probability & Measure Theory Robert L. Wolpert 7 Convergence in R d and in Metric Spaces A sequence of elements a n of R d converges to a limit a if and only if, for each ǫ > 0, the sequence a
More informationSTA205 Probability: Week 8 R. Wolpert
INFINITE COIN-TOSS AND THE LAWS OF LARGE NUMBERS The traditional interpretation of the probability of an event E is its asymptotic frequency: the limit as n of the fraction of n repeated, similar, and
More information4 Expectation & the Lebesgue Theorems
STA 205: Probability & Measure Theory Robert L. Wolpert 4 Expectation & the Lebesgue Theorems Let X and {X n : n N} be random variables on a probability space (Ω,F,P). If X n (ω) X(ω) for each ω Ω, does
More information4 Expectation & the Lebesgue Theorems
STA 7: Probability & Measure Theory Robert L. Wolpert 4 Expectation & the Lebesgue Theorems Let X and {X n : n N} be random variables on the same probability space (Ω,F,P). If X n (ω) X(ω) for each ω Ω,
More information17. Convergence of Random Variables
7. Convergence of Random Variables In elementary mathematics courses (such as Calculus) one speaks of the convergence of functions: f n : R R, then lim f n = f if lim f n (x) = f(x) for all x in R. This
More informationProbability and Measure
Part II Year 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2018 84 Paper 4, Section II 26J Let (X, A) be a measurable space. Let T : X X be a measurable map, and µ a probability
More informationSTA 711: Probability & Measure Theory Robert L. Wolpert
STA 711: Probability & Measure Theory Robert L. Wolpert 6 Independence 6.1 Independent Events A collection of events {A i } F in a probability space (Ω,F,P) is called independent if P[ i I A i ] = P[A
More informationLecture 4: September Reminder: convergence of sequences
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 4: September 6 In this lecture we discuss the convergence of random variables. At a high-level, our first few lectures focused
More informationErgodic Theorems. Samy Tindel. Purdue University. Probability Theory 2 - MA 539. Taken from Probability: Theory and examples by R.
Ergodic Theorems Samy Tindel Purdue University Probability Theory 2 - MA 539 Taken from Probability: Theory and examples by R. Durrett Samy T. Ergodic theorems Probability Theory 1 / 92 Outline 1 Definitions
More information1 Sequences of events and their limits
O.H. Probability II (MATH 2647 M15 1 Sequences of events and their limits 1.1 Monotone sequences of events Sequences of events arise naturally when a probabilistic experiment is repeated many times. For
More informationAn Introduction to Laws of Large Numbers
An to Laws of John CVGMI Group Contents 1 Contents 1 2 Contents 1 2 3 Contents 1 2 3 4 Intuition We re working with random variables. What could we observe? {X n } n=1 Intuition We re working with random
More informationProblem set 1, Real Analysis I, Spring, 2015.
Problem set 1, Real Analysis I, Spring, 015. (1) Let f n : D R be a sequence of functions with domain D R n. Recall that f n f uniformly if and only if for all ɛ > 0, there is an N = N(ɛ) so that if n
More informationNotes 1 : Measure-theoretic foundations I
Notes 1 : Measure-theoretic foundations I Math 733-734: Theory of Probability Lecturer: Sebastien Roch References: [Wil91, Section 1.0-1.8, 2.1-2.3, 3.1-3.11], [Fel68, Sections 7.2, 8.1, 9.6], [Dur10,
More informationMetric Spaces and Topology
Chapter 2 Metric Spaces and Topology From an engineering perspective, the most important way to construct a topology on a set is to define the topology in terms of a metric on the set. This approach underlies
More information8 Laws of large numbers
8 Laws of large numbers 8.1 Introduction We first start with the idea of standardizing a random variable. Let X be a random variable with mean µ and variance σ 2. Then Z = (X µ)/σ will be a random variable
More informationInference for Stochastic Processes
Inference for Stochastic Processes Robert L. Wolpert Revised: June 19, 005 Introduction A stochastic process is a family {X t } of real-valued random variables, all defined on the same probability space
More information1/12/05: sec 3.1 and my article: How good is the Lebesgue measure?, Math. Intelligencer 11(2) (1989),
Real Analysis 2, Math 651, Spring 2005 April 26, 2005 1 Real Analysis 2, Math 651, Spring 2005 Krzysztof Chris Ciesielski 1/12/05: sec 3.1 and my article: How good is the Lebesgue measure?, Math. Intelligencer
More informationProbability Theory. Richard F. Bass
Probability Theory Richard F. Bass ii c Copyright 2014 Richard F. Bass Contents 1 Basic notions 1 1.1 A few definitions from measure theory............. 1 1.2 Definitions............................. 2
More informationA D VA N C E D P R O B A B I L - I T Y
A N D R E W T U L L O C H A D VA N C E D P R O B A B I L - I T Y T R I N I T Y C O L L E G E T H E U N I V E R S I T Y O F C A M B R I D G E Contents 1 Conditional Expectation 5 1.1 Discrete Case 6 1.2
More informationX n D X lim n F n (x) = F (x) for all x C F. lim n F n(u) = F (u) for all u C F. (2)
14:17 11/16/2 TOPIC. Convergence in distribution and related notions. This section studies the notion of the so-called convergence in distribution of real random variables. This is the kind of convergence
More informationg 2 (x) (1/3)M 1 = (1/3)(2/3)M.
COMPACTNESS If C R n is closed and bounded, then by B-W it is sequentially compact: any sequence of points in C has a subsequence converging to a point in C Conversely, any sequentially compact C R n is
More informationIntroduction and Preliminaries
Chapter 1 Introduction and Preliminaries This chapter serves two purposes. The first purpose is to prepare the readers for the more systematic development in later chapters of methods of real analysis
More informationLecture 6 Basic Probability
Lecture 6: Basic Probability 1 of 17 Course: Theory of Probability I Term: Fall 2013 Instructor: Gordan Zitkovic Lecture 6 Basic Probability Probability spaces A mathematical setup behind a probabilistic
More informationThe Borel-Cantelli Group
The Borel-Cantelli Group Dorothy Baumer Rong Li Glenn Stark November 14, 007 1 Borel-Cantelli Lemma Exercise 16 is the introduction of the Borel-Cantelli Lemma using Lebesue measure. An approach using
More informationMATH 140B - HW 5 SOLUTIONS
MATH 140B - HW 5 SOLUTIONS Problem 1 (WR Ch 7 #8). If I (x) = { 0 (x 0), 1 (x > 0), if {x n } is a sequence of distinct points of (a,b), and if c n converges, prove that the series f (x) = c n I (x x n
More informationAdvanced Probability
Advanced Probability Perla Sousi October 10, 2011 Contents 1 Conditional expectation 1 1.1 Discrete case.................................. 3 1.2 Existence and uniqueness............................ 3 1
More information1 Topology Definition of a topology Basis (Base) of a topology The subspace topology & the product topology on X Y 3
Index Page 1 Topology 2 1.1 Definition of a topology 2 1.2 Basis (Base) of a topology 2 1.3 The subspace topology & the product topology on X Y 3 1.4 Basic topology concepts: limit points, closed sets,
More informationStatistical Inference
Statistical Inference Robert L. Wolpert Institute of Statistics and Decision Sciences Duke University, Durham, NC, USA Week 12. Testing and Kullback-Leibler Divergence 1. Likelihood Ratios Let 1, 2, 2,...
More informationLecture 1: Overview of percolation and foundational results from probability theory 30th July, 2nd August and 6th August 2007
CSL866: Percolation and Random Graphs IIT Delhi Arzad Kherani Scribe: Amitabha Bagchi Lecture 1: Overview of percolation and foundational results from probability theory 30th July, 2nd August and 6th August
More information18.175: Lecture 2 Extension theorems, random variables, distributions
18.175: Lecture 2 Extension theorems, random variables, distributions Scott Sheffield MIT Outline Extension theorems Characterizing measures on R d Random variables Outline Extension theorems Characterizing
More informationCLASSICAL PROBABILITY MODES OF CONVERGENCE AND INEQUALITIES
CLASSICAL PROBABILITY 2008 2. MODES OF CONVERGENCE AND INEQUALITIES JOHN MORIARTY In many interesting and important situations, the object of interest is influenced by many random factors. If we can construct
More information1 Stat 605. Homework I. Due Feb. 1, 2011
The first part is homework which you need to turn in. The second part is exercises that will not be graded, but you need to turn it in together with the take-home final exam. 1 Stat 605. Homework I. Due
More informationReal Analysis Notes. Thomas Goller
Real Analysis Notes Thomas Goller September 4, 2011 Contents 1 Abstract Measure Spaces 2 1.1 Basic Definitions........................... 2 1.2 Measurable Functions........................ 2 1.3 Integration..............................
More informationMath 320-2: Midterm 2 Practice Solutions Northwestern University, Winter 2015
Math 30-: Midterm Practice Solutions Northwestern University, Winter 015 1. Give an example of each of the following. No justification is needed. (a) A metric on R with respect to which R is bounded. (b)
More informationIEOR 6711: Stochastic Models I Fall 2013, Professor Whitt Lecture Notes, Thursday, September 5 Modes of Convergence
IEOR 6711: Stochastic Models I Fall 2013, Professor Whitt Lecture Notes, Thursday, September 5 Modes of Convergence 1 Overview We started by stating the two principal laws of large numbers: the strong
More informationSolutions: Problem Set 4 Math 201B, Winter 2007
Solutions: Problem Set 4 Math 2B, Winter 27 Problem. (a Define f : by { x /2 if < x
More informationCourse 212: Academic Year Section 1: Metric Spaces
Course 212: Academic Year 1991-2 Section 1: Metric Spaces D. R. Wilkins Contents 1 Metric Spaces 3 1.1 Distance Functions and Metric Spaces............. 3 1.2 Convergence and Continuity in Metric Spaces.........
More informationHomework 11. Solutions
Homework 11. Solutions Problem 2.3.2. Let f n : R R be 1/n times the characteristic function of the interval (0, n). Show that f n 0 uniformly and f n µ L = 1. Why isn t it a counterexample to the Lebesgue
More informationI. ANALYSIS; PROBABILITY
ma414l1.tex Lecture 1. 12.1.2012 I. NLYSIS; PROBBILITY 1. Lebesgue Measure and Integral We recall Lebesgue measure (M411 Probability and Measure) λ: defined on intervals (a, b] by λ((a, b]) := b a (so
More informationMeasure and integration
Chapter 5 Measure and integration In calculus you have learned how to calculate the size of different kinds of sets: the length of a curve, the area of a region or a surface, the volume or mass of a solid.
More informationTools from Lebesgue integration
Tools from Lebesgue integration E.P. van den Ban Fall 2005 Introduction In these notes we describe some of the basic tools from the theory of Lebesgue integration. Definitions and results will be given
More informationLecture Notes 3 Convergence (Chapter 5)
Lecture Notes 3 Convergence (Chapter 5) 1 Convergence of Random Variables Let X 1, X 2,... be a sequence of random variables and let X be another random variable. Let F n denote the cdf of X n and let
More informationWe are going to discuss what it means for a sequence to converge in three stages: First, we define what it means for a sequence to converge to zero
Chapter Limits of Sequences Calculus Student: lim s n = 0 means the s n are getting closer and closer to zero but never gets there. Instructor: ARGHHHHH! Exercise. Think of a better response for the instructor.
More informationPart II Probability and Measure
Part II Probability and Measure Theorems Based on lectures by J. Miller Notes taken by Dexter Chua Michaelmas 2016 These notes are not endorsed by the lecturers, and I have modified them (often significantly)
More informationP-adic Functions - Part 1
P-adic Functions - Part 1 Nicolae Ciocan 22.11.2011 1 Locally constant functions Motivation: Another big difference between p-adic analysis and real analysis is the existence of nontrivial locally constant
More informationF (x) = P [X x[. DF1 F is nondecreasing. DF2 F is right-continuous
7: /4/ TOPIC Distribution functions their inverses This section develops properties of probability distribution functions their inverses Two main topics are the so-called probability integral transformation
More informationLecture 2: Convergence of Random Variables
Lecture 2: Convergence of Random Variables Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Introduction to Stochastic Processes, Fall 2013 1 / 9 Convergence of Random Variables
More informationPart IA Probability. Definitions. Based on lectures by R. Weber Notes taken by Dexter Chua. Lent 2015
Part IA Probability Definitions Based on lectures by R. Weber Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly) after lectures.
More informationNumerical Sequences and Series
Numerical Sequences and Series Written by Men-Gen Tsai email: b89902089@ntu.edu.tw. Prove that the convergence of {s n } implies convergence of { s n }. Is the converse true? Solution: Since {s n } is
More informationProbability and Measure
Chapter 4 Probability and Measure 4.1 Introduction In this chapter we will examine probability theory from the measure theoretic perspective. The realisation that measure theory is the foundation of probability
More information1 The Glivenko-Cantelli Theorem
1 The Glivenko-Cantelli Theorem Let X i, i = 1,..., n be an i.i.d. sequence of random variables with distribution function F on R. The empirical distribution function is the function of x defined by ˆF
More informationMTH 202 : Probability and Statistics
MTH 202 : Probability and Statistics Lecture 5-8 : 15, 20, 21, 23 January, 2013 Random Variables and their Probability Distributions 3.1 : Random Variables Often while we need to deal with probability
More informationMATHS 730 FC Lecture Notes March 5, Introduction
1 INTRODUCTION MATHS 730 FC Lecture Notes March 5, 2014 1 Introduction Definition. If A, B are sets and there exists a bijection A B, they have the same cardinality, which we write as A, #A. If there exists
More information1 Independent increments
Tel Aviv University, 2008 Brownian motion 1 1 Independent increments 1a Three convolution semigroups........... 1 1b Independent increments.............. 2 1c Continuous time................... 3 1d Bad
More informationTopology. Xiaolong Han. Department of Mathematics, California State University, Northridge, CA 91330, USA address:
Topology Xiaolong Han Department of Mathematics, California State University, Northridge, CA 91330, USA E-mail address: Xiaolong.Han@csun.edu Remark. You are entitled to a reward of 1 point toward a homework
More information3 (Due ). Let A X consist of points (x, y) such that either x or y is a rational number. Is A measurable? What is its Lebesgue measure?
MA 645-4A (Real Analysis), Dr. Chernov Homework assignment 1 (Due ). Show that the open disk x 2 + y 2 < 1 is a countable union of planar elementary sets. Show that the closed disk x 2 + y 2 1 is a countable
More informationConvergence of random variables, and the Borel-Cantelli lemmas
Stat 205A Setember, 12, 2002 Convergence of ranom variables, an the Borel-Cantelli lemmas Lecturer: James W. Pitman Scribes: Jin Kim (jin@eecs) 1 Convergence of ranom variables Recall that, given a sequence
More informationIntroduction to Dynamical Systems
Introduction to Dynamical Systems France-Kosovo Undergraduate Research School of Mathematics March 2017 This introduction to dynamical systems was a course given at the march 2017 edition of the France
More informationAnalysis Qualifying Exam
Analysis Qualifying Exam Spring 2017 Problem 1: Let f be differentiable on R. Suppose that there exists M > 0 such that f(k) M for each integer k, and f (x) M for all x R. Show that f is bounded, i.e.,
More informationAnalysis Finite and Infinite Sets The Real Numbers The Cantor Set
Analysis Finite and Infinite Sets Definition. An initial segment is {n N n n 0 }. Definition. A finite set can be put into one-to-one correspondence with an initial segment. The empty set is also considered
More information2 Measure Theory. 2.1 Measures
2 Measure Theory 2.1 Measures A lot of this exposition is motivated by Folland s wonderful text, Real Analysis: Modern Techniques and Their Applications. Perhaps the most ubiquitous measure in our lives
More informationLecture Notes in Advanced Calculus 1 (80315) Raz Kupferman Institute of Mathematics The Hebrew University
Lecture Notes in Advanced Calculus 1 (80315) Raz Kupferman Institute of Mathematics The Hebrew University February 7, 2007 2 Contents 1 Metric Spaces 1 1.1 Basic definitions...........................
More informationMathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio ( )
Mathematical Methods for Neurosciences. ENS - Master MVA Paris 6 - Master Maths-Bio (2014-2015) Etienne Tanré - Olivier Faugeras INRIA - Team Tosca October 22nd, 2014 E. Tanré (INRIA - Team Tosca) Mathematical
More informationPart 2 Continuous functions and their properties
Part 2 Continuous functions and their properties 2.1 Definition Definition A function f is continuous at a R if, and only if, that is lim f (x) = f (a), x a ε > 0, δ > 0, x, x a < δ f (x) f (a) < ε. Notice
More informationIntegration on Measure Spaces
Chapter 3 Integration on Measure Spaces In this chapter we introduce the general notion of a measure on a space X, define the class of measurable functions, and define the integral, first on a class of
More informationII - REAL ANALYSIS. This property gives us a way to extend the notion of content to finite unions of rectangles: we define
1 Measures 1.1 Jordan content in R N II - REAL ANALYSIS Let I be an interval in R. Then its 1-content is defined as c 1 (I) := b a if I is bounded with endpoints a, b. If I is unbounded, we define c 1
More informationConvergence of Random Variables
1 / 15 Convergence of Random Variables Saravanan Vijayakumaran sarva@ee.iitb.ac.in Department of Electrical Engineering Indian Institute of Technology Bombay March 19, 2014 2 / 15 Motivation Theorem (Weak
More informationMath LM (24543) Lectures 01
Math 32300 LM (24543) Lectures 01 Ethan Akin Office: NAC 6/287 Phone: 650-5136 Email: ethanakin@earthlink.net Spring, 2018 Contents Introduction, Ross Chapter 1 and Appendix The Natural Numbers N and The
More informationFinite-dimensional spaces. C n is the space of n-tuples x = (x 1,..., x n ) of complex numbers. It is a Hilbert space with the inner product
Chapter 4 Hilbert Spaces 4.1 Inner Product Spaces Inner Product Space. A complex vector space E is called an inner product space (or a pre-hilbert space, or a unitary space) if there is a mapping (, )
More informationMcGill University Math 354: Honors Analysis 3
Practice problems McGill University Math 354: Honors Analysis 3 not for credit Problem 1. Determine whether the family of F = {f n } functions f n (x) = x n is uniformly equicontinuous. 1st Solution: The
More informationNotions such as convergent sequence and Cauchy sequence make sense for any metric space. Convergent Sequences are Cauchy
Banach Spaces These notes provide an introduction to Banach spaces, which are complete normed vector spaces. For the purposes of these notes, all vector spaces are assumed to be over the real numbers.
More informationA PECULIAR COIN-TOSSING MODEL
A PECULIAR COIN-TOSSING MODEL EDWARD J. GREEN 1. Coin tossing according to de Finetti A coin is drawn at random from a finite set of coins. Each coin generates an i.i.d. sequence of outcomes (heads or
More informationMeasure and Integration: Solutions of CW2
Measure and Integration: s of CW2 Fall 206 [G. Holzegel] December 9, 206 Problem of Sheet 5 a) Left (f n ) and (g n ) be sequences of integrable functions with f n (x) f (x) and g n (x) g (x) for almost
More informationLecture 7. Sums of random variables
18.175: Lecture 7 Sums of random variables Scott Sheffield MIT 18.175 Lecture 7 1 Outline Definitions Sums of random variables 18.175 Lecture 7 2 Outline Definitions Sums of random variables 18.175 Lecture
More information1. Probability Measure and Integration Theory in a Nutshell
1. Probability Measure and Integration Theory in a Nutshell 1.1. Measurable Space and Measurable Functions Definition 1.1. A measurable space is a tuple (Ω, F) where Ω is a set and F a σ-algebra on Ω,
More information1 Probability theory. 2 Random variables and probability theory.
Probability theory Here we summarize some of the probability theory we need. If this is totally unfamiliar to you, you should look at one of the sources given in the readings. In essence, for the major
More informationIntroduction to Real Analysis Alternative Chapter 1
Christopher Heil Introduction to Real Analysis Alternative Chapter 1 A Primer on Norms and Banach Spaces Last Updated: March 10, 2018 c 2018 by Christopher Heil Chapter 1 A Primer on Norms and Banach Spaces
More informationUseful Probability Theorems
Useful Probability Theorems Shiu-Tang Li Finished: March 23, 2013 Last updated: November 2, 2013 1 Convergence in distribution Theorem 1.1. TFAE: (i) µ n µ, µ n, µ are probability measures. (ii) F n (x)
More informationLarge Sample Theory. Consider a sequence of random variables Z 1, Z 2,..., Z n. Convergence in probability: Z n
Large Sample Theory In statistics, we are interested in the properties of particular random variables (or estimators ), which are functions of our data. In ymptotic analysis, we focus on describing the
More informationCompact operators on Banach spaces
Compact operators on Banach spaces Jordan Bell jordan.bell@gmail.com Department of Mathematics, University of Toronto November 12, 2017 1 Introduction In this note I prove several things about compact
More informationMetric Spaces Math 413 Honors Project
Metric Spaces Math 413 Honors Project 1 Metric Spaces Definition 1.1 Let X be a set. A metric on X is a function d : X X R such that for all x, y, z X: i) d(x, y) = d(y, x); ii) d(x, y) = 0 if and only
More informationProblem Set 2: Solutions Math 201A: Fall 2016
Problem Set 2: s Math 201A: Fall 2016 Problem 1. (a) Prove that a closed subset of a complete metric space is complete. (b) Prove that a closed subset of a compact metric space is compact. (c) Prove that
More informationREVIEW OF ESSENTIAL MATH 346 TOPICS
REVIEW OF ESSENTIAL MATH 346 TOPICS 1. AXIOMATIC STRUCTURE OF R Doğan Çömez The real number system is a complete ordered field, i.e., it is a set R which is endowed with addition and multiplication operations
More informationLecture Notes for MA 623 Stochastic Processes. Ionut Florescu. Stevens Institute of Technology address:
Lecture Notes for MA 623 Stochastic Processes Ionut Florescu Stevens Institute of Technology E-mail address: ifloresc@stevens.edu 2000 Mathematics Subject Classification. 60Gxx Stochastic Processes Abstract.
More informationOn the Set of Limit Points of Normed Sums of Geometrically Weighted I.I.D. Bounded Random Variables
On the Set of Limit Points of Normed Sums of Geometrically Weighted I.I.D. Bounded Random Variables Deli Li 1, Yongcheng Qi, and Andrew Rosalsky 3 1 Department of Mathematical Sciences, Lakehead University,
More informationOn the convergence of sequences of random variables: A primer
BCAM May 2012 1 On the convergence of sequences of random variables: A primer Armand M. Makowski ECE & ISR/HyNet University of Maryland at College Park armand@isr.umd.edu BCAM May 2012 2 A sequence a :
More informationWhy study probability? Set theory. ECE 6010 Lecture 1 Introduction; Review of Random Variables
ECE 6010 Lecture 1 Introduction; Review of Random Variables Readings from G&S: Chapter 1. Section 2.1, Section 2.3, Section 2.4, Section 3.1, Section 3.2, Section 3.5, Section 4.1, Section 4.2, Section
More information1: PROBABILITY REVIEW
1: PROBABILITY REVIEW Marek Rutkowski School of Mathematics and Statistics University of Sydney Semester 2, 2016 M. Rutkowski (USydney) Slides 1: Probability Review 1 / 56 Outline We will review the following
More informationWeak convergence. Amsterdam, 13 November Leiden University. Limit theorems. Shota Gugushvili. Generalities. Criteria
Weak Leiden University Amsterdam, 13 November 2013 Outline 1 2 3 4 5 6 7 Definition Definition Let µ, µ 1, µ 2,... be probability measures on (R, B). It is said that µ n converges weakly to µ, and we then
More information1 Probability space and random variables
1 Probability space and random variables As graduate level, we inevitably need to study probability based on measure theory. It obscures some intuitions in probability, but it also supplements our intuition,
More informationFundamental Inequalities, Convergence and the Optional Stopping Theorem for Continuous-Time Martingales
Fundamental Inequalities, Convergence and the Optional Stopping Theorem for Continuous-Time Martingales Prakash Balachandran Department of Mathematics Duke University April 2, 2008 1 Review of Discrete-Time
More informationSome Background Material
Chapter 1 Some Background Material In the first chapter, we present a quick review of elementary - but important - material as a way of dipping our toes in the water. This chapter also introduces important
More information1 Measurable Functions
36-752 Advanced Probability Overview Spring 2018 2. Measurable Functions, Random Variables, and Integration Instructor: Alessandro Rinaldo Associated reading: Sec 1.5 of Ash and Doléans-Dade; Sec 1.3 and
More informationLecture I: Asymptotics for large GUE random matrices
Lecture I: Asymptotics for large GUE random matrices Steen Thorbjørnsen, University of Aarhus andom Matrices Definition. Let (Ω, F, P) be a probability space and let n be a positive integer. Then a random
More informationExercises Measure Theoretic Probability
Exercises Measure Theoretic Probability 2002-2003 Week 1 1. Prove the folloing statements. (a) The intersection of an arbitrary family of d-systems is again a d- system. (b) The intersection of an arbitrary
More informationST213 Mathematics of Random Events
ST213 Mathematics of Random Events Wilfrid S. Kendall version 1.0 28 April 1999 1. Introduction The main purpose of the course ST213 Mathematics of Random Events (which we will abbreviate to MoRE) is to
More informationINTRODUCTION TO REAL ANALYSIS II MATH 4332 BLECHER NOTES
INTRODUCTION TO REAL ANALYSIS II MATH 4332 BLECHER NOTES You will be expected to reread and digest these typed notes after class, line by line, trying to follow why the line is true, for example how it
More informationMeasures. 1 Introduction. These preliminary lecture notes are partly based on textbooks by Athreya and Lahiri, Capinski and Kopp, and Folland.
Measures These preliminary lecture notes are partly based on textbooks by Athreya and Lahiri, Capinski and Kopp, and Folland. 1 Introduction Our motivation for studying measure theory is to lay a foundation
More informationStatistics 612: L p spaces, metrics on spaces of probabilites, and connections to estimation
Statistics 62: L p spaces, metrics on spaces of probabilites, and connections to estimation Moulinath Banerjee December 6, 2006 L p spaces and Hilbert spaces We first formally define L p spaces. Consider
More informationPreliminaries. Probability space
Preliminaries This section revises some parts of Core A Probability, which are essential for this course, and lists some other mathematical facts to be used (without proof) in the following. Probability
More information