4 Expectation & the Lebesgue Theorems

Transcription

1 STA 7: Probability & Measure Theory Robert L. Wolpert 4 Expectation & the Lebesgue Theorems Let X and {X n : n N} be random variables on the same probability space (Ω,F,P). If X n (ω) X(ω) for each ω Ω, does it follow that E[X n ] E[X]? That is, may we exchange expectation and limits in the equation lim E[X n] =? E [ ] lim X n? () In general, the answer is no. For a simple example take Ω = (0,], the unit interval, with Borel sets F = B(Ω) and Lebesgue measure P = λ, and for n N set X n (ω) := 2 n (0,2 n ](ω). (2) For each ω Ω, X n (ω) = 0 for all n > log 2 (/ω), so X n (ω) 0 as n for every ω, but E[X n ] = for all n. We will want to find conditions that allow us to compute expectations by taking limits, i.e., to force equality in Eqn(). The two most famous of these conditions are both attributed to Henri Lebesgue: the Monotone Convergence Theorem (MCT) and the Dominated Convergence Theorem (DCT). We will see stronger results later in the course but let s look at these two now. First, we have to define expectation. 4. Expectation Let E be the linear space of real-valued F-measurable random variables taking only finitelymany values (these are called simple), and let E + be the positive members of E. Each X E may be represented in the form X(ω) = k a j Aj (ω) j= for some k N, {a j } R and {A j } F. The representation is unique if we insist that the {a j } be distinct and that the {A j } form a partition i.e., be disjoint with Ω = j A j (why?) in which case X E + if and only if each a j 0. In general we will not need uniqueness of the representation, so don t demand that the {a j } be distinct nor that the {A j } be disjoint. We define the expectation for simple functions in the obvious way:

2 STA 7 Week 4 R L Wolpert EX := k a j P[A j ]. j= For this to be a definition we must verify that the right-hand side doesn t depend on the (non-unique) representation. That s easy you should do it. Now we extend the definition of expectation to all non-negative F-measurable random variables as follows: Definition The expectation of any nonnegative random variable Y 0 on (Ω,F,P) is The expectation can be evaluated using: Proposition EY := sup{ex : X E +, X Y}. EY = lim EX n for any simple sequence X n E + such that X n (ω) ր Y(ω) for each ω Ω. Proof. First let s check that such a sequence of simple random variables exists and that the limit makes sense. In a homework exercise you re asked to prove that X n := min ( 2 n, 2 n 2 n Y ) is simple and nonnegative, and increases monotonically to Y. Thus at least one such sequence exists. By monotonicity the expectations E[X n ] are increasing, so lime[x n ] = supe[x n ] is just their least upper bound and always exists in the extended positive reals R + = [0, ]. Now let s show that EX n for any such sequence converges to EY (note EY may be infinite). Fix any λ < EY and any ǫ > 0. By the definition of EY, find X E + with X Y and EX λ. Since X E takes only finitely many values, it must be bounded for all ω by 0 X B for some 0 < B <. Because X n X n+ and X n (ω) Y(ω) X (ω) as n for each ω Ω, the events A n := {ω : X n (ω) < X (ω) ǫ} are decreasing (i.e., A n A n+ ) with empty intersection A n =, so P[A n ] 0. Fix N ǫ large enough that P[A n ] ǫ/b for all n N ǫ. Then for n N ǫ, EX n = EX ǫ+e(x n X +ǫ) = EX ǫ+e(x n X +ǫ) An +E(X n X +ǫ) A c n EX ǫ+e(x n X +ǫ) An Page 2

3 STA 7 Week 4 R L Wolpert since (X n X +ǫ) 0 on A c n and, since (X n +ǫ) An 0, EX ǫ EX An. Since X B, EX An BP[A n ] ǫ and, for all n N ǫ, Thus sup n EX n λ 2ǫ. EX n EX ǫ BP[A n ] EX 2ǫ λ 2ǫ. Since this is true for all ǫ > 0, also sup n EX n λ; since that holds for all λ < EY, lim n EX n = sup n EX n EY as claimed. Now that we have EX well-defined for random variables X 0 we may extend the definition of expectation to all (not necessarily non-negative) RVs X by EX := EX + EX as long as either of the nonnegative random variables X + := (X 0), X := ( X 0) has finite expectation. If both EX + and EX are infinite, we must leave EX undefined. If both are finite, call X integrable and note that EX EX + +EX = E X. 4.. Examples Let Ω = N 0 := {0,,...} with F = 2 Ω and probability measure determined by P[{ω}] = 2 ω, ω Ω. The random variable ζ(ω) = ω has the geometric distribution ζ Ge(/2) with P[ζ = n] = 2 n, but we ll be interested in the random variables Y(ω) := 2 ω and X n := Y {ω<n}. Then Y 0 and X n E + with X n ր Y as n, so and, by Prop., n EX n = 2 ω P[{ω}] = n/2, ω=0 EY = lim EX n =. The distribution of Y has a colorful history. Known as the St. Petersburg Paradox, it led to the invention of idea of utility in decision theory. The random variable Z(ω) := ( 2) ω is well-defined and finite, but does not have an expectation because EZ + = EZ =. Page 3

4 STA 7 Week 4 R L Wolpert 4..2 Properties of Expectation Expectation is a linear operation in the sense that, if a,a 2 R are two constants and X,X 2 are two random variables on (Ω,F,P), then E[a X +a 2 X 2 ] = a E[X ]+a 2 E[X 2 ] provided the right-hand side is well-defined (not of the form ). It follows that expectation respects monotonicity, in the sense that X X 2 E[X ] E[X 2 ] and, as special cases, that E[X] E [ X ] and X 0 E[X] 0. Wewill encounter many more identities and inequalities for expectations in Section(5). Expectation is unaffected by changes on null-sets if P[X Y] = 0, then EX = EY. How would you prove this? 4..3 A Small Extension The definition of expectation extends without change to random variables X that take values in the extended real numbers R := [, ]. Obviously EX = + if P[X = + ] > 0 and P[X = ] = 0, EX = if P[X = + ] = 0 and P[X = ] > 0, and EX is undefined if both P[X = + ] > 0 and P[X = ] > 0. Otherwise, if P[ X = ] = 0, then X (and any function of X) have the same expectation as if X were replaced by the real-valued RV X defined to be X(ω) when X(ω) < and otherwise zero, since then P[X X ] = 0. With this extension, we can always consider the expectations of quantities like limsupx n and liminfx n, which might take on the values ± for some RV sequences {X n } Lebesgue Summability Counterexample Does the alternating sum = k N ( ) k+ k (3) converge? Let s look closely the answer depends on what you mean by converge. First, define n n S(n) = k log(n) = x dx. By summing k= k < x < k + k + < x < k k + < k+ from k = to n, and from k = 2 to n, note that for all n N log(n+) < S(n) log(n)+. Page 4 k x dx < k,

5 STA 7 Week 4 R L Wolpert Thus the harmonic series S(n) = n k= k logn. In fact [S(n) logn] converges as n to a finite limit, the Euler-Mascheroni constant γ e Thus in the Lebesgue sense, the alternating series of Eqn(3) does not converge, since its negative and positive parts S (n) := n/2 j= 2j S + (n) := n/2 j= 2j = 2 S(n/2) = S(n) 2 S(n/2) = 2 [log(n/2)+γ e]+o() = 2 [log(2n)+γ e]+o() each approach as n. Notice however that the even partial sums are 2n ( ( ) k+ = k ) ( ) ( ) n + = 6 (2j )(2j), k= bounded above by π 2 /8 for all n (why?), making the example interesting. More precisely, the difference n ( ) k+ = S + (n) S (n) = ] log(2n) log(n/2) +o() k 2[ k= converges to log2 as n. What do you think happens with n k= ξ k/n, for independent random variables ξ k = ± with probability /2 each? j= 4.2 Lebesgue s Convergence Theorems Theorem (MCT) Let X and X n 0 be random variables (not necessarily simple) for which X n (ω) ր X(ω) for each 2 ω Ω. Then [ ] lim E[X n] = EX = E lim X n, i.e., Eqn() is satisfied. If E X <, then also E X n X 0. For the proof we must find for each n an approximating sequence Y n (m) ր X n as m and, from it, construct a single sequence Y (m) n Z m := max n m Y (m) n E + E + such that The little oh notation o() means that any remaining terms converge to zero as n. More generally, f = o(g) means that ( ǫ > 0)( N ǫ < )( x > N ǫ ) f(x) ǫg(x) roughly, that f(x)/g(x) 0. 2 In fact it is enough to assume that P[X n 0] = and P[X n ր X] =, i.e., that X n are nonnegative and increase to X outside of a null set N F, since X n N c and X N c have the same expectations as X n and X. Page 5

6 STA 7 Week 4 R L Wolpert that satisfies Z m X m for each m (this is true because, for each n m, Y n (m) X n X m ) and Z m ր X as m (to see this, take ω Ω and ǫ > 0; first find n such that X n (ω) X(ω) ǫ, then find m n such that Y n (m) (ω) X n (ω) ǫ, and verify that Z m (ω) X(ω) 2ǫ), and verify that lim E[X n] lim E[Z m] = EX lim E[X n ]. m Theorem 2 (Fatou s Lemma) Let X n 0 be random variables. Then [ ] E lim inf X n liminf E[X n]. To prove this, just set Y n := inf m n X m. Then Y n Y := liminfx n by definition, and {Y n } is increasing, so the MCT and the inequality Y n X n give [ ] [ ] E lim inf X n := E lim Y n = E[Y] = liminf E[Y n] liminf E[X n] Notice that equality may fail, as in the example of Eqn(2). The condition X n 0 isn t entirely superfluous, but it can be weakened to X n Z for any integrable random variable Z (i.e., one with E Z < ). For indicator random variables X n := An of events {A n }, since EX n = P[A n ], Fatou s lemma asserts that ( ) ( ) P lim inf A n liminf P[A n] limsupp[a n ] P lim supa n Corollary Let {X n }, Z be random variables on (Ω,F,P) with X n Z and E Z <. Then [ ] E lim inf X n liminf E[X n]. That is, we may weaken the condition X n 0 to X n Z L in the statement of Fatou s lemma. To prove this, apply Fatou to (X n Z) and add EZ to both sides. Corollary 2 Let {X n }, Z be random variables on (Ω,F,P) with X n Z and E Z <. Then [ ] E lim supx n limsupe[x n ]. To prove this, use the identity (limsupa n ) = liminf( a n ) (true for any real numbers {a n }) and apply Fatou s lemma to the nonnegative sequence (Z X n ). Finally we have the most important result of this section: Page 6

7 STA 7 Week 4 R L Wolpert Theorem 3 (DCT) Let X and X n be random variables (not necessarily simple or positive) for which P[X n X] =, and suppose that P [ X n Y ] = for some integrable random variable Y with EY <. Then lim E[X n] = EX = E [ lim X n i.e., Eqn() is satisfied if {X n } is dominated by Y L. Moreover, E X n X 0. Proof. To show this just apply Fatou Corollaries and 2 with Z = Y and Z = Y, respectively: EX = E[liminfX n ] liminfe[x n ] ], limsupe[x n ] E[limsupX n ] = EX For the moreover part, apply DCT separately to the positive and negative parts of X, (X n X) + := 0 (X n X) and (X n X) := 0 (X X n ); each is dominated by 2Y and converges to zero as n. Then use E X n X = E(X n X) + +E(X n X) 0. We will see later that the condition P[X n X] =, known as almost sure convergence, can be weakened to convergence in probability: ( ǫ > 0) P[ X n X > ǫ] 0. The domination condition in the DCT can be weakened too, and the MCT positivity condition X n 0 can be weakened to X n Z for some RV Z with E Z <. Counter-examples For both examples, let (Ω,F,P) be the unit interval Ω = (0,] with the Borel sets and Lebesgue measure. Undominated, No convergence The sequence {X n (ω) := 2 n (0,2 n ](ω)} of Eqn(2) does not satisfy equation Eqn(), so theremust not exist a dominating Y with X n Y ande Y <. The smallest dominating function is Y := sup n = n 0X 2 n (2 n,2 n ] n 0 whose expectation is EY = n 0 2 n (2 n 2 n ) = n 02 =. Page 7

8 STA 7 Week 4 R L Wolpert This can also be seen from the relation so EY 0 dω =. 2ω Undominated, Convergence 2ω Y < ω, Now consider the sequence {Y n (ω) := n ( n+, ] on the same (Ω,F,P). Again there is no n dominatation by an integrable RV, since the smallest dominating RV Y := sup n = n NY n = n NY n ( n+, ], n n N has expectation EY := n Nn [ n ] = n+ n+ =. n N Still, Y n (ω) 0 for every ω Ω and EY n = 0. This shows that domination is n+ sufficient but not necessary to ensure that equality holds in Eqn(). Last edited: September 20, 207 Page 8