Lecture 21: Expectation of CRVs, Fatou s Lemma and DCT Integration of Continuous Random Variables

Transcription

1 EE50: Probability Foundations for Electrical Engineers July-November 205 Lecture 2: Expectation of CRVs, Fatou s Lemma and DCT Lecturer: Krishna Jagannathan Scribe: Jainam Doshi In the present lecture, we will cover the following three topics: Integration of Continous Random Variables Fatou s Lemma Dominated Convergence Theorem (DCT) 2. Integration of Continuous Random Variables Theorem 2. Consider a probability space (, F, P). Let X : R be a continuous random variable. Let g be a measurable function which is either non-negative or satisfies g dp X <.Then, Eg(X) gf X dλ. In particular, if g(x) x, i.e. the identity map, we have EX xf X dλ. Proof: Let us first consider the case of g being a simple function i.e. g K a i I Ai disjoint subsets A i over the real line. We then have for some measurable Eg(X) g dp X f X dλ From Radon-Nikodym Theorem i X i a i P X (A i ) g is a simple function a i A i a f dλ a is a constant A i (a i I Ai f X )dλ I Ai is the indicator random variable of event A i 2-

2 2-2 Lecture 2: Expectation of CRVs, Fatou s Lemma and DCT (a i I Ai f X )dλ Interchanging finite summation and integral ( K ) (a i I Ai ) f X dλ (gf X )dλ. Thus we have proved the above theorem for simple functions. We now assume g to be a non-negative measurable function which may not necessarily be simple. Let g n be an increasing sequence of non-negative simple functions that converge to g point wise. One way of coming up with such a sequence was discussed in the previous lecture. We then have, Eg(X) lim g n dp X From MCT lim g n f X dλ From result for simple functions gf X dλ. From MCT, since g n f X gf X For arbitrary g which are absolutely integrable, a similar proof can be worked out by writing g g + g and proceeding. Example : Let X be an exponential random variable with parameter µ. Find EX and EX 2. Solution: Recall that for an exponential random variable with parameter µ, f X (x) µe µx. Thus, we have EX xf X dλ xµe µx dx µ. 0 EX 2 x 2 f X dλ x 2 µe µx dx 2 µ 2. 0 Example 2: Let X N (µ, σ 2 ). Find EX and EX 2. Solution: Recall that the density of X is given by f X (x) σ (x µ) 2 e 2σ 2. Thus, we have 2π EX xf X dλ x σ (x µ) 2 2π e 2σ 2 dx µ. EX 2 x 2 f X dλ x 2 σ (x µ) 2 2π e 2σ 2 dx µ 2 + σ 2.

3 Lecture 2: Expectation of CRVs, Fatou s Lemma and DCT 2-3 Example 3: Let X be a one-sided Cauchy random variable i.e. f X (x) 2 π +x for x 0. Find EX. 2 Solution: We have EX xf X dλ x 2 dx. π + x2 0 Example 4: Let X be a two-sided Cauchy random variable i.e., f X (x) π +x for x R. Find EX. 2 Solution: In this case the random variable X takes both positive and negative values. Hence, we need to find EX + and EX seperately and then evaluate EX EX + EX. Recall that X + max(x, 0) and X min(x, 0). Thus, Similarly, X + (ω) 0 for ω A {ω X(ω) < 0}, X + (ω) X(ω) for ω A c. X (ω) 0 for ω B {ω X(ω) > 0}, X (ω) X(ω) for ω B c. It is easy to see that P(A) P(B) 0.5. Next, we have EX + xdp X+ Similarly, EX + 0 P(A) + EX xdp X 0 x π dx. + x2 EX 0 P(B) + 0 Thus, we have a case of and EX is undefined. x π dx. + x2 Note that in Example 2 also, X takes both positive and negative values and we should find EX + and EX seperately and evaluate EX EX + EX. But in that case both EX + and EX are finite, allowing us to integrate with respect to the pdf f X (x) from to directly. Note: For the two sided Cauchy, dx lim π + x2 M 2 M M 2 M The above limit does not exist and hence the integral is not defined. π + x 2 dx. 2.2 Fatou s Lemma Before we state Fatou s lemma, let us motivate it with an elementary result.

4 2-4 Lecture 2: Expectation of CRVs, Fatou s Lemma and DCT Lemma 2.2 Let X and Y be random variables. Then, E min(x, Y ) min (EX, EY ). E max(x, Y ) max (EX, EY ). Proof: By definition, we have min(x, Y ) X. min(x, Y ) Y. Taking expectations on both the sides, E min(x, Y ) EX. E min(x, Y ) EY. Combining the above two equations, we get E min(x, Y ) min (EX, EY ). The other statment of the lemma involving maximum of X and Y can be proved in a similar way and is left to the reader as an exercise. The above lemma can be generalized to any finite collection of random variables and a similar proof can be worked out. Fatou s Lemma generalizes this idea for a sequence of random variables. Lemma 2.3 Fatou s Lemma: Let Y be a random variable that satisfies E Y <. Then the following holds, If Y X n, for all n, then E lim inf X n lim inf If Y X n, for all n, then E lim sup EX n. X n lim sup EX n. Proof: Let us start by proving the first statement. For some n we have inf X k Y X m Y, m n. k n Taking expectations, E inf X k Y EX m Y, m n. k n Taking infimum with respect to m on R.H.S, we obtain E inf X k Y inf EX m Y, m n. k n m n Let Z n inf k n X k Y. Note that Z n 0 since X m Y m and Z n is a non-decreasing sequence of random variables.

5 Lecture 2: Expectation of CRVs, Fatou s Lemma and DCT 2-5 Also, Z lim Z n lim inf X n Y. By MCT, we have E lim inf X n Y lim inf EX n Y. As E Y <, we can invoke linearity of expectation to get the first result of Fatou s lemma. The second statement can be proved similarly and is left to the reader as an exercise. 2.3 Dominated Convergence Theorem The DCT is an important result which asserts a sufficient condition under which we can interchange a limit and integral. Theorem 2.4 Consider a sequence of random variables X n that converges almost surely to X. Suppose there exists a random variable Y such that X n Y almost surely for all n and EY <. Then, we have lim EX n EX. Proof: We have X n Y which implies Y X n Y. We can now apply Fatou s lemma to obtain EX E lim inf X n lim inf EX n lim sup EX n E lim sup X n EX. Thus, all the inequalities in the above equation must be met with equalities and we have EX lim inf EX n lim sup EX n, which proves that the limit, lim EX n exists and is given by lim EX n EX. Thus we see that Dominated Convergence theorem (DCT) is a direct consequence of Fatou s Lemma. The name dominated is intuitive because we need X n to be bounded by some random variable Y almost surely for every n. However, we do not require X n s to be monotonically increasing as in the case of MCT. Corollary 2.5 A special case of DCT is known as Bounded Convergence theorem (BCT). Here, the random variable Y is taken to be a constant random variable. BCT states that if there exists a constant c R such that X n c almost surely for all n, then lim EX n EX.

6 2-6 Lecture 2: Expectation of CRVs, Fatou s Lemma and DCT 2.4 Exercise. MIT OCW problem set A workstation consists of three machines, M, M 2 and M 3, each of which will fail after an amount of time T i which is an independent exponentially distributed random variable, with parameter. Assume that the times to failure of the different machines are independent. The workstation fails as soon as both of the following have happened: (a) Machine M has failed. (b) Atleast one of the machines M 2 or M 3 has failed. Find the expected value of the time to failure of the workstation. 2. Assignment problem, University of Cambridge Let Z be an exponential random variable with parameter λ and Z int Z. Compute EZ int. 3. Prof. Pollak, Purdue University Suppose S k and S n are the prices of a financial instrument on days k and n, respectively. For k < n, the gross return G k,n between days k and n is defined as G k,n Sn S k and is equal to the amount of money you would have on day n if you invested $ on day k. Let G k,k+ be lognormal random variable with parameters µ and σ 2, k, and the random variables G j,j+ and G k,k+ are independent and identically distributed k j. Find the expected total gross return from day to day n.