Chapter 6 Expectation and Conditional Expectation. Lectures Definition 6.1. Two random variables defined on a probability space are said to be

Transcription

1 Chapter 6 Expectation and Conditional Expectation Lectures In this chapter, we introduce expected value or the mean of a random variable. First we define expectation for discrete random variables and then for general random variable. Finally we introduce the notion of conditional expectations using conditional probabilities. Definition 6.1. Two random variables defined on a probability space are said to be equal almost surely (in short a.s.) if Now we give a useful characterization of discrete random variables. Theorem Let be a discrete random variable defined on a probability space. Then there exists a partition of and such that where may be. Proof. Let be the distribution function of. Let be the set of all discontinuities of. Here may be. Since is discrete, we have Set Then is pairwise disjoint and Now define Then is a partition of and

2 Remark If is a discrete random variable on a probability space, then the 'effective' range of is at the most countable. Here 'effective' range means those values taken by which has positive probability. This leads to the name 'discrete' random variable. Remark If that is a discrete random variable, then one can assume without the loss of generality Since if, then set and for. Theorem Let be such that is a countable partition of. Then if then Proof. For each, set Then clearly Also if then. Therefore This completes the proof. Definition 6.2. Let be a discrete random variable represented by. Then expectation of denoted by is defined as

3 provided the right hand side series converges absolutely. Remark In view of Remark , if has range, then Example Let be a Bernoulli( ) random variable. Then Example Let be a Binomial random variable. Then Here we used the identity Example Let be a Poisson ( ) random variable. Then Example Let be a Geometric ( ) random variable. Then

4 Theorem (Properties of expectation) Let and be discrete random variables with finite means. Then (i) If, then. (ii) For Proof. (i) Let be a representation of. Then implies for all. (ii) Let has a representation. Now by setting one can use same partition for and. Therefore Definition 6.3. (Simple random variable) A random variable is said to be simple if it is discrete and the distribution function has only finitely many discontinuities. Theorem Let be random variable in such that, then there exists a sequence of simple random variables satisfying (i) For each,.

5 (ii) For each as. Proof. For, define simple random variable as follows: Then 's satisfies the following: Lemma Let be a non negative random variable and be a sequence of simple random variables satisfying (i) and (ii) of Theorem Then exists and is given by Proof. Since, we have (see exercise). exists. Also since 's are simple, clearly, Therefore to complete the proof, it suffices to show that for simple and, Let where is a partition of. Fix, set for and, Since, we have for each. Also

6 From the definition of we have (6.0.1) Using continuity property of probability, we have Now let, in (6.0.1), we get Since, is arbitrary, we get This completes the proof. Definition 6.4. The expectation of a non negative random variable is defined as (6.0.2) where is a sequence of simple random variables as in Theorem Remark One can define expectation of, non negative random variable, as But we use Definition 6.4., since it is more handy. Theorem Let be a continuous non negative random variable with pdf. Then Proof. By using the simple functions given in the proof of Theorem , we get (6.0.3)

7 where is the point given by the mean value theorem. Definition 6.5. Let be a random variable on. The mean or expectation of is said to exists if either or is finite. In this case is defined as where Note that is the positive part and is the negetive part of Theorem Let be a continuous random variable with finite mean and pdf. Then Proof. Set

8 Then is a sequence of simple random variables such that Similarly, set Then Now (6.0.4) and (6.0.5) The last equality follows by the arguments from the proof of Theorem Combining (6.0.4) and (6.0.4), we get Now as in the proof of Theorem , we complete the proof. We state the following useful properties of expectation. The proof follows by approximation argument using the corresponding properties of simple random variables Theorem Let be random variables with finite mean. Then (i) If, then.

9 (ii) For, (iii) Let be a random variable such that. Then has finite mean and. In the context of Riemann integration, one can recall the following convergence theorem. `` If is a sequence of continuous functions defined on the such that uniformly in, then i.e., to take limit inside the integral, one need uniform convergence of functions. In many situations in it is highly unlikely to get uniform convergence. In fact uniform convergence is not required to take limit inside an integral. This is illustrated in the following couple of theorem. The proof of them are beyond the scope of this course. Theorem (Monotone convergence theorem) Let be an increasing sequence of nonnegative random variables such that. Then [Here means.] Theorem (Dominated Convergence Theorem) Let be random variables such that (i) has finite mean. (ii) (iii) Then Definition 6.6. (Higher Order Moments) Let be a random variable. Then is called the th moment of and is called the th central moment of. The second central moment is called the variance. Now we state the following theorem whose proof is beyond the scope of this course. Theorem Let be a continuous random variable with pdf and be a continuous function such that the integral is finite. Then The above theorem is generally referred as the ``Law of unconscious statistician'' since often users treat the

10 above as a definition itself. Now we define conditional expectation denoted by E[Y X] of the random variable Y given the information about the random variable X. If Y is a Bernoulli (p) random variable and X any discrete random variable, then we expect E[Y X = x] to be P{Y = 1 X = x}, since we know that EY = p = P{Y = 1}. i.e., Where is the conditional pmf of Y given X. Now since we expect conditional expectation to be liner and any discrete random variable can be written as a liner combination of Bernoulli random variable we get the following definition. Definition 6.7. Let are discrete random variable with conditional pmf. Then conditional expectation of given is defined as Example Let be independent random variables with geometric distribution of parameter Set. Calculate, where For Now Therefore i.e., Now

11 When X and Y are discrete random variable. E[Y X] is defined using conditional pmf of Y given X. we define E[Y X] when X and Y are continuous random variable with joint pdf f in a similar way as follows. Definition 6.8. Let be continuous random variable with conditional pdf. Then conditional expectation of given is defined as Remark One can extend the defination of E[Y X] when X is any random variable (discrete, continuous or mixed) and Y is a any random variable with finite mean. But it is beyound the scope of this course. Theorem (i) Let be discrete random variables with joint pmf, marginal pmfs and respectively. Then if has finite mean, then (ii) Let be continuous random variables with joint pdf, marginal pdfs and respectively. Then if has finite mean, then Proof. We only prove (ii). Example Let be continuous random variables with joint pdf given by Find and hence calculate. Note that and elsewhere. for,

12 Also elsewhere. Therefore