Prof. Thistleton MAT 505 Introduction to Probability Lecture 18

Size: px

Start display at page:

Download "Prof. Thistleton MAT 505 Introduction to Probability Lecture 18"

Janis Pierce
5 years ago
Views:

1 Prof. Thistleton MAT 505 Introduction to Probability Lecture Sections from Text and MIT Video Lecture: 6., 6.4, Topics from Syllabus: Jointly Distributed Random Variables, Conditional and Total Expectation Review and Looking Ahead We now know a fair amount about basic probability models (exponential, normal, Poisson, etc.) and their derived distributions. We even have a few practical (e.g probability plots) and theoretical (e.g. moment generating functions) tools. In the rest of the course we consider several random variables at a time, rather than just one at a time. This is important because we often, for example, try to predict a random outcome when we know some related quantity. For example, some variables are difficult or even impossible to measure directly but they may be related to other variables which are easier to measure. As an almost trivial example, we could try to predict your GPA across 4 years of college (a future event) by using your SAT score and your high school GPA. We now have 3 random variables in play. Surely they are related somehow. As another example, it is common to try and understand things like a person s job satisfaction with a paper and pencil survey instrument. You can take a look at a survey such as the Nursing Home Nurse Aide Job Satisfaction Questionnaire (NHNA-JSQ) (Castle, N.. Assessing Job Satisfaction of Nurse Aides in Nursing Homes. Journal of Gerontological Nursing, May, 2000, 4-47). This is a 2-question instrument with subscales measuring job satisfaction along several dimensions including: Satisfaction with Coworkers, Workplace Support, Work Content, Work- Load or Work Schedule, Training, Rewards, and a CNA s perceived Quality of Resident Care. Are these constructs related? SUNY POLY Page

2 Prof. Thistleton MAT 505 Introduction to Probability Lecture Random Vectors You get the idea: we may often associate more than one numerical measure with the outcome of an experiment. Technically, we consider multiple mappings from the sample space to the reals. For example, given a piece of steel we may be interested in its hardness, measured on some scale, or in its tensile strength (greatest longitudinal stress born before breaking apart). Maybe both! As another example, we may be interested to determine whether a relationship exists between educational level and annual income for 40 year old women. The underlying idea here is that, given a sample space, we may define more than one random variable on this sample space and consider these random variables together. So, let S be a sample space and let X and Y be random variables defined on S. We would call (X, Y) a 2-dimensional random vector, or a 2 dimensional random variable. We can also similarly define an n dimensional random vector as (X, X 2,, X n ) or even infinite dimensional random vectors. Start with the 2-d case. Let (X, Y) be a 2 dimensional random vector. Then the random vector (X, Y) is said to be discrete if it assumes a finite or a countably infinite number of values. With each possible outcome of (X, Y), say (, ) we associate a number, f(, ), which has the value f(, ) = P(X = and Y = ) Note that it is not necessarily true that P(X = and Y = ) = P(X = )P(Y = ). This is a special case called independence which will be discussed below. We will, however, say that f is a probability function (probability mass function) for some discrete random vector (X, Y) if. f(, ) 0 i, j 2. i j f(, ) = SUNY POLY Page 2

3 Prof. Thistleton MAT 505 Introduction to Probability Lecture A Trivial Example to Organize our Thoughts Suppose you toss a fair coin 3 times. Let the random variable X indicate how many HEADS you obtained on the first two tosses, and let Y indicate how many HEADS you obtained on the last two tosses. Calculate the joint probability mass function for (X, Y). I ve listed outcomes from S together with their associated probabilities. Use the classical notion. For instance, f(x = 2, Y = ) = f(2,) = f(x =, Y = ) = f(,) = 2 (from HHT) (from THT, HTH) X=number of heads on first two tosses 0 2 Y=number of heads on last two tosses 0 TTT, TTH, HTT, 2 0 THH, THT, HTH, 2 HHT, HHH, Another Example This example is from Meyer's Introductory Probability and Statistical Applications. In what follows, imagine yourself at a factory, and let the random variable X represent the number of items produced by Line I and let the random variable Y represent the number of items produced by Line II on a given day. SUNY POLY Page 3

4 Prof. Thistleton MAT 505 Introduction to Probability Lecture X=number of items from Line I Y= number of items from Line II Marginal Distributions Let (X, Y) be a 2 dimensional random vector with a joint probability mass function f(, ). We may define marginal distributions for X and Y with probability mass functions f X ( ) = f(, ) j f Y ( ) = f(, ) i For the marginal of X we sum across all possible Y values, and similarly for Y.. Calculate the probability that Line II produces exactly 2 items. Since the random variable Y counts production from Line II, we aggregate or accumulate across all of the possibilities on the random variable X. That is, we add along the row corresponding to the event Y = 2. This gives us 5 P(Y = 2) = f Y (2) = f(, 2) = 0.25 i=0 SUNY POLY Page 4

5 Prof. Thistleton MAT 505 Introduction to Probability Lecture 2. Calculate the probability that Line I produces exactly 4 items. This is very similar. 3 P(X = 4) = f X (4) = f(4, ) = = Calculate the probability that Line I produces more than Line II. The event in play here is X > Y. You could write this as j=0 f(, ) > = = Calculate the probability that total production exceeds 6. The event in play here Is now X + Y > 6. You could write this as f(, ) + >6 = = Calculate the probability distribution of Z X + Y We can make a table as follows. Just work off of the diagonals running from SW to NE. k p Z (k) = = SUNY POLY Page 5

6 Prof. Thistleton MAT 505 Introduction to Probability Lecture 6. Calculate the average amount produced by Line I. We will work the marginal of Line I X=number of items from Line I p( ) E[X] = = 3.39 Remember this number for later- we ll need it. Conditional Distributions Recall that Prob(A B) Prob(A B) P(B) We can use this idea with our random variables. In general, we calculate P(X = Y = ). Consider again the example from Meyer about the two production lines. Calculate. P( Y = 0 X = 2) Park yourself along the column X = 2 and work as you always do. P( Y = 0 X = 2) = P(Y = 0 X = 2 ) P(X = 2) = f(2,0) f X (2) = = P( Y = X = 2) P( Y = X = 2) = P(Y = X = 2 ) P(X = 2) = f(2,) f X (2) = =.25 SUNY POLY Page 6

7 Prof. Thistleton MAT 505 Introduction to Probability Lecture 3. P( Y = 2 X = 2) P( Y = 2 X = 2) = P(Y = 2 X = 2 ) P(X = 2) = f(2,2) f X (2) =. 6 = P( Y = 3 X = 2) P( Y = 3 X = 2) = P(Y = 3 X = 2 ) P(X = 2) = f(2,3) f X (2) = =.25 This idea allows us to define a new random variable: Work with Y parked somewhere this time. Given that Y has occurred with outcome Y =, define the random variable X Y = with probability mass function f X Y=yj ( ) f X,Y(, ) f Y ( ) You can remember this as joint over marginal. This really is a legitimate random variable. All the possible probabilities are non-negative, and the mass function sums to. This is easily seen with f X Y=yj ( ) f X,Y (, ) = = f Y ( ) f x Y ( ) f X,Y(, ) = f i x Y ( ) f Y( ) = i Conditional Expectation Since we have a random variable, it seems natural to consider what happens on average. Let your definitions guide your work and refer back to the Meyer example to make this more concrete. We define the conditional expectation of X given an outcome y of random variable Y as the sum of outcomes times their probabilities SUNY POLY Page 7

8 Prof. Thistleton MAT 505 Introduction to Probability Lecture E[X Y = ] = f X Y=yj ( ) Try to calculate what happens to the output of Line I if we know that line II has produced 3 items. Here is our new restricted universe: X=number of items from Line I Y= number of items from Line II f X Y=3 ( 3) E[X Y = 3] = f X Y=yj ( 3) = = So, if we know that Line II has produced 3 items we can say that, on average, Line I will produce around 3.2 items. SUNY POLY Page

9 Prof. Thistleton MAT 505 Introduction to Probability Lecture Something interesting happens when we look at the conditional expectation of X over all the possible Y values. That is, look at the average of X when Y=0, then with Y=, and so on. See if you can calculate (Excel really helps here!) each of the conditional expectations in the table below E[X Y = ] P(Y = ) f X Y=0 ( 0) f X Y= ( ) f X Y=2 ( 2) f X Y=3 ( 3) We can think of each of the conditional outcomes as an outcome in its own right and define a new random variable with outcomes given as the conditional expectations and associated probabilities coming from the events that we condition on. This is shown in the last two columns in the table. I wonder what the average of the conditional averages might be? We will form E[ E[X Y = ] ] TOTAL EXPECTATION = E[X Y = ] P(Y = ) Not to be too chatty here, but we can reproduce the table and take a sum: SUNY POLY Page 9

10 Prof. Thistleton MAT 505 Introduction to Probability Lecture E[X Y = ] P(Y = ) E[X Y = ] P(Y = ) Σ 3.39 This is an example of one of our most important results in action. This is the famous Total Expectation Theorem. Total Expectation Theorem E[ E[X Y = ] ] E[X Y = ] P(Y = ) = E[X] I wonder if we could close out this lecture by proving this celebrated result. It s more like unpacking notation and bookkeeping than complicated mathematics, so here goes: Form a random variable in a fairly natural way by associating the outcomes obtained as the conditional expectations of X on Y (i. e. E[X Y = ]) with the probabilities of the outcomes of Y ( i. e. P(Y = )). Then, by the definition of expected value E[ E[X Y = ] ] E[X Y = ] P(Y = ) SUNY POLY Page 0

11 Prof. Thistleton MAT 505 Introduction to Probability Lecture A quick substitution for the conditional expectation E[X Y = ] = f X Y=yj ( ) gives, as we note that P(Y = ) is another name for f Y ( ) and f X Y=yj ( ) f X,Y(, ) f Y ( ) E[ E[X Y = ] ] = f X Y=yj ( ) P(Y = ) f X,Y (, ) = f Y ( ) f Y ( ) Cancel and interchange the summations to be done. E[ E[X Y = ] ] = f X,Y (, ) = f X,Y (, ) = f X ( ) = E[X] SUNY POLY Page

Prof. Thistleton MAT 505 Introduction to Probability Lecture 13

Prof. Thistleton MAT 505 Introduction to Probability Lecture 13 Prof. Thistleton MAT 55 Introduction to Probability Lecture 3 Sections from Text and MIT Video Lecture: Sections 5.4, 5.6 http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-4- probabilisticsystems-analysis-and-applied-probability-fall-2/video-lectures/lecture-8-continuousrandomvariables/