JOINT PROBABILITY DISTRIBUTIONS

Size: px

Start display at page:

Download "JOINT PROBABILITY DISTRIBUTIONS"

Patricia Ford
6 years ago
Views:

1 MTH/STA 56 JOINT PROBABILITY DISTRIBUTIONS The study of random variables in the previous chapters was restricted to the idea that a random variable associates a single real number with each possible outcome of an experiment. Subsequently, probability distribution is de ned for the one-dimensional random variable, based upon the probability measure for the experiment. Now we equally well de ne rules that would associate two numbers, three numbers, or n numbers with possible outcomes of an experiment. These rules actually de ne two-dimensional, three-dimensional, or for the general case, n-dimensional random variables (or random vectors). Again, the probability distribution for the n-dimensional random variable (random vector) derives directly from the probability measure for the experiment. As an example, consider an experiment of tossing a fair coin and a balanced die together once. Let Y be an binary variable that takes on the value if the coin turns up to be a head; and the value otherwise. Also, let Y be the number of dots appearing on the die. The sample space associated with the experiment is S f(y ; y ) j y ; and y ; ; ; ; 5; 6g : Assume that all sample points are equally likely. Then the random variable Y and Y associates two numbers with each possible outcome of the experiment as follows: Let y and y denote the realized values of Y and Y, respectively, we then have Sample Point (y ; y ) Sample Point (y ; y ) (Head; ) (; ) (Head; ) (; ) (T ail; ) (; ) (T ail; ) (; ) (Head; ) (; ) (Head; 5) (; 5) (T ail; ) (; ) (T ail; 5) (; 5) (Head; ) (; ) (Head; 6) (; 6) (T ail; ) (; ) (T ail; 6) (; 6) and, accordingly, the bivariate probability distribution for Y and Y is given by p (y ; y ) P fy y ; Y y g for y ; and y ; ; ; ; 5; 6. It is also clear that P fy ; Y 5g p (; ) + p (; ) + p (; 5) : It should noted here that the vector (Y ; Y ) thus de ned can be thought of as a rule that associates an ordered pair (y ; y ) of realized values with each sample point of the sample

2 space S, and is referred to as a two-dimensional random vector. Since (Y ; Y ) is a twodimensional random vector, we can visualize the possible observed values (y ; y ) as being points in a two-dimensional space and an event is a collection of points or a region in the two-dimensional space. In general, an n-dimensional random vector may be de ned as follows. De nition. Let S be the sample space of an experiment. An n-dimensional random vector (Y ; Y ; ; Y n ) is a rule (or, equivalently, an ordered collection of n rules) that associates an n-tuple with each sample point of S. We will equivalently say that Y, Y,, Y n are jointly distributed random variables. If (Y ; Y ; ; Y n ) is an n-dimensional random vector, we can visualize the possible observed values (y ; y ; ; y n ) as being points in an n-dimensional space. For an n-dimensional random vector, events are collections of points or some regions in the n-dimensional space. I. Joint Probability Distributions for Discrete Random Variables Based upon the discussions in the preceding paragraphs, it is natural to de ne the bivariate probability distribution for discrete random variables Y and Y as follows. De nition. The function p (y ; y ) is a bivariate probability distribution for discrete random variables Y and Y if the following properties holds: () p (y ; y ) for all real numbers y and y. () X X p (y ; y ), where the sum ranges over all values (y ; y ) that are assigned y y nonzero probabilities. () p (y ; y ) P fy y ; Y y g. () For any region A in the two-dimensional space, P f(y ; Y ) Ag X X p (y ; y ) : (y ;y )A Below are examples with regard to the bivariate probability distribution for two discrete random variables. Example. Suppose that random variables Y and Y have the joint probability distribution given by / y +y for y p (y ; y ) ; ; ; and y ; ; elsewhere Clearly, X X y y y +y X y y y X y ;

3 where the two summations in the second slot of the above equation are geometric series with ratio. Example. A local supermarket has three checkout counters. Two customers, say A and B, arrive at the counters at di erent times when the counters are serving no other customers. Each customer chooses a counter at random, independently of the other. Let Y be the number of customers who choose counter, and Y the number of customers who choose counter. The all possible arrivals of the two customers are listed in the following table: Counter Counter Counter (y ; y ) A B (; ) A B (; ) A B (; ) B A (; ) B A (; ) B A (; ) AB (; ) AB (; ) AB (; ) Then the bivariate probability distribution for Y and Y is given by y y Example. Consider an experiment of drawing two marbles at random from an urn which contains blue marbles, red marbles, and green marbles. Let Y be the number of blue marbles drawn, and Y the number of red marbles drawn. Then (Y ; Y ) takes on values (y ; y ) with y,,, y,,, and y + y. The total number 8 of equally likely ways of drawing any two marbles from the eight is 8; and the number of ways of drawing y blue marbles, y red marbles, and y y green marbles is. Hence, the bivariate probability distribution for Y and Y is y y y y given by p (y ; y ) y y 8 y y for y,,, y,,, and y + y ; that is, (y ; y ) (; ), (; ), (; ), (; ), (; ), (; ). More speci cally, p (y ; y ) is given by

4 y y Hence, the probability that at least one green marble will be drawn is P fy + Y g p (; ) + p (; ) + p (; ) : When (Y ; Y ; ; Y n ) is a discrete n-dimensional random vector; that is, Y, Y,, Y n are n discrete (one-dimensional) random variables, the corresponding joint probability function, or joint probability distribution, of Y, Y,, Y n may be speci ed by p (y ; y ; ; y n ) P fy y ; Y y ; ; Y n y n g which gives the probability of occurrence of individual points (in the n-dimensional space); as in the one-dimensional case, the probability of any event is computed by summing the probabilities of occurrence of individual points that belong to the event of interest. Evidently, the joint probability function of Y, Y,, Y n must satisfy properties in the following de nition: De nition. The function p (y ; y ; ; y n ) is a joint probability distribution for discrete random variables Y, Y,, Y n if the following conditions hold: () p (y ; y ; ; y n ) for all real numbers y, y,, y n. () X X X p (y ; y ; ; y n ), where the sum ranges over all values (y ; y ; ; y n ) y y y n that are assigned nonzero probabilities. () p (y ; y ; ; y n ) P fy y ; Y y ; ; Y n y n g. () For any region A in the n-dimensional space, P f(y ; Y ; ; Y n ) Ag X X X p (y ; y ; ; y n ) : (y ;y ; ;y n)a II. Joint Distribution Functions Remember that the one-dimensional continuous random variable and its univariate probability density function are de ned through its univariate distribution function. In a similar fashion, an n-dimensional random vector and its joint probability density function can also be derived from its joint distribution function. With this in mind, it is necessary to de ne the joint distribution function for the n-dimensional random vector. To do so, let us rst

5 de ne the bivariate distribution function (which works for both the discrete and continuous cases) and, subsequently, its bivariate probability density function for the continuous case only. Naturally this bivariate distribution function can be readily extended to be the joint distribution function for an n-dimensional random vector. In analogue to the distribution function F (y) P fy yg for a one-dimensional random variable Y, the bivariate distribution function for any two random variables Y and Y (discrete or continuous) can be easily de ned as follows: De nition. For any two random variables Y and Y, the bivariate distribution function F (y ; y ) is given by for < y < and < y <. F (y ; y ) P fy y ; Y y g Likewise, the joint distribution function random variables Y, Y,, Y n (discrete or continuous) is de ned in a similar form as follows: De nition 5. The joint distribution function F (y ; y ; ; y n ) of random variables Y, Y,, Y n is given by F (y ; y ; ; y n ) P fy y ; Y y ; ; Y n y n g for < y i < (i ; ; ; n). It seems to be relatively easier to evaluate the bivariate distribution function for the discrete case than for the continuous case. Typical evaluation of such distribution functions for the discrete case relies upon counting possible discrete pairs (y ; y ) of realized values of the random vector (Y ; Y ) as demonstrated in the example below. Example. Refer to the coin-die-tossing example, we have F (; ) P fy ; Y g p (; ) + p (; ) + p (; ) + p (; ) + p (; ) + p (; ) 6 : Refer to Example, we obtain F ( ; ) P fy ; Y g P () and F (:5; ) P fy :5; Y g p (; ) + p (; ) + p (; ) + p (; ) + p (; ) + p (; ) : 5

6 Refer to Example, we get F (:5; :6) P fy :5; Y :6g p (; ) + p (; ) + p (; ) + p (; ) + p (; ) + p (; ) : As seen in the distribution function for one-dimensional random variable, is possible to show that the distribution function F (y ; y ; ; y n ) is nondecreasing and continuous at least from the right with respect to every variable. Since fy < g \ fy < y g \ \ fy n < y n g fy < y g \ fy < g \ \ fy n < y n g... fy < y g \ fy < y g \ \ fy n < g are impossible events, it is clear that F ( ; y ; ; y n ) F (y ; ; ; y n ) F (y ; y ; ; ) : Moreover, since f < Y < g \ f < Y < g \ \ f < Y n < g S; where S is the sample space, it satis es the equality F (; ; ; ) P f < Y < ; < Y < ; ; < Y n < g P (S) : For a two-dimensional random vector (Y ; Y ), it should be noticed that we also have the following equality: P fa < Y a ; b < Y b g P fy a ; Y b g P fy a ; Y b g as illustrated in the gure below. P fy a ; Y b g + P fy a ; Y b g F (a ; b ) F (a ; b ) F (a ; b ) + F (a ; b ) 6

7 A good demonstration for the above result is laid out in the following example. Example 5. Refer to Example, it is clear that the event f < Y ; < Y g contains only one possible pair (; ) and so P f < Y ; < Y g p (; ) On the other hand, it follows from the above formula that where P f < Y ; < Y g F (; ) F (; ) F (; ) + F (; ) F (; ) p (; ) + p (; ) + p (; ) + p (; ) + p (; ) + p (; ) F (; ) p (; ) + p (; ) + p (; ) F (; ) p (; ) + p (; ) + p (; ) + p (; ) F (; ) p (; ) + p (; ) : In contrast to the one-dimensional distribution function, in order that the function F (y ; y ) be the distribution function of a two-dimensional random vector, it is not suf- cient that this function be continuous from the right, nondecreasing with respect to each of the variables, and satisfy the following conditions: To see this, consider the function F ( ; y ) F (y ; ) and F (; ) : F (y ; y ) for y + y < for y + y, that is, the function F (y ; y ) takes on the value for the points on and above the line y y, and the value for the points below the line. This function is nondecreasing, continuous from the right with respect to y and y, and F ( ; y ) F (y ; ) and F (; ) : However, it does not satisfy the equality P fa < Y a ; b < Y b g F (a ; b ) F (a ; b ) F (a ; b ) + F (a ; b ) : 7

8 For instance, P f < Y ; < Y g F (; ) F (; ) F ( ; ) + F ( ; ) + < : Remark. A real-valued function F (y ; y ) is a distribution function of a two-dimensional random vector if and only if the following conditions hold: () F (y ; y ) is nondecreasing and continuous at least from the right with respect to both arguments y and y. () F ( ; ) F (y ; ) F ( ; y ) and F (; ). () If a a and b b, then P fa < Y a ; b < Y b g F (a ; b ) F (a ; b ) F (a ; b ) + F (a ; b ) : We shall mainly consider multi-dimensional random vectors of the discrete or continuous type. De nition 6. A two-dimensional random vector (Y ; Y ) is said to be discrete if, with probability, it takes on pairs of values belonging to a set A of pairs that is at most countable, and every pair (a; b) is taken with positive probability P fy a; Y bg. We call theses pairs of values jump points, the their probabilities jumps. The joint distribution function F (y ; y ) of two discrete random variables Y and Y is given by F (y ; y ) X X p (t ; t ) t y t y and the joint probability function p (y ; y ) is de ned to be p (y ; y ) P fy y ; Y y g : In the discrete case, the joint distribution function for random variables Y, Y,, Y n is given by F (y ; y ; ; y n ) X X X p (t ; t ; ; t n ) t y t ny n t y and the joint probability function p (y ; y ; ; y n ) is de ned to be p (y ; y ; ; y n ) P fy y ; Y y ; ; Y n y n g : 8

9 III. Joint Probability Density Functions for Continuous Random Variables As seen in the univariate continuous case, two random variables Y and Y are said to be jointly continuous if their joint distribution function F (y ; y ) is continuous in both arguments. We now formally de ne the notion of a two-dimensional random vector of the continuous type. De nition 7. A two-dimensional random vector (Y ; Y ) is said to be continuous if there exists a nonnegative function f (y ; y ) such that F (y ; y ) y y f (t ; t ) dt dt for all pairs (y ; y ) of real numbers, where F (y ; y ) is the joint distribution function of Y and Y. The function f (y ; y ) is called the joint probability density function or bivariate probability density function for continuous random variables Y and Y. Consider a continuous two-dimensional random vector (Y ; Y ). The corresponding bivariate probability density function, f (y ; y ), for Y and Y is proportional to the probability that the random vector is equal to the argument (y ; y ); that is, P fy Y y + y ; y Y y + y g f (y ; y ) y y : Let A be any event (a region in the two-dimensional space). To evaluate the probability of any event A, we simply integrate the density function over the region de ned by A. At the continuity points of (y ; y ), we write f (y ; y ) P fy Y y + y ; y Y y + y g lim : y! y y y! If the joint density function f (y ; y ) is continuous at the point (y ; y ), then f (y ; @y F (y ; y ) : The bivariate probability density function for continuous random variables Y and Y should satisfy the following conditions. Theorem. If the function f (y ; y ) is a bivariate probability density function for continuous random variables Y and Y, then the following properties holds: () f (y ; y ) for all real numbers y and y. () f (y ; y ) dy dy. 9

10 () P f(y ; Y ) Ag A f (y ; y ) dy dy for any region A in the xy-plane. When (Y ; Y ; ; Y n ) is a continuous n-dimensional random vector; that is, Y, Y,, Y n are n continuous (one-dimensional) random variables, the corresponding joint probability density function, or joint density distribution, of Y, Y,, Y n may be denoted by f (y ; y ; ; y n ) which is proportional to the probability that the random vector is equal to the argument (y ; y ; ; y n ); that is, P fy Y y + y ; y Y y + y ; ; y n Y n y n + y n g f (y ; y ; ; y n ) y y y n : We evaluate the probability of any event A (a region in the n-dimensional space) by integrating the density function over the region de ned by A. At the continuity points of (y ; y ; ; y n ), we write f (y ; y ; ; y n ) lim y! y! y n! P fy Y y + y ; y Y y + y ; ; y n Y n y n + y n g y y : If the joint density function f (y ; y ; ; y n ) for the n-dimensional continuous random vector is continuous at the point (y ; y ; ; y n ), then f (y ; y ; ; y n n F (y ; y ; ; y n ) : In summary, the joint density function of Y, Y,, Y n must satisfy properties in the following de nition: De nition 8. The function f (y ; y ; ; y n ) is a joint probability density function for continuous random variables Y, Y,, Y n if and only if () f (y ; y ; ; y n ) for all real numbers y, y,, y n. () f (y ; y ; ; y n ) dy dy dy n. () For any region A in the n-dimensional space, P f(y ; Y ; ; Y n ) Ag f (y ; y ; ; y n ) dy dy dy n : (y ;y ; ;y n)a Presented below are examples for bivariate probability density functions of continuous random variables.

11 Example 6. Let Y and Y denote the proportions of time, out of one workday, that employees and, respectively, actually spend on performing their assigned tasks. The joint probability density function is given by y + y f (y ; y ) for y and y elsewhere. Then and P f (y ; y ) dy dy Y < ; Y > (y + y ) dy dy + y dy (y + y ) dy dy y y + y y dy y y + y y y y dy 8 y + y y y + y y dy y y y : Example 7. Let Y and Y have the joint probability density function given by cy f (y ; y ) y for < y < and < y < elsewhere. Then the value of c can be obtained through c yy dy dy c y dy y dy c y y y Hence, c. Also, since the probability is zero if Y, it follows that P < Y < ; < Y < 5 P < Y < ; < Y < y y c y yy dy dy y dy y dy 8 56 y y y 8 y y y :

12 It should be noted that this probability is the volume under the surface f (y ; y ) y y and above the rectangular set (y ; y ) j < y < ; < y < 5 in the y y -plane. Example 8. Consider random variables Y and Y having the joint probability density function given by y y f (y ; y ) e y y for < y < and < y < elsewhere. This is a legitimate joint density function because y y e y y dy dy y e y dy y e y dy Also, P f < Y < ; < Y < g h lim a! i y a e y lim y b! h y y e y y dy dy e y i y b y y e y dy : y e y dy h e y i y y h e y i y y e e e 9 : Note that this probability is the volume under the surface f (y ; y ) y y e y y and above the rectangular set f(y ; y ) j < y < ; < y < g in the y y -plane. Example 9. Let Y and Y have the joint probability density function given by cy for y f (y ; y ) y elsewhere. (a) Find the value of c. (b) Evaluate P Y ; Y >.

13 Solution. (a) Since f (y ; y ) is a joint probability density function for Y and Y, it must have that Hence, c. y cy dy dy (b) Also, we can evaluate cy [y ] y y y dy cy dy c y y c y : P Y ; Y > y " y y dy dy 8 y dy y # : y [y ] y y y dy " y 8 y y 8 # y y dy This probability is the volume under the surface f (y ; y ) y and above the triangular set f(y ; y ) j y y g in the y y -plane. Example. Let Y and Y have the joint probability density function given by y y f (y ; y ) for y and y elsewhere. Then P < Y ; < Y 8 8 y y dy dy y dy y 8 y y 9 56 : y y y y dy

14 On the other hand, F ; P Y ; Y y y dy y y ; y y dy dy y y y y dy F ; P Y ; Y y y dy y y y y dy dy 9 6 ; y y y y dy F ; P Y ; Y y 8 y dy 6 y y 6 ; y y dy dy y y y y dy Hence, F ; P Y ; Y y 8 y dy 6 y y 6 P < Y ; < Y F ; F ; F y y dy dy ; 9 56 : + F ; y y y y dy 56 :

15 Example. < y <, F (y ; y ) Let f (y ; y ) be de ned as in Example 8. Then for < y < and y y t t e t t dt dt y t e t dt y t e t dt Hence, F (y ; y ) h e t i t y t h ( e y e y e t i t y t elsewhere. e y e y for < y < and < y < Example. For the joint density function de ned in Example 7, for < y < and < y <, the joint distribution function is given by y y y y F (y ; y ) t t dt dt t dt t dt Hence, t t y t t t y t y y : 8 < for y and y F (y ; y ) y : y for < y < and < y < elsewhere. Example. Given a joint density function ( (n )(n ) f (y ; y ) for y (+y +y ) n ; y ; n > elsewhere, the joint distribution function is given by + for y F (y ; y ) (+y ) n (+y ) n (+y +y ) n ; y ; n > elsewhere, since, for y and y, we have y y F (y ; y ) y (n ) (n ) ( + t + t ) n dt dt y n (n ) ( + t ) n ( + y + t ) n dt t y (n ) ( + t + t ) n dt t t y ( + t ) n + ( + y + t ) n t ( + y ) n ( + y ) n + ( + y + y ) n : 5

MULTINOMIAL PROBABILITY DISTRIBUTION

MTH/STA 56 MULTINOMIAL PROBABILITY DISTRIBUTION The multinomial probability distribution is an extension of the binomial probability distribution when the identical trial in the experiment has more than