Department o Electrical Engineering University o Arkansas ELEG 3143 Probability & Stochastic Process Ch. 4 Multiple Random Variables Dr. Jingxian Wu wuj@uark.edu
OUTLINE 2 Two discrete random variables Two continuous random variables Statistical independence and correlation Functions o two random variables Moment generating unction
TWO DISCRETE RANDOM VARIABLES 3 Joint PMF o two discrete random variables Consider two discrete RVs, X and Y. The joint probability mass unction (PMF) o X and Y is deined as
TWO DISCRETE RANDOM VARIABLES 4 Marginal PMF: the PMF o each individual RV (exactly the same as in Ch. 2) Joint PMF marginal PMF (using the total probability equation) Pr( X x ) Pr( X x, Y y ) i j the summation is over all the possible value o Y i j Pr( Y y ) Pr( X x, Y y ) j i the summation is over all the possible value o X i j Recall: total probability
TWO DISCRETE RV 5 Example Consider an urn contains 4 white balls and 6 black balls. Pick two balls without replacement. Let X denote the result rom the irst pick, and Y the result rom the 2 nd pick, i.e., deine two random variables (X, Y) as {(B, B)} (X=0, Y=0), {(B, W)} (X=0, Y=1), {(W, B)} (X=1, Y=0), {(W, W)} (X = 1,Y = 1) (a) Find the joint PMF (b) Find the marginal PMF o X (c) Find the marginal PMF o Y i j P XY ( x, y ) 1 i j
TWO DISCRETE RV 6 Review: conditional probability Two events A and B P( A B) P( AB) P( B) Conditional PMF Given event B, what is the probability that X x i? E.g. two RV X and Y B: Y y j B: Y > y
TWO DISCRETE RV 7 Example Consider an urn contains 4 white balls and 6 black balls. Pick two balls without replacement. Let X denote the result rom the irst pick, and Y the result rom the 2 nd pick, i.e., deine two random variables (X, Y) as {(B, B)} (X=0, Y=0), {(B, W)} (X=0, Y=1), {(W, B)} (X=1, Y=0), {(W, W)} (X = 1,Y = 1) Find P( X xi Y y j ) i P( X x i B) 1
TWO DISCRETE RV 8 Example The joint PMF o two RV X and Y are given as ollows. Find the marginal PMF and the conditional PMF
OUTLINE 9 Two discrete random variables Two continuous random variables Statistical independence and correlation Functions o two random variables Moment generating unction
TWO CONTINUOUS RANDOM VARIABLES 10 Joint CDF The joint cumulative distribution unction (CDF) o two random variables X and Y is deined as The above deinition is true or both discrete RV and continuous RV Marginal CDF The CDF o each individual RV (exactly the same as in Ch. 2) joint CDF marginal CDF F X ( x) F ( x, ) XY F Y ( y) F (, y) XY Why?
TWO CONTINUOUS RV 11 Example F XY (x,) F XY (, y) F XY (, )
TWO CONTINUOUS RV 12 Example Two RVs X and Y have a joint CDF as ollows. Find the CDF o X
TWO CONTINUOUS RV 13 Joint pd o two continuous RVs d ( x, y) 2 F( x, y) dxdy
TWO CONTINUOUS RV 14 Properties o joint pd Probability P( a1 X b1, a2 X b2 ) b 2 a 2 b a 1 1 XY ( x, y) dxdy CDF XY F ( x, y) ( x, y) dxdy y x XY Marginal pd X ( x) XY ( x, y) dy Y ( y) XY ( x, y) dx Recall: marginal PMF or discrete RV P ( X x ) P( X x, Y y ) i j i j
TWO CONTINUOUS RV 15 Example Two RVs with joint pd given as ollows Find the marginal pd o X The probability that X > 0.5 and Y < 0.3
TWO CONTINUOUS RV 16 Example Two RVs with the joint pd as ollows Find the constant c Find the marginal pd o x and y
TWO CONTINUOUS RV Conditional probability Recall 17 ) ( ), ( ) ( y y x y x Y XY Y X ) ( ) ( ) ( B P AB P B A P ) ( ) ( ), ( y y x y x Y X Y XY ) ( ) ( ) ( B P B A P AB P
TWO CONTINUOUS RV 18 Conditional pd Conditional pd is still a pd 0 ( x y) 1 X Y X Y ( x y) dx 1 x X Y ( x y) P( X x Y y) X Y F ( u y) du Dierence between P( X x Y y) and P( X x Y y) P( X x Y y)
TWO CONTINUOUS RV 19 Example Two RVs with joint pd as ollows Find Find Find X Y ( x y) P( X 4 y 3) P( X 2 y 3)
TWO CONTINUOUS RV 20 Example Two RVs with a joint pd as ollows 6 (1 x XY ( x, y) 5 0, Find X Y ( x y) Veriy that 2 y), X Y ( x y) dx 0 x 1,0 x 1, 1 otherwise
TWO CONTINUOUS RVS 21 Joint moment E g( X, Y)] g( x, y) XY ( x, y) dxdy [ Linearity o the expectation operator E[ ax by] ae[ X ] be[ Y] Proo:
TWO CONTINUOUS RVS 22 Example Two RVs X and Y with mean m and. Deine Z = 2X + 5Y + X 5 m Y 6 7. Find E[Z].
OUTLINE 23 Two discrete random variables Two continuous random variables Statistical independence and correlation Functions o two random variables Moment generating unction
INDEPENDENCE AND CORRELATION 24 Independence Tow continuous RVs are called independent i and only i XY ( x, y) ( x) ( y) X Y Recall: or two RVs X and Y XY ( x, y) ( x y) ( y) X Y Thereore, i X and Y are independent, then ( x y) ( X Y X x ) Y Recall: two events A and B are independent i and only i
INDEPENDENCE AND CORRELATION 25 Example Two RVs with joint pd as ollows XY ke ( x, y) ( x y1) 0,, 0 x,1 y otherwise, Find the value o k Are they independent
INDEPENDENCE AND CORRELATION 26 Correlation The correlation between two RVs X and Y is deined as XY E xy XY ( x, y) dxdy Covariance The covariance between two RVs X and Y is deined as E Recall: variance o X: E ( X mx )( Y my E ( X mx ) 2 ) ( X mx )( Y my ) E( XY ) mxmy
INDEPENDENCE AND CORRELATION Correlation coeicient (normalized covariance) The correlation coeicient o two RV X and Y is deined as 27 Y X Y X Y X Y X m m XY E m Y m X E ) ( ) )( (
INDEPENDENCE AND CORRELATION 28 Example Consider two RVs with the joint pd as x y, 0 x 1,0 ( x, y) 0, o. w. y 1 Find the correlation, covariance, and the correlation coeicient
INDEPENDENCE AND CORRELATION 29 Uncorrelated Two RVs X and Y are said to be uncorrelated i E XY E( X ) E( Y ) I two RVs are uncorrelated, then the covariance is, their correlation coeicient is. Independence correlation Independent uncorrelated (it s not true the other way around) I X and Y are independent EXY I two RVs are uncorrelated, they might not be independent.
INDEPENDENCE AND CORRELATION 30 Example Let be uniormly distributed in [ 0,2 ]. X cos() Y sin() Are they uncorrelated?
INDEPENDENCE AND CORRELATION 31 Example Two independent RVs, X and Y, have variance 2. Find X 4, 2 Y 16 the variance o U = 2X + 3Y.
OUTLINE 32 Two discrete random variables Two continuous random variables Statistical independence and correlation Functions o two random variables Moment generating unction
FUNCTIONS OF TWO RVS 33 pd o the sum o two RVs Consider two RVs X and Y, with the joint pd ( x, y) Deine a new RV Z = X + Y. What is the pd o Z? Sol: Find the CDF o Z, then dierentiate it with respect to z. XY Z ( z) XY ( x, z x) dx
FUNCTIONS OF TWO RVS 34 Pd o the sum o two independent RVs I X and Y are independent, then the pd o Z = X+Y is Z ( z) ( x, z x) dx ( x) ( z x) dx ( z) ( z) XY The pd o Z is the convolution o the pds o X and Y X Y X Y
FUNCTIONS OF TWO RVS Example Two independent RVs have pd as Find the pd o Z = X + Y 35.. 0, 0, ) ( 1 1 o w x e x x X.. 0, 0, ) ( 2 2 o w y e y y Y
FUNCTIONS OF TWO RVS 36 Pd o unctions o two RVs Consider two RVs, X and Y, with joint pd Let Z g (, ), 1 X Y W g2( X, Y) Or, inversely, X h ( Z, ), Y h ( Z, ) The joint pd o Z and W is 1 W 2 W ( z, w) XY h1 ( z, w), h2 ( z, w J ZW ) XY ( x, y) J x z y z x w y w
FUNCTION OF TWO RVS 37 Example Two random variables X and Y have a joint pd 1, 0 x 1,0 y 1 XY ( x, y) 0, otherwise Find the pd o Z = XY
FUNCTIONS OF TWO RVS 38 Example A box contains resistors whose values are independent and uniormly distributed between 10 Ohm and 20 Ohm. I two resistors are selected in random and connected in series The pd o the resistance o the serial circuit.
OUTLINE 39 Two discrete random variables Two continuous random variables Statistical independence and correlation Functions o two random variables Moment generating unction
MOMENT GENERATING FUNCTION 40 Moment generating unction (MGF) The moment generating unction o a random variable X is Calculation o MGF or a continuous RV This is the Laplace transorm o the pd. Calculation o MGF or a discrete RV
MOMENT GENERATING FUNCTION 41 Example: Find the MGF o the exponential RV
MOMENT GENERATING FUNCTION 42 Example Find the MGF o a Bernoulli RV with parameter p
MOMENT GENERATING FUNCTION 43 Theorem We can calculate the n-th moment o a random variable X by using its MGF as
MOMENT GENERATING FUNCTION 44 Example Find the irst and second moments o an exponential RV by using the MGF
MOMENT GENERATING FUNCTION 45 MGF o sum o independent RVs
MOMENT GENERATING FUNCTION 46 Example For a Poisson RV with PMF The MGF is Find the distribution o the sum o N independent Poisson RVs
MOMENT GENERATING FUNCTION 47 Example