Random Vectors Part A

Transcription

1 Random Vectors Part A Page 1 Outline Random Vectors Measurement of Dependence: Covariance Other Methods of Representing Dependence Set of Joint Distributions Copulas Common Random Vectors Functions of Random Vectors One-to-one mappings Many-to-one mappings

2 Random Vectors Page 2 A random vector [XX 1, XX 2,, XX NN ] is made up of dependent random variables XX 1, XX 2,, XX NN. Ex: Temperature and wind speeds in the next two days in Dallas: [TT 0, VV 0, TT 1, VV 1 ] Dependence between TT 1 and VV 1 ; temperature and wind of tomorrow Dependence between TT 0 and TT 1 ; temperatures of today and tomorrow Joint cdf of random variables FF 1,2 xx 1, xx 2 = P(XX 1 xx 1, XX 2 xx 2 ). The random vector [XX 1, XX 2 ] is defined through its joint cdf FF 1,2 (xx 1, xx 2 ). Joint cdf properties Nondecreasing: FF 1,2 xx 1, xx 2 FF 1,2 xx 1 + uu, xx 2 + vv for uu, vv 0 Right-continuous: lim uu,vv 0 + FF 1,2(xx 1 + uu, xx 2 + vv)=ff 1,2 xx 1, xx 2 0-probability: lim xx 1 FF 1,2(xx 1, xx 2 )= lim xx 2 FF 1,2 xx 1, xx 2 = 0 1-probability: lim xx 1,xx 2 FF 1,2(xx 1, xx 2 )=1 Reduction to Marginal cdfs: FF 1 xx 1 = lim FF 1,2 xx 1, xx 2 xx 2 FF 2 xx 2 = lim FF 1,2 xx 1, xx 2 xx 1 and xx 2 + vv xx 2 XX 22 Surface is ff 1,2 (xx 1, xx 2 ) Submodularity: FF 1,2 xx 1 + uu, xx 2 + vv +FF 1,2 xx 1, xx 2 FF 1,2 xx 1 + uu, xx 2 FF 1,2 xx 1, xx 2 + vv 0 for uu, vv 0 Moment generating function of [XX 1,, XX NN ]: exp nn=1 tt nn XX nn Joint cdf is more informative than marginals Marginals uniquely the Joint Joint non-uniquely marginals; more on this later xx 1 xx 1 + uu XX 11 xx 1 +uu xx 2 +vv 1 xx 2ff1,2 0 ff 1,2 aa, bb dddddddd + xx aa, bb dddddddd xx 1 +uu 2ff1,2 xx 1 xx 2 +vv xx aa, bb dddddddd ff 1,2 aa, bb dddddddd

3 Discrete Random Vector Ex: Tossing of a fair coin 3 times, XX: the total number of Heads and YY: the number Heads on the first toss. XX {0,1,2,3}, YY {0,1} and XX YY, the inequality hints at dependence between XX and YY Probabilities P(XX = 0, YY = 0) = P(TTTTTT) = 1/8, P(XX = 1, YY = 0) = P(TTTTTT or TTTTTT) = 2/8, P(XX = 1, YY = 1) = P(HHTTTT) = 1/8, P XX = 2, YY = 0 = P(TTTTTT) = 1/8, P(XX = 2, YY = 1) = P(HHTTHH or HHHHTT) = 2/8, P(XX = 3, YY = 1) = P(HHHHHH) = 1/8. Cdf over {0,1,2,3} {0,1} {0,1} : 7 points FF XX,YY (0,0) = P(XX = 0, YY = 0) = 1/8, FF XX,YY 1,0 = FF XX,YY (0,0) + P(XX = 1, YY = 0) = 3/8, FF XX,YY (1,1) = FF XX,YY (1,0) + P(XX = 1, YY = 1) = 4/8, FF XX,YY (2,0) = FF XX,YY (1,0) + P(XX = 2, YY = 0) = 4/8, FF XX,YY 3,0 = FF XX,YY (2,0) = 4/8, FF XX,YY (2,1) = FF XX,YY (2,0) + P(XX = 1, YY = 1) + P(XX = 2, YY = 1) = FF XX,YY (1,1) + P(XX = 2, YY = 0) + P(XX = 2, YY = 1) = 7/8, FF XX,YY 3,1 = FF XX,YY (2,1) + P(XX = 3, YY = 1) = 8/8. Marginal pmf of XX P(XX = 0) = P(XX = 0, YY = 0) = 1/8, P(XX = 1) = P(XX = 1, YY = 0) + P(XX = 1, YY = 1) = 3/8, P(XX = 2) = P(XX = 2, YY = 0) + P(XX = 2, YY = 1) = 3/8, P(XX = 3) = P(XX = 3, YY = 1) = 1/8. Marginal pmf of YY: P(YY = 0) = 4/8 and P(YY = 1) = 4/8 2 Page 3 Y Cdf takes the value indicated by black dots not the value indicated by yellow dots 3 X

4 A Property of Joint CDF and the Tail Page 4 Ex: For the joint tail probability FF XX xx 1, xx 2 = P [XX 1 xx 1 ] [XX 2 xx 2 ], we have FF XX xx 1, xx 2 = FF 1 xx 1 + FF 2 xx 2 + FF XX xx 1, xx 2 1 Starting with 1 P [XX 1 xx 1 ] [XX 2 xx 2 ] = P [XX 1 xx 1 [XX 2 xx 2 ]) = P [XX 1 xx 1 ) + P([XX 2 xx 2 ]) P [XX 1 xx 1 [XX 2 xx 2 ]) Reorganizing P [XX 1 xx 1 [XX 2 xx 2 ]) = P [XX 1 xx 1 ) + P XX 2 xx P [XX 1 xx 1 ] [XX 2 xx 2 ] This yields the desired equality Ex: FF XX xx 1, xx 2 + FF XX xx 1, xx 2 < 1 when P [XX 1 xx 1 ] [XX 2 xx 2 ] > 0 or P [XX 1 xx 1 ] [XX 2 xx 2 ] > 0

5 Expected Value of Sums of Dependent RVs Page 5 Regardless of independence of XX 1, XX 2, the linearity of expectation holds: E gg 1 XX 1 + gg 2 XX 2 = E gg 1 XX 1 + E gg 2 XX 2 for functions gg 1, gg 2 Ex: An OM course is taken by 5 students. Independent of his classmates, each of the students pursue a degree program: SCM, Finance, Marketing, Accounting, MBA and PhD. A student in this course can be in any one of the programs with equal probability. What is the expected number of degree programs represented in this course? Let XX SSSSSS = 1 if at least 1 student in the course pursues MS in SCM. Similarly define XX FFFFFF, XX MMMMMM, XX AAAAAA, XX MMMMMM, XX PPPPP. The number of degree programs represented is XX XX = XX SSSSSS + XX FFFFFF + XX MMMMMM + XX AAAAAA + XX MMMMMM + XX PPPPP [XX SSSSSS, XX FFFFFF, XX MMMMMM, XX AAAAAA, XX MMMMMM, XX PPPPP ] are not independent: 5 students and 6 degree programs, 1 XX SSSSSS + XX FFFFFF + XX MMMMMM + XX AAAAAA + XX MMMMMM + XX PPPPP 5 Dependence because of Probability of XX SSSSSS = 1 depends on conditioned events 5 students & 6 degree programs P XX SSSSSS = 1 XX FFFFFF = XX MMMMMM = XX AAAAAA = XX MMMMMM = XX PPPPP = 0 = 1 > 0 = P XX SSSSSS = 1 XX FFFFFF = XX MMMMMM = XX AAAAAA = XX MMMMMM = XX PPPPP = 1 7 students & 6 degree programs P XX SSSSSS = 1 XX FFFFFF = XX MMMMMM = XX AAAAAA = XX MMMMMM = XX PPPPP = 0 = 1 > = 1 P Neither in SCM = P At least 1 of 2 students in SSSSSS = 6 P XX SSSSSS = 1 XX FFFFFF = XX MMMMMM = XX AAAAAA = XX MMMMMM = XX PPPPP = 1 Dependent program representation Independent choice by each student

6 Expected Number of Programs Represented in the Class Page 6 Regardless of independence of XX 1, XX 2, the linearity of expectation holds: E gg 1 XX 1 + gg 2 XX 2 = E gg 1 XX 1 + E gg 2 XX 2 for functions gg 1, gg 2 Ex: Despite dependence, E(XX) = E(XX SSSSSS ) + E(XX FFFFFF ) + E(XX MMMMMM ) + E(XX AAAAAA ) + E(XX MMMMMM ) + E(XX PPPPP ) Since XX SSSSSS is binary E XX SSSSSS =P(One or more SCM students)= 1 P(No SCM student) o P(No SCM student)= P(Student 1, Student 2,, Student 5 pursue other than SCM) Each student independently pursues a program: P(Student 1, Student 2 pursue Finance)= P(Student 1 in Finance)P(Student 2 in Finance) A student pursue any one of the programs with equal probability: P(Student 1, Student 2 pursue other than SCM)=(1-1/6)(1-1/6) o P(No SCM student)= P(Student 1, Student 2,, Student 5 pursue other than SCM)= Dependent program representation Independent choice by each student E XX SSSSSS = 1 P(No SCM student)= E(XX) = = E(XXFFFFFF ) = E XX MMMMMM = E(XX AAAAAA ) = E(XX MMMMMM ) = E(XX PPPPP )

7 Generalizing the Example of Number of Degree Programs Page 7 Ex: A course is taken by mm students. Independent of his classmates, each of the students pursue a degree program out of nn programs. A student in this course can be in any one of the programs with equal probability. What is the expected number of degree programs represented in this course? Let XX ii = 1 if at least 1 student in the course pursues program ii. The number of degree programs XX = ii=1 [XX 1, XX 2,, XX nn ] are not independent: mm students and nn degree programs, 1 nn ii=1 XX ii min{mm, nn} Despite dependence, E XX = nn ii=1 E(XX ii ) Since XX ii is binary E XX ii =P(One or more students pursue program ii) = 1 P(No students for program ii) o Because of Each student independently pursues a program: A student pursue any one of the programs with equal probability: P(No students for program ii)= 1 1 nn E XX ii = 1 P(No students for program ii)= nn mm mm and E(XX) = nn nn mm nn XX ii A lot of degree programs nn, every program gets at most one student, some get exactly one, lim E XX = lim nn nn nn 1 n mm = lim n mm 1 1 nn 1 n 2 mm 1 1 nn 2 = lim n mm 1 1 nn mm = mm = Number of students A lot of students mm, every program gets at least one student, lim E XX = lim nn mm = nn lim mm = nn = Number of degree programs mm mm nn m nn

8 Independence of Random Variables Page 8 Although expected values are useful, they are not sufficiently detailed to assess probabilities. Then we must work with probability functions. When random variables XX, YY are independent, i.e., XX YY, P XX aa, YY bb = P XX aa P(YY bb) which implies for discrete rvs, pp XX,YY aa, bb = P XX = aa, YY = bb = pp XX aa pp YY (bb) aa bb continuous rvs, ffxx,yy xx, yy dddddddd = P XX aa, YY bb = P XX aa P(YY bb) = aa bb ffxx xx ff YY (yy)ddyyddxx, which in turn implies ff XX,YY xx, yy = ff XX xx ff YY (yy). We have the following equalities for the marginal and joint density aa ffxx xx dddd = P XX aa = lim P(XX aa, YY bb) = bb ffxx,yy xx, yy ddyyddxx for every aa Then the marginal density is ff XX xx dddd = ffxx,yy xx, yy dddd. aa Ex: Are XX, YY independent if ff XX,YY xx, yy = I xx,yy 0 4xxxx exp( xx 2 yy 2 )? We check for marginal pdfs, ff XX xx = 0 4xxxx exp xx 2 yy 2 dddd = 2xx exp xx 2 0 2yy exp yy 2 dddd = 2xx exp xx 2. By symmetry, ff YY yy = I yy 0 2yy exp yy 2. Hence, ff XX,YY xx, yy = ff XX xx ff YY (yy) and rvs are independent.

9 Covariance: Measurement of Dependence Page 9 Covariance is a measure of dependence Cov XX, YY = E XX E XX YY E YY = E XXYY E XX)E(YY Ex: Cov NN ii=1 aa ii XX ii, MM jj=1 bb jj YY jj = NN ii=1 = ii=1 MM jj aa ii bb jj E(XX ii YY jj ) NN ii=1 MM jj aa ii bb jj E(XX ii )E(YY jj ) NN jj MM aa ii bb jj Cov(XX ii, YY jj ) Independence is more informative than Covariance=0 Independence Covariance = 0 Dependent random variables can also have Covariance = 0, see the next example Ex: Let XX, YY discrete random variables have the joint pmf in the following table. pp XX,YY (xx, yy) yy = 00 yy = 11 yy = 22 pp XX (xx) xx = 00 1/4 0 1/4 1/2 xx = /2 0 1/2 pp YY (yy) 1/4 1/2 1/4 Variables are dependent: pp XX,YY 0,1 = = pp XX 0 pp YY (1). Cov XX, YY = 0: E XXXX = = 1 2, E XX = = 1 2 and E YY = = 1 This is unfortunate we cannot in general deduce independence from covariance.

10 Covariance: Alternative Formula Ex: For covariance, integrate the difference between joint tail and products of marginal tails CCCCCC XX, YY = Let XX 1, YY 1, (XX 2, YY 2 ) be iid with (XX, YY). [P XX uu, YY vv P XX uu P YY vv ] dddddddd Page 10 CCCCCC XX, YY = 1 2 E XX 1YY 1 E XX 1 E YY 1 + E XX 2 YY 2 E XX 2 E YY 2 = 1 2 E XX 1YY 1 E XX 1 E YY 2 + E XX 2 YY 2 E XX 2 E YY 1 = 1 2 E XX 1 XX 2 YY 1 YY 2 xx For xx 1 xx 2 left-below, xx 1 xx 2 = 1 xx2 dddd = 11xx2 uu xx 1 dddd = (11uu xx1 11 uu xx2 )dddd - xx 2 + xx 1 For xx 1 xx 2 right-above, xx 1 xx 2 = xx1 + - xx 1 xx 2 ( 1)dddd = 11xx1 uu xx 2 dddd = (11uu xx1 11 uu xx2 )dddd CCCCCC XX, YY = 1 E XX 2 1 XX 2 YY 1 YY 2 = 1 E 2 (11uu XX1 11 uu XX2 )dddd (11vv YY1 11 vv YY2 )dddd = 1 2 E((11uu XX1 11 uu XX2 )(11 vv YY1 11 vv YY2 ))dddddddd xx 2 = 1 2 E(11uu XX1 11 vv YY1 ) E 11 uu XX1 )E(11 vv YY2 E 11 uu XX2 )E(11 vv YY1 + E(11 uu XX2 11 vv YY2 )dddddddd by iid blue = green = E(11uu XX1 11 vv YY1 ) E 11 uu XX1 )E(11 vv YY2 dddddddd, which is the desired equality

11 Other Methods for Representing Dependence Set of Joint Distributions Marginal non-unique a joint cdf Ex: For given marginal {FF 1, FF 2 }, consider two alternative cdfs FF 12 xx 1, xx 2 = min{ff 1 xx 1, FF 2 (xx 2 )} and GG 12 xx 1, xx 2 = FF 1 xx 1 FF 2 (xx 2 ) Both joint cdfs correspond to the given marginals!! Both have non-decreasing, right-continuous, 0-probability, 1-probability and submodularity. More importantly FF 12 and GG 12 match marginals lim FF 1,2 xx 1, xx 2 = lim min FF 1 xx 1, FF 2 xx 2 = FF 2 xx 2, similarly for FF 1 xx 1 xx 1 lim GG 1,2 xx 1, xx 2 = lim FF 1 xx 1 FF 2 (xx 2 ) = FF 2 xx 2, similarly for FF 1 xx 1 xx 1 Given marginals, a joint cdf is not unique Page 11 Given marginals, a set of cdfs RR{FF 1, FF 2,, FF NN } Ex: Each FF RR{FF 1, FF 2 } has a Lower Bound and an Upper Bound max FF 1 xx 1 + FF 2 xx 2 1,0 FF xx 1, xx 2 min{ff 1 xx 1, FF 2 xx 2 } For the lower bound, FF xx 1, xx 2 = P [XX 1 xx 1 ] [XX 2 xx 2 ] = 1 P [XX 1 xx 1 ] [XX 2 xx 2 ] because blue=red c Also FF xx 1, xx 2 1 P XX 1 xx 1 P XX 2 xx 2 = 1 1 FF 1 xx 1 (1 FF 2 (xx 2 )) = FF 1 xx 1 + FF 2 xx For the upper bound see exercises. For >2 random variables see notes.

12 Achieving the Upper Bound: A Random Vector whose CDF is Upper Bound Page 12 Upper Bound min{ff 1 xx 1, FF 2 xx 2 } is a cdf for a random vector X [FF 1 1 UU, FF 1 2 (UU)] where UU is a uniform rv over (0,1). FF 1 xx 1 FF 2 xx 2 is a cdf for a random vector [FF 1 1 UU 1, FF 2 1 (UU 2 )] where UU 1, UU 2 are iid uniform rv over (0,1). FF 1 FF 2 UU FF 1 FF 2 UU 1 UU 2 XX 1 = FF 1 1 UU XX 2 = FF 2 1 (UU) FF 1 1 UU 1 FF 2 1 (UU 2 ) FF XX xx 1, xx 2 = P FF 1 1 UU 1 xx 1, FF 2 1 UU 2 xx 2 FF XX xx 1, xx 2 = P FF 1 1 UU xx 1, FF 2 1 UU xx 2 = P UU FF 1 xx 1, UU FF 2 xx 2 = P UU min FF 1 xx 1, FF 2 xx 2 = min{ff 1 xx 1, FF 2 xx 2 } = P UU 1 FF 1 xx 1, UU 2 FF 2 xx 2 = P UU 1 FF 1 xx 1 )P(UU 2 FF 2 xx 2 = FF 1 xx 1 FF 2 xx 2 min{ff 1 xx 1, FF 2 xx 2 } See Exercises for achieving the lower bound.

13 Other Methods for Representing Dependence Achieving Unique Joint Probability: Copula Page 13 Given marginals FF 1, FF 2, FF XX RR{FF 1, FF 2 } is not unique for XX = [XX 1, XX 2 ]. Use a copula function CC: 0,1 0,1 [0,1] to relate joint cdf to the marginals FF XX (xx 1, xx 2 ) = CC(FF 1 (xx 1 ), FF 2 xx 2 ) Copula describes the dependence structure when random variables are together, whereas marginals describe the behavior of each random variable on its own. Ex: Independence copula: CC uu 1, uu 2 = uu 1 uu 2. Comonotonic copula: CC uu 1, uu 2 = min{uu 1, uu 2 } Copula function must satisfy Non-decreasing & right-continuous cdf non-decreasing and right-continuous copula function CC 0- & 1-probability for cdf lim uu1 0 CC(uu 1, uu 2 )= lim uu2 0 CC(uu 1, uu 2 )=0, lim uu1 1 CC uu 1, uu 2 = uu 2, lim uu2 1 CC uu 1, uu 2 = uu 1 Submodularity for cdf CC vv 1, vv 2 + CC uu 1, uu 2 CC uu 1, vv 2 + CC vv 1, uu 2 for uu 1, uu 2 vv 1, vv 2 Sklar s theorem: Copula approach does not miss anything. Each FF XX RR{FF 1, FF 2 } is representable with a unique copula. Given the joint cdf FF XX for vector XX = [XX 1, XX 2 ], there is a Copula CC satisfying FF XX (xx 1, xx 2 ) = CC(FF 1 (xx 1 ), FF 2 xx 2 ). This copula is the joint cdf of the random vector XX FF = [FF 1 XX 1, FF 2 (XX 2 )]. FF 1 (XX 1 ) is a random variable because it is a function of random variable.» The cdf of FF 1 (XX 1 ) is FF FF1 XX 1 aa P FF 1 XX 1 aa = P XX 1 FF 1 1 aa = FF 1 FF 1 1 aa = aa, so FF 1 (XX 1 ) is a uniform random variable. Marginals of XX FF are uniform. The cdf of XX FF is FF XXFF aa, bb : = P(FF 1 (XX 1 ) aa, FF 2 (XX 2 ) bb) = P XX 1 FF 1 1 aa, XX 2 FF 1 2 bb = FF XX FF 1 1 aa, FF 1 2 bb = CC FF 1 FF 1 1 aa, FF 2 FF 1 2 bb = CC(aa, bb)

14 Comonotonic Copula Page 14 For two monotone increasing (or decreasing) functions gg 1, gg 2 and a random variable YY If XX 1 = gg 1 (YY) and XX 2 = gg 2 (YY), then XX 1 and XX 2 are comonotonic. Ex: Comonotonic copula CC uu 1, uu 2 = min uu 1, uu 2 yields comonotonic rvs. With the comonotonic copula XX = [XX 1, XX 2 ] has the joint cdf FF XX xx 1, xx 2 = CC FF XX1 xx 1, FF XX2 xx 2 = min FF XX1 xx 1, FF XX2 xx 2 From earlier discussion, min FF XX1 xx 1, FF XX2 xx 2 is the cdf of XX = [FF XX1 UU, FF XX2 UU ] for uniformly distributed UU over [0,1]. Letting gg 1 uu = FF 1 XX1 uu and gg 2 uu = FF 1 XX2 uu XX 1 = gg 1 UU and XX 2 = gg 2 UU. and YY = UU, we see that XX 1 and XX 2 are comonotonic.

15 Marshall-Olkin Bivariate Exponential Copula Page 15 For independent YY 1 = EEEEEEEE λλ 1, YY 2 = EEEEEEEE λλ 2, ZZ = EEEEEEEE λλ, consider the vector XX = XX 1, XX 2 = [min YY 1, ZZ, min{yy 2, ZZ}] YY 1, YY 2 lifetimes of two necessary components of a system that receives a fatal shock at ZZ Tail probability for XX ii : FF ii (xx ii ) = P XX ii xx ii = P min{yy ii, ZZ} xx ii = P YY ii xx ii )P(ZZ xx ii = exp λλ ii + λλ xx ii Tail probability for XX: FF XX (xx 1, xx 2 ) = P XX 1 xx 1, XX 2 xx 2 = P YY 1 xx 1, YY 2 xx 2, ZZ max{xx 1, xx 2 } = P YY 1 xx 1 )P(YY 2 xx 2 )P(ZZ max{xx 1, xx 2 } = exp( λλ 1 xx 1 ) exp( λλ 2 xx 2 ) exp λλmax xx 1, xx 2 = exp( (λλ 1 + λλ)xx 1 ) exp( (λλ 2 +λλ)xx 2 ) min{exp λλxx 1, exp(λλxx 2 )} = FF 1 (xx 1 ) FF 2 (xx 2 )min{exp λλxx 1, exp(λλxx 2 )} where max aa, bb = aa + bb min{aa, bb} and monotonicity of exp(xx) are used respectively in the last two equalities. The cdf for XX: FF XX xx 1, xx 2 = FF 1 xx 1 + FF 2 xx 2 + FF XX xx 1, xx 2 1 = 1 FF 1 xx FF 2 xx 2 + FF 1 (xx 1 ) FF 2 (xx 2 )min{exp λλxx 1, exp(λλxx 2 )} 1 = 1 FF 1 xx 1 FF 2 xx 2 + FF 1 (xx 1 ) FF 2 (xx 2 )min{ FF 1 xx 1 λλ/(λλ 1 +λλ), FF 2 xx 2 λλ/(λλ 2 +λλ) } = 1 FF 1 xx 1 FF 2 xx 2 +min{ FF 2 xx 2 FF 1 xx 1 1 λλ/(λλ 1 +λλ), FF 1 xx 1 FF 2 xx 2 1 λλ/(λλ 2 +λλ) } 1 λλ 1 = 1 (1 FF 1 ) (1 FF 2 ) +min{(1 FF 2 ) 1 FF λλ1+λλ 1, (1 FF 1 ) 1 FF 2 =: CC(FF 1, FF 2 ) This holds for Marshall-Olkin copula CC: 0,1 0,1 [0,1] and 0 αα ii = λλ λλ ii +λλ 1 FF λλ ii xx λλ ii ii +λλ λλ = exp λλ ii + λλ xx λλ ii ii +λλ = exp(λλxx ii ) λλ λλ2+λλ } CC uu 1, uu 2 = 1 1 uu 1 1 uu 2 + min{ 1 uu 2 1 uu 1 1 αα 1, 1 uu 1 1 uu 2 1 αα 2 }

16 Summary Page 16 Random Vectors Measurement of Dependence: Covariance Other Methods of Representing Dependence Set of Joint Distributions Copulas Common Random Vectors Functions of Random Vectors One-to-one mappings Many-to-one mappings