1.12 Multivariate Random Variables

Transcription

1 112 MULTIVARIATE RANDOM VARIABLES Multivariate Random Variables We will be using matrix notation to denote multivariate rvs and their distributions Denote by X (X 1,,X n ) T an n-dimensional random vector whose components are random variables Then, all the definitions given for bivariate rvs extend to the multivariate case For example, ifx is continuous, then we may write and F X (x 1,,x n ) P(X A) xn x1 wherea X and X R n is the support off X f X (x 1,,x n )dx 1 dx n f X (x 1,,x n )dx 1 dx n, A Example 15 LetX (X 1,X 2,X,X 4 ) T be a four-dimensional random vector with the joint pdf given by f X (x 1,x 2,x,x 4 ) 4 (x2 1 +x 2 2 +x 2 +x 2 4)I X, wherex {(x 1,x 2,x,x 4 ) R 4 : < x i < 1,i 1,2,,4} Calculate: 1 the marginal pdf of(x 1,X 2 ); 2 the expectatione(x 1 X 2 ); the conditional pdff ( x,x 4 x 1 1,x 2 ) 2 ; 4 the probabilityp ( X 1 < 1,X 2 2 <,X 4 4 > 2) 1 Solution: 1 Here we have to calculate the double integral of the joint pdf with respect tox and x 4, that is, f(x 1,x 2 ) (x2 1 +x2 2 )+ 1 2 f X (x 1,x 2,x,x 4 )dx dx 4 4 (x2 1 +x2 2 +x2 +x2 4 )dx dx 4

2 6 CHAPTER 1 ELEMENTS OF PROBABILITY DISTRIBUTION THEORY 2 By definition of expectation we have E(X 1 X 2 ) 1 1 x 1 x 2 f(x 1,x 2 )dx 1 dx 2 x 1 x 2 ( 4 (x2 1 +x 2 2)+ 1 2 By definition of a conditional pdf we have, Hence, f ( ) f X (x 1,x 2,x,x 4 ) x,x 4 x 1,x 2 f(x 1,x 2 ) 4 (x2 1 +x2 2 +x2 +x2 4 ) 4 (x2 1 +x 2 2)+ 1 2 x2 1 +x2 2 +x2 +x2 4 x 2 1 +x ) dx 1 dx f ( x,x 4 x 1 1,x 2 ) 2 ( 1 2 ( ) + 2 ) 2 +x 2 +x 2 4 ( 1 ) 2 ( + 2 ) x x2 4 4 Here we use (indirectly) the marginal pdf for(x 1,X 2,X 4 ): P ( X 1 < 1 2,X 2 < 4,X 4 > ) 4 (x2 1 +x 2 2 +x 2 +x 2 4)dx 1 dx 2 dx dx The following results will be very useful in the second part of this course They are extensions of Definition 118, Theorem 11, and Theorem 114, respectively, tonrandom variablesx 1,X 2,,X n Definition 122 LetX (X 1,X 2,,X n ) T denote a continuousn-dimensional rv with joint pdf f X (x 1,x 2,,x n ) and marginal pdfs f Xi (x i ), i 1,2,,n The random variables are called mutually independent (or just independent) if f X (x 1,x 2,,x n ) f Xi (x i )

3 112 MULTIVARIATE RANDOM VARIABLES 61 It means that all pairs X i,x j, i j, are independent Example 16 Suppose that Y i Exp(λ) independently for i 1,2,,n Then the joint pdf ofy (Y 1,Y 2,,Y n ) T is f Y (y 1,,y n ) λe λy i λ n e λ n y i Theorem 121 For g j (X j ), a function of X j only, j 1,2,,m, m n, we have ( m ) m E g j (X j ) E ( g j (X j ) ) Theorem 122 LetX (X 1,X 2,,X n ) T be a vector of mutually independent rvs with mgfsm X1 (t),m X2 (t),,m Xn (t) and leta 1,a 2,,a n andb 1,b 2,,b n be fixed constants Then the mgf of the random variablez n (a ix i +b i ) is M Z (t) e t b i M Xi (a i t) Exercise 12 Proof Theorem 122 Example 17 Calculate the mean and the variance of the random variable Y n X i, wherex i Gamma(α i,λ) independently First, we will find the mgf of Y and then generate the first and second moments using this mgf (Theorem 17) X i are independent, hence, by Theorem 122 we have M Y (t) M Xi (t) The pdf of a single rvx Gamma(α,λ) is f X (x) λα Γ(α) xα 1 e λx I [, ) (x)

4 62 CHAPTER 1 ELEMENTS OF PROBABILITY DISTRIBUTION THEORY Thus, by the definition of the mgf we have Hence, M Y (t) M X (t) E ( e tx) λα Γ(α) λα Γ(α) e tx x α 1 e λx dx x α 1 e (λ t)x dx (λ t)α λ α x α 1 e (λ t)x dx (λ t) α Γ(α) λ α (λ t) α x α 1 e (λ t)x dx (λ t) α Γ(α) }{{} 1, (pdf of a Gamma rv) ( ) α ( λ 1 t α λ t λ) M Xi (t) ( 1 λ) t αi ( 1 t ) n α i λ This has the same form as the mgf of a Gamma random variable with parameters n α i and λ, that is, ( n ) Y Gamma α i,λ The mean and variance of a Gamma rv can be obtained calculating the derivatives of the mgf att, see Theorem 17 ForX Gamma(α,λ) we have ( M X (t) 1 t ) α λ EX α λ EX 2 α(α+1) λ 2 var(x) EX 2 [EX] 2 α λ 2 Hence, fory Gamma( n α i,λ) we get EY n α i λ and var(y) n α i λ 2

5 112 MULTIVARIATE RANDOM VARIABLES 6 The following definition is often used when we consider realizations of rvs (samples) coming from populations having the same distribution Definition 12 The random variablesx 1,X 2,,X n are identically distributed if their distribution functions are identical, that is, F X1 (x) F X2 (x) F Xn (x) for all x R If they are also independent then we denote this briefly as IID, which means Independently, Identically Distributed For example, notation {X i },2,,n IID means that the variablesx i are IID but the type of the distribution is not specified We will often use IID normal rvs denoted by X i iid N(µ,σ 2 ), i 1,2,,n Exercise 121 Find the pdf of the random variablex 1 n n X i, where X i iid N(µ,σ 2 ), i 1,2,,n 1121 Expectation and Variance of Random Vectors The expectation of a random vector X is a vector of expectations of its components, that is, X 1 E(X 1 ) µ 1 X 2 E(X 2 ) µ 2 E(X) E X n E(X n ) µ n µ The variance-covariance matrix ofx is V Var(X) E [ (X E(X))(X E(X)) T] var(x 1 ) cov(x 1,X 2 ) cov(x 1,X n ) cov(x 2,X 1 ) var(x 2 ) cov(x 2,X n ) cov(x n,x 1 ) cov(x n,x 2 ) var(x n ) (12)

6 64 CHAPTER 1 ELEMENTS OF PROBABILITY DISTRIBUTION THEORY The following theorem shows a basic property of the variance-covariance matrix Theorem 12 If X is a random vector then its variance-covariance matrix V is a non-negative definite matrix, that is for any constant vector b the quadratic formb T V b is non-negative Proof For any constant vector b R n we can construct a one-dimensional variabley b T X whose variance is var(y) E [ (Y E(Y)) 2] E [ (b T X E(b T X)) 2] E [ (b T X E(b T X))(b T X E(b T X)) T] E [ b T (X E(X))(X E(X)) T b ] b T E [ (X E(X))(X E(X)) T] b b T Var(X)b b T V b That is b T V b and sov is a non-negative definite matrix The proof of the above theorem shows that the variance of a combination Y n b ix i of random variables X i is a quadratic form of the variance-covariance matrix of X and the vector of the coefficients of the combination b More generally, if X is n-dimensional rv, B is an m n constant matrix and a is a real m 1vector, then the expectation and the variance of the random vector Y a+bx are, respectively and E(Y) a+be(x) a+bµ, Var(Y) BVarXB T The covariance of two random vectors, n-dimensionalx and m-dimensional Y, is defined as Cov(X,Y) E [ (X E(X))(Y E(Y)) T] It is an n m-dimensional matrix

7 112 MULTIVARIATE RANDOM VARIABLES Joint Moment Generating Function Definition 124 Let X (X 1,X 2,,X n ) T be a random vector We define the joint mgf as M X (t) E [ e tt X ], where t (t 1,t 2,,t n ) T is an n-dimensional argument of M Similarly as in the univariate case, there is a unique relationship between the joint pdf and the joint mgf The mgf related to a marginal distribution of a subset of variables X i1,,x is can be obtained by setting t j for all j not in the set {i 1,,i s } Note also that if the variablesx 1,X 2,,X n are mutually independent, then the joint mgf is a product of the marginal mgfs, that is M X (t) E [ e ] tt X E [ e n t jx j ] E e t jx j M Xj (t j ) Another useful property of the joint mgf is given in the following theorem Theorem 124 Let X (X 1,X 2,,X n ) T be a random vector If the joint mgf ofx can be written as a product of some functionsg j (t j ), j 1,2,,n, that is M X (t) g j (t j ), then the variablesx 1,X 2,,X n are independent Proof Let t i for all i j Then, the marginal mgfm Xj (t j ) is M Xj (t j ) g j (t j ) i j g i () Also, note that ift i for all i 1,2,,n, then M X (t) E [ e n t jx j ] E [ e ] 1 This gives 1 M X (t) j () g i j g i () 1 g j ()

8 66 CHAPTER 1 ELEMENTS OF PROBABILITY DISTRIBUTION THEORY Therefore, and hence M X (t) g j (t j ) M Xj (t j ) g j(t j ) g j () g j ()M Xj (t j ) 1 M Xj (t j ) This means that the joint pdf can also be written as a product of marginal pdfs, each with the marginal mgf equal to M Xj (t j ) g j(t j ) g j Hence, the random variablesx 1,X 2,,X n are independent () 112 Transformations of Random Vectors LetX (X 1,X 2,,X n ) T be a continuous random vector and letg : R n R n be a one-to-one and onto function denoted by g(x) (g 1 (x),g 2 (x),,g n (x)) T, where x (x 1,x 2,,x n ) T and g i : R n R Then, for a transformed random vectory g(x) we have the following result Theorem 125 The density of Y g(x) is given by f Y (y) f X ( h(y) ) Jh (y), whereh(y) g 1 (y) and J h (y) denotes the absolute value of the Jacobian J h (y) det h(y) det y y 1 h 1 (y) y 1 h 2 (y) y 1 h n (y) y 2 h 1 (y) y 2 h 2 (y) y 2 h n (y) y n h 1 (y) y n h 2 (y) y n h n (y) Another useful form of the Jacobian is J h (y) [ J g ( h(y) )] 1,

9 112 MULTIVARIATE RANDOM VARIABLES 67 where J g (x) det x g(x) Exercise 122 Let A be a non-singular n n real matrix and let X be an n- dimensional random vector Show that the linearly transformed random variable Y AX has the joint pdf given by 1 f Y (y) deta f ( X A 1 y ) 1124 Multivariate Normal Distribution A random variablex has a multivariate normal distribution if its joint pdf can be written as { 1 f X (x 1,,x n ) (2π) n/2 detv exp 1 } 2 (x µ)t V 1 (x µ), where the mean is µ (µ 1,,µ n ) T, and the variance-covariance matrix has the form (12) Exercise 12 Use the result from Exercise 122 to show that if X N n (µ,v) then Y AX has n-dimensional normal distribution with expectation Aµ and variance-covariance matrixava T Lemma 1 If X N n (µ,v), B is an m n matrix, and a is a real m 1 vector, then the random vector is also multivariate normal with and the variance-covariance matrix, Y a+bx E(Y ) a+be(x) a+bµ, V Y BV B T

10 68 CHAPTER 1 ELEMENTS OF PROBABILITY DISTRIBUTION THEORY Note that taking B b T, where b is an n 1 dimensional vector and a we obtain Y b T X b 1 X 1 ++b n X n, and Y N(b T µ,b T V b)