Chater General Random Variables. Law of a Random Variable Thus far we have considered onl random variables whose domain and range are discrete. We now consider a general random variable X! defined on the robabilit sace ( F P). Recall that F is a -algebra of subsets of. IP is a robabilit measure on F, i.e., IP (A) is defined for ever A F. A function X! is a random variable if and onl if for ever () (the -algebra of orel subsets of ), the set fx g 4 X () 4 f! X(!) g F i.e., X! is a random variable if and onl if X is a function from () to F(See Fig..) Thus an random variable X induces a measure X on the measurable sace ( ()) defined b X () IP X () 8 () where the robabili on the right is defined since X () F. X is often called the Law of X in Williams book this is denoted b L X.. Densit of a Random Variable The densit of X (if it exists) is a function f X![0 ) such that X () f X (x) dx 8 () 3
4 X R {X ε } Ω Figure. Illustrating a real-valued random variable X. We then write d X (x) f X (x)dx where the integral is with resect to the Lebesgue measure on. f X is the Radon-Nikodm derivative of X with resect to the Lebesgue measure. Thus X has a densit if and onl if X is absolutel continuous with resect to Lebesgue measure, which means that whenever () has Lebesgue measure zero, then IP fx g 0.3 Exectation Theorem 3.3 (Exectation of a function of X) Let h! be given. Then IEh(X) 4 h(x(!)) dip (!) h(x) d X (x) h(x)f X (x) dx Proof (Sketch). If h(x) (x) for some, then these equations are IE (X) 4 P fx g X () f X (x) dx which are true b definition. Now use the standard machine to get the equations for general h.
CHAPTER. General Random Variables 5 (X,Y) C { (X,Y) ε C} Ω x Figure. Two real-valued random variables X Y..4 Two random variables Let X Y be two random variables! defined on the sace ( F P). Then X Y induce a measure on ( ) (see Fig..) called the joint law of (X Y ), defined b X Y (C) 4 IP f(x Y ) Cg 8C ( ) The joint densit of (X Y ) is a function f X Y![0 ) that satisfies X Y (C) C f X Y (x ) dxd 8C ( ) f X Y is the Radon-Nikodm derivative of X Y with resect to the Lebesgue measure (area) on. We comute the exectation of a function of X Y in a manner analogous to the univariate case IEk(X Y ) 4 k(x(!) Y(!)) dip (!) k(x ) d X Y (x ) k(x )f X Y (x ) dxd
6.5 Marginal Densit Suose (X Y ) has joint densit f X Y. Let be given. Then where Y () IP fy g IP f(x Y ) g X Y ( ) f Y () 4 Therefore, f Y () is the (marginal) densit for Y. f X Y (x ) dxd f Y () d f X Y (x ) dx.6 Conditional Exectation Suose (X Y ) has joint densit f X Y. Let h! be given. Recall that IE[h(X)jY ] 4 IE[h(X)j(Y )] deends on! through Y, i.e., there is a function g() (g deending on h) such that IE[h(X)jY ](!) g(y (!)) How do we determine g? We can characterize g using artial averaging Recall that A (Y )()A fy g for some (). Then the following are equivalent characterizations of g g(y ) dip A A h(x) dip 8A (Y ) (6.) (Y )g(y ) dip (Y )h(x) dip 8 () (6.) ()g() Y (d) ()h(x) d X Y (x ) 8 () (6.3) g()f Y () d h(x)f X Y (x ) dxd 8 () (6.4)
CHAPTER. General Random Variables 7.7 Conditional Densit A function f XjY (xj)![0 ) is called a conditional densit for X given Y rovided that for an function h! (Here g is the function satisfing and g deends on h, butf XjY g() does not.) h(x)f XjY (xj) dx (7.) IE [h(x)jy ]g(y ) Theorem 7.33 If (X Y ) has a joint densit f X Y, then f XjY (xj) f X Y (x ) (7.) f Y () Proof Just verif that g defined b (7.) satisfies (6.4) For () h(x)f XjY (xj) dx {z } g() Notation. Let g be the function satisfing The function g is often written as f Y () d IE[h(X)jY ]g(y ) g() IE[h(X)jY ] h(x)f X Y (x ) dxd and (7.) becomes IE[h(X)jY ] h(x)f XjY (xj) dx In conclusion, to determine IE[h(X)jY ] (a function of!), first comute g() h(x)f XjY (xj) dx and then relace the dumm variable b the random variable Y IE[h(X)jY ](!) g(y (!)) Examle. (Jointl normal random variables) Given arameters > 0 > 0 <<. Let (X Y ) have the joint densit x f X Y (x ) ex ( ) x +
8 The exonent is x x + ( ) ( ) ( ) x + x ( ) # We can comute the Marginal densit of Y as follows f Y () e ( ) e u due using the substitution u e Thus Y is normal with mean 0 and variance. Conditional densit. From the exressions f X Y (x ) ( e ) x x x dxe, du e dx we have f Y () e f XjY (xj) f X Y (x ) f Y () e ( ) x In the x-variable, f XjY (xj) is a normal densit with mean and variance ( ). Therefore, IE[XjY ] xf XjY (xj) dx IE Y # X x f XjY (xj) dx ( )
CHAPTER. General Random Variables 9 From the above two formulas we have the formulas IE Taking exectations in (7.3) and (7.4) ields IE IE[XjY ] Y (7.3) X Y Y # ( ) (7.4) IEX IEY 0 (7.5) X Y # ( ) (7.6) ased on Y, the best estimator of X is Y. This estimator is unbiased (has exected error zero) and the exected square error is ( ). No other estimator based on Y can have a smaller exected square error (Homework roblem.)..8 Multivariate Normal Distribution Please see Oksendal Aendix A. Let X denote the column vector of random variables (X X X n ) T, and x the corresonding column vector of values (x x x n ) T. X has a multivariate normal distribution if and onl if the random variables have the joint densit f X (x) det A n o () n ex (X )T A(X ) Here, 4 ( n ) T IEX 4 (IEX IEX n ) T and A is an n n nonsingular matrix. A is the covariance matrix A IE h (X )(X ) Ti i.e. the (i j)th element of A is IE(X i i )(X j j ). The random variables in X are indeendent if and onl if A is diagonal, i.e., where j IE(X j j ) is the variance of X j. A diag( n )
30.9 ivariate normal distribution Take n in the above definitions, and let Thus, f X X (x x ) A 4 IE(X )(X ) A 4 ( ) ( ) det A ( ex # ( ) ( ) and we have the formula from Examle., adjusted to account for the ossibl non-zero exectations (x ) ( ) 3 5 + (x ) (x )(x ) #).0 MGF of jointl normal random variables Let u (u u u n ) T denote a column vector with comonents in, and let X have a multivariate normal distribution with covariance matrix A and mean vector. Then the moment generating function is given b IEe ut X e ut X fx X X (x x x n ) dx dx n n n ex ut A u + u T o If an n random variables X X X n have this moment generating function, then the are jointl normal, and we can read out the means and covariances. The random variables are jointl normal and indeendent if and onl if for an real column vector u (u u n ) T IEe ut X 4 IE ex 8 9 8 < nx < nx u j X j ex j j 9 [ j u j + u j j ]