Random Vectors. 1 Joint distribution of a random vector. 1 Joint distribution of a random vector

Transcription

1 Random Vectors Joint distribution of a random vector Joint distributionof of a random vector Marginal and conditional distributions Previousl, we studied probabilit distributions of a random variable. However, in man occasions we are interested on studing more than one variable in a random eperiment. For eample, signals (sent or received) can be classified, attending to their qualit as: low, medium, high. We define number of signals of low qualit, and number of signals of high qualit. Mean Variance, Covariance, Correlation In general, if and are random variables, the probabilit distribution that describe them simultaneousl is called joint probabilit distribution Multivariate Normal distribution Joint distribution of a random vector Joint distribution of a random vector Discrete variables Given two discrete r.v.,, we define their probabilit function as: p(, ) Pr(, ) As in the univariate case, this function should satisfied that: p (, ) 0 p (, ) A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: Number of bit acceptable Number of suspicious bit The joint distribution function is: F(, ) Pr(, ) Pr(, )

2 Joint distribution of a random vector Joint distribution of a random vector A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: p (, ) 0 p (, ) Pr(, ) Joint distribution of a random vector Joint distribution of a random vector Continuous variables Continuous variables Given two continuous r.v.,, their joint densit function is defined as: f (, ) The probabilit is now a volume: Pr( a b d b, c d ) f (, ) dd a c As in the univariate case, this function should satisfie that: f(, ) The joint distribution function: f(, ) dd f(, ) df (, ) dd Pr(,.5.5) 0 0 ( 0, 0) Pr( 0, 0) (, ) F f dd 7 8

3 Joint distribution of a random vector The probabilit is now a volume: Continuous variables Pr( a b d b, c d ) f (, ) dd a c Joint distribution of a random vector Let be a random variable which represents the time until our PC connects to a server and, the time until the server recognize ou as a user. The joint densit function is given b: f (, ) 0 ep( ) 0 < < Pr(,.5.5) Pr( < 000, < 000)? 9 0 Joint distribution of a random vector Joint distribution of a random vector f (, ) 0 ep( ) 0 < < f (, ) 0 ep( ) 0 < < Pr( < 000, < 000)? Pr( < 000, < 000)? Region where the densit funtion is not Region used to compute that probabilit

4 Joint distribution of a random vector Joint distribution of a random vector f (, ) 0 ep( ) 0 < < f (, ) 0 ep( ) 0 < < 000 Pr( < 000, < 000) f (, ) dd Pr( < 000, < 000) f (, ) dd+ f(, ) dd Random Vectors Marginal distributions Joint distribution of a random vector Marginal and conditional distributions Mean Variance, Covariance, Correlation If we observe than one r.v. in an eperiment, it is important to distinguish between the joint probabilit distribution, and the probabilit distribution of each of them separatel. The distribution of each variable is called marginal distribution. Discrete variables Given two discrete r.v.,, with joint probabilit function p(, ) the marginal probabilit functions are given b: p ( ) Pr( ) Pr(, ) p ( ) Pr( ) Pr(, ) The are probabilit functions We can compute their mean, variance, etc. Multivariate Normal distribution 5

5 Marginal distributions Marginal distributions A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: The marginal probabilit functions are obtained b adding in both directions Number of bits acceptables Number of suspicious bits A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: The marginal probabilit functions are obtained b adding in both directions Number of bits acceptables Number of suspicious bits Marginal distributions Marginal distributions Continuous variables Given two continuous r.v.,, With joint densit function f (, ) the marginal densit functions are given b: f ( ) f(, ) d f ( ) f(, ) d + + The are densit functions We can compute their mean, variance, etc. 0.4 Let be a random variable which represents the time until our PC connects to a server and, the time until the server recognize ou as a user. The joint densit function is given b: f (, ) 0 ep( ) 0 < < 0.3 Pr( > 000)? f()

6 Marginal distributions f (, ) 0 ep( ) 0 < < Pr( > 000)? We can solve ir in two was: Integrate the densit function over the appropriate region Compute the marginal densit of and use it compute the probabilit Marginal distributions f (, ) 0 ep( ) 0 < < Pr( > 000)? We can solve ir in two was: Integrate the densit function over the appropriate region Pr( > 000) f (, ) dd Marginal distributions Marginal distributions f (, ) 0 ep( ) 0 < < f (, ) 0 ep( ) 0 < < Pr( > 000)? We can solve ir in two was: Pr( > 000)? We can solve ir in two was: Compute the marginal densit of and use it to compute the probabilit ( ) (, ) 0 ( ) > 0 0 f f d e e Compute the marginal densit of and use it to compute the probabilit + Pr( > 000) f ( ) d

7 Conditional distributions When we observe more than one r.v. in an eperiment, one variable ma affect the probabilities associated with the other. Conditional distributions When we observe more than one r.v. in an eperiment, one variable ma affect the probabilities associated with the other. Remember from previous lectures (Probabilit): Pr ( B A) Pr Pr ( A I B) ( A) Measures the size of one event with respect to the other 5 We can compute the mean, variance, etc. Discrete variables Given two discrete r.v.,, with joint probabilit function p(, ) the conditional probabilil function of given 0 : A B A p (, 0) Pr(, 0) p ( 0 ) p ( 0) > 0 p ( ) Pr( ) For an given 0 0 p(, ) Pr(, ) p ( ) p ( ) Pr( ) p(, ) p( ) p ( ) Conditional distributions Conditional distributions A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: Onl 4 bits are transmitted, so if 4, then0 if 3, then 0 ó M Knowing the value of changes the probabilit associated with the values of Number of bit acceptable Number of suspicious bit 7 A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: Pr( 0, 3) Pr( 0 3) 0. Pr( 3) 0.9 Pr(, 3) Pr( 3) 0.8 Pr( 3) 0.9 Number of bit acceptable Number of suspicious bit Onl 4 bits are transmitted, if 3, then 0 ó Pr( 0 3) + Pr( 3) E [ 3] Epected number of suspicious bits when the number of acceptable bits is 3 8

8 f( ) Conditional distributions f (, ) f ( ) Continuous variables Given two continuous r.v.,, With joint densit function f (, ) the marginal densit functions are given b: The are densit functions We can compute their mean, variance, etc. f (, ) f( ) f ( ) Conditional distributions Let be a random variable which represents the time until our PC connects to a server and, the time until the server recognize ou as a user. The joint densit function is given b: f (, ) 0 ep( ) 0 < < What is the probabilit that the time until the server recognize ou as user is mote than 000, if our PC has taken 500 to connect to the server? Pr( > )? 9 30 Conditional distributions Conditional distributions f (, ) 0 ep( ) 0 < < f (, ) 0 ep( ) 0 < < Pr( > )? f( ) + f f d e f (, ) f ( ) ( ) (, ) > ( ) 0.00 f e 0 < < Pr( > )? ( ) 0.00 f e 0 < < Pr( > ) f( 500) d e d 3

9 Random Vectors Joint distribution of a random vector When we observe more than one r.v. in an eperiment, one variable ma not affect the probabilities associated with the other. Marginal and conditional distributions Remember from previous lectures (Probabilit): Mean Variance, Covariance, Correlation Pr ( AI B) Pr( A) Pr( B) Pr( A B) Pr( A) Pr( B A) Pr( B) Multivariate Normal distribution Discrete variables Continuous variables Two variables, are independent if: Two variables, are independent if: p ( ) p( ) p ( ) p( ) f( ) f ( ) f ( ) f ( ) p (, ) p ( ) p ( ) p ( ) p ( ), f (, ) f( ) f ( ) f ( ) f ( ), 35 3

10 Let be a random variable which represents the time until our PC connects to a server and, the time until the server recognize ou as a user. The joint densit function is given b: f (, ) 0 ep( ) 0 < < Random Vectors Joint distribution of a random vector Marginal and conditional distributions ( ) 0.00 f e 0 < < Mean Variance, Covariance, Correlation For all values of f e e ( ) 0 ( ) > 0 37 Multivariate Normal distribution 38 Given n r.v.,, K, n an n-dimensional random vector is M n The probabilit/densit function of the vector is the joint probabilit/densit function of the vector components. Mean We define the vector of means as the vector whose components are the means of each component. μ [ ] [ ] [ ] E E M E E [ ] n 39 Covariance We start b defining the covariance between two variables: It is measure of the linear relationship between two variables ( [ ])( [ ]) [ ] [ ] [ ] Cov(, ) E E E E E E Properties If, are independent Cov(, ) 0 since E [ ] E [ ] E [ ] If Cov(, ) 0, are independent Z a + b If we change scale and origin: Cov( Z, W ) abcov(, ) W c + d 40

11 Covariance ( [ ])( [ ]) [ ] [ ] [ ] Cov(, ) E E E E E E How do we compute this? We need to compute the mean of a function of two random variables: Positive Covariance Zero Covariance E[ h(, )] + + hp (, ) (, ) h (, ) f( dd, ) The are related, but not linearl 4 Estadística. Profesora: Negative María Covariance Durbán Zero Covariance 4 A new receptor for the transmission of digital information receives bit that Are classified as acceptable, suspicious or non-acceptable, depending on The qualit of the signal received. 4 bits are transmitted, and two r.v. are defined: Number of bit acceptable Number of suspicious bit Is the covariance between and positive or negative? We know that + 4 when is close to 4, is close to 0 Therefore, the covariance is negative. Correlation The correlation between two variables is also a measure of the linear relationship between two variables. ρ (, ) Cov(, ) [ ] [ ] Var Var If, are independent ρ(, ) 0 since Cov(, ) 0 ρ(, ) If a + b ρ(, ) 43 44

12 Random Vectors Vaciance-Covariance Matri Given n r.v.,, K, n the variace-covariance matri of vector is an n n matri: Joint distribution of a random vector Marginal and conditional distributions [ ] [, ] L [, n ] [, ] [ ] M M Var Cov Cov Cov Var M E ( -μ)( -μ) M M O M Cov[, n] Var [ n] L L Properties Mean Variance, Covariance, Correlation Smmetric Positive semi-definite 45 Multivariate Normal distribution 4 As in the univariate case, sometimes we will need to obtain the probabilit distribution of a function of two or more r.v. Given a random vector with joint densit function f ( ) and it is transformed into another random vector of the same dimension, b a function g g(, K, n ) The inverse g(, K, n ) transformations M eist n gn(, K, n) d d L f( ) f( g ( )) d d dn d M M d d dn dn L d d n 47 if has lower dimension than, we complete with elements of until both have the same dimension. f Calculate the densit function of. Define. Find the joint densit of 3. Find the marginal densit of 4 0 <, < (, ) 0 elsewhere + (, ) 48

13 f 4 0 <, < (, ) 0 elsewhere 0 <, < Find the joint densit of (, ) f f g d ( ) ( ( )) d g( ) ( +, { ) g ( ) (, ) 443 d d 0 f( ) 4( ) In which region is defined? f g ( ( )) 4( ) + 0< < 0 < < 0 < < < < 0 <, < 0 < < 0 < < 0 <, < ( ) 4( ) f 0< < 0< < + 0< < 0 < < 0 < < < < -< < < < 0 < < < < -< < 5 5

14 Convolution of and Find the marginal densit of If and are independent randon variables with densit functions f 0 ( ) 3 3 4( ) 0 < < ( ) + 4 < < 3 f ( ) and f (, the densit function of is ) + ( f * f ) f ( ) f ( ) f ( ) An special case is the computation of the mean and variance of a linear transformation: A m m n n m n [ ] AE[ ] [ ] M E Var A A An special case is the computation of the mean and variance of a linear transformation: A m m n n m n [ ] AE[ ] [ ] M E Var A A [ ] [ ] + [ ] Var [ ] [ ] ( ) E E E + ( ) Cov(, ) Var Var Var Cov [ ] [ ] (, ) Cov(, ) Var + + [ ] 55 [ ] [ ] [ ] Var [ ] [ ] ( ) E E E ( ) Cov(, ) Var Var Var Cov [ ] [ ] (, ) Cov(, ) Var + [ ] 5

15 Random Vectors An special case is the computation of the mean and variance of a linear transformation: Joint distribution of a random vector A m m n n m n [ ] AE[ ] [ ] M E Var A A Marginal and conditional distributions Normal Distribution ( ) ~ N μ,,, independent i i σ i i K n a + a + K+ a Normal [ ] iμi [ ] iσ i n E a Var a i i n n n 57 Mean Variance, Covariance, Correlation Multivariate Normal distribution 58 Multivariate Normal distribution Multivariate Normal distribution If a random vector follow a bivariante Normal distibution μ with vector of means variance-covariance matri σ ρσ σ Σ it has densit function: μ μ ρσ σ σ f ( ) σ ρσ σ ep ( μ)' Σ ( μ) / ( π ) Σ ρσ σ σ Σ σ ( ρ ) ρ σσ ρ σσ σ σ σ ( ρ ) f ( ) ep ( μ)' Σ ( μ) / ( π ) Σ f μ μ μ μ, ep + ρ ( ) ( π) σ ( ) σ ( ρ ) ρ σ σ σ σ 59 0

16 The Bivariate Normal Distribution σ σ f(,) σ σ σ σ Multivariate Normal distribution μ ρ 0 ρ 0.90 Contour Plots of the Bivariate Normal Distribution σ σ ρ 0 σ σ σ σ Densit ρ 0.9 function ρ 0.9 μ μ μ Scatterplot μ μ Scatter Plots of data from the Bivariate Normal Distribution σ σ ρ 0 σ σ ρ 0.9 σ σ ρ 0.9 ρ 0.90 μ μ μ Properties σ ρσ σ ρσ σ Σ σ ρ 0, independent ( μ σ ) N( μ σ ) ~ N, ~, are Normal μ μ μ μ μ μ