Two-dimensional Random Vectors

Transcription

1 1 Two-dimensional Random Vectors Joint Cumulative Distribution Function (joint cd) [ ] F, ( x, ) P xand Properties: 1) F, (, ) = 1 ),, F (, ) = F ( x, ) = 0 3) F, ( x, ) is a non-decreasing unction 4) F, ( x, ) = F ( x) 5) [ ] P x < x, = F ( x, ) F ( x, ) 1,, 1 6) [ ] P x < x, < = F ( x, ) F ( x, ) F ( x, ) + F ( x, ) 1 1,, 1, 1, 1 1 (, ) F x, 1 (, ) F x, x1 x x (, ) F x, F, ( x, 1) F ( a, ) = lim F ( x, ) 7), +, x a

2 Joint Probabilit Densit Function (joint pd), ( x, ) x F, ( x, ) Properties 1), ( x, ) 0 ), ( xd, ) dxd= 1 3) P[ x 1 < x, < = 1 ] = x x 1 1, ( x, ) dx d x F (, ) = 4), 5), [, ( u, v) du dv ( xdxd=, ) = P x x+ dx and + d ] Marginal pd rom the joint pd For continuous rvs, ( x ) For discrete rvs, =, ( xd, ) p ( x ) = p, ( x, )

3 3 Conditional cd [ = ] F ( x) P x c1 [ = ] = lim [ +Δ ] P x P x x x = = = Δx 0 lim Δx 0 lim Δx 0, [, +Δ ] P[ x x+δx] P x x x ( x, u) Δxdu, ( x) Δx ( x, u) du ( x) ( c) Conditional pd d ( x) F ( x) d From eq. c, d, ( x, u) du d d F ( x ) = d ( x) = As the consequence,, ( x, ) ( x) ( x, ) = ( x ) ( x) = ( ) ( x ),

4 4 Example. Reer. Prob and Stoch Proc b ates and Goodman. x F ( x, ) ( uvdudv, ) =,,

5 5 Example. Conditional pd

6 6 Example. Jointl Gaussian. and are said to be jointl Gaussian i their joint pd is given b, 1/ πσ σ(1 ρ ) x μ x μ μ μ ρ + σ σ σ σ (1 ρ ) 1 ( x, ) = e g1 Note that there are ive parameters: μ, σ, μ, σ, and ρ. ρ is reerred to as the correlation coeicient between and. x μ ( x μ ) μ μ ρ + ( ) σ Exponent o. 1 σ σ σ eq g = (1 ) ρ x μ (1 )( x μ ) ( x μ ) μ μ ρ + ρ ρ + ( ) σ σ σ σ σ = (1 ρ ) x μ (1 )( x μ ) μ ρ + ρ σ σ σ = (1 ρ ) σ μ ρ ( x μ ) ( x μ ) σ = σ σ (1 ρ ) The joint pd in eq. g1 can be written as the product o two Gaussian pd's 1, ( x, ) = e πσ ( x μ ) σ 1 1/ πσ (1 ρ ) σ μ ρ ( x μ ) σ σ (1 ) e ρ ( g) Eq. g shows the relation: ( x, ) = ( x) ( x)., The irst term is the marginal pd o, ( x), which is Gaussian N μ, σ. The second term is the conditional pd, ( x), which is also Gaussian σ Nμ + ρ ( x μ ), σ(1 ρ ). σ Alternativel we could write eq. g to show ( x, ) = ( x ).,

7 7 In summar, When and are jointl Gaussian, the random variables and are marginall Gaussian. σ The conditional pd ( x) is Gaussian with mean μ + ρ x μ and variance σ 1 ρ. σ The conditional variance depends on ρ but does not depend on the condition = x.

8 8 Moments o Bi-variate Random Variables Conditional Mean E[ = x] = ( x) d Example. Discrete Bi-variate Random Variables Find the conditional mean x Correlation E[ ] x ( x, ) dxd =, and are said to be orthogonal i correlation is zero. Example Discrete Bi-variate Random Variables. Find the correlation between and x

9 9 Example Find the correlation between the jointl Gaussian Random Variables. Solution. and are jointl Gaussian i their joint pd is given b 1 ( x, ) = e, 1/ πσ σ(1 ρ ) x μ x μ μ μ ρ + σ σ σ σ (1 ρ ) We have shown or jointl Gaussian and, ( μ σ ) ( μ σ ) ( x) N,. N,. σ x Nμ + ρ ( x μ ), σ(1 ρ ). σ = x ( x, ) dxd =, dx x ( x) ( x) d σ = dx x x μ + ρ x μ σ ( ) σ = μ dx x x + ρ dx x μx ( x) σ σ = μ μ + ρ μ σ σ = μ μ + ρ σ σ = μ μ + ρσ σ

10 10 Covariance cov(, ) E[( )( )] ( )( ) Properties = ( x )( ) ( x, ) dx d, cov(, ) = and are said to be uncorrelated i cov(, ) = 0. Example For jointl Gaussian bi-variate random variables = μ. = μ. = μ μ + ρσ σ cov, = = μ μ + ρσ σ μ μ = ρσ σ Homework. Discrete Bi-variate Random Variables. Find the covariance between and x

11 11 Correlation Coeicientt (or Normalized Covarian ce) ρ, cov(, ) σ σ Example For jointl Gaussian Random Variables, ρ, cov(, ) ρσ = = σ σ σ σ σ = ρ. Plots o a jointl Gaussian pd or dierent values o correlation coeicient.

12 1 Independent Random Variables and are said to be independent i x ( x ) ( ) = or all values o,. I and are independent, ( x ) = ( x) and, ( x, ) = ( x ). Propert Independent implies uncorrelated. independent ( x, ) = ( x), cov(, ) = 0 cov(, ) ρ, = = 0. σ σ Thus and are uncorrelated. (, ) = x x dxd = x x dxd = The converse is not necessaril true. However, or jointl Gaussian random variables, the converse is true. proo σ (, ) x = Nμ+ ρ x μ, σ 1 ρ. σ I uncorrelated, ρ = 0 and thus ( μ σ ) (, x) = N, = Thus and are independent.

13 13 Cauch-Schwarz Inequalit For an pair o random variables and, E [ ] E[ ] E[ ] Homework. Prove the Cauch-Schwarz inequalit ( λ ) Hint: 0 or an random variables and, and or an constant λ. Bounds on the Correlation Coeicient For an pair o random variables and, proo 1 ρ 1, ( μ )( μ ) ( μ ) ( μ ) σ Notation: = μ and VAR = σ. cov(, ) =, using the Cauch-Schwarz inequalit, = σ cov(, ) Thereore 1 1 ρ, 1 σ σ

14 14 Variance o a Sum o Random Variables VAR( ± ) = VAR( ) + VAR( ) ± cov(, ) proo VAR( + ) = + + ( ) = + ( ) ( ) ( )( ) = + + = VAR( ) + VAR( ) + cov(, ) Properties I and are independent, cov(, ) = 0 and thus VAR( + ) = VAR( ) = VAR( ) + VAR( )