5 Operations on Multiple Random Variables

Transcription

1 EE360 Random Signal analysis Chapter 5: Operations on Multiple Random Variables 5 Operations on Multiple Random Variables Expected value of a function of r.v. s Two r.v. s: ḡ = E[g(X, Y )] = g(x, y)f X,Y (x, y)dxdy N r.v. s: ḡ = E[g(X, X2,,XN)] = g(x, x2,,xn)fx,x2,,xn (x, x2,,xn)dxdx2 dxn Engineering Dept.-JUST. EE360 Signal Analysis-Electrical - Electrical Engineering Department. EE360-Random Signal Analysis-Electrical Engineering Dept.-JUST. EE360-Random Signal Analysis-Electrical Engineering Department-JUST. EE360-Random

2 Chapter 5: Operations on Multiple Random Variables 2 Ex. 5.- g(x, X 2,,X N ) = N i= α ix i =weighted sum of r.v. s E[g(X, X 2,,X N )] = E[ N i= α ix i ] = N i= α ie[x i ]

3 Chapter 5: Operations on Multiple Random Variables 3 Joint Moments about the Origin joint moment m nk = E[X n Y k ] = xn y k f X,Y (x, y)dxdy m 0 = E[X], m 0 = E[Y ] Second order moment = correlation of X and Y = m m = E[XY ] = R XY = xyf X,Y (x, y)dxdy If R XY = E[XY ] = E[X]E[Y ] X, Y are uncorrelated. If X, Y are independent, then X, Y are uncorrelated but converse is not true (except for gaussian) If R XY = 0 X, Y are orthogonal.

4 Chapter 5: Operations on Multiple Random Variables 4 Ex Y = 6X + 22, X = 3, σ 2 X = 2 m 20 = E[X 2 ] = σ 2 X + ( X) 2 = = Ȳ = E[ 6X + 22] = 6 X + 22 = = 4 R XY = E[XY ] = E[ 6X X] = 6() + 22(3) = 0 X, Y are orthogonal. R XY E[X]E[Y ] X, Y are correlated

5 Chapter 5: Operations on Multiple Random Variables 5 Example If Y = ax + b, then X, Y are always correlated if a 0. R X,Y = E[XY ] = E[aX 2 + bx] = ae[x 2 ] + be[x]. If we want X, Y to be orthogonal, i.e. R X,Y = 0 = E[aX 2 + bx] b = ae[x 2 ]/E[X]

6 Chapter 5: Operations on Multiple Random Variables 6 Joint Moments about the origin N-dim. case m n,n 2,,n N = E[X n = Xn 2 2 Xn N with n i = 0,, i =, 2,,N N ] x n xn N N f X,,X N (x,,x N )dx dx N

7 Chapter 5: Operations on Multiple Random Variables 7 Joint Central Moments µ nk = E[(X X) n (Y Ȳ )k ] = (x X) n (y Ȳ )k f X,Y (x, y)dxdy µ 20 = E[(X X) 2 ] = σ 2 X and µ 02 = E[(Y Ȳ )2 ] = σ 2 Y Covariance of X, Y : C XY = µ = E[(X X)(Y Ȳ )] = E[XY XY Ȳ X + XȲ ] = R XY XȲ Ȳ X + XȲ C XY = R XY XȲ

8 Chapter 5: Operations on Multiple Random Variables 8 Comments on covariance of X,Y : C XY = R XY XȲ X, Y uncorrelated, i.e. R XY = XȲ C XY = 0 X, Y orthogonal, i.e. R XY = 0 C XY = XȲ X, Y orthogonal and ( X = 0)or(Ȳ = 0) C XY = 0

9 Chapter 5: Operations on Multiple Random Variables 9 Correlation Coefficient ρ ρ = µ µ20 µ 02 = C XY σ X σ Y = E Using Cauchy-Schwarz inequality [( X X σ X )( Y Ȳ σ Y )] E[U 2 ]E[V 2 ] (E[UV ]) 2 show that ρ Let U = X X σ X and V = Y Ȳ σ Y we get E[( X X σ X ) 2 ]E[( Y Ȳ σ Y ) 2 ] (E[ X X σ X Y Ȳ σ Y ]) 2 σ 2 X σ 2 X σ 2 Y σ 2 Y ρ 2 ρ 2

10 Chapter 5: Operations on Multiple Random Variables 0 N r.v. s µ n n 2 n N For N r.v. s X, X 2,,X N the (n + n n N )-order joint central moment is defined by µ n n 2 n N = E [ (X X ) n (X 2 X 2 ) n2 (X N X N ) n ] N

11 Chapter 5: Operations on Multiple Random Variables Ex Let Y = N i= α ix i, with α i are real weights. Find σ 2 Y Ȳ = E[Y ] = N i= α ie[x i ] = N i= α X i i Y Ȳ = N i= α i(x i X i ) and [ σy 2 = E[(Y Ȳ N )2 ] = E i= α i(x i X i ) N j= α j(x j X ] j ) = N N i= j= α iα j E[(X i X i )(X j X j )] = N N i= j= α iα j C Xi X j σ 2 Y = N i= N α i α j C Xi X j j= For X i are uncorrelated, i.e. C Xi X j = σ 2 X i δ(i j) we get σ 2 Y = N i= α2 i σ2 X i The variance of a weighted sum of uncorrelated random variables (weights α i ) equals the weighted sum of the variances of the random variables (weights α 2 i )

12 Chapter 5: Operations on Multiple Random Variables 2 Joint Characteristic Function - 2D Fourier Transform Φ X,Y (ω, ω 2 ) = E[e jω X+jω 2 Y ] with ω, ω 2 are real numbers. Φ X,Y (ω, ω 2 ) = f X,Y (x, y)e jω x+jω 2 y dxdy f X,Y (x, y) = (2π) 2 Marginal Characteristic fcns: Φ X (ω ) = Φ X,Y (ω, 0), Φ Y (ω 2 ) = Φ X,Y (0, ω 2 ) Φ X,Y (ω, ω 2 )e jω x jω 2 y dω dω 2 Joint m nk moment: m nk = ( j) n+k n+k Φ X,Y (ω,ω 2 ) ω n ωk ω =ω 2 =0 2

13 Chapter 5: Operations on Multiple Random Variables 3 Ex on using m nk, m nk = ( j) n+k n+k Φ X,Y (ω,ω 2 ) ω n ωk ω =ω 2 =0 2 Given Φ X,Y (ω, ω 2 ) = e 2ω2 8ω2 2, find X, Ȳ, R XY, C XY X = m 0 = j Φ X,Y (ω,ω 2 ) ω ω =ω 2 =0 = j( 4ω )e 2ω2 8ω2 2 ω =ω 2 =0 = 0 Ȳ = m 0 = j Φ X,Y (ω,ω 2 ) ω 2 ω =ω 2 =0 = j( 6ω 2 )e 2ω2 8ω2 2 ω =ω 2 =0 = 0 R XY = m = ( j) 2 2 ω ω 2 e (2ω2 +8ω2 2 ) ω =ω 2 =0 = 0 C XY = R XY XȲ = 0 X, Y are Uncorrelated

14 Chapter 5: Operations on Multiple Random Variables 4 Joint Characteristic function for N r.v.s X, X 2,,X N r.v.s, then Φ X,,X N (ω,,ω N ) = E(e jω X + +jω N X N ) and the joint moments are obtained from m n n 2 n N = ( j) n + +n N n + +n N Φ X,,X N (ω, ω 2,,ω N ) ω n ωn 2 2 ωn N N all ωk =0

15 Chapter 5: Operations on Multiple Random Variables 5 Example Y = X + X X N where X i, i =, 2,,N are statistically independent r.v.s with f Xi (x i ) and Φ Xi (ω i ) Φ X,,X N (ω,,ω N ) = N Φ Xi (ω i ) i=

16 Chapter 5: Operations on Multiple Random Variables 6 Jointly Gaussian Two R.V.s f X,Y (x, y) = = 2πσ X σ e Y ρ 2 [ (x X) 2 σ 2 X ] 2ρ(x X)(y Ȳ ) (y Ȳ )2 + σ X σ Y σ 2 Y 2( ρ 2 ) ( [ x ) X 2 2ρ ( x X σ X 2πσ X σ e Y ρ 2 )( y ( Ȳ y Ȳ )+ σ X σ Y 2( ρ 2 ) σ Y ) 2 ] with ρ = C XY σ X σ Y Maximum occurs at x = X, y = Ȳ, i.e. f X,Y (x, y) f X,Y ( X, Ȳ ) = 2πσ X σ Y ρ 2

17 Chapter 5: Operations on Multiple Random Variables 7 Jointly Gaussian Uncorrelated R.V.s are Independent Case of uncorrelated X, Y, i.e. ρ = 0 f X,Y (x, y) = = 2πσ X σ e Y ρ 2 [ e 2πσ X σ Y (x X) 2 σ 2 X [ (x X) 2 σ 2 X ] (y Ȳ )2 + σ 2 Y ] 2ρ(x X)(y Ȳ ) (y Ȳ )2 + σ X σ Y σ 2 Y 2( ρ 2 ) Uncorrelated Gaussian X, Y f X,Y (x, y) = f X (x)f Y (y) X, Y independent. 2

18 Chapter 5: Operations on Multiple Random Variables 8 Can we remove correlation between 2 r.v. s by proper rotation θ? Ex For any X, X 2 r.v.s, we can form two new r.v.s Y, Y 2 by rotating the axes an angle θ to make Y, Y 2 uncorrelated. X2 Y x2 (x,x2)=(y,y2) Y2 y Y = X cos θ + X 2 sinθ Y 2 = X sinθ + X 2 cos θ Want θ that makes C Y Y 2 = 0. y2 θ x X

19 Chapter 5: Operations on Multiple Random Variables 9 C Y Y 2 = µ = E[(Y Ȳ)(Y 2 Ȳ2)] = E[{(X X ) cos θ + (X 2 X 2 ) sinθ} { (X X ) sinθ + (X 2 X 2 ) cos θ}] = E[ (X X ) 2 cos θ sinθ + (X 2 X 2 ) 2 sinθ cos θ + (X X )(X 2 X 2 )(cos 2 θ sin 2 θ)] = σx 2 sinθ cos θ + C X X 2 cos 2 θ C X X 2 sin 2 θ + σx 2 2 sinθ cos θ = 2 (σ2 X σx 2 2 ) sin(2θ) + C X X 2 cos(2θ) Set C Y Y 2 = 0 we get 2 (σ2 X σ 2 X 2 ) sin(2θ) = C X X 2 cos(2θ) = ρσ X σ X2 cos(2θ) tan(2θ) = 2ρσ X σ X2 σ 2 X σ 2 X 2 θ = 2 tan ( 2ρσX σ X2 σ 2 X σ 2 X 2 )

20 Chapter 5: Operations on Multiple Random Variables 20 Jointly Gaussian - N - r.v.s X,X 2,,X N f X,,X N (X,,X N ) = [C X] /2 e { (2π) N/2 2 (x X) t [C X ] (x X)} where (x X) = x X x 2 X 2. x N X N C ij = E[(X i X i )(X j X j )] =, C X = σ 2 X i C Xi X j C C 2 C N C 2 C 22 C 2N... C N C N2 C NN i = j i j

21 Chapter 5: Operations on Multiple Random Variables 2 Case of N = 2 [C X ] = σ 2 X ρσ X σ X2 ρσ X σ X2 σ 2 X 2, [C X ] = ( ρ 2 )σ 2 X σ 2 X 2 [C X ] = ρ 2 σ 2 X f X,X 2 (x, x 2 ) = σ 2 exp{ t 2 (x X) ρ 2 f X,X 2 (x, x 2 ) = ρ σ X σ X2 ρ σ X σ X2 σ 2 X 2 X σ 2 X 2 ( ρ 2 ) σ 2 X exp{ 2πσ 2 X 2πσ 2 ( ρ 2 ) X 2 (2π) 2/2 ρ σ X σ X2 ρ σ X σ X2 σ 2 X 2 (x X)} can verify [ (x X ) 2 2σ 2 X + (x 2 X 2 ) 2 2σ 2 X 2 2ρ(x X )(x 2 X 2 ) 2σ X σ X2 ]}

22 Chapter 5: Operations on Multiple Random Variables 22 Notes on Gaussian r.v.s. Only mean, variance, and covariance are needed to completely characterize gaussian r.v.s. 2. Uncorrelated statistically independent, 3. X i, i =, 2,,n are gaussian, n i= a ix i is gaussian. 4. Any k-dim marginal density is also gaussian. 5. Conditional density is also gaussian, i.e., f X,X 2,,X k (x, x 2,,x k {X k+ = x k+,,x N = x N }) gaussian.

23 Chapter 5: Operations on Multiple Random Variables 23 Linear Transformation of Multiple r.v.s Y = TX where Y is an N vector, T is an N N matrix, X is an N vector E[Y ] = TE[X] E[Y Y t ] = E[TXX t T t ] R Y = TR X T t also E[(Y Ȳ )(Y Ȳ )t ] = E[T(X X)(X X) t T t ] from which we get C Y = TC X T t

24 Chapter 5: Operations on Multiple Random Variables 24 Ex Transformation of Multiple r.v.s Gaussian X N(0, 4), X 2 N(0, 9), C X X 2 = 3. Let Y = X 2X 2, Y 2 = 3X + 4X 2. Find means, variances, and covariance of Y and Y 2. E[Y ] = E[X ] 2E[X 2 ] = 0, E[Y 2 ] = 0. E[Y 2 ] = E[X 2 ] 4C X X 2 + 4E[X 2 2] = = 28 E[Y 2 2 ] = E[9X 2 ] + 24C X X 2 + 6E[X 2 2] = = 252 E[Y Y 2 ] = E[3X 2 2X X 2 8X 2 2] = = 66

25 Chapter 5: Operations on Multiple Random Variables 25 Example Gaussian random vector, X N(µ, C X ) with 0 µ = 5, C X = Find. pdf of X : marginal of jointly gaussian is gaussian. X N(, ) 2. pdf of X 2 + X 3 : here C 23 = C 32 = 0 X 2, X 3 uncorrelated, since gaussian independent. Sum of two jointly gaussian is also gaussian. Mean will add and variance will add. X 2 + X 3 N(7, 3). 3. pdf of 2X + X 2 + X 3 : linear combination of gaussian r.v.s, i.e.

26 2X + X 2 + X 3 = [2,, ] Chapter 5: Operations on Multiple Random Variables 26 X X 2, X 3 mean=µ = 2 X + X 2 + X 3 = 2() + (5) + (2) = σ 2 = [2,, ] 4 0 = [3, 6, 9] = X + X 2 + X 3 N(9, 2) 4. pdf of X 3 (X, X 2 ) = f(x 3 X, X 2 ) =? C 23 = C 3 = 0 X 3, X 2 stat. independent and X 3, X stat. independent, f(x 3 X, X 2 ) = f(x 3 ) X 3 (X, X 2 ) N(2, 9) 5. P {2X + X 2 + X 3 < 0} =? Y = 2X + X 2 + X 3 as in previous part N(9, 2)

27 Chapter 5: Operations on Multiple Random Variables 27 P {Y < 0} = Φ( 0 Ȳ σ Y Y = TX, with T = 2 Ȳ = T X = C Y = TC X T t = ) = Φ( 9 2 ) = Φ(.96) = Φ(.96) = 2 hence Y N( Ȳ, C Y ) = =

28 Chapter 5: Operations on Multiple Random Variables 28 Sampling and Some Limit Theorems Sampling and Estimation Estimation of Mean, Power, and Variance Given N samples x n representing values of independent (at least pair-wise) identically distributed X n, n =, 2,,N. Define the r.v. ˆ XN as follows ˆ x N = N N n= x n its mean E[ ˆ XN ] = N N E[X n ] = X, for any N n= This is an Unbiased estimator mean of estimate=mean of the r.v. and its variance σ 2ˆ = E[( ˆ XN X) 2 ] = E[ ˆ X2 X N 2 X ˆ XN + X 2 ] N

29 Chapter 5: Operations on Multiple Random Variables 29 = ˆ X2 + E[ N = ˆ X2 + N 2 N n= N n= m= X n N N N m= X m ] E[X n X m ] Since X n and X m are pairwise independent identically distributed, then X 2 for m n E[X n X m ] = E[X 2 ] for m = n N N E[X n X m ] = NE(X 2 ) + (N 2 N) X 2 n= m=

30 Chapter 5: Operations on Multiple Random Variables 30 Hence σ 2ˆ = ˆ X2 + X N N 2 [NE(X2 )+(N 2 N) X 2 = N [E(X2 ) X 2 ] = σ2 X N 0 as N. hence σ ˆ X2 N Using Chebychev s inequality, P { ˆ XN X σ 2ˆ X < ǫ} ( N ǫ 2 Consistent estimator: Weak Law of Large Numbers: = σ2 X Nǫ 2 ) as N ˆ X N X with probability as N lim P { ˆ XN X < ǫ} =, for any ǫ > 0 N Strong Law of Large Numbers: P { lim ( ˆ XN ) = X } N =

31 Chapter 5: Operations on Multiple Random Variables 3 Complex Random Variables Z = X + jy, X, Y are real r.v.s E[g(Z)] = g(z)f X,Y (x, y)dxdy Z = X + jȳ σ 2 Z = E[ Z E[Z] 2 ] For two complex r.v.s Z m, Z n : joint pdf f Xm,Y m,x n,y n (x m, y m, x n, y n ) If f Xm,Y m,x n,y n (x m, y m, x n, y n ) = f Xm,Y m (x m, y m )f Xn,Y n (x n, y n ) Z m, Z n statistically independent. Can extend to N r.v.s

32 Chapter 5: Operations on Multiple Random Variables 32 Complex vars.: Correlation, Covariance, Independence, Orthogonal Correlation R Zm Z n = E[Z mz n ], n m Covariance C Zm Z n = E[(Z m Z m ) (Z n Z n )], n m Uncorrelated Complex r.v.s R Zm Z n = E[Z m]e[z n ], n m, C Zm Z n = 0 Independence Uncorrelation Orthogonal R Zm Z n = E[Z mz n ] = 0

33 Chapter 5: Operations on Multiple Random Variables 33 Summary Extend chapter 3 to work on multiple random variables. Topics extended were: Expected values were developed of functions of random variables, which included both joint moments about the origin and central moments, as well as joint characteristic functions that are useful in finding moments. New moments of special interest were correlation and covariance. Multiple gaussian random variables were defined. Transformation results were used to show how linear transformation of jointly gaussian random variables is especially important, as it produces random variables that are also joint gaussian. Some new material on the basics of sampling and estimation of mean, power, and variance was given. Definition of complex random variables and their characteristics.