SIMG-713 Homework 5 Solutions

Transcription

1 SIMG-73 Homework 5 Solutions Spring 00. Potons strike a detector at an average rate of λ potons per second. Te detector produces an output wit probability β wenever it is struck by a poton. Compute te DQE of te detector. Solution: Te input X is a Poisson random variable wit mean and variance µ x = σ x = λ. Te output is also a Poisson random process, but now te rate is βλ, so te output as mean and variance µ y = σ y = βλ. Te DQE is te ratio of te output SNR to te input SNR, or µ SNRo DQE = = µ y/σ y SNR I µ x/σ x = βλ λ = β. Tis problem examines te correlation and covariance of two random variables X and Y. Te covariance function as been deþned as cov(x, Y )=E (X µ x ) Y µ y were µ x = E [X] and µ y = E [Y]. Te correlation coefficient is deþned to be te ratio ρ xy = were, respectively, σ x and σ y are te variances of X and Y. cov (X, Y) σ x σ y (a) Tis is a result tat will be needed in some oter parts of tis problem. Sow tat te function v = au + bu + c is positive for all values of u provided c>0 and b < 4ac. Te quantity d = b 4ac is called te discriminant. Te condition tat v>0 for all values of u is equivalent to d<0. Solution: v(0) = c, so c 0. To Þnd te minimum point solve v 0 (u) =au + b =0wic yields u min = b/a. Ten v (u min )=a b /4a b (b/a)+c = b /4a + c 0 implies d = b 4ac 0. Anoter metod is to solve for te roots of v by te quadratic formula. Tis yields roots at b ± b 4ac /a. Te roots will be real, and terefore v will cross into negative territory, unless te term under te radical is negative. Tis requires d 0. (X (b) Sow tat cov(x, Y) σ x σ y. Hint: Look at E µx ) u + i Y µ y as a function of u. Make use of te discriminant. (X Solution: Clearly E µx ) u + i Y µ y 0. We now need to identify te discriminant. Expand te expression: (X E µx ) u + i Y µ y = E (X µ x ) i u +E (X µ x ) Y i Y µ y u + E µy = σ Xu +cov(x, Y)u + σ y Identify te discriminant terms as a = σ X,b=cov(X, Y) and c = σ y. Since we know tat d 0 we ave 4cov (x, y) 4σ X σ y. Te result follows. (c) Sow tat ρxy. Solution: Since ρ =cov(x, Y) /σ X σ y, tis result follows directly from te above.

2 (d) Sow tat for any two random variables X and Y, E [XY] E X E Y. Solution: Let v = E (Xu + Y) i and follow te metod of part (b). (e) Sow tat if E X = E Y ten S = X + Y and D = X Y are ortogonal random variables. Solution: E [SD] =E [(X + Y)(X Y)] = E X E Y =0. (f) Sow tat if random variables X and Y are uncorrelated ten σ s = σ x +σ y,weres = X + Y. Extend tis to te sum of any number of uncorrelated random variables. Note tat we once proved tis for statistically independent random variables, but tat tis requirement is new because it is weaker. Solution: First compute te mean value µ s = µ x + µ y. Tis is always true. Ten σ s = E (S µ s ) i = X i E + Y µx µ y = E (X µ x ) i E (X µ x ) Y i Y µ y + E µy. If X and Y are uncorrelated ten te middle term is zero, and te result follows. 3. If X and Y are jointly normal random variables, ten so are random variables S =ax + by and D =cx+dy for any real coefficients (a, b, c, d). (a) Given µ x =0,µ y =0, σ x =, σ y =, ρ xy =0.5 construct a contour plot of for f XY (x, y) sowing te locus of constant probability points Solution: Te joint probability density function is f XY (x, y) = = p πσ x σ y ( ρ ) exp " π () ().75 exp x 0 ³ x µx σ x ³ ³ ³ ρ x µx y µy y µy σ x σ y + ( ρ ) x 0 (0.75) # (y)+y σ y Te probability is constant on contours were te exponential term is constant. Tis requires tat µ µ x 0 x 0 (y)+y = C

3 Contours of constant probabilities form ellipses as sown in te Þgure below. (b) Find te probability density function f SD (x, y) for te random variables S = X + Y and D = X Y. Solution: We can Þnd te parameters µ s,µ d, σ s, σ d and ρ sd and ten substitute into te general form for a bivariate normal distribution. We Þnd µ s = µ x + µ y = 0,µ d = µ x µ y = 0, σ s = E (S µ s ) i = (X E µx )+ i Y µ y = E (X µ x ) i +E (X µ x ) Y i Y µ y + E µy = σ x +ρ xy σ x σ y + σ y =4+ ( 0.5) + = 7. Similarly, σ d (D µ = E d ) i (X = E µx ) i Y µ y = σ x ρ xy σ x σ y + σ y =4 ( 0.5) + = 3 cov (S, D) =E [(S µ s )(D µ d )] = E (X µ x )+ Y µ y (X µx ) Y µ y = E (X µ x ) i Y i E µy = σ x σ y =3. Finally, ρ sd =cov(s, D) /σ s σ d =3/ 3 7=0.65

4 (c) Construct a contour plot of for f SD (x, y) sowing te locus of constant probability points. 4. Let X and Y be random variables wit a joint probability density function f XY (x, y). Let Ŷ = g (X) be a predictor of Y. (a) Sow tat te mean-squared prediction error can be expressed as ³ E Y Ŷ = E Y E [g 0 (X) g (X)] + E g (X) were g 0 (X) =E [Y X]. Solution: Expand te expectation and substitute for Ŷ = g (X). E ³Y Ŷ = E Y i E Y Ŷ + i E Ŷ = E Y E [Yg(X)] + E g (X). We only need to sow tat te middle term yields E [Yg(X)] = E [g 0 (X) g (X)]. Write out te expectation in terms of te joint probability function: E [Yg(X)] = X X Yg(X) P (X, Y ) x y = X g (X) P (X) X YP(Y X) x y = X x P (X)(g (X) E [Y X]) = E [g (X) g 0 (X)] were E [Y X] = g 0 (X) = P y YP(Y X) is a function of X obtained by computing te mean value of te conditional probability function P (Y X). Tis function is equal to te mean value of te conditional probability function on Y wen X is given and sown below to be te best estimate based on te given information about X.

5 (b) Let X and Y be random variables wit a joint probability density function f XY (x, y). Let Ŷ0 = g 0 (X) =E [Y X] be a predictor of Y. Sow tat te mean-squared prediction error can be expressed as ³ E Y Ŷ0 = E Y E g0 (X) ³ Solution: E Y Ŷ0 = E Y E [Yg 0 (X)] + E g0 (X). Temiddletermnowbecomes E [Yg 0 (X)] = X x = X x X Yg 0 (X) P (X, Y ) y g 0 (X) P (X) X YP(Y X) y = X P (X)(g 0 (X) E [Y X]) x = E g0 (X) ³ and terefore combines wit te last term. Tis yields E Y Ŷ0 = E Y E g0 (X) (c) Sow tat te previous two problems establis tat E ³ Y Ŷ E ³ Y Ŷ0 for any prediction function Ŷ = g (X). Tat is, te conditional expectation of Y given X gives te least-meansquare prediction of Y. Tis sows tat g 0 (X) =E [Y X] is te optimum predictor function and provides a tool to Þnd te optimum predictor. Solution: By subtraction of te above results we Þnd ³ ³ E Y Ŷ E Y Ŷ0 = E g (X) E [g (X) g 0 (X)] + E g0 (X) = E (g (X) g 0 (X)) i 0 5. Suppose tat Y = αx + β + Z were Z is a random variable statistically independent of X wit mean E [Z] =µ z. Wat is te optimum predictor function Ŷ = g (X), based on te above results? Solution: g 0 (X) =E [αx + β + Z X] = P z (αx + β + Z) P (Z X) =P z (αx + β + Z) P (Z) because Z is statistically independent of X. Ten Ŷ0 = g 0 (X) =αx + β + µ z. 6. In ;tis problem we will construct a formulation of te probability density function for te bivariate normal distribution based on te covariance matrix and mean values. Tis approac extends to any number of dimensions and is very useful in constructing algoritms. We begin by assuming tat X =[X,X ] T is a column vector wose elements are statistically independent normal random variables. (a) Sow tat f X (x) = π det (Γ) / e (x m x) T Γ (x m x )/

6 were m =[E [X ],E[X ]] T is a column vector of te mean values and Γ is te covariance matrix var (X Γ = ) cov(x,x ) σ = 0 cov(x,x ) var(x ) 0 σ In reality, tis is just a compact way to express te equation ³ ³ i f XX (x,x )= exp x m x + m s s πσ σ Solution: Te inverse is Γ /σ = 0 0 /σ as can be veriþed by computing ΓΓ = I. Ten (x m x ) T Γ (x m x ) = /σ X m X m 0 X m 0 /σ X m = (X m ) /σ (X m ) /σ X m X m = (X m ) /σ +(X m ) /σ wic is just te exponential term for te bivariate distribution wen X and X are statistically independent. Te term det (Γ) / = σ σ wic is te required factor in te denominator for te multiplier of te exponential function. Wen te substitutions are made te result is te known bivariate pdf above. (b) Let G be a square matrix wit det (G) 6= 0and let Y = GX. Tat is, Y is a vector of random variables formed by a linear combination of elements of X. Te only restriction we are making is tat te transformation sould ave an inverse. Sow tat m y = Gm x. Tis means tat Y m y = G (X m x ). Solution: Eac component of Y is just a function of te elements of X. For example, Y = g X + g X. Tis is just a function of two random variables, so tat we can calculate te mean value by te usual metod: E [Y ]=g E [X ]+g E [X ]=g m x + g m x. Similarly, E [Y ]=g E [X ]+g E [X ]=g m x + g m x. Te same results are produced by multiplying m x by G. (c) Sow tat te covariance matrix for Y is Λ = GΓG T Solution: var (Y ) = E (Y m y ) i = E (g (X m x )+g (X m x )) i = ge (X m x ) i +g g E [(X m x )(X m x )] + ge (X m x ) i = g var (X )+g g cov (X,X )+g var (X )

7 Similarly, var (Y )=g var (X )+g g cov (X,X )+g var (X ). Finally, cov (Y,Y ) = E [(Y m y )(Y m y )] = E [(g (X m x )+g (X m x )) (g (X m x )+g (X m x ))] = g g var(x )+(g g + g g )cov(x,x )+g g var (X ) Terefore, var (Y Λ = ) cov(y,y ) g g = var (X ) cov(x,x ) g g cov(y,y ) var(y ) g g cov(x,x ) var(x ) g g as can be veriþed by multiplying out te matrix expression on te rigt. (d) One can make a cange of variables in n dimensions by f Y (y) =f X G y det G Te exponent is transformed by (x m x ) T Γ (x m x ) = G T y m y G ΛG T G y m y = T y m y G T G T Λ GG y m y = T y m y Λ y m y Also, det (Γ) =det G ΛG T =det(λ)det G. Wen all tis is substituted back we Þnd f Y (y) = π det (Λ) / e (y m y) T Λ (y m y )/ Tis is exactly te same form, but now it accommodates random variables tat are not uncorrelated. Tis is a demonstration tat a linear transformation of normal random variables produces anoter set of normal random variables. Assume tat we are working in D and tat X and X are statistically independent normal random variables. Find expressions for m y,m y, σ y, σ y, and ρ in erms of m x,m x, σ x, σ x and te elements of te transformation matrix g g G = g g Assume det G 6=0. Write an expression for f Y (y) in terms of m y,m y, σ y, σ y, and ρ. Solution: Above we computed expressions for m y = g m x + g m x my = g m x + g m x σ y = g σ x +g g cov (X,X )+g σ x σ y = g σ x +g g cov (X,X )+g σ x cov (Y,Y ) = g g σ x +(g g + g g )cov(x,x )+g g σ x We can compute te correlation coefficient ρ y = cov (Y,Y ) σ y σ y using te above components. From tis te probability density function can be assembled. Incidentally, te above equations can be used to ceck te results of Problem 3(b).