Solutions Homework 7 November 16, 2015

Size: px

Start display at page:

Download "Solutions Homework 7 November 16, 2015"

Luke Carr
5 years ago
Views:

1 Solutions Homework 7 November 6, 25 Solution to Exercise 2..5 If x and p < q, then x p x q. This follows since the exponential function is monotone increasing, so x p exp[p log x] exp[q log x] x q since p log x q log x when p < q and log x, i.e. x. Thus E[ X p ] E[ X p I [,] ( X )] + E[ X p I (, ) ( X )] E[ I [,] ( X )] + E[ X q I (, ) ( X )] P [ X ] + E[ X q ]. Clearly, E[ X q ] < implies E[ X p ] <. Solution to Exercise 2..4 (a) This problem seems to very difficult for students, but I claim it is quite easy if one proceeds by visualizing the distribution and the geometry of the transformations needed to satisfy the requirements. The distribution is concentrated on three points which determine a plane (they re obviously not collinear), and by Proposition the range space of the covariance matrix has to be this plane where the distribution lives. All we have to do to construct A is to find (firstly) a matrix which diagonalizes the covariance (which one can do by a rotation around the x-axis, as is clear from visualizing the distribution), then follow with a transformation (multiply on the left with a matrix) that rescales things. The distribution is shown in Figure. The requirement Cov[AX + b] diag(,, ) means that the support of the transformed distribution will be in a plane which is parallel to the xy plane (since the z-component has variance ). A simple rotation around the x-axis by 45 will bring the point at (,, ) down to the y-axis while leaving the other two points unchanged. The rotation matrix would be 2/2 2/2 2/2. 2/2 (How did I know this? I recall that cos and sin of 45 are 2/2, and I just fudged in the signs so the z component would disappear). In fact, we

2 2 don t need to use this one; the following will work: A /2 /2 /2 /2. Then we have A, and A (or more correctly, the linear transformation represented by the matrix A ) leaves (,, ) and (,, ) unchanged, so Law[A X] 3 { δ(,,) + δ (,,) + δ (,,) }. Now there is clearly correlation in these three points: the least squares fit is going to have a downward trend. We can get rid of the correlation by rotating in the xy-plane by 45 so that we go from an L shape to a V shape. Again, we don t actually have to do the rotation and keep track of the 2 s. It suffices to move (,, ) to (,, ) and (,, ) to (,, ). Thus, we use Then we have A 2. Law[A 2 A X] 3 { δ(,,) + δ (,,) + δ (,,) }. Thus, E[A 2 A X] /3 2/3, + +

3 3 and E[(A 2 A X)(A 2 A X) T ] /3 /3 /3 /3 [ ] + [ ] + [ ] 2/3 2/3, + /3 so Cov[A 2 A X] 2/3 2/3 2/3 2/9. 2/3 [ 2/3 ] We are almost there we have a diagonal covariance matrix but the diagonal entries are not (,, ). Ok, but this can be accomplished with a simple rescaling. Letting 3/2 A 3 3/ 2, then the desired matrix and translation vector are A A 3 A 2 A

4 4 b A 3 3/2 3/2 3/ /2 3/ 2 3/ /3 2/ 2 Note that the minus sign comes into b because adding b has to amount to subtracting the mean of AX. Let s check it all, in Splus: > Amatrix(c(,,,,.5,-.5,,.5,.5),ncol3) > A [,] [,2] [,3] [,].. [2,].5.5 [3,] > A2matrix(c(,,,-,,,,,),ncol3) > A2 [,] [,2] [,3] [,] - [2,] [3,] > A3diag(c(sqrt(3/2),3/sqrt(2),)) > A3 [,] [,2] [,3] [,] [2,] [3,].. > AA3%*%A2%*%A > A [,] [,2] [,3] [,] [2,] [3,]... > Xmatrix(c(,,,,,,,,),ncol3) > X.,

5 [,] [,2] [,3] [,] [2,] [3,] > \#note that the points where the distibution lives form > \#the columns of X > YA%*%X > b-apply(y,,mean) > YY+matrix(b,nrow3,ncol3) > Y point point point [,] [2,] [3,]... > \# we know Y is centered, so just compute average of outer products > \# of the columns to get the covariance. By the way, the outer > \# product of 2 vectors x and y is x%*%t(y) (in Splus-ese), or just > \# outer(x,y). > covymatrix(,nrow3,ncol3) > for(j in :3) covycovy+outer(y[,j],y[,j]) > covycovy/3 > covy > \# Voila! (b) Now, we suppose A is any 3 3 matrix and b and 3-dimensional vector such that if Y AX +b satisfies (i) E[Y ] and (ii) Cov[Y ] diag(,, ). We are supposed to show that the components of Y are uncorrelated but not independent. The uncorratedness is immediate since the covariance matrix is diagonal. To show it they are not independent, consider that the support of the distribution of X is 3 points. Assuming that the matrix A is nonsingular (so the linear transformation it represents is bijective or one to one and onto), then the support of Y is 3 points. Now if a probability measure is a product measure we would look at the marginal distributions which must be discrete and have as their support, 2, or 3 points, and then the product 5

6 6 measure would be supported on some number of points which is obtained by multiplying the number of points of support together. Since Law[Y ] is supported on 3 points and 3 is a prime, then two of the marginals must be supported on point and the other on 3 points. Then the two marginals supported on point are degenerate and have variance, but we see that two of the marginals have positive variance (since Cov[Y ] diag(,, )) which is a contradiction. Thus, Law[Y ] is not a product measure. Solution to Exercise 2..2: (a) The claim is that ψ(t) t log t t, which we should verify. The figure shows a plot of ψ(t) and t and it appears that the following are true: (i) t is tangent to ψ(t) at t ; (ii) ψ(t) is a strictly convex function; (iii) it is always above its tangent. Let s verify these three claims. We claim that the function ψ(t) t log t is strictly convex. The first and second derivatives are ψ (t) log t +, ψ (t) /t. As the second derivative is strictly positive, the claim (ii) follows. Also, ψ() and ψ (), so the line y t is the tangent at t (since the values and slopes agree). Claim (iii) is a general property of convex functions, but it is clear in case the function is differentiable (as it is here) since then we know the first derivative is increasing, so ζ(t) ψ(t) (t ) satisfies ζ (), ζ (t) < for t <, and ζ (t) > for t >, so ζ has a unique minimum at t. To show the integral defining K(Q, P ) exists, we will show that the integral of the negative part is finite. Now we can t do anything with log ( ) dq dq until we know it exists because none of our theorems apply. As mentioned in class, we can show the integral of the negative part is finite. There are a couple of trivial little facts about negative parts we need: Trivial Fact #. f g then f g. Trivial Fact #2. if f then (fg) fg.

7 7 So, we have ( [ ]) dq log dq. ( [ ]) dq dq log by Proposition.4.2 (a) ( [ ] ) dq dq log using Trivial Fact #2 since dq ( ) dq using Trivial Fact # and the inequality on t log t ( ) since dq (b) Now we can apply the usual results on integration as in Proposition.2.5 since we have shown the integral is well defined. ( ) dq ( ) dq dq log dq log ( ) dq by the inequality proved above dq dq. (c) Suppose K(Q, P ), and we will show P Q. Since ψ is strictly convex, by Jensen s inequality ( ) ( dq ) dq ψ ψ ψ(), with strict inequality if and only if dq is constant, P -a.s. One observes that the l.h.s. of the displayed inequality is K(Q, P ), and the r.h.s. is ψ ( dq)

8 8 log. Also, if dq is constant, P -a.s., then the constant must be because that s the integral of dq dq w.r.t. P. But, P -a.s., means Q P. Solution to Exercise 2.2. For any complex number z z +iz 2 it is clear that the modulus z z where z (z, z 2 ) is a 2-D vector. Therefore, the result will follow if we show that for any f : (Ω, F) (IR d, B d ), we have fdµ f dµ. It will be useful to have this more general result. If µ is a probability measure, then the result follows easily from Jensen s inequality. Note that the map ψ(x) x is convex since if t [, ], ψ (tx + ( t)y) tx + ( t)y tx + ( t)y t x + ( t) y tψ(x) + ( t)ψ(y). Thus, writing E[ ] in place of dµ and X in place of f we have ψ(e[x]) E[X] E[ψ(X)] E[ X ]. However, if µ is not a probability measure, this argument doesn t apply, and we are asked to prove the more general result. I have tried to find a simple proof, but it doesn t seem to exist. When you are stumped on some general result about integrals, think of simple functions. Let φ be a vector valued simple function, i.e. φ (φ,... φ d ) where each component function φ j is a simple function. For convenience, we can assume the sets A i in the representations of each φ j are the same: φ j i a ij I Ai. In a vector notation: φ i a i I Ai. Now by the triangle inequality, φdµ a i µ (A i ) i a i µ (A i ) i φ dµ,

9 which is the desired result. Now we consider a general vector valued function f. Writing f (f,..., f d ), we can find simple functions φ jn f j as n for each j, j d, and φ jn f j for all j and n. This latter inequality implies φ n f where φ n (φ n,..., φ dn ) is a vector valued simple function. Note that we can assume f dµ <, since otherwise the inequality we are trying to prove is trivial (assuming fdµ is defined; which for vector valued functions would require all components are finite, which would imply f dµ < anyway). Thus by DCT we have φ jn dµ f j dµ for j d, hence φ n dµ fdµ, and since ψ(x) x is a continuous function, we have 9 φ n dµ fdµ. Now it is easy to see that φ n is a simple function and φ n f, so by DCT φ n dµ f dµ. By the result we just proved for simple functions, we have φ n dµ φn dµ, so by the limiting results above we conclude fdµ f dµ. Solution to Exercise φ Y (v) E [ exp ( iv T Y )] E [ exp ( iv T (AX + b) )] E [ exp { i ( v T AX + v T b })] [ E exp {i ( A T v ) } T X exp { iv T b }] exp { iv T b } [ E exp {i ( A T v ) }] T X e ivt b φ X ( A T v ). The proof for the m.g.f. is basically the same - just take away the i in the derivation above.

10 Figure : Plot of the three points where the distribution in Exercise 2..4 lives. Note that the origin appears in the lower center of the diagram, the x-axis goes off to the right and the y-axis to the left.

11 x*log(x) x Figure 2: Plot of y x log(x) with line y x overlaid.

If g is also continuous and strictly increasing on J, we may apply the strictly increasing inverse function g 1 to this inequality to get

If g is also continuous and strictly increasing on J, we may apply the strictly increasing inverse function g 1 to this inequality to get 18:2 1/24/2 TOPIC. Inequalities; measures of spread. This lecture explores the implications of Jensen s inequality for g-means in general, and for harmonic, geometric, arithmetic, and related means in