APPM/MATH 4/5520 Solutions to Exam I Review Problems. (a) f X (x ) f X,X 2 (x,x 2 )dx 2 x 2e x x 2 dx 2 2e 2x x was below x 2, but when marginalizing out x 2, we ran it over all values from 0 to and so there was no upper bound on x. The final answer for the marginal pdf of X is f X (x) 2e 2x I (0, ) (x). That is, X exp(rate 2). f X2 (x 2 ) f X,X 2 (x,x 2 )dx x 2 0 2e x x 2 dx 2e x2 2e 2x 2 x 2 was above x, but since there was no lower bound (other than 0) on x when marginalizing out x, x 2 ends up going all the way from 0 to. The final answer for the marginal pdf of X 2 is f X2 (x) 2e x2 ( e x2 )I (0, ) (x). This distribution does not have a name. (So sad!) (b) No, X and X 2 are not independent because f X,X 2 f X (x ) f X2 (x 2 ). You can also tell that they are dependent because of the triangular region indicated by the indicators. (c) We have y g (x,x 2 ) 2x and y 2 g 2 (x,x 2 ) x 2 x which gives us x g (y,y 2 ) y /2 and x 2 g2 (y,y 2 ) y /2+y 2. The Jacobian of this transformation is x x y y 2 /2 0 J /2 /2 x 2 y x 2 y 2 Now f Y,Y 2 (y,y 2 ) f X,X 2 (g (y,y 2 ),g 2 (y,y 2 )) J 2e y /2 (y 2 y /2) I (0,y /2+y 2 )(y /2)I (0, ) (y /2+y 2 ) /2 e y y 2 I (0,y /2+y 2 )(y /2)I (0, )
The first indicator: I (0,y /2+y 2 )(y /2) 0 < y /2 < y /2+y 2 y /2 < 0 < y 2 So we need to at least have y > 0 and y 2 > 0. The second indicator: I (0, ) (y /2+y 2 ) 0 < y /2+y 2 < While this does not mean that, necessarily, y > 0 and y 2 > 0, we already know that y > 0 and y 2 > 0 and this second indicator does not constrain things any further. Therefore, I (0,y /2+y 2 )(y /2) I (0, ) (y /2+y 2 ) I (0, ) (y ) I (0, ) (y 2 ). Hence, the joint pdf becomes f Y,Y 2 (y,y 2 ) e y y 2 I (0, ) (y ) I (0, ) (y 2 ) which factors into a y part and a y 2 part. This shows that Y and Y 2 are independent. 2. First note that P(X x) ( p) x pi {0,,2,...}. So, P(Y y) P(X + y) P(X y ) ( p) y pi {0,,2,...,} (y ) ( p) y pi {,2,3,...,} (y) Therefore, Y geom (p).
3. y g(x) e x x g (y) ln(y +) Here f X (x) λe λx I (0, ) (x). So, f Y (y) f X (g (y)) y g (y) λe λln(y+) I (0, ) (ln(y +)) λe ln(y+) λ I (0, ) (y) y+ y+ λ(y +) λ I (0, ) (y) y+ (We can drop the absolute value since the indicator tells us that the thing inside will be positive.) So, λ f Y (y) (y +) λ+ I (0, )(y) which implies that Y P areto(λ). 4. (a) f X (x) γ (+x) γ+ I (0, ) (x). y g(x) ln(x+) x g (y) e y The pdf for Y is then f Y (y) f X (g (y)) d dy g (y) γ (+e y ) γ+ I (0, ) (e y ) e y For the indicator, 0 < e y < < e y <. Taking ln all the way across gives 0 < y <. Simplifying everything gives f Y (y) γe γy I (0, ) (y) which is the pdf of an exponential distribution with rate γ. Thus, Y exp(rate γ)
(b) You could go through part(a) of this problem and try to figure out what you could have done at the start that would cancel the γ out of that problem and then think again about how you would have introduced the λ in the right spot. Or, start from the end where you concluded that Y exp(rate γ). If you think about the g procedure on this, it becomes clear that γy will be an exponential random variable with rate and then that Z γy/λ will be an exponential random variable with rate λ. Since the original Y was Y ln( +X), we will need Z (γ/λ)y (γ/λ)ln(+x) in order to end up with an exponential distribution with rate λ. That is, z g(x) (γ/λ)ln(+x). 5. So, M X (t) E[e tx ] E[e t n n i X i ] iid [M X (t/n)] n [ ] λ n [ ] λn n λ t/n λn t X Γ(n,λn) (Alternatively, you could say that n i X i Γ(n,λ) and then use the g inverse method to find the pdf of /n times Y where Y Γ(n,λ). ) 6. M Y (t) E[e ty ] E[e t(x +X 2 ) ] E[e tx e tx 2 ] indep E[e tx ] E[e txt ] M X (t) M X2 (t) Since Y χ 2 (n) Γ(n/2,/2), we know that the mgf for Y is M Y (t) ( ) /2 n/2. /2 t Similarly, since X χ 2 (n ) Γ(n /2,/2), we know that the mgf for X is M X (t) ( ) /2 n /2. /2 t Now M Y (t) M X (t) M X2 (t) M X2 (t) M Y (t)/m X (t)
( ) /2 n/2 /2 t ( ) /2 /2 t n /2 ( /2 /2 t which we recognize as the mgf for a Γ( n n 2, 2 Therefore, X 2 χ 2 (n n ). ) (n n )/2 ) χ 2 (n n ). 7. Using moment generating functions, M Y (t) iid [M X (t)] n [ ] pe t n ( p)e t which is the moment generating function for a negative binomial distribution. (The second one on the table that starts from r.) 8. (a) The Beta(, ) distribution is the same thing as the unif(0, ) distribution. You know this even if you do not know what the Beta function is because the pdf is some constant on the interval (0,). The only possibility for that constant is for it to be! (b) We first need to find the pdf for the maximum. First, note that the cdf for any one X unif(0,) is 0, x < 0 F(x) x, 0 x <, x Thus, F X(n) (x) P(X (n) x) P(max(X,X 2,...,X n ) x) P(X x,x 2 x,...,x n x) indep P(X x) P(X 2 x) P(X n x) ident [P(X x)] n unif x n So, f X(n) (x) d dx F X (n) (x) nx n.
To complete the pdf, we include the domain of X (n) : f X(n) (x) nx n I (0,) (x). If we didn t recognize this pdf, we would proceed to find the expected value and variance of X (n) by computing the appropriate integrals. However, this is the Beta pdf with parameters a n and b. That is X (n) Beta(n,). So, we can just look up the mean and variance! E[X (n) ] n n+ and Var[X (n) ] n (n+) 2 (n+2) (c) E[X (n) ] Var[X (n) ]+(E[X (n) ]) 2 ( ) n n 2 (n+) 2 (n+2) + n+ 9. (a) Let s try the sample mean X m mi X i. Then (b) E[X] E[X i ] np. So, putting a /n on front of X will do the trick. In other words, we have an unbiased estimator ˆp n X m X i. nm (Note: There is more than one correct answer. For example, another unbiased estimator is ˆp 2 n X.) i MSE[ˆp] MSE[X] unbiased [ ] Var n X n 2 Var[X] n 2 Var[X ] m n 2 np( p) m p( p) nm 0. There are many possible answers. For example, the mean of this distribution is θ/2. Since X is always unbiased for the mean, we know that E[X] θ 2. Therefore, ˆθ : 2X is an unbiased estimator of θ.
Another estimator that makes sense to use here since θ is the upper endpoint for possible values in the sample is the maximum value in the sample. Is X (n) unbiased though? F X(n) (x) P(X (n) x) P(X x,x 2 x,...,x n x) indep P(X x)p(x 2 x) P(X n x) ident [P(X x)] n [ ] x n θ So f X(n) (x) d dx F X (n) (x) n θ ( ) x n I θ (0,θ)(x) Notethattheindicatorwasjusttacked ontotheend thedomainwasfiguredoutby common sense. ie: All the individual x s had to be between 0 and θ so the max of the x s has to be between 0 and θ! Now for the expectation. So, another unbiased estimator of θ is E[X (n) ] x f X (n) (x)dx θ ( x n 0 xn θ θ) n θ n θ 0 xn dx n θ n θn+ n+ n n+ θ ˆθ 2 n+ n X (n).. M X (t) E[e tx ] x0 e tx P(X x) e (0)(t) ( p)+e ()(t) p p+pe t
2. Let Y X /X 2 and let Y 2 be anything. When dealing with a ratio, it is usually convenient to let Y 2 be the denominator, so I will do just that. Let Y 2 X 2. So Solving for x and x 2 : y g (x,x 2 ) x /x 2, y 2 g 2 (x,x 2 ) x 2. This is how you define the inverse functions: The Jacobian of the transformation is x J So, the joint pdf for Y and Y 2 is x 2 y 2, x y x 2 y y 2. x : g (y,y 2 ) y y 2, x 2 : g 2 (y,y 2 ) y 2. x y y 2 x 2 x 2 y y 2 y 2 y 0 y 2 f Y,Y 2 (y,y 2 ) f X,X 2 (g (y,y 2 ),g 2 (y,y 2 )) J To simplify the indicators, write them out: 8(y y 2 )y 2 I (0,y2 )(y y 2 )I (0,) (y 2 ) y 2 8y y 2 2 y 2 I (0,y2 )(y y 2 )I (0,) (y 2 ) 0 < y y 2 < y 2, 0 < y 2 <. Sincethe second inequality tells us that y 2 > 0, wecan divideeverything in thefirstinequality through by y 2 and not have to worry about dividing by zero or flipping inequalities because of dividing by a negative. We now have 0 < y <, 0 < y 2 <. I m going to replace the indicators in the pdf and, at the same time, drop the absolute value since we now know that y 2 > 0. The final joint pdf is then f Y,Y 2 (y,y 2 ) 8y y 3 2 I (0,) (y )I (0,) (y 2 ). Finally, we only wanted the pdf of Y X /X 2 so we will marginalize out y 2 : f Y (y ) f Y,Y 2 (y,y 2 )dy 2 8y y 3 2 I (0,)(y )I (0,) (y 2 )dy 2 0 8y y 3 2 I (0,)(y )dy 2 2y y2 4I (0,)(y ) 2y I (0,) (y 2 ) y 2 y 2 0 (Note: We are done since I didn t ask for a name, but this is a Beta(2,) pdf!)
3. Let Y X /X 2 and let Y 2 X 2. We will find the joint pdf of Y and Y 2 and then integrate out the y 2. By independence of the X s, We have which implies that f X,X 2 (x,x 2 ) f X (x ) f X2 (x 2 ) 2e 2x I (0, ) (x ) 2e 2x 2 I (0, ) (x 2 ) 4e 2(x +x 2 ) I (0, ) (x )I (0, ) (x 2 ) y g (x,x 2 ) x /x 2 and y 2 g 2 (x,x 2 ) x 2 x g (y,y 2 ) y y 2 and x 2 g 2 (y,y 2 ) y 2. The Jacobian of the transformation is x J x y y 2 x 2 x 2 y y 2 y 2 y 0 y 2. So, f Y,Y 2 (y,y 2 ) f X.X 2 (g (y ),y 2,g 2 (y,y 2 )) J 4e 2(y y 2 +y 2 ) I (0, ) (y y 2 )I (0, ) (y 2 ) y 2 The first indicator says that 0 < y y 2 < which means that we either have both y > 0 and y 2 > 0, or, we have both y < 0 and y 2 < 0. However, the second indicator says that y 2 > 0, thereby ruling the second ( negative negative ) possibility out. Therefore, the product of indicators is equivalent to the product I (0, ) (y ) I (0, ) (y 2 ). Since y 2 is positive, we can drop the absolute value on y 2 in the joint pdf for Y and Y 2. So f Y,Y 2 (y,y 2 ) 4y 2 e 2(y y 2 +y 2 ) I (0, ) (y )I (0, ) (y 2 ) Now f Y (y ) f Y,Y 2 (y,y 2 )dy 2 0 4y 2 e 2(y y 2 +y 2 ) I (0, ) (y )dy 2 4I (0, ) (y ) 0 y 2 e 2(y +)y 2 dy 2 The integral is almost that of the pdf of the Γ(2,2(y + )) distribution and we can put in (and adjust for) the correct constants in order to get an integral of.
To change things up a bit, note also that the integral is almost like the mean of an exponential random variable with rate 2(y +) (which would be /[2(y +)]). We can put in the rate: f Y (y ) 4I (0, ) (y ) 2(y +) 4I (0, ) (y ) 2(y +) (y +) 2 I (0, ) (y ) 0 y 2 2(y +)e 2(y +)y 2 dy 2 2(y +) This is the pdf of the Pareto distribution with parameter γ. That is, Y X /X 2 Pareto(). 4. (a) E[X] x f X(x)dx 0 x Γ(α) βα x α e βx dx 0 Γ(α) βα x α e βx dx This is almost like a Γ(α+,β) pdf. To get it just right, we need Γ(α+) in place of Γ(α) and we need another β. To this end, let s write it as E[X] 0 Γ(α) βα x α e βx dx. Γ(α+) Γ(α) fty β 0 Γ(α+) βα+ x α e βx dx Now we are integrating the Γ(α+,β) pdf over 0 to. This must be. So, the answer is (b) The mgf is E[X] Γ(α+) Γ(α) The derivative with respect to t is M X (t) β α Γ(α) Γ(α) β α β. [ ] β α. β t [ ] β α ( ) M X(t) β α β t (β t) 2 E[X] M X (0) α β β 2 α β.
5. By independence of the X s, We have which implies that f X,X 2 (x,x 2 ) f X (x ) f X2 (x 2 ) e x I (0, ) (x ) e x 2 I (0, ) (x 2 ) e (x +x 2 ) I (0, ) (x )I (0, ) (x 2 ) y g (x,x 2 ) x x 2 and y 2 g 2 (x,x 2 ) x +x 2 x g (y,y 2 ) 2 (y +y 2 ) and x 2 g 2 (y,y 2 ) 2 (y 2 y ). The Jacobian of the transformation is x J x y y 2 x 2 x 2 y y 2 2 2 2 2 2. So, f Y,Y 2 (y,y 2 ) f X.X 2 (g (y ),y 2,g 2 (y,y 2 )) J e ( 2 (y +y 2 )+ 2 (y 2 y )) I (0, ) ( 2 (y +y 2 ))I (0, ) ( 2 (y 2 y )) That exponent simplifies to y 2. As for the indicators 0 < 2 (y +y 2 ) < 0 < y +y 2 < y < y 2 which is shown in the shaded region in Figure. The second indicator 0 < y 2 y < which implies that y < y 2. This region is indicated by the horizontal line shading of Figure 2. The intersection where both indicators are on at the same time is the upper V shape in Figure 2. This may be represented as I (0, ) (y 2 ) I ( y2,y 2 )(y ). Putting this all together, we have the joint pdf for Y and Y 2 is f Y,Y 2 (y,y 2 ) 2 e y 2 I ( y2,y 2 )(y )I (0, ) (y 2 ). 2
Figure : 0 < y +y 2 < Figure 2: 0 < y y 2 < on top of 0 < y +y 2 < 6. (a) There are many answers that will work here. Since λ is the mean for the Poisson distribution, one obvious estimator is the sample mean, X since it is always unbiased for the mean of the distribution. So ˆλ X. Note that for the Poisson distribution, λ is also the variance of the distribution. Since we know that the sample variance S 2 is unbiased for the distribution variance, we have another unbiased estimator for λ: ˆλ 2 S 2. HOWEVER, looking ahead to parts (b) and (c), I don t want to have to findthe variance of S 2! So, I ll come up with another unbiased estimator.
How about just X? Let ˆλ 2 X. (b) It s boring but it works. Var[ˆλ ] Var[X] Var(X ) n Var[ˆλ 2 ] Var[X ] λ λ n So, the first estimator is better in terms of variance (smaller variance). (c) There is nothing to do here! If the estimators are unbiased, then the MSE is the same as the variance! So, the first estimator is better in terms of MSE (smaller MSE). 7. X is always unbiased for the true mean which is in this case θ /λ. So, ˆθ X. 8. Based on the previous problem, we should probably try the estimator /X. Note that E[/X] /E[X]. Instead, where Y Γ(n,λ). So, So, E[ Y ] E[/X] E[n/ X i ] ne[/y] y f Y(y)dy 0 0 To get an unbiased estimator, we use y Γ(n) λn y n e λy dy Γ(n) λn y n 2 e λy dy }{{} like Γ(n,λ) Γ(n ) Γ(n) λ 0 Γ(n) λn y n 2 e λy dy Γ(n ) Γ(n) λ (n 2)! (n )! λ λ n. E[/X] ne[/y] n n λ. ˆλ n n X.
9. The real numbers a n converge to the real number a if, for any ε > 0, there exists a natural number N such that a n a < ε for all n N. Thus, for all n N we have P( a n a,ε) and therefore, lim P( a n a,ε). n So, a n P a, as desired. 20. We know that, for any distribution with finite variance, the sample mean X converges to the mean µ of the distribution. In this case, µ E[X ] λ. So, X P λ. 2. (a) Um... this looks familiar. (See the previous problem.) We know that X P λ since we know that, for any distribution with finite variance, the sample mean X converges to the mean µ of the distribution. Since g(x) x 2 is a continuous function, we then have that X 2 g(x) P g(λ) λ 2. (b) We will try That Theorem. ] E[Y n ] E[ X n+ n(n+) E[ n i X i ] ni n(n+) E[X i ] Poisson ni n(n+) iλ n(n+) n(n+) λ 2 λ 2. We wanted to see λ. Note though, that 2Y n is unbiased for λ. Also, note that ] Var[2Y n ] 4Var[Y n ] 4Var[ X n+ 4 n 2 (n+) 2 Var[ n i X i ] indep 4 n 2 (n+) 2 ni Var[X i ] Poisson 4 n 2 (n+) 2 ni iλ 4 λ n(n+) n 2 (n+) 2 2 2 n(n+) λ.
Since this goes to 0 as n, we have, by That Theorem, that 2Y n P λ. Thus, we get Y n P λ/2. We can see this as either Y n 2 }{{} /2 2Y n }{{} P λ P 2 λ (Since we know that X n P a and Yn P b implies that Xn Y n P ab. Combine this with problem 9.) OR, we can put 2Y n P λ through the continuous function g(x) x/2. 22. The cdf for this distribution is F(x) e (x θ) for x > θ. The cdf for the minimum is F X() (x) P(X () x) P(min(X,X 2,...,X n x) x) P(min(X,X 2,...,X n ) > x) iid [P(X > x)] n unif e n(x θ). Taking the derivative with respet to x, we see that X () has an exponential distribution with rate n that has been shifted θ units to the right. Thus E[X () ] n +θ. This is not an unbiased estimator, which is sad because we d like to use That Theorem to show convergence in probability, but That Theorem requires an unbiased estimator. Consider for a moment the estimator θ 2 X () /n. This is an unbiased estimator with variance Var[ θ 2 ] Var[X () /n] Var[X () ] n 2. (Here I have twice used the fact that adding or subtracting a constant to a random variable or a distribution does not affect its variance!) Since E[ θ 2 ] θ and Var[ θ 2 ] 0 as n, we know, by That Theorem, that X () n θ 2 P θ. So, finally, we conclude that X () θ + n P θ+0 θ
as desired. (Here we are using the Theorem that said that X n P a and Yn P b implies X n +Y n P a+b and problem 9 on this review that addresses convergence in probability for non-random things.) 23. F Yn (y) P(Y n y) P(nln(X () +) y) P(X () e y/n ) Since P(X () x) P(X () > x) iid [P(X > x)] n [ Pareto ] n (+x) γ we have that (+x) nγ, F Yn (y) P(X () e y/n ) Finally, (+e y/n ) nγ (e y/n ) nγ e yγ e γy. lim n F Y n (y) lim n [ e γy ] e γy which is the cdf of an exponential distribution with rate γ. So, where Y exp(rate γ). Y n d Y
24. F Yn (y) P(Y n y) P(nX () y) So, P(X () y/n) P(X () > y/n) iid [P(X > y/n)] n [e λy/n] n e λy lim n F Y n (y) lim n [ e λy ] e λy which is the cdf of an exponential distribution with rate λ. Therefore, where Y exp(rate λ). Y n d Y 25. Using mgfs, M Y (t) E[e ty ] E[e t X i ] E[e txi ] ni ] E[ e tx i indep n i E[e tx i ] n i M Xi (t) ( ) n /2 ni /2 i /2 t ( /2 t) ( ni )/2. This is the mgf of the χ 2 ( n i n i ) distribution. Thus, n Y χ 2 ( n i ). i 26. (a) S 2 ni (X i X) 2 n
(b) E[(X i X) 2 ] E[X 2 i 2X ix +X 2 ] We know that We also know that E[X 2 i ] 2E[X ix]+e[x 2 ] E[X 2 i ] Var[X i]+(e[x i ]) 2 σ 2 +µ 2. E[X 2 ] Var[X]+(E[X]) 2 σ2 n +µ2. The other term is a bit tricky since X i is not independent of X. X i is, however, independent of most of the terms in X. [ ] E[X i X] E X nj i n X j n E[X X i + X i X i +X 2 i +X ix i+ + X n X i ] We already know that E[X 2 i ] σ2 +µ 2. The other n terms have the form E[X i X j ] where j i. In this case E[X i X j ] indep E[X i ]E[X j ] µ µ µ 2. Thus, E[X i X] n [ ] (n )µ 2 +(σ 2 +µ 2 ) n [nµ2 +σ 2 ] µ 2 +σ 2 /n. Putting it all together, we have E[(X i X) 2 ] E[Xi 2 2X ix +X 2 ] E[Xi 2] 2E[X ix]+e[x 2 ] σ 2 +µ 2 2[µ 2 +σ 2 /n]+σ 2 /n+µ 2 (n )σ2 n (c) Thus, E[S 2 ] E [ n ] i (X i X) 2 n n E[ n i (X i X) 2 ] n ni E[(X i X) 2 ] ni (n )σ 2 n n (n )σ2 n n n σ 2