MATH2715: Statistical Methods

Size: px

Start display at page:

Download "MATH2715: Statistical Methods"

Kathryn Ross
5 years ago
Views:

1 MATH275: Statistical Methods Exercises VI (based on lectre, work week 7, hand in lectre Mon 4 Nov) ALL qestions cont towards the continos assessment for this modle. Q. The random variable X has a discrete niform distribtion, taking vales and +, each with probability 0.5. Show that X has mean zero and variance nity. Show that the moment generating fnction of X is m X (t) = E[e tx ] = 2 e t + 2 et. Q2. Sppose that X,...,X n are mtally independent discrete niform random variables taking vales and +, each with probability 0.5. Let S n = X + + X n. The moment generating fnction of X is m X (t) = 2 e t + 2 et. Write down the moment generating fnction of S n. Sppose that Y = S n n has moment generating fnction m Y (t). Show that m Y (t) = ( 2 e t/ n + 2 et/ n ) n. Obtain log m Y (t) and dedce that, as n, log m Y (t) 2 t2. What do yo conclde abot the distribtion of Y for large n? Hint: Recall that e θ = + θ + θ2 2! + θ3 3! +. Q3. The random variables X and Y are independent and each has an exponential distribtion with parameter λ =. Let U = X Y. Prove that the moment generating fnction of U is m U (t) = t2. Obtain the mean and variance of U. Hint: If X exponential(λ), then the moment generating fnction is m X (t) = λ λ t for t < λ. Q4. The random variables X and Y are independent and each has an exponential distribtion with parameter λ =. Let U = X Y. Prove that the characteristic fnction of U is φ U (t) = + t 2. Hint: Recall that φ U (t) = E[e itu ]. Q5. The random variables X and Y are independent and each has an exponential distribtion with parameter λ =. Let U = X Y. It can be shown that U has probability density fnction f U () = 2 e for < <. The Forier inversion theorem relates a density fnction f U () and the corresponding characteristic fnction φ U (t) by f U () = 2π e it φ U (t)dt. By considering what yo get if yo pt z = t, dedce that the characteristic fnction of the Cachy distribtion is e t. Hint: Yo don t need to evalate any integrals! 69

2 Backgrond Notes: Lectre. Central limit theorem Another integration reslt NOT necessary to learn e itx π( + x 2 ) = = = 2 π 0 cos(tx) + i sin(tx) π( + x 2 ) cos(tx) π( + x 2 as the imaginary part cancels ot, ) cos(tx) = e t + x2 on sing Cachy s reside theorem (see MATH206). History of the central limit theorem NOT necessary to learn For the case where X Bin(n,θ = 2 ) and n is large De Moivre ( ) by 730 had discovered the approximation pr { 2 X = 2 n} nπ. The reslt follows becase Bin(n,θ = 2 ) N( 2 n, 4n). By 733 he had extended the reslt to showing, in modern notation, that if X Bin(n,θ = 2 ), then { lim pr a < X 2 n } < b = b e 2 t2 dt. n n 2π In 82 Laplace generalised this reslt to Bin(n,θ) random variables, { } lim pr a < X nθ < b = b e 2 t2 dt. n nθ( θ) 2π This is sometimes referred to as the De Moivre-Laplace theorem. 2 Recall that X Bin(n,θ) can be thoght of as the sm, X = X +X 2 + +X n, of n independent Bernolli trials, where { with probability θ, X i = 0 with probability θ. Ths the De Moivre-Laplace theorem can be written as { } lim pr a < X + X X n nθ < b n nθ( θ) a a = b e 2 t2 dt. 2π In 887 Chebyshev generalised this reslt bt his proof was improved by one of his stdents, Markov. In 90 another of his stdents, Aleksandr Lyapnov (857-98), frther generalised the De Moivre-Laplace theorem. Later workers removed some of the assmptions made in earlier proofs and in 922 Jarl Lindeberg ( ) proved what is now known as the central limit theorem, that a seqence of n independent and identically distribted random variables X, X 2,..., X n, each with mean µ and finite variance σ 2, satisfies { } lim pr X + X X n nµ < b = b e 2 t2 dt. n nσ 2 2π a 70

3 Frther reading for lectre Rice, J.A. (995) Mathematical Statistics and Data Analysis (2nd edition), sections 3.6, 4., 4.5, 5.3. Hogg, R.V., McKean, J.W. and Craig, A.T. (2005) Introdction to Mathematical Statistics (6th edition), sections.9, 4.4. Larsen, R.J. and Marx, M.L. (200) An Introdction to Mathematical Statistics and its Applications (5th edition), sections 4.3. Miller, I. and Miller, M. (2004) John E. Frend s Mathematical Statistics with Applications, sections

4 MATH275: Statistical Methods Worked Examples VI Worked Example: The random variables X and Y are independent and have moment generating fnction m X (t) and m Y (t) respectively. Obtain the moment generating fnction of U = ax + by. Answer: m U (t) = E[e tu ] = E[e t(ax+by ) ] = E[e (at)x e (bt)y ] = E[e (at)x ]E[e (bt)y ] by independence of X and Y. Ths m U (t) = m X (at)m Y (bt). Worked Example: Let X,...,X n be independent and identically distribted exponential(λ) random variables. If Xn = n X i, show that, as n, Z n = X n E[ X n ] d N(0,). n Stdev[ X n ] i= Answer: If X i exponential(λ), then E[X i ] = /λ, Var[X i ] = /λ 2 so E[ X n ] = λ, Var[ X n ] = nλ 2. The moment generating fnction of X i is m Xi (t) = ( ) λ n of S n = X + + X n is m Sn (t) =. λ t Now notice Z n = X n E[ X n ] Stdev[ X n ] = λ λ t S n n λ λs = n n. n nλ 2 Recalling that U = ay + b has moment generating fnction so that the moment generating fnction m U (t) = E[e tu ] = E[e t(ay +b) ] = E[e bt e aty ] = e bt E[e (at)y ] = e bt m Y (at) shows that, with Y = S n, U = Z n, a = λ/ n, and b = n, ( ) n ( m Zn (t) = e nt = e nt λ λ (λt/ n) (t/ n) ) n. Hence Recall that, for small δ, log m Zn (t) = ( nt n log t ). n log( δ) = δ 2 δ2 3 δ3 so log m Zn (t) = ( ) t nt + n + t2 n 2n + t3 3n.5 + t22 t3 = + 3 n +. As n, log m Zn (t) 2 t2 so m Zn (t) e 2 t2 which is the moment generating fnction of a N(0,) random variable. 72

5 Worked Example: Sppose X,...,X n are independent and identically distribted Poisson(µ) random variables. Show that, as n, Z n = X n E[ X n ] Stdev[ X n ] d N(0,). Answer: If X i Poisson(µ), then E[X i ] = µ and Var[X i ] = µ so E[ X n ] = µ and Var[ X n ] = µ n. The moment generating fnction of X i is m Xi (t) = exp( µ( e t )) so that the moment generating fnction of S n = X + + X n is Now notice Z n = X n E[ X n ] Stdev[ X n ] m Sn (t) = ( exp( µ( e t )) ) n = exp( nµ( e t )). = S n n µ = S n nµ. µ nµ n Recalling that U = ay + b has moment generating fnction m U (t) = E[e tu ] = E[e t(ay +b) ] = E[e bt e aty ] = e bt E[e (at)y ] = e bt m Y (at) shows that, with Y = S n, U = Z n, a = / nµ and b = nµ, m Zn (t) = e ( nµ)t exp( nµ( e t/ nµ )). Hence log m Zn (t) = ( ( ) nµ)t nµ e t/ nµ = ( nµ)t nµ + nµe t/ nµ. Recall that, for small δ, e δ = + δ + 2! δ2 + 3! δ3 + so giving log m Zn (t) = ( nµ)t nµ + nµ ( + t + t2 nµ 2nµ + t 3 ) 6n.5 µ.5 + = t22 + t 3 6n 0.5 µ As n, log m Zn (t) 2 t2 so m Zn (t) e 2 t2 which is the moment generating fnction of a N(0,) random variable. Worked Example: Random variables X and Y are independent and each has a niform(0, ) distribtion. Let U = X Y. Obtain the characteristic fnction and probability density fnction f U () of U. Use the Forier inversion theorem to dedce the characteristic fnction of the random ( cos z) variable Z with probability density fnction f Z (z) = πz 2, where < z < +. Hint: The Forier inversion theorem relates a density fnction f U () and the characteristic fnction φ U (t) by Answer: f U () = e it φ U (t)dt. 2π 73

6 Characteristic fnction of U: If X niform(0,), then f X (x) = for 0 < x <. Hence X (and Y ) has moment generating fnction m X (t) = E[e tx ] = x= e tx f X (x) = x=0 [ e e tx tx = t ] x=0 = et, < t <. t The mgf of U is m U (t) = E[e tu ] = E[e t(x Y ) ] = E[e tx e ty ] = E[e tx ] E[e ( t)y ] = m X (t)m Y ( t) as X, Y are independent. Hence ( e t )( e t ) m U (t) = = 2 et e t t t t 2. Ths the characteristic fnction of U is φ U (t) = E[e itu ] = m U (it) = 2 eit e it 2 (cos t + isin t) (cos t isin t) 2( cos t) (it) 2 = t 2 = t 2. Joint pdf of (U,V ): As X and Y are independent, f XY (x,y) = f X (x)f Y (y) =, 0 < x,y <. Pt = x y, v = y so x = + v and y = v with Jacobian J = (x, y) x x (, v) = v = 0 =. y y v Hence f UV (,v) = f XY (x,y) J =. Range of U and V : 0 < x <, 0 < y < = < x y < so < < while 0 < v <. Bt also 0 < x < = y < x y < y so v < < v giving < v <. Clearly max(0, ) < v < min(, ). Figre 30(left) shows the (x,y) region on the left. The elemental strip B 66 with > 0 is defined for y (0, ). The elemental strip A 67 with < 0 is defined for y (,). Figre 30(right) shows the mapping in the (, v) plane. The corresponding elemental strips are now vertical strips. When > 0, the elemental strip B is defined for v (0, ). When < 0, the elemental strip A is defined for v (,). y A <0 A* v= v=y B >0 v=- v=- B* 0 x - 0 v=0 =x-y Figre 30: mapping = x y, v = y for x, y 0 showing (left) x-y and (right) -v planes. 66 The line y = x for 0 < x < with > The line y = x for 0 < x < with < 0. 74

7 Probability density fnction of U: f U () = v= f UV (,v)dv = min(, ) v=max(0, ) dv = [ ] min(, ) v. v=max(0, ) There are ths two cases to consider. [ ] v= v = + if < 0, v= f U () = = for < <. [ ] v= v = if 0. v=0 Forier inversion theorem: The Forier inversion theorem gives f U () = 2π Ths = e it 2( cos t) 2π t= t 2 dt. Ptting t = z and dt = dz shows that z= e iz( cos z) πz 2 dz = t= e it φ U (t)dt. so that the characteristic fnction of a random variable Z having probability density fnction ( cos z) f Z (z) = πz 2 is φ Z () =. Clearly f Z (z) is a probability density fnction since f Z (z) 0 for all z and φ Z (0) =. It is shown in figre 3(right) Pdf f U () Pdf f Z (z) z Figre 3: (left) plot of f U () = for < <, (right) plot of probability density fnction f Z (z). 68 The probability density fnction f Z(z) can be plotted sing the following R command: crve((-cos(x))/(pi*x*x),-30,30,xlab="z",ylab="pdf") 75

8 MATH275: Soltions to exercises III Q. f X (x) = for 0 < x <. If = log x, then x = e and d = e so that f U () = f X (x) d = e = e, > 0. This is the pdf of an exponential random variable with parameter λ = and mean /λ =. Q2. (a) f X (x) = for 0 < x <. The transformation = x 2 gives x = so d = 2 2 so that f U () = f X (x) d = 2 = 2, 0 < <. The range for follows becase if 0 < x <, then 0 < <. (b) f X (x) = for 2 < x < 2. The transformation = x2 now gives x = ± so d = ± 2 2. The transformation is no longer a mapping. Both x = + and x = give the same -vale. Hence split the problem into the two cases x < 0 and x > 0 where the mapping is. This gives f U () = f X (x = ) d + f X (x = + ) x= d. x=+ In both cases the absolte vale of the Jacobian is the same. 69 Ths f U () = =, 0 < < 4. The range of U follows becase 2 < x < 2 implies 0 < < 4. Q3. = log x so x = e and d = e. Since x > 0 then = log x satisfies < < +. f U () = f X (x) d = e x e = exp( e )e = exp( e ), < <. df U () Sketching pdf: = ( e )exp( e ) so df U() = 0 at = 0. As ±, f U () 0. d d The pdf can be evalated for different vales and is shown in figre 32. Q4. (a) f X (x) = 2 e 2 x, for x > 0. The mapping = x is a mapping with inverse transformation x = 2 and d = 2. f U () = f X (x) d = 2 e 2 x 2 = e 2 2, > In some more complicated sitations the gradient making p the Jacobian might be very different in the different parts of the mapping. For example, consider the mapping = x 2 if x > 0 bt = x if x < 0. The two vales x = ± map to the same = vale bt the gradient d eqals 2 if x = + and eqals if x =. 76

9 f U () = exp( e ) Pdf Figre 32: pdf f U () = e e, < <. U is said to have a Rayleigh distribtion 70. (b) If Z N(0,), then E[Z 2 ] =, so Ths E[U] = (c) The R code... f U ()d = =0 z= z 2 e 2 z2 dz = and z 2 e 2 z2 dz = 2π z=0 2π 2. 2 e 2 2 d = π 2π 2 e 2 2 d = =0 2π 2 2π = 2. x=rexp(000,0.5) =sqrt(x) mean() #...gave me.2603 while sqrt(pi/2)= Q5. (a) Noting that X and Y are independent, their joint pdf is f XY (x,y) = f X (x)f Y (y) = e 2 x2 e 2 y2 = 2π 2π 2π e 2 (x2 +y 2), < x,y <. If = x 2 + y 2, v = x/y, then (, v) (x, y) = x v x y v y = 2x 2y /y x/y 2 = 2(x2 /y 2 + ). The mapping is not since ( x, y) and (x,y) map to give the same vale of (,v). Hence consider the two cases {x < 0} and {x 0} separately. Since J = (x, y) (, v) = (,v) (x, y) = ( 2(x 2 /y 2 + )) = 2(v 2 + ), then J = 2( + v 2 and the ) joint pdf f UV (,v) satisfies f UV (,v) = f XY ( x, y) J + f XY (x,y) J = 2 2π e 2 (x2 +y 2) 2 2( + v 2 ) = e 2π( + v 2 ). 70 Often sed to model wind velocity and sea-wave heights. For example, if X and Y are independent orthogonal components of wind velocity having normal distribtions, then the magnitde X 2 + Y 2 has a Rayleigh distribtion. Qestion 4 shows that X 2 + Y 2 has a Rayleigh distribtion and qestion 5 shows that X 2 +Y 2 exponential( 2 ). 77

10 The joint pdf is defined for > 0, < v <. (b) Independence 7 of U and V follows as the joint pdf factorises to give f UV (,v) = f U ()f V (v) where f U () = 2 e 2, > 0, f V (v) = π( + v 2 ), < v <. Clearly these are recognisable 72 pdfs so that U exponential(λ = 2 ) while V Cachy. 7 Contors of f XY (x, y) are circles centred on the origin as shown in the plot below where r = p x 2 + y 2 and θ = tan x/y. The qestion shows that U and V = tanθ (and hence R = U and Θ) are independent. y θ r (x,y) x 72 If yo do not recognise them as pdfs, then the pdfs of U and V cold be obtained by integrating the joint pdf: f U() = Z for > 0. Also f V (v) = Z 0 f UV (, v) dv = 2 e 2 Z f UV (, v) d = Z dv π( + v 2 ) = 2 e 2 Z π( + v 2 2 ) e 2 d = 0 v=0 2dv h π( + v 2 ) = 2 e 2 2 i π tan v = 2 e 2, 0 h e i π( + v 2 2 = ) =0, for < v <. π( + v 2 ) 78

11 MATH275: Statistical Methods Examples Class VI, week 7 The qestions below will be looked at in Examples Class VI held in week 7. Q. Sppose that X and Y are independent Poisson random variables with means λ and µ respectively. Use moment generating fnctions to dedce the distribtion of U = X + Y. Q2. The random variables X and X 2 are independent discrete random variables satisfying pr{x i = 0} = pr{x i = } = 2 for i =,2. Obtain the moment generating fnction of X i. Obtain the moment generating fnction of U = X + X 2 and of V = X X 2. Dedce the probability fnctions of U and V. 79

MATH2715: Statistical Methods

MATH2715: Statistical Methods MATH275: Statistical Methods Exercises III (based on lectres 5-6, work week 4, hand in lectre Mon 23 Oct) ALL qestions cont towards the continos assessment for this modle. Q. If X has a niform distribtion