STAT 801: Mathematical Statistics. Distribution Theory

Transcription

1 STAT 81: Mathematical Statistics Distribution Theory Basic Problem: Start with assumptions about f or CDF of random vector X (X 1,..., X p ). Define Y g(x 1,..., X p ) to be some function of X (usually some statistic of interest). How can we compute the distribution or CDF or density of Y? Univariate Techniques Method 1: compute the CDF by integration and differentiate to find f Y. Example: U Uniform[, 1] and Y logu. so Y has standard exponential distribution. Example: Z N(, 1), i.e. and Y Z 2. Then F Y (y) P(Y y) P( log U y) P(log U y) P(U e y ) { 1 e y y > y. f Z (z) 1 2π e z2 /2 F Y (y) P(Z 2 y) { y < P( y Z y) y. Now differentiate to get Then P( y Z y) F Z ( y) F Z ( y) f Y (y) d dy F Z( y) [ y < d dy FZ ( y) F Z ( y) ] y > undefined y. f Z ( y) d y dy 1 ( exp ( ) y) 2 1 /2 2π 2 y 1/ πy e y/2. (Similar formula for other derivative.) Thus 1 2πy e y/2 y > f Y (y) y < undefined y. We will find indicator notation useful: 1(y > ) { 1 y > y which we use to write f Y (y) 1 2πy e y/2 1(y > ) 1

2 (changing definition unimportantly at y ). Notice: I never evaluated F Y before differentiating it. In fact F Y and F Z are integrals I can t do but I can differentiate then anyway. Remember fundamental theorem of calculus: d dx x at any x where f is continuous. Summary: for Y g(x) with X and Y each real valued a f(y)dy f(x) P(Y y) P(g(X) y) P(X g 1 (, y]). Take d/dy to compute the density f Y (y) d f X (x)dx. dy {x:g(x) y} Often can differentiate without doing integral. Method 2: Change of variables. Assume g is one to one. I do: g is increasing and differentiable. Interpretation of density (based on density F ): and f Y (y) P(y Y y + δy) lim δy δy lim δy F Y (y + δy) F Y (y) δy P(x X x + δx) f X (x) lim. δx δx Now assume y g(x). Define δy by y + δy g(x + δx). Then Get Take limit to get or P(y Y g(x + δx)) P(x X x + δx). P(y Y y + δy)) δy P(x X x + δx)/δx {g(x + δx) y}/δx. f Y (y) f X (x)/g (x) f Y (g(x))g (x) f X (x). Alternative view: Each probability is integral of a density. The first is the integral of the density of Y over the small interval from y g(x) to y g(x + δx). The interval is narrow so f Y is nearly constant and Since g has a derivative the difference P(y Y g(x + δx)) f Y (y)(g(x + δx) g(x)). g(x + δx) g(x) δxg (x) and we get P(y Y g(x + δx)) f Y (y)g (x)δx. 2

3 Same idea applied to P(x X x + δx) gives P(x X x + δx) f X (x)δx so that or, cancelling the δx in the limit If you remember y g(x) then you get f Y (y)g (x)δx f X (x)δx f Y (y)g (x) f X (x). f X (x) f Y (g(x))g (x). Or solve y g(x) to get x in terms of y, that is, x g 1 (y) and then f Y (y) f X (g 1 (y))/g (g 1 (y)). This is just the change of variables formula for doing integrals. Remark: For g decreasing g < but Then the interval (g(x), g(x + δx)) is really (g(x + δx), g(x)) so that g(x) g(x + δx) g (x)δx. In both cases this amounts to the formula Mnemonic: Example: X Weibull(shape α, scale β) or f X (x) α β f X (x) f Y (g(x)) g (x). f Y (y)dy f X (x)dx. ( ) α 1 x exp { (x/β) α }1(x > ). β Let Y log X or g(x) log(x). Solve y log x: x exp(y) or g 1 (y) e y. Then g (x) 1/x and 1/g (g 1 (y)) 1/(1/e y ) e y. Hence ( ) e y α 1 exp { (e y /β) α }1(e y > )e y. f Y (y) α β For any y, e y > so indicator 1. So β f Y (y) α β α exp {αy eαy /β α }. Define φ log β and θ 1/α; then, f Y (y) 1 θ exp { y φ θ { }} y φ exp. θ Extreme Value density with location parameter φ and scale parameter θ. (Note: several distributions are called Extreme Value.) Marginalization Simplest multivariate problem: (or in general Y is any X j ). X (X 1,..., X p ), Y X 1 3

4 Theorem 1 If X has density f(x 1,..., x p ) and q < p then Y (X 1,..., X q ) has density f Y (x 1,...,x q ) f(x 1,..., x p )dx q+1...dx p. f X1,...,X q is the marginal density of X 1,...,X q and f X the joint density of X but they are both just densities. Marginal just to distinguish from the joint density of X. Example The function f(x 1, x 2 ) Kx 1 x 2 1(x 1 >, x 2 >, x 1 + x 2 < 1) is a density provided The integral is P(X R 2 ) so K 24. The marginal density of x 1 is f X1 (x 1 ) K x1 24x 1 x 2 f(x 1, x 2 )dx 1 dx 2 1. x 1 x 2 dx 1 dx 2 K 1 x 1 (1 x 1 ) 2 dx 1 /2 K(1/2 2/3 + 1/4)/2 K/24 1(x 1 >, x 2 >, x 1 + x 2 < 1)dx 2 1 x1 x 1 x 2 1( < x 1 < 1)dx 2 12x 1 (1 x 1 ) 2 1( < x 1 < 1). This is a Beta(2, 3) density. General problem has Y (Y 1,..., Y q ) with Y i g i (X 1,...,X p ). Case 1: q > p. Y won t have density for smooth g. Y will have a singular or discrete distribution. Problem rarely of real interest. (But, e.g., residuals have singular distribution.) Case 2: q p. We use a change of variables formula which generalizes the one derived above for the case p q 1. (See below.) Case 3: q < p. Pad out Y add on p q more variables (carefully chosen) say Y q+1,..., Y p. Find functions g q+1,..., g p. Define for q < i p, Y i g i (X 1,..., X p ) and Z (Y 1,...,Y p ). Choose g i so that we can use change of variables on g (g 1,..., g p ) to compute f Z. Find f Y by integration: f Y (y 1,..., y q ) Change of Variables f Z (y 1,...,y q, z q+1,..., z p )dz q+1... dz p Suppose Y g(x) R p with X R p having density f X. Assume g is a one to one ( injective ) map, i.e., g(x 1 ) g(x 2 ) if and only if x 1 x 2. Find f Y : Step 1: Solve for x in terms of y: x g 1 (y). Step 2: Use basic equation: f Y (y)dy f X (x)dx 4

5 and rewrite it in the form Interpretation of derivative dx dy when p > 1: which is the so called Jacobian. Equivalent formula inverts the matrix: This notation means f Y (y) f X (g 1 (y)) dx dy. dx dy dy dx det ( ) det xi y j f Y (y) f X(g 1 (y)) dy dx y 1 x 1 y 1 x 2 y p x 1. y p x 2 but with x replaced by the corresponding value of y, that is, replace x by g 1 (y). Example: The density f X (x 1, x 2 ) 1 { 2π exp x2 1 + } x2 2 2 is the standard bivariate normal density. Let Y (Y 1, Y 2 ) where Y 1 X X2 2 and Y 2 < 2π is angle from the positive x axis to the ray from the origin to the point (X 1, X 2 ). I.e., Y is X in polar co-ordinates. Solve for x in terms of y: so that It follows that X 1 Y 1 cos(y 2 ) X 2 Y 1 sin(y 2 ) y 1 x p y p x p g(x 1, x 2 ) (g 1 (x 1, x 2 ), g 2 (x 1, x 2 )) ( x x2 2, argument(x 1, x 2 )) g 1 (y 1, y 2 ) (g1 1 (y 1, y 2 ), g2 1 (y 1, y 2 )) (y 1 cos(y 2 ), y 1 sin(y 2 )) ( dx dy det cos(y2 ) y 1 sin(y 2 ) sin(y 2 ) y 1 cos(y 2 ) y 1. f Y (y 1, y 2 ) 1 { } 2π exp y2 1 y 1 1( y 1 < )1( y 2 < 2π). 2 Next: marginal densities of Y 1, Y 2? Factor f Y as f Y (y 1, y 2 ) h 1 (y 1 )h 2 (y 2 ) where h 1 (y 1 ) y 1 e y2 1 /2 1( y 1 < ) ) and h 2 (y 2 ) 1( y 2 < 2π)/(2π). 5

6 Then f Y1 (y 1 ) h 1 (y 1 ) h 1 (y 1 )h 2 (y 2 )dy 2 h 2 (y 2 )dy 2 so marginal density of Y 1 is a multiple of h 1. Multiplier makes f Y1 1 but in this case so that h 2 (y 2 )dy 2 2π (2π) 1 dy 2 1 f Y1 (y 1 ) y 1 e y2 1 /2 1( y 1 < ). (Special case of Weibull or Rayleigh distribution.) Similarly f Y2 (y 2 ) 1( y 2 < 2π)/(2π) which is the Uniform((, 2π) density. Exercise: W Y1 2 /2 has standard exponential distribution. Recall: by definition U Y1 2 has a χ 2 distribution on 2 degrees of freedom. Exercise: find χ 2 2 density. Note: We show below factorization of density is equivalent to independence. 6