18 Bivariate normal distribution I

Size: px

Start display at page:

Download "18 Bivariate normal distribution I"

Willa Wheeler
5 years ago
Views:

1 8 Bivariate normal distribution I 8 Example Imagine firing arrows at a target Hopefully they will fall close to the target centre As we fire more arrows we find a high density near the centre and fewer further away We can imagine contours showing the density of hits about the centre Regard these contours as defining the pdf surface of variables X and X, where: Suppose X, X ind N, σ so X deviation from centre in x -direction, X deviation from centre in x -direction f Xi x i πσ exp { } x i, i, σ Since X, X are independent, the joint pdf of X, X is f X X x, x f X x f X x { exp πσ πσ exp x σ { x + x σ } exp πσ { } x σ } 8 X Low pdf High pdf X Pdf X X ind Figure 8: bivariate normal distribution with X, X N, σ ; joint pdf; right joint pdf surface Write X X X x, x x so equation 8 gives the bivariate normal pdf f X x πσ exp, x x x x, x x 69 { } x x σ x + x, left contours of

2 8 Linear transformation c c Suppose U CX, where C c c is the distribution of U? Here u Cx gives is a non-singular matrix of constants What u c x + c x, u c x + c x, and u u, u u x, x x x u u c c c c C, x x where C is the determinant of C Notice u Cx gives du dx C so du dx C Now recall 6, f U u f X x x, x u, u f Xx u, u x, x Since u Cx, then x C u so x x C u C u u C C u Taking the absolute value of the determinant gives f U u { } { πσ exp x x σ C πσ C exp u C C } u σ This is a bivariate normal distribution with zero mean and variance-covariance matrix Σ where Σ σ C C so here Σ σ CC Write U N,Σ, with pdf f U u { exp } π Σ / u Σ u Here Σ is a real symmetric and positive definite matrix so Σ > Writing Σ σ ij, diagonal elements σ ii Var[U i ], off-diagonal elements σ ij covu i, U j Example 8 Let C Hence C Then U CX X X X X X + X Also, C 5, C C 9 9 Σ C C σ σ Thus Var[U ] σ, Var[U ] 5σ, covu, U σ 9, so Σ σ

3 Revision: X, X are independent so covx, X, U X X Var[U ] Var[X ] + Var[X ] σ U X + X Var[U ] Var[X ] + Var[X ] 5σ covu, U covx X, X + X Var[X ] Var[X ] covx, X σ Example 8 Suppose Σ Iσ where I is a identity matrix This gives equation 8 since Σ σ σ σ and Σ σ 8 Orthogonal transformation Suppose U CX where C is an orthogonal matrix so that C C CC I Example 8 cos θ sin θ The transformation C sin θ cosθ an angle θ to give new axes U, U rotates the X, X axes anti-clockwise through X U θ U X Figure 9: rotation of X, X through θ to give new axes U, U and showing the pdf contours of f X X x, x In the orthogonal case C C so C C C C CC I Using properties of determinants, } C C C C C C C ± C I We take absolute value so C Hence from 8, U N, Iσ with pdf f U u { πσ exp } u σ u 7

4 9 Bivariate normal distribution II 9 σ ρσ Suppose X N, Σ with Σ ρσ σ The pdf of X is then 9 f X x x Σ x x, x σ ρ, Σ σ ρ σ σ ρ Then ρ ρ σ ρ x ρx, x ρx σ ρ x + x ρx x x x exp π Σ x Σ x πσ ρ exp x + x ρx x, < x σ ρ, x < The contours of constant pdf satisfy f X x k, a constant, and are ellipses with centre, given by x + x ρx x C, a constant x x X X Pdf X X Figure 5: bivariate normal distribution with σ, ρ 5; left contours of joint pdf, dashed line ellipse principal axes, dotted line X ρx ; right pdf surface 7

5 9 Marginal distributions The marginal distributions of X and X are normal random variables f X x x f X,X x, x dx x πσ ρ exp + x ρx x σ ρ dx e x πσ πσ ρ exp σ x + x ρx x x ρx + x ρ x ρx σ ρ exp πσ ρ exp x ρx σ ρ If Y Nρx, σ ρ, f Y y dy e x σ, < x <, πσ x ρ dx σ ρ dx exp y ρx dy πσ ρ σ ρ so X N, σ Similarly, X N, σ The converse is not true, viz if X and X are normal random variables, then X, X is not necessarily a bivariate normal distribution 9 Conditional distributions Conditional probability density function of X given X x is fx x X x f X X x, x f X x exp x +x ρx x πσ ρ σ ρ x πσ exp σ exp x + x ρx x + x πσ ρ σ ρ σ x exp ρx x + ρ x πσ ρ σ ρ exp x ρx, < x πσ ρ σ ρ < 7

6 Hence X X x Nρx, σ ρ The conditional mean and variance of X given X x are thus E[X X x ] ρx, Var[X X x ] σ ρ The conditional variance is constant for all values of x It is said to be homoscedastic X X Pdf X Figure 5: bivariate normal distribution with σ, ρ 5; left contours of joint pdf, dashed line ellipse principal axes, dotted line regression line X ρx ; right showing conditional pdf for X given X x is a normal density 95 Bivariate normal distribution More generally X Nµ,Σ with and pdf f X x µ µ µ σ, Σ ρσ σ ρσ σ σ { πσ σ exp x µ ρ x µ x µ + x } µ ρ ρ σ σ σ σ The contours of constant pdf are ellipses with centre µ It can be shown that X Nµ, σ, X Nµ, σ, covx, X ρσ σ, corrx, X ρ, and X X x N µ + ρσ x µ, σ σ ρ The line X µ + ρσ σ x µ 7,

7 X X 8 6 Pdf X X Figure 5: bivariate normal distribution with µ µ, σ, σ, ρ 5; left contours of joint pdf, dashed line ellipse principal axes, dotted line regression line X ρσ X /σ ; right pdf surface is called the curve of regression of X given X and gives the mean value of X for a given value X x When X, X have a bivariate normal distribution, the curve is a straight line Example 9 CD cell counts in AIDS patients The CD cell count of a patient is used as a marker of HIV infection Let Y Y, Y denote the CD cell count at times and For some transformation U uy can model U Nµ,Σ where µ depends on the age of the patient, the treatment used and the stage of disease reached The distribution of Y can then be derived knowing the Jacobian of the transformation Source: Lipsitz, SR, Ibrahim, J, and Molenberghs, G Using a Box-Cox transformation in the analysis of longitudinal data with incomplete responses, Applied Statistics, 9, pp87-96 Example 9 Heights and weights Rosenbaum 95 gives the heights and weights of 587 National servicemen, born in 9 and recruited in 95 Source: Rosenbaum, S 95 Heights and weights of the army intake, 95, Journal of Royal Statistical Society, series A, 5, 7 75

8 Weight lbs Y Class 8 7 Marginal Mid-point y j total for X X x i Height inches Marginal total for Y Total 587 x 6769, ȳ 766 lbs, s X 7995, s XY 579, s Y 58 Weight lbs Height inches 5 Frequency Height inches Weight lbs Figure 5: left frequency contours of,,, 5 men, right frequency plotted as a surface Could model the data as from a bivariate normal distribution with non-zero mean Distribution of height for a given weight is then a normal rv as is distribution of weight for a given height For each height grouping we can determine the mean weight Plotted as a function of height this gives the curve of regression of weight given height Similarly the curve of regression of height given weight can be obtained The curves of regression are close to straight lines The best estimated straight line fit is the least squares regression line Least squares regression line for Y given X x is y ȳ + x x s XY s X x If x 68, predict y ???? lbs x

9 Frequency 5 5 Frequency Height inches 5 5 Weight lbs Figure 5: frequency polygon for men left weighing 55 lbs, right 68 tall Weight lbs 5 5 Mean height for given weight Mean weight for given height Height inches Figure 55: curve of regression for solid line height given weight, dashed line weight given height 77

10 Least squares regression line for X given Y y is x x + y ȳ s XY s X y y 58 If y 7 lbs, predict x ??? 96 Independence and correlation If X and X are independent normal variables, then X X, X is bivariate normal and also ρ If X Nµ,Σ and ρ, then X and X are independent normal variables If X and X are uncorrelated normal variables, so ρ, then X and X are not necessarily independent and X, X is not necessarily bivariate normal 78

11 Bivariate normal distribution III If X, X have a bivariate normal distribution, then X and X have marginal distributions which are normal distributions; see 9 The converse is not true If the marginal distributions of X and X are normal random variables, then X, X is not necessarily a bivariate normal distribution Example Consider the joint pdf shown in figure 56, f X,X x, x x +x f X x π e x π e π e x f X,X x, x dx x +x dx + + x x e x +x, < x, x < π x x e x +x dx π e x dx + π x e x If Y N, with pdf fy y π e y, E[Y ] π e x, < x < Hence X N, Similarly X N, Suppose X N, Σ with Then covx, X ρσ so corrx, X σ ρσ Σ ρσ σ covx, X Var[X ]Var[X ] ρσ σ σ ρ π x e x dx yf Y y dy Recall: if µ E[X ], µ E[X ], then covx, X E[X µ X µ ] E[X X ] µ µ If µ µ, then covx, X E[X X ] We could then show by integration that E[X X ] x x f X,X x, x dx dx ρσ If X and X are independent, then they are uncorrelated so ρ corrx, X Proof: First year revision E[X X ] E[X ]E[X ] as X, X independent Hence covx, X E[X X ] µ µ and in turn ρ 79

12 5 Pdf 5 X X X Figure 56: the pdf f X X x, x π e x +x + x x e x +x X ind If X, X are joint normal random variables and ρ, then X, X normal Proof: If ρ, then for all x and x, f X,X x, x πσ exp x + x σ e x πσ σ e x σ f X x f X x πσ Hence X and X are independent Recall: Continuous random variables X and Y are independent if f X,Y x, y f X xf Y y for all x, y Suppose X and X have marginal normal distributions and corrx, X This does not imply X and X are independent Example Consider the joint pdf shown in figure 57, f X,X x, x π e x +x { + x x 9 x x e x +x }, < x, x < Then X N,, X N,, corrx, X since E[X X ], and yet f X,X x, x f X x f X x 8

13 5 Pdf 5 X X X { } Figure 57: the pdf f X X x, x π e x +x + x x 9 x x e x +x X f X x x π e x f X,X x, x dx 9 π x e x e x dx + π π x e x If Y N, π x e x dx π e x, < x < Hence X N, Similarly X N, π x e x dx with pdf fy y π e y, E[Y ] E[Y ] E[X X ] x 9 x x e x dx π x x f X,X x, x dx dx x e x dx π x e x dx + π x π e x dx x e x dx π x π e x dx If Y N, with pdf fy y π e y, E[Y ], E[Y ] 8

Statistics 351 Probability I Fall 2006 (200630) Final Exam Solutions. θ α β Γ(α)Γ(β) (uv)α 1 (v uv) β 1 exp v }

Statistics 351 Probability I Fall 2006 (200630) Final Exam Solutions. θ α β Γ(α)Γ(β) (uv)α 1 (v uv) β 1 exp v } Statistics 35 Probability I Fall 6 (63 Final Exam Solutions Instructor: Michael Kozdron (a Solving for X and Y gives X UV and Y V UV, so that the Jacobian of this transformation is x x u v J y y v u v