The general linear model (and PROC GLM)

Transcription

1 The general linear model (and PROC GLM) Solving systems of linear equations 3"! " " œ 4" " œ 8! " can be ritten as 3 4 "! œ " 8 " or " œ c. No look at the matrix 3. One can see that œ œ 0 0 œ I, the identity matrix. The matrix " œ is called the inverse matrix for the matrix. If such a matrix can be found, then the solution to the linear equations becomes 3 " " 8 " œ I" œ " œ c œ "#.

2 This solution is a unique solution. No suppose e have instead 3"! " " œ 6" " œ 8! " 3 that is, œ. Note that the equations have no 6 solution. The determinant of is det œ 36 œ 0. is said to be singular. If e just consider the first equation, there are an infinite number of solutions (an arbitrary solution to the equation can be found just by setting " œ constant, say, " œ 3, and then " œ 3 results. " "! General linear model Y œ X" % here Y is the R vector of random variables (the responses), X is the R 5 design matrix (ith R5 Ñ, " is the 5 column vector of parameters, and % is an R column vector of independent normal0, 5 # random variables. Let y denote the vector of data (recorded values of the variables in Y). Parameter estimates satisfy a system of linear equations (the normal equations): XX" s œ Xy

3 " If XXis nonsingular, its inverse XX exists, and the equations have a solution. Matrix theory result: if the columns of X are linearly independent, that is, if one column cannot be ritten as a linear combination of the other columns, then XX is nonsingular. OV: means coding Suppose there are 3 treatment levels, ith means.",.#, and. $, and 4 observations in each cell. The model is ] 34 œ. 3 % 34 The means coding for the design matrix is X œ

4 Note that an indicator column is left off for the third treatment; otherise the three last ros add up to the first ro. The parameters are "! œ. $ " " œ. " " œ.! " "! # # OV: effects coding Suppose there are 3 treatment levels, ith means.",.#, and. $ no parameterized as." œ. α ",.# œ. α#,. $ œ. α$, here the α3s sum to zero, and 4 observations in each cell. The model is ] 34 œ. α 3 % 34 The effects coding for the design matrix is X œ

5 The parameters are "! œ. "" œ α" "# œ α# " " œ α " # $

6 To factors: suppose there is another factor ith levels, in a completely randomized factorial design, observations per cell. Main effects model is ] 345 œ. α 3 # 4 % 345 The design matrix (effects coding) is X œ Parameters are "! œ. "" œ α" "# œ α# " " œ α " $ œ #" " œ # " # $ $ #

7 PROC GLM PROC GLM is set up mainly for testing of statistical hypotheses. It uses a less than full rank coding for the indicator variables in the design matrix. For instance, its design matrix for to factors (3 levels & levels, obs. per cell) is X œ Ô Ö Ù 0 0 Õ 0 0 Ø Interactions ould be coded as six extra columns (products of cols -4 ith 5-6). The matrix XX is singular, and the normal equations do not have a solution. GLM uses a generalized inverse solution to the normal equations. hich allos a partial

8 The ordinary inverse of produces #" œ I property of the ordinary inverse is #" #" #" œ generalized inverse of the matrix, denoted #, is any matrix such that # # # One such matix is found by finding a smaller matrix hich can be inverted. Write œ œ "" "# #" ## here "" is an invertible matrix (say, 7 7). Then #" # "" 0 œ 0 0 "# #" ## is a generalized inverse, here the 0' s are matrices of zeros. For example, recall the matrix generalized inverse is œ # œ

9 partial solution to a system of equations is given by " œ c b œ # c This amounts to zeroing out as many equations and variables necessary to get a solvable system of equations. For instance, our system of equations given by 3"! ) " " œ 6"! ) " " œ 8 becomes, ith the above generalized inverse, b # œ c œ œ In other ords, drop the second equation, set solve for. "! The normal equations are " " œ 0, and axxb" s œ Xy GLM calculates a partial solution ( b, an 7 vector) to the normal equations in the form # b œ ax Xb X y

10 Why? Statistical hypotheses for the general linear model can be ritten in matrix form as L" œ 0 here L is a ro vector of constants. For instance, in a linear regression model, the hypothesis of zero slope results from L œ c0 d It turns out that for certain forms of L (linear combinations of the " 4 s called estimable functions) the linear function Lb is an unbiased estimate of L". ctually, L can even be a matrix, ith L" œ 0 giving a hole set of simultaneous hypotheses on estimable functions. The sums of squares for the hypotheses are # SSaL" œ 0b œ albb clax Xb L dalbb test of the hypotheses (as H ) is provided by the F! statistic given by Jœ SSaL" œ 0bÎ7 SSaerror b/(df for unrestricted model) here SS(error) is calculated from the generalized inverse: # SSaerrorb œ y ci# XaX Xb X dy

11 The partial solutions in b can be obtained ith the SOLUTION option in the MODEL statement, for instance: MODEL Y= B *B / SOLUTION; Hoever, those values are not of much interest (unless you have advanced interests). If one ants to report a model form, for use in prediction say, that contains categorical predictor variables, one might consider the folloing process: () develop the model (i.e. hat variables to include) in PROC GLM, then () code full rank indicator variables corresponding to the model for use in PROC REG. Use the coefficients reported by PROC REG.