REGRESSION Notaton Ths rocedure erforms multle lnear regresson wth fve methods for entry and removal of varables. It also rovdes extensve analyss of resdual and nfluental cases. Caseweght (CASEWEIGHT) and regresson weght (REGWGT) can be secfed n the model fttng. The followng notaton s used throughout ths chater unless stated: y c g Deendent varable for case wth varance σ g Caseweght for case ; c f CASEWEIGHT s not secfed Regresson weght for case ; g f REGWGT s not secfed l w W Number of dstnct cases cg l w Number of ndeendent varables C Sum of caseweghts: c x l The th ndeendent varable for case X l Samle mean for the th ndeendent varable: X wx W Y l Samle mean for the deendent varable: Y wy W h Leverage for case
REGRESSION h g W + h S j S yy S y Samle covarance for X and X j Samle varance for Y Samle covarance for X and Y Number of coeffcents n the model. f the ntercet s not ncluded; + R The samle correlaton matrx for X, K, X and Y Descrtve Statstcs r K r r y r K r r y R K! where ry K ry ryy " $ # r j S S j S jj and r y r y S S y S yy The samle mean X and covarance S j are comuted by a rovsonal means algorthm. Defne W w cumulatve weght u to case
REGRESSION 3 then 6 6 6 X X + x X w W and, f the ntercet s ncluded, w Cj6 Cj 6 + x X 6 xj X j 6 w W Otherwse, 6 6 Cj Cj + wxxj where 6 X x and C j 6 0 The samle covarance S j s comuted as the fnal C j dvded by C. Swee Oeratons (Demster, 969) For a regresson model of the form Y β + β X + β X + + β X + e 0 L swee oeratons are used to comute the least squares estmates b of β and the assocated regresson statstcs. The sweeng starts wth the correlaton matrx R.
4 REGRESSION Let R be the new matrx roduced by sweeng on the th row and column of R. The elements of R are r r r r r, r rj j, r j and rr j r rj rj,, j r If the above swee oeratons are reeatedly aled to each row of R n R R R R R where R contans ndeendent varables n the equaton at the current ste, the result s R R R R R R R R R R The last row of RR contans the standardzed coeffcents (also called BETA), and R RR R
REGRESSION 5 can be used to obtan the artal correlatons for the varables not n the equaton, controllng for the varables already n the equaton. Note that ths routne s ts own nverse; that s, exactly the same oeratons are erformed to remove a varable as to enter a varable. Varable Selecton Crtera Let r j be the element n the current swet matrx assocated wth X and X j. Varables are entered or removed one at a tme. X s elgble for entry f t s an ndeendent varable not currently n the model wth r t (tolerance wth a default of 0.000) and also, for each varable X j that s currently n the model, rj rj rjj t r The above condton s mosed so that entry of the varable does not reduce the tolerance of varables already n the model to unaccetable levels. The F-to-enter value for X s comuted as F to enter V ryy V wth and degrees of freedom, where s the number of coeffcents currently n the model and V ryry r The F-to-remove value for X s comuted as F to remove V ryy wth and degrees of freedom.
6 REGRESSION Methods for Varable Entry and Removal Fve methods for entry and removal of varables are avalable. The selecton rocess s reeated untl the maxmum number of stes (MAXSTEP) s reached or no more ndeendent varables qualfy for entry or removal. The algorthms for these fve methods are descrbed below. Stewse If there are ndeendent varables currently entered n the model, choose X such that F to remove s mnmum. X s removed f F to remove < Fout (default.7) or, f robablty crtera are used, PF to remove 6 > Pout (default 0.). If the nequalty does not hold, no varable s removed from the model. If there are no ndeendent varables currently entered n the model or f no entered varable s to be removed, choose X such that F to enter s maxmum. X s entered f F to enter > Fn (default 3.84) or, PF to enter6 < Pn (default 0.05). If the nequalty does not hold, no varable s entered. At each ste, all elgble varables are consdered for removal and entry. Forward Ths rocedure s the entry hase of the stewse rocedure. Bacward Ths rocedure s the removal hase of the stewse rocedure and can be used only after at least one ndeendent varable has been entered n the model. Enter (Forced Entry) Choose X such that r s maxmum and enter X. Reeat for all varables to be entered.
REGRESSION 7 Remove (Forced Removal) Statstcs Summary Choose X such that r s mnmum and remove X. Reeat for all varables to be removed. For the summary statstcs, assume ndeendent varables are currently entered n the equaton, of whch a bloc of q varables have been entered or removed n the current ste. Multle R R r yy R Square R r yy Adjusted R Square R adj R 4 R 9 R Square Change (when a bloc of q ndeendent varables was added or removed) current revous R R R
8 REGRESSION F Change and Sgnfcance of F Change F R 3 8 q Rcurrent & K 3 8 R 3 q8 qrrevous ' K 3 8 for the removal of q ndeendent varables for the addton of q ndeendent varables the degrees of freedom for the addton are q and, whle the degrees of freedom for the removal are q and q. Resdual Sum of Squares 6 SSe ryy C S yy wth degrees of freedom. Sum of Squares Due to Regresson SS R C S R 6 yy wth degrees of freedom.
REGRESSION 9 ANOVA Table Analyss of Varance df Sum of Squares Mean Square 6 e6 4 9 Regresson SS R SSR w SS e SS C Varance-Covarance Matrx for Unstandardzed Regresson Coeffcent Estmates A square matrx of sze wth dagonal elements equal to the varance, the below dagonal elements equal to the covarance, and the above dagonal elements equal to the correlatons: 6 var b rryysyy S 3 8 3 j8 cov b, b 3 j8 cor b, b r r S S S r j r r jj j yy jj yy 3 8 Selecton Crtera Aae Informaton Crteron (AIC) SSe AIC C ln C +
0 REGRESSION Amemya s Predcton Crteron (PC) PC 4 R 94C+ 9 Mallow s C (CP) e CP SS + C $ σ * where $σ s the mean square error from fttng the model that ncludes all the varables n the varable lst. Schwarz Bayesan Crteron (SBC) SSe SBC C ln C C + ln 6 Collnearty Varance Inflaton Factors VIF r Tolerance Tolerance r
REGRESSION Egenvalues, l The egenvalues of scaled and uncentered cross-roduct matrx for the ndeendent varables n the equaton are comuted by the QL method (Wlnson and Rensch, 97). Condton Indces η λ j max λ Varance-Decomoston Proortons Let 3 8 v v, K, v be the egenvector assocated wth egenvalue λ. Also, let Φ j v j λ and Φ j Φ j The varance-decomoston roorton for the jth regresson coeffcent assocated wth the th comonent s defned as π j Φ j Φ j Statstcs for Varables n the Equaton Regresson Coeffcent b b r y S S yy for, K,
REGRESSION The standard error of b s comuted as r r S yy yy $σ b S A 95 confdence nterval for b s constructed from b ± $ σ t b 0. 05, C If the model ncludes the ntercet, the ntercet s estmated as b y b X 0 The varance of b 0 s estmated by 6 $ ryysyy σ $ b + Xσb + X X est. b, b 0 cov C j+ j 3 8 j j Beta Coeffcents Beta r y The standard error of Beta s estmated by ryyr $σ Beta
REGRESSION 3 F-test for Beta Beta F Beta $σ wth and degrees of freedom. Part Correlaton of X wth Y 6 Part Corr X ry r Partal Correlaton of X wth Y 6 Partal Corr X r y r r r r yy y y Statstcs for Varables Not n the Equaton Standardzed regresson coeffcent Beta f X enters the equaton at the next ste ry Beta r The F-test for Beta F ry r ryy ry wth and degrees of freedom
4 REGRESSION Partal Correlaton of X wth Y PartalX 6 r r y r yy Tolerance of X Tolerance r Mnmum tolerance among varables already n the equaton f X enters at the next ste s mn, r j r r r r 3 8 jj j j Resduals and Assocated Statstcs There are 9 temorary varables that can be added to the actve system fle. These varables can be requested wth the RESIDUAL subcommand. Centered Leverage Values For all cases, comute h 0 5 & K ' K0 5 g X X X X r j j j C SjjS j g X X r j j C SjjS j 3 83 8 f ntercet s ncluded
REGRESSION 5 For selected cases, leverage s h ; for unselected case wth ostve caseweght, leverage s h & K ' K! + 6 g + + + W h W h W + h h g " $# f ntercet s ncluded Unstandardzed Predcted Values $Y b X & K b0 + ' K b X f no ntercet Unstandardzed Resduals e Y Y $ Standardzed Resduals ZRESID & K ' K e s SYSMIS f no regresson weght s secfed where s s the square root of the resdual mean square.
6 REGRESSION Standardzed Predcted Values ZPRED & K ' K Y$ Y sd SYSMIS f no regresson weght s secfed where sd s comuted as sd l c Y Y $ C Studentzed Resduals SRES & K K ' K e s h e s + h g g for selected cases wth c > 0 Deleted Resduals DRESID &K 'K e h c > 0 e for selected cases wth
REGRESSION 7 Studentzed Deleted Resduals SDRESID & K K ' K DRESID s 6 e s + h g for selected cases wth c > 0 where s6 s comuted as 6 s s h DRESID Adjusted Predcted Values ADJPRED Y DRESID DfBeta ge DFBETA b b 6 6 XWX h X t where &K 3 'K 8 3 8, X t, K, X X X, K, X f ntercet s ncluded ottherwse and W dag w, K, w l 6.
8 REGRESSION Standardzed DfBeta SDBETA j 6 bj bj t s6 4XWX9 jj where bj bj 6 s the jth comonent of b b6. DfFt DFFIT he X b b6 h Standardzed DfFt SDFIT DFFIT s 6 h Covrato COVRATIO 6 s s h Mahalanobs Dstance For selected cases wth c > 0, MAHAL h & 6 f ntercet s ncluded ' Ch
REGRESSION 9 For unselected cases wth c > 0 MAHAL & ' 6 Ch C+ h f ntercet s ncluded Coo s Dstance (Coo, 977) For selected cases wth c > 0 COOK 'K DRESID hg s + &K 6 4DRESID hg9 4s 9 f ntercet s ncluded For unselected cases wth c > 0 COOK & K ' K + 6 DRESID h + s W DRESID h 4s 9 f ntercet s ncluded where h s the leverage for unselected case, and s s comuted as C SS e h e + W s & K! + K SSe + e h 6 + ' K " $# f ntercet s ncluded
0 REGRESSION Standard Errors of the Mean Predcted Values For all the cases wth ostve caseweght, SEPRED &K 'K s h s h g f ntercet s ncluded g 95 Confdence Interval for Mean Predcted Resonse LMCIN Y$ t SEPRED 0. 05, UMCIN Y$ + t SEPRED 0. 05, 95 Confdence Interval for a Sngle Observaton LICIN UICIN &K 'K Y$ t s h + g 0. 05, Y$ t s h + g &K 'K 6 0. 05, 6 Y$ + t s h + g 0. 05, Y$ + t s h + g 0. 05, f ntercet s ncluded f ntercet s ncluded Durbn-Watson Statstc DW 6 l e e l ce where e e g.
REGRESSION Partal Resdual Plots Mssng Values References The scatterlots of the resduals of the deendent varable and an ndeendent varable when both of these varables are regressed on the rest of the ndeendent varables can be requested n the RESIDUAL branch. The algorthm for these resduals s descrbed n Velleman and Welsch (98). By default, a case that has a mssng value for any varable s deleted from the comutaton of the correlaton matrx on whch all consequent comutatons are based. Users are allowed to change the treatment of cases wth mssng values. Coo, R. D. 977. Detecton of nfluental observatons n lnear regresson, Technometrcs, 9: 5 8. Demster, A. P. 969. Elements of Contnuous Multvarate Analyss. Readng, Mass.: Addson-Wesley. Velleman, P. F., and Welsch, R. E. 98. Effcent comutng of regresson dagnostcs. The Amercan Statstcan, 35: 34 4. Wlnson, J. H., and Rensch, C. 97. Lnear algebra. In: Handboo for Automatc Comutaton, Volume II, J. H. Wlnson and C. Rensch, eds. New Yor: Srnger-Verlag.