LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur
Aalytcal methods for selectg a trasformato o study varable The Box-Cox method Suppose the ormalty ad/or costat varace of study varable y ca be corrected through a power trasformato o y. Ths meas y s to be trasformed as y where s the parameter to be determed. For example, f 0.5, the the trasformato s square root ad y s used as study varable place of y. Now the lear regresso model has parameters smultaeously the βσ, ad. Box ad Cox method tells how to estmate ad parameters of the model usg the method of maxmum lkelhood. y 0 Note that as approaches zero, approaches to. So there s a problem at because ths makes all the observato y to be uty. It s meagless that all the observato o study varable are costat. So there s a y dscotuty at. Oe approach to solve ths dffculty s to use as a study varable. 0 Note that as y 0, l y y for 0 W l y for 0.. So a possble soluto s to use the trasformed study varable as
3, So famly W s cotuous. Stll t has a drawback. As chages, the value of W chage dramatcally. So t s dffcult to obta the best value of. If dfferet aalyst obta dfferet values of the t wll ft dfferet models. It may the ot be approprate to compare the models wth dfferet values of y for 0 ( ) y V y* y* l y for 0 y y yy y / * (... ) where y * s the geometrc mea of s as whch s costat.. So t s preferable to use a alteratve form For calculato purpose, we ca use l y* l y. Whe V s appled to each y, we get we use t to ft a lear model V ( V V V ) V Xβ + ε,,..., ' as a vector of observato o trasformed study varable ad usg least squares or maxmum lkelhood method. The quatty y the deomator s related to the th power of Jacoba of trasformato. See how: *
4 We wat to covert y to Let ( ) y ( ) y y W ; 0. ( ) as y y, y,..., y ', W ( W, W,..., W )'. Note that f W W y W y y, the y y 0. I geeral, W y f j y j 0 f j. The Jacoba of trasformato s gve by y J( y W). W W y y
5 W W W y y y y y W W W JW ( y) y y y W W W y y y y y Y J( y W) y. JW ( ) 0 0 0 0 0 0 0 0 0 y Sce ths s a Jacoba whe we wat to trasform the whole vector y to whole vector W. If a dvdual y s to be trasform to W, the take ts geometrc mea as The quatty JY ( W) y J( y W) y. esures that ut volume s preserved movg from the set of y to the set of V. Ths s a factor whch scales ad esures that the resdual sum of squares obtaed from dfferet values of ca be compared.
6 To fd the approprate famly, cosder where ( ) y V X β + ε y y, ~ N (0, I ). ( ) ε σ y * Applyg method of maxmum lkelhood for lkelhood fucto for ( y ), ( ) ε L y exp πσ σ εε ' exp πσ σ ( ) ( ) ( y Xβ)'( y Xβ) exp πσ σ σ ( ) ( ) ( ) ( y Xβ)'( y Xβ) l L y l σ ( gorg costat).
7 Solvg ( ) l L y β ( ) l L y σ gves the maxmum lkelhood estmators ( ) ˆ( ) ( ' ) X X X ' β y 0 0 for a gve value of. ( ) ( ) y ' Hy ˆ σ ( ) y ' I X( X ' X) X ' y ( ) ( ) Substtutg these estmates the log lkelhood fucto L( ) l ˆ σ l res( ) [ SS ] where s the sum of squares due to resduals whch s a fucto of. Now maxmze wth respect to It s dffcult to obta ay closed form of the estmator of ( ) l L y SS ( ) re s L( ).. So we maxmze t umercally. gves The fucto l res( ) [ SS ] s called as the Box-Cox objectve fucto.
8 Let be the value of whch maxmzes the Box-Cox objectve fucto. The uder farly geeral codtos, for ay other max has approxmately [ ] [ ] l SS ( ) l SS ( ) re s re s max χ () Ths s explaed as follows: dstrbuto. Ths result s based o the large sample behavour of the lkelhood rato statstc. The lkelhood rato test statstc our case s Max L Ωo η η Max L Ω Max Ω o σ Max Ω σ ˆ σ ( ) ˆ σ ( max ) / SSres( ) / SSres( max )
9 lη SS ( ) re s max l SSres( ) SS ( ) res lη l SSres( max ) l re s( ) l re s( max ) [ SS ] [ SS ] L( ) + L( ) max where L( ) l re s( ) [ SS ] L( max ) l [ SSre s( max )]. lη χ () Sce uder certa regularty codtos, coverges dstrbuto to whe the ull hypothess s true, so l η ~ χ () χ () or l η ~ χ () or L( max ) L( )~.