Chapter 5 Transformation and Weighting to Correct Model Inadequacies

Size: px

Start display at page:

Download "Chapter 5 Transformation and Weighting to Correct Model Inadequacies"

June Richardson
5 years ago
Views:

1 Chapter 5 Trasformato ad Weghtg to Correct Model Iadequaces The graphcal methods help detectg the volato of basc assumptos regresso aalss. Now we cosder the methods ad procedures for buldg the models through data trasformato whe some of the assumptos are volated. Varace stablzg trasformatos I regresso aalss, t s assumed that the varace of dsturbaces s costat,.e., Var = = Suppose ths assumpto s volated. A commo reaso for such solato s that ( ε ) σ,,,...,. the stud varable follows a probablt dstrbuto whch the varace s fuctoall related to mea. For example, f stud varable ( ) the model s Posso radom varable a smple lear regresso model, the ts varace s same as mea. Sce mea of s related to explaator varable x, so the varace of wll be proportoal to x. I such cases, varace stablzg trasformatos are useful. I aother example, f s proporto,.e., the such cases the varace of s proportoal to E( )[ E( )]. I such case, the varace stablzg trasformato s useful. Some commol used varace-stablzg trasformatos the order of ther stregth are as follows: Relato of σ to E( ) Trasformato σ σ costat E( ) σ E σ [ E( )] σ [ E( )] 3 ( )[ E( )] = (o trasformato) = (Posso data) = (Bomal proporto ) s ( ) = l( ) = / σ [ E( )] 4 = After makg the sutable trasformato, use as a stud varable respectve case. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur

2 The stregth of a trasformato depeds o the amout of curvature preset the curve betwee stud ad explaator varable. The trasformato metoed here rage from relatvel mld to relatvel strog. The square root trasformato s relatvel mld ad recprocal trasformato s relatvel strog. The square root trasformato s relatvel mld ad recprocal trasformato s relatvel strog. I geeral, a mld trasformato appled whe the mmum ad maxmum values do ot rage much (e.g. max / m <,3) ad such trasformato has lttle effect o the curvature. O the other had whe the mmum ad maxmum var much the, a strog trasformato s eeded that wll have a strog effect o the aalss. I the presece of o-costat varace, the OLSE wll rema ubased but wll looses the mmum varace propert. Whe the stud varable has bee trasformed as, the the predcted values are the trasformed scale. It s ofte ecessar to covert the predcted values back to the orgal uts ( ). Whe verse trasformato s drectl appled to the orgal values, the t gves a estmate of the meda of the dstrbuto of stud varable stead of mea. So oe eeds to be careful whle dog so. Cofdece terval ad predcto terval ma be drectl coverted from oe metrc to aother. Reaso beg that the terval estmates are percetle of a dstrbuto ad percetles are uaffected b trasformato. Oe ma ote that the resultg tervals ma or ma ot be or rema the shortest possble tervals. Trasformatos to learze the model The basc assumpto lear regresso aalss s that the relatoshp betwee stud varable ad explaator varables s lear. Suppose ths assumpto s volated. Such volato ca be checked b scatter plot matrx, scatter dagrams, partal regresso plots, lack of ft test etc. I some cases, a olear model ca be learzed b usg a sutable trasformato. Such olear models are called trscall or trasformabl lear. The advatage of trasformg the olear fucto to lear fucto s that the statstcal tools are developed for the case of lear regresso model. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur

3 For example, exact tests for test of hpothess, cofdece terval estmato etc. are developed for the case of lear regresso model. Oce the olear fucto s trasformed to a lear fucto, all such tools ca be readl appled ad there s o eed to develop them separatel. Some learzable fuctos are as follows:. If the curve betwee ad x s lke as follows: the the possble learzable fucto s of the form = β x β. Usg the trasformato = l, x = l x,.e., b takg log o both sdes, the model becomes log = log β + β log x or = β + β x where β = log β ad the model becomes a lear model. Note that the parameter β chages to log β the trasformed model. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 3

4 . If the curve betwee ad x s lke as follows the the possble learzable fucto s of the form = β exp ( β x) Takg log (l) e o both sdes, l = l β + β x or = β + x β where = l ad β = l β. So = l s the trasformato eeded ths case. The tercept term β becomes l β the trasformed model. 3. If the curve betwee ad x s lke as follows Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 4

5 the the possble learzable fucto s of the form = β + β log x whch ca be wrtte as = β + β x usg the trasformato x = log x. 4. If the curve betwee ad x s lke as follows the the possble learzable fucto s of the form x = β x β whch ca be wrtte as β = β x or β β = + x. whch becomes a lear model b usg the trasformato =, x = x. Wth the observed behavour of the plots, oe ca choose a such curve ad use the learzed form of the fucto. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 5

6 Whe such trasformatos are used, ma tmes the form of ε also gets chaged. For example, case of or = β exp( β x) ε l = l β + β x+ l ε = β + β x+ ε. Ths mples that the multplcatve error orgal model s log ormall dstrbuted the trasformed model. Ma tmes, we gore ths aspect ad cotue to assume that the radom errors are stll ormall dstrbuted. I such cases, the resduals from the trasformed model should be checked for the valdt of the assumptos. Whe such trasformatos are used, the OLSE has the desred propertes wth respect to the trasformed data ad ot the orgal data. Aaltcal methods for selectg a trasformato o stud varable The Box-Cox method Suppose the ormalt ad/or costat varace of stud varable ca be corrected through a power trasformato o. Ths meas s to be trasformed as where s the parameter to be determed. For example, f =.5, the the trasformato s square root ad s used as stud varable place of. Now the lear regresso model has parameters βσ, ad. Box ad Cox method tells how to estmate smultaeousl the ad parameters of the model usg the method of maxmum lkelhood. Note that as approaches zero, approaches to. So there s a problem at = because ths makes all the observato to be ut. It s meagless that all the observato o stud varable are costat. So there s a dscotut at =. Oe approach to solve ths dffcult s to use as a stud varable. Note that as, l. So a possble soluto s to use the trasformed stud varable as Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 6

7 W for = l for =. So faml W s cotuous. Stll t has a drawback. As chages, the value of W chage dramatcall. So t s dffcult to obta the best value of. If dfferet aalst obta dfferet values of, the t wll ft dfferet models. It ma the ot be approprate to compare the models wth dfferet values of. So t s preferable to use a alteratve form for ( ) = V = l for = where s the geometrc mea of ' s as / = (... ) whch s costat. For calculato purpose, we ca use l = l. = Whe V s appled to each varable ad we use t to ft a lear model V = Xβ + ε. V V V V =,,..., ' as a vector of observato o trasformed stud, we get ( ) usg least squares or maxmum lkelhood method. The quatt the deomator s related to the th power of Jacoba of trasformato. See how: We wat to covert to ( ) as ( ) = W = ;. =,,..., ', W = ( W, W,..., W )'. Let ( ) Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 7

8 Note that f W =, the W = = I geeral, W =. W f = j = j f j. The Jacoba of trasformato s gve b J( W) = = =. W W W W W = W W W JW ( ) = = = W W W = = J( W) = =. JW ( Y ) = Sce ths s a Jacoba whe we wat to trasform the whole vector to whole vector W. If a dvdual s to be trasform to W, the take ts geometrc mea as J( W) =. = Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 8

9 The quatt JY ( W) = = esures that ut volume s preserved movg from the set of to the set of V. Ths s a factor whch scales ad esures that the resdual sum of squares obtaed from dfferet values of ca be compared. To fd the approprate faml, cosder ( ) V X = = β + ε where, ~ N (, I ). ( ) = ε σ Applg method of maxmum lkelhood for lkelhood fucto for Solvg ( ) ε = L = exp πσ σ εε ' = exp πσ σ ( ) ( ) ( Xβ)'( Xβ) exp = πσ σ β β = σ ( ), ( ) ( ) ( ) ( X )'( X ) l L l σ (gorg cos ( ) l L = β ( ) l L = σ gves the maxmum lkelhood estmators ˆ( β ) = ( X ' X ) X ' ( ) ( ) ( ) ' H ˆ σ ( ) = ' I X( X ' X) X ' = for a gve value of. ( ) ( ) tat). Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 9

10 Substtutg these estmates the log lkelhood fucto L( ) = l ˆ σ = l res( ) [ SS ] ( ) l L gves where SS ( ) re s s the sum of squares due to resduals whch s a fucto of. Now maxmze L( ) wth respect to. It s dffcult to obta a closed form of the estmator of. So we maxmze t umercall. res s called as the Box-Cox objectve fucto. The fucto l [ SS ( ) ] Let max be the value of whch mmzes the Box-Cox objectve fucto. The uder farl geeral codtos, for a other [ ] [ ] l SS ( ) l SS ( ) has approxmatel re s re s max χ () dstrbuto. Ths result s based o the large sample behavour of the lkelhood rato statstc. Ths s explaed as follows: The lkelhood rato test statstc our case s Max L Ωo η η = Max L Ω ˆ ( ) ˆ ( max ) Max Ωo σ = Max Ω σ σ = σ / SSres( ) = / SSres( max ) Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur

11 where lη SS ( ) re s max = l SSres( ) SS ( ) res lη = l SSres( max ) = l re s( ) l re s( max ) = L( ) + L( ) [ SS ] [ SS ] max L( ) = l [ SSres( ) ] L( max ) = l [ SSres( max )]. Sce uder certa regulart codtos, hpothess s true, so η χ l ~ () χ () or l η ~ χ () or L( max ) L( ) ~. lη coverges dstrbuto to χ () whe the ull Computatoal procedure The maxmum- lkelhood estmate of correspods to the value of for whch resdual sum of squares from the ftted model SS ( ) e s a mmum. To determe such, we proceed computatoall as follows: r s - Ft ( ) for varous values of. For example, start wth values (-, ) the take the values (-, ) ad so o. Take about 5 to values of whch are expected to be suffcet for the estmato of optmum value. - Plot SS ( ) re s versus. - Fd the value of whch mmzes SS ( ) e from the graph. - A secod terato ca be performed usg a fer mesh of values of desred. r s Note that the value of ca ot be selected b drectl comparg the resdual sum of squares from the regresso of o x because for each, the resdual sum of squares s measured o a dfferet scale. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur

12 It s better to use smple values of. For example, the practcal dfferece betwee =.5 ad =.58 s lkel to be small but =.5 s much easer to terpret. Oce s selected, the use as a stud varable f l as a stud varable f =. It s etrel acceptable to use ( ) as respose for fal model. Ths model wll have a scale dfferece ad a org shft comparso to model usg (or l ) as the respose. A approxmate cofdece terval for We ca fd a approxmate cofdece terval for the trasformato parameter. Ths terval helps selectg the fal value of. For example, f ˆ =.58 s the value of whch s mmzg the sum of squares due to resdual. But f =.5 s the cofdece terval, the oe ma use the square root trasformato because t s easer to expla. Furthermore f = s the cofdece terval, the t ma be cocluded that o trasformato s ecessar. I applg the method of maxmum lkelhood to the regresso model, we are essetall maxmzg L( ) = l res( ) [ SS ] or equvaletl, we are mmzg SS ( ) re s. A approxmate (- α ) % cofdece terval for cossts of those values of that satsf ˆ χ α () L( ) L( ) where χ α () s the upper α % pot of the Ch-square dstrbuto wth oe degree of freedom. The approxmate cofdece terval s costructed usg the followg steps: - Draw a plot of L( ) versus. - Draw a horzotal le at heght ˆ χ α () L( ) o the vertcal scale. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur

13 - Ths le would cut the L( ) at two pots. - The locato of these two pots o the -axs defes the two ed pots of the approxmate cofdece terval. - If sum of squares due to resduals s mmzed ad SS ( ) e versus s plotted, the the le must be plotted at the heght ˆ χ () ( )exp α = SS SS r e s where ˆ s the value of whch mmzes the sum of squares due to resduals. See how: r s ˆ χα() ˆ χα() L( ) = l SSres( ) () l { ˆ χ α = SSRes ( ) } + () l { SS ˆ χ α = res( ) } + l exp ˆ χα () = l SSres( ).exp = l SS. Usg the expaso of expoetal fucto as t exp( t) = + t+ +...! + t, we ca approxmate ad replace exp χ α () cofdece terval procedure, we ca use the followg: χ () b + α. So place of exp χ α () applg the Z Z + or + γ α/ α/ t t or + or + γ α/ α/ χ χ or + or + γ α/ α/ where γ s the degrees of freedom assocated wth sum of squares due to resduals. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 3

14 These expressos are based o the fact that χ () = Z t γ f γ s small. It s debatable to use ether γ or but practcall the dfferece s ver lttle betwee the cofdece terval results. Box-Cox trasformato was orgall troduced to reduce the oormalt the data. It also helps s reducg the oleart. The approach s to fd out the trasformatos whch attempts to reduce the resduals assocated wth outlers ad also reduce the problem of o costat error varace f there was o acute oleart to beg wth. Trasformato o explaator varables: Box ad Tdwell procedure Suppose the relatoshp betwee ad oe or more of the explaator varables s olear. Other usual assumptos ormall ad depedetl dstrbuted stud varable wth costat varace are at least approxmatel satsfed. We wat to select a approprate trasformato o the explaator varable so that the relatoshp betwee ad trasformed explaator varable s as smple as possble. Box ad Tdwell procedure descrbes a geeral aaltcal procedure for determg the form of trasformato o x Suppose that the stud varable s related to the power of explaator varables. Box ad Tdwell procedures for explaator varables chooses the varables as z j α j xj whe α j, =,,.., ; j =,,..., k = α j l xj whe α j =. We eed to estmate α ' s. Sce the depedet varable s ot beg trasformed, we eed ot worr about the chages of scale ad mmze [ ] β βz... βk zk = b usg olear least squares techques. j Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 4

15 We cosder ths for smple lear regresso model stead of olear regresso model. Assume s related to ξ = x α as E( ) = f( ξ, β, β) = β + βξ α x f α where ξ = l x f α = where β, β ad α are the ukow parameters. Suppose α s the tal guess of costat α. Usuall, frst guess s α = so that ξ = x or o trasformato s appled the frst terato. Expad about the tal guess a Talor seres ad gorg terms of order hgher them oe gves df ( ξ, β, β ) E( ) ( ξ, β, β ) ( α α ) = f + dα ξ= ξ α= α df ( ξ, β, β ) = β + β x + α ( ). dα ξ= ξ α= α Suppose the term df ( ξ, β, β ) d α ξ= ξ α= α s kow, the t ca be treated just lke as a addtoal explaator varable. The the parameters β, β ad α ca be estmated b least squares method. The estmate of α ca be cosdered as a mproved estmate of the trasformato parameter. Ths term ca be wrtte as df ( ξ, β, β) df ( ξ, β, β) dξ = dα dξ dα ξ ξ ξ ξ α α α α = = = =. Sce the form of trasformato s kow,.e., ξ = x α, dξ so x l x. dα = Furthermore Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 5

16 df ( ξ, β, β) d( β + βx) = = β. dξ ξ= ξ dx So β ca be estmated b fttg the model ŷ = ˆ β + ˆ β x b least squares method. The a adjustmet to tal guess α = s computed b defg a secod regresso varable as ω = xl estmatg the parameter x E( ) = β + β x+ ( α ) βω = β + β + γω x b least squares. Ths gves the followg: or ˆ = ˆ β ˆ ˆ + βx+ γω ˆ γ = ( α ) ˆ β ˆ γ α = + ˆ β as the revsed estmate of α. Note that ˆβ s obtaed from ŷ = ˆ β ˆ + βx ad ˆ γ s obtaed from Geerall, ˆ β ad ˆ β wll dffer. ŷ = ˆ β + ˆ β x+ ˆ γω. Ths procedure ma be repeated usg a ew regresso x = x α the calculato. Ths procedure geerall coverges rapdl. Usuall, the frst stage result α s a satsfactor estmate of α. The roud-off error s a potetal problem. If eough decmal places are ot take care, the the successve values of α ma oscllate badl. If the stadard devato of error ( σ ) s large or the rage of explaator varable s ver small relatve to ts mea the the estmator ma face covergece problems. Ths stuato mples that the data do ot support the eed for a trasformato. Regresso Aalss Chapter 5 Trasf. Weght. Correct Model Iadequaces Shalabh, IIT Kapur 6

LINEAR REGRESSION ANALYSIS

LINEAR REGRESSION ANALYSIS MODULE V Lecture - Correctg Model Iadequaces Through Trasformato ad Weghtg Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur Aalytcal methods for