Regularized Nonlinear Least Trimmed Squares Estimator in the Presence of Multicollinearity and Outliers

Transcription

1 Amerca Joural of Theoretcal ad Aled Stattc 8; 7(4): 56-6 htt:// do:.648/.ata ISSN: (Prt); ISSN: (Ole) Regularzed Nolear Leat Trmmed Square Etmator the Preece of Multcollearty ad Outler George Kembo Kru Ketay, Aada Omutokoh Kube, Joeh Mutua Mutya, Fud Dael Murth Deartmet of Stattc ad Actuaral Scece, Keyatta Uverty (KU), Narob, Keya Emal addre: To cte th artcle: George Kembo Kru Ketay, Aada Omutokoh Kube, Fud Dael Murth, Joeh Mutua Mutya. Regularzed Nolear Leat Trmmed Square Etmator the Preece of Multcollearty ad Outler. Amerca Joural of Theoretcal ad Aled Stattc. Vol. 7, No. 4, 8, do:.648/.ata Receved: Arl 7, 8; Acceted: May 9, 8; Publhed: Jue 9, 8 Abtract: Th tudy rooe a regularzed robut Nolear Leat Trmmed quare etmator that rele o a Elatc et ealty olear regreo. Regularzato arameter electo wa doe ug a robut cro-valdato crtero ad etmato through Newto Raho terato algorthm for the ormal model coeffcet. Mote Carlo mulato wa coducted to verfy the theoretcal roerte outled the methodology both for cearo of reece ad abece of multcollearty ad extece of outler. The rooed rocedure erformed well comared to the NLS ad NLTS a vewot of yeldg relatvely lower value of MSE ad Ba. Furthermore, a real data aaly demotrated atfactory erformace of the uggeted techque. Keyword: Elatc Net, Multcollearty, Regularzato, Nolear Leat Trmmed Square, Outler. Itroducto The kowledge of olear regreo oe of the wdely ued model aalyzg the effect of exlaatory varable o a reoe varable. For examle, Khademzadeh et al. [9] ued MaReduce to model large-cale olear regreo roblem, Ramalho ad Ramalho [3] ued momet baed etmato of olear regreo model wth boudary outcome ad edogeety to oegatve ad fractoal reoe. Moreover, Tabataba et al. [5] ued robut olear regreo to etmate drug cocetrato, ad tumor ze-metata. The comutatoal feld of tattc ha develoed over tme wth the troducto of the comuter whch ha creaed tattcal method ad algorthm wth umerou alcato caable of geeratg ad torg gfcat amout of data. Varou tude Geomc, Medce, Edemology, Marketg ad bac cece meet data et whereby the effect of collearty are aaly amogt ther tudy varable []. Regularzato techque have wdely bee ued for the oluto of ll-oed roblem occurrg the ue of maxmum lkelhood or leat quare method ad have roved ucceful everal feld cludg regreo aaly ad mache learg [6]. The methodology volve the addto of retrcto or ealty to a model [4], wth the obectve of revetg overfttg []. It alo mrove redctve accuracy tuato where there ext may redctor varable, hgh collearty, eekg for a are oluto or accoutg for varable groug the hgh-dmeoal dataet [5]. Tkhoov [7] develoed th mathematcal techque of regularzato whle workg o the oluto of ll-oed roblem. I vat lterature, choce of ealty fucto uch a,,, or orm are avalable. Ado, Koh ad Imoto [] troduced radal ba fucto wth hyerarameter ad cotructed olear regreo model wth the hel of regularzato. Tateh, Matu ad Koh [6] cotructed olear regreo model wth Gaua ba fucto ug weghted tye regularzato for aalyzg data wth comlex tructure. Farooh, Ghaema ad Fard [4] rooed a weghted rdge ealty o a fuzzy olear regreo model ug fuzzy umber ad Gaua ba fucto. Jag, Jag ad Sog [7] develoed weghted comote regreo etmato ad ued the Adatve Lao ad SCAD regularzato to acheve

2 57 George Kembo Kru Ketay et al.: Regularzed Nolear Leat Trmmed Square Etmator the Preece of Multcollearty ad Outler a multaeou arameter model etmato ad electo. Zucker et al. [9] develoed a aroxmate vero of the Stefak-Nakamura corrected core aroach, ug the method of regularzato to obta a aroxmate oluto of the relevat tegral equato ad Hag et al. [6] rooed a grah regularzed olear rdge regreo (RR) model for remote eg data aaly, cludg hyer-ectral mage clafcato ad atmoherc aerool retreval. Although thee regularzato method have how excetoal erformace varou feld, t ue the leat quare lo fucto whch flueced by outler []. Outler retat regularzed method have bee develoed by relacg the leat quare lo fucto wth the robut techque [5]. Amogt thee method are Huber M-etmator, MM-etmator, Leat Trmmed Square, Leat Meda Square etmator. Lm [] rooed robut Rdge regreo etmato rocedure for olear model wth varyg varace tructure. Regularzato arameter electo a eetal roblem Lao-tye method ce effectve varable electo, ad model etmato deed o aduted arameter. AIC, BIC, Cro-valdato, Mallow C crtero ad geeralzed formato crtero have bee uggeted for choog regularzato arameter. However, the Lao-tye ealty caot be aalytcally derved becaue they are ot dfferetable ad local quadratc aroxmato, LARS, ad Coordate decet algorthm have bee develoed to ettle th ue. Elatc Net ealty, a regularzato method, ha bee etablhed to ecourage groug effect whe redctor are hghly correlated ad alo ueful whe there ext a large umber of redctor tha that of the obervato lear regreo [8]. Secto troduce the rooed regularzed robut olear Leat Trmmed Square etmator wth a Elatc et ealty olear regreo model. I ecto 3, the rocedure for etmato ad arameter electo of the rooed methodology derved. Latly, the effcecy of the trategy vetgated through Mote Carlo tudy ad real data aaly ecto 4.. Regularzed Nolear Leat Trmmed Square Regreo The olear regreo model ha the form y f( x, ) + ε.,,, () where y are reoe value, x covarate, f( x, ) a olear fucto of the arameter, a ukow dmeoal vector of arameter ad ε a radom error aumed to have a dtrbuto fucto F() wth fte varace. The mot oular crtero for obtag arameter model etmate the Nolear Leat Square crtero (NLS), gve by m ( y (, )) f x. () However, the error dtrbuto the model () may exhbt heavy tal ad aymmetry due to data adequace ofte caued by meauremet or recordg error. Robut techque are commoly ued to accommodate volato of the aumto that uch error caue. Nolear Leat Trmmed Square etmator (NLTS) a hgh breakdow regreo techque whch afeguard agat wld obervato [3]. It derve the arameter of the model () by mmzg the obectve fucto of the form Where the breakdow ot, r() ˆ NLTS argm ( ) (3) () r are the order tattc of the quared redual ad. Model etmate of the obectve fucto (3) are derved ug a teratve rocedure becaue t caot be exreed a cloed form. A aroach to olear modelg to aroxmate f( x, ) by a frt order Taylor ere exao about a tal value gve by f( x, ) f( x, ) f( x, ) ( ) ad lead to the aroxmato + ˆ f( x, ) NLTS argm ( y f( x, ) ( )).. Th f( x, ) whe the Jacoba of full rak, the te uque ad reeted a f( x, ) f( x, ) ( ). o ( y f x ) f( x, ).. (, ) However, due to the reece of multcollearty, the vero may ot be ueful. The tudy rooe to obta arameter model etmate to ehace the etmato erformace of NLTS by mmzg the obectve fucto r() ˆ NLTS ELet argm[ + λ + λ ] Where λ + λ (4) the Elatc et ealty. The obectve fucto (4) above combe NLTS lo fucto wth a Elatc et ealty evaged to roduce

3 Amerca Joural of Theoretcal ad Aled Stattc 8; 7(4): table model etmate whe multcollearty reet wth the extece of outler. The NLTS-ELet etmator baed oly o obervato havg mall redual to relax the effect of outler atfyg <. The trmmg cotat defe the breakdow ot of the etmator, ad a breakdow ot of.75 codered, to clude a uffcet umber of obervato [8]. 3. Parameter Selecto ad Etmato of the NLTS-Elet Etmator The obectve fucto (4) above ca be mlfed to Where, ˆ NLTS Elet argm r() + λp α ( ). (5) P α α ( ) + ( α), (6) ad α λ/( λ + λ), α ad λ>. However, the ealty term (6) caot be aalytcally derved ce t ot dfferetable at zero. The local quadratc aroxmato of a Lao-tye ealty ha bee rooed to ettle th drawback. Suoe gve a a tal value of the mmzer of equato (5), the the local quadratc aroxmato of the dervatve of the ealty term gve by Where for evaluated at Pλ( ) Pλ( ) q q λ P ( ) λ, P ( ) λq λ. The regreo arameter vector ˆ derved ug Newto-Raho teratve method gve by ew old NLTS Elet NLTS Elet (7) ˆ ˆ E. T (8) The tudy rooe to ue the robut Geeralzed Cro Valdato aroach to elect regularzato arameter of the uggeted rocedure. The crtero gve by Where y fˆ( λ, x ) the redcted value ad H the hat matrx. A otmal et of the regularzato arameter that mmze equato (8) baed o the etmated ˆ by the NLTS-Elet at each et of tug arameter λ ad λ are elected ug grd earch. 4. Data Aaly ad Dcuo I th ecto, Mote Carlo tudy coducted to vetgate the behavour of the rooed NLTS-Elet etmator o mulated ad real data wth a comaro of t erformace to NLS ad NLTS. The Mea Squared Error ad Ba are codered evaluatg ther erformace. A breakdow ot of.75 codered to clude a uffcet umber of obervato. Stadard ormal error dtrbuto wth 3% Uform (-, 4) ad Studet t (3) cotamato are codered. Subecto (4.) tude the roerte of NLTS-Elet, NLTS ad NLS etmator ug mulated data et. Subecto (4.) exhbt the behavour of NLTS-Elet, NLS ad NLTS method o SENIC data et obtaed from the Hotal Ifecto Program for the tudy erod. 4.. Study o the Regularzed Leat Trmmed Square Etmator I th ubecto, a Mote Carlo tudy carred out to evaluate the erformace of the rooed olear regreo etmator. Samle of ze,, 4, 8 ad are geerated from a exoetal regreo model () below adoted from [7]. y + ex( cx ) + ε. () where X x for,, 3, ad x ~N (, ), ε the error,, ad c (, 3, 4) are model arameter. The true value of the arameter are,.5 ad c (-.6, -.8, -.7). The redctor ued model (8) are geerated wth Correlato betwee x l ad x m comoet choe a.99 for all l m. MSE ad Ba of arameter etmate are comuted ug the followg equato ( ˆ ) ˆ MSE( ), GCV { y fˆ( λ, x)} { trh} (9) ad, ˆ Ba( )( ˆ ).

4 59 George Kembo Kru Ketay et al.: Regularzed Nolear Leat Trmmed Square Etmator the Preece of Multcollearty ad Outler where ˆ the th etmate for the th relcato ad the actual value of the th coeffcet. Table. Parameter etmate, Mea quared error ad Ba for NLS uder Stadard ormal error dtrbuto wth 3% Uform (-, 4) ad Studet t (3) cotamato ad correlato.99, where the actual value of.5, -.6, -.8 ad -.7. ρ 3 4 Samle ze Method N (, ) U (-, 4) Studet t (3) NLS ˆ MSE Ba NLS ˆ MSE Ba NLS ˆ MSE Ba NLS ˆ MSE Ba NLS ˆ MSE Ba Table. Parameter etmate, Mea quared error ad Ba for NLTS uder Stadard ormal error dtrbuto wth 3% Uform (-, 4) ad Studet t (3) cotamato ad correlato.99, where the true value of.5, -.6, -.8 ad -.7. ρ 3 4 Samle ze Method N (, ) U (-, 4) Studet t (3) NLTS ˆ MSE Ba NLTS ˆ MSE

5 Amerca Joural of Theoretcal ad Aled Stattc 8; 7(4): Samle ze Method N (, ) U (-, 4) Studet t (3) Ba NLTS ˆ MSE Ba NLTS ˆ MSE Ba NLTS ˆ MSE Ba Table 3. Parameter etmate, Mea quared error ad Ba for NLTS-Elet uder Stadard ormal error dtrbuto wth 3% Uform (-, 4) ad Studet t (3) cotamato ad correlato.99, where the true value of.5, -.6, -.8 ad -.7. ρ 3 4 Samle ze Method N (, ) U (-, 4) Studet t (3) NLTS-Elet ˆ MSE Ba NLTS-Elet ˆ MSE Ba NLTS-Elet ˆ MSE Ba NLTS-Elet ˆ MSE Ba NLTS-Elet ˆ MSE Ba

6 6 George Kembo Kru Ketay et al.: Regularzed Nolear Leat Trmmed Square Etmator the Preece of Multcollearty ad Outler Table 4. Parameter Model etmate ug NLS ad NLTS-Elet Method for SENIC data. Parameter Method Etmated Parameter ˆ NLS NLTS NLTS-Elet NLS NLTS NLTS-Elet NLS NLTS NLTS-Elet NLS.9643 NLTS.6734 NLTS-Elet Table 5. MSPE etmate of NLS ad NLTS-Elet Method for SENIC data et. METHOD RMSPE NLS.3444 NLTS.3333 NLTS-Elet The examle for th table. The examle for th table. The examle for th table. The examle for th table. The examle for th table. The examle for th table. Table (), () ad (3) dlay the MSE ad Ba uder tadard ormal errror dtrbuto, ad the reece of Uform ad tudet t error cotamato. Moreover, arameter etmate of model (8) are dlayed where the three covarate x, x, ad x 3 are hghly correlated ( ρ.99). From table (), the olear leat quare method how low etmate of MSE ad ba uder the codered error dtrbuto ad amle ze. Throughout the coderd error dtrbuto, the method erfomed oorly uder mall amle ze, roducg etmate far from the true value, ad etmate well wth the amle of ze. From table (), the olear leat trmmed quare etmator how hgh value of MSE ad ba etmate for mall amle ze ad reduce gfcatly for larger amle ze. Geerally, the value of MSE are lower tha thoe for NLS. From table (3), the rooed regularzed NLTS method gve bet etmate of MSE, ba ad arameter of the model. Thee value became better whe creaed from to 4, whch gve true value from 8 to uder all the codered error dtrbuto. The methodologe erform oorly whe ad mrove a the amle creae. Furthermore, NLTS-Elet ha the lowet value of MSE uder all the codered error dtrbuto. It value alo eem to decreae cotetly from to excet for U (-, 4) ulke for NLS ad NLTS. Moreover, the model etmate of the rooed method aear to aroach a table value fater tha thoe for the codered method. 4.. Real Data Alcato I th ecto, the NLTS-Elet, NLS ad NLTS etmator are aled to SENIC data et obtaed from the Hotal Ifecto Program for the tudy erod. Model () below ued to model the relatoh betwee the fecto ( x ) ( x ) ( x 3 ) x ) rate (y) ad legth of tay, route culturg, umber of bed ad the average daly ceu for hotal dexed,, 3,, ad 3. NLTS-Elet, NLS method are comared arameter model etmato. y ex( x). + ex( x) () Frt, Varace Iflato Factor (VIF) whch meaure multcollearty deedet varable wa comuted ug the formula, where R the multle correlato coeffcet whe regreed agat all the other exlaatory varable the model. The average VIF value wa calculated a.5478 whch a dcato of hghly correlated varable. Mea Squared Predcto Error (MSPE), the ftted ad etmated obervato are codered evaluatg the erformace of NLTS-Elet, NLS ad NLTS method. The MSPE gve by y where ad deote the actual ad etmated obervato reectvely. The reult are ummarzed Table (4) ad (5). From Table (4), the etmated coeffcet for - were egatve ad otve for, howg that the fecto rate VIF MSPE ŷ x 4 R ( y. yˆ ), ( 4 x x 3 x

7 Amerca Joural of Theoretcal ad Aled Stattc 8; 7(4): creae wth the average umber of atet the hotal er day ad decreae wth the legth of tay, route culturg ad the average umber of bed durg the tudy erod of the Hotal Ifecto Program. Probably, th dect the actual cearo becaue a hgh average umber of atet the hotal creae the chace of cro fecto amog atet whch ca be mmzed by decreaed legth of tay, route culturg ad the average umber of bed. The Mea Squared Predcto Error value Table (5) are comarably the ame ad lowet for the rooed methodology. 5. Cocluo Regularzed robut Nolear Leat Trmmed Square etmator ha bee rooed th tudy by addg a Elatc et ealty to the NLTS lo fucto. Robut geeralzed cro-valdato wa ued to elect the regularzato arameter robutly. It ca be ee through Mote Carlo tudy ad real-world data examle that the rooed methodology erform well everal tuato comared to the NLS ad NLTS the vewot of yeldg relatvely lower value of MSE ad Ba whe there the reece of multcollearty. A breakdow ot of.75 wa redetermed to ehace effcecy by cludg the maorty of the data ot. Future work rema to be doe toward coderg the multaeou electo of otmal value of regularzato arameter ad the breakdow ot. Alo, varable electo behavour of the rooed etmator a large umber of redctor could be a oble roblem whch eed to be exlored. Referece [] Ado, T., Koh, S., Imoto, S. (8). Nolear regreo modelg va regularzed radal ba fucto etwork, Joural of Stattcal Plag ad Iferece, Volume 38, Iue, Page [] Batterham, A., Tolfrey, K., ad George, K. Nevll (997). exlaato of kleber.75 ma exoet: a artfact of collearty roblem leat quare model? Joural of Aled Phyology, 8: [3] Czek, P. (). Nolear leat trmmed quare. SFB Dcuo aer Humboldt Uverty, 5/. [4] Farooh, R., J. Ghaema, J., Fard, O. S. (). Com. ad Al. Math. - o., 3, [5] Hahem, H. (4). Regularzed ad robut regreo method for hgh-dmeoal data. Deartmet of Mathematcal Scece, Bruel Uverty. [6] Hag, R., Lu, J. J., Q., Sog, H., Zhu, F., ad Pe, H. (7). Grah Regularzed Nolear Rdge Regreo for Remote Seg Data Aaly. IEEE, : [7] Jag, X., Jag, J., ad Sog, X. (). Oracle model electo for olear model baed o weghted comote quatle regreo. Stattca Sca, : [8] Kamal, D. ad Al, B. (5). H. Robut lear regreo ug l-ealzed mm-etmato for hgh dmeoal data. Amerca Joural of Theoretcal ad Aled Stattc, 4(3): [9] Khademzadeh, A., Khademzadeh, A., D, P. C. P., D, M. S. P., ad Aagotooulo, G. C. (3). Large-cale o-lear regreo wth the mareduce framework. [] Lm, C. (5). Robut rdge regreo etmator for olear model wth alcato to hgh throughut creeg aay data. Stattc medce. [] Ohlo, H. (). Regularzato for aree ad moothe alcato ytem detcato ad gal roceg. Lkog Uverty Electroc Pre. [] Park, H. (3). Robut regreo modellg va l tye regularzato. Det. of Mathematc Chuo Uverty. [3] Ramalho, E. ad Ramalho, J. (4). Momet-baed etmato of olear regreo model wth boudary outcome ad edogeety, wth alcato to oegatve ad fractoal reoe. CEFAGE-UE Workg Paer 4/9. [4] Sma, D. (Ar. 6). Regularzato techque modelg ad arameter etmato. PhD the. [5] Tabataba, M. A., Kegwoug-Keumo, J. J., Eby, W. M., Bae, S., ad Mae, U. (4). A ew robut method for olear regreo. J Bomet Botat, 5. [6] Tateh, S., Matu, H., ad Koh, S. (9). Nolear regreo va the lao-tye regularzato. Joural of tattcal lag ad ferece. [7] Tkhoov, A. N. (943). O the tablty of vere roblem. Dokl. Akad. Nauk SSSR, 39: [8] Zou, H. ad Hate, T. (5). Regularzato ad varable electo va the elatc et. Joural of the Royal Stattcal Socety, B67: 3-3. [9] Zucker, D. M., Gorfe, M., L, Y., Tadee, M., & Segelma, D. (3). A Regularzato Corrected Score Method for Nolear Regreo Model wth Covarate Error. Bometrc, 69(), 8 9.