Inernaonal Mahemacal Forum, Vol. 8, 3, no., 7 - HIKARI Ld, www.m-hkar.com hp://dx.do.org/.988/mf.3.3488 New M-Esmaor Objecve Funcon n Smulaneous Equaons Model (A Comparave Sudy) Ahmed H. Youssef Professor of Appled Sascs and Economercs Insue of Sascal Sudes and Research, Caro Unversy, Egyp Ahmed Amn El-Shekh Assocaed Professor of Appled Sascs and Economercs Insue of Sascal Sudes and Research, Caro Unversy, Egyp Eman Taha Hassan Mohammed Maser suden of Appled Sascs and Economercs, Insue of Sascal Sudes and Research, Caro Unversy, Egyp Copyrgh 3 Ahmed H. Youssef e al. Ths s an open access arcle dsrbued under he Creave Commons Arbuon Lcense, whch perms unresrced use, dsrbuon, and reproducon n any medum, provded he orgnal work s properly ced. Absrac In hs paper, a suggesed M-esmaor (S-M) objecve funcon wll be nroduced. In addon, A comparave sudy among dfferen wo-sage M- esmaon funcons ncludng S-M and wo sage leas squares (SLS) has been nroduced usng Mone Carlo Smulaons o nvesgae he properes of hese funcons, when he errors erm of he reduced form follows heavy al dsrbuons. Keywords: Two-sage M- esmaon, smulaneous equaons model, wo sage leas squares (SLS), robus esmaon, heavy al dsrbuon, Huber M- esmaor, Tukey M-esmaor
8 Ahmed H. Youssef e al. -Inroducon Leas squares (LS) esmaes when he errors are heavy-aled dsrbued has ceran dsadvanages ha LS has low breakdown pon, whch ends o zero as he sample sze ncreases. Also, LS esmaors may no be effcen or asympocally effcen whch mean ha LS esmaes wll no be BLUE. One reamen s o remove nfluenal observaons from he leas squares, whch s called dagnosc echnque. Robus regresson analyss provdes an alernave approach o he LS regresson model when he fundamenal assumpons are unfulflled by he naure of he daa. M-esmaon mehod s one of he mos common mehod of robus regresson. Huber (964) has nroduced as a new approach owards a heory of robus esmaon; He reaed he asympoc heory of esmang a locaon parameer for conamnaed normal dsrbuons. Also, n (973) he nroduced M- esmaor of regresson. Km and Muller (7) have nroduced wo sage Huber esmaor (SH) o ge a smple, a robus and an effcen esmaor n he case of random regressors and asymmerc errors sage. They concluded ha SH esmaor has smaller sandard errors han (SLS) and wo sage devaons (SLAD). Smulaneous equaons model can be consdered as a ool for nvesgang he nerdependence beween varables ha s dffcul o explan by sngle-equaon models. The mehod of wo sage leas squares (SLS) can be consdered he mos common mehod used for esmang smulaneous equaons models whch based on he leas squares regresson esmaon. Greene (8) Ths paper ams o nroduced a suggesed M- esmaor objecve funcon, apply o esmae smulaneous wo equaons model, and sudy he properes of hs funcon compared o dfferen M- esmaor objecve funcons SH and Tukey M-esmaor. The effecve of vercal oulers n endogenous varable (heavy al errors) on properes of esmaors wll be nvesgaed usng Mone Carlo smulaons whch are execued o evaluae he performance of dfferen robus M-esmaors. - M- Esmaor M-esmaor s consdered as he nex sep n he drecon of he robus esmaon, where Huber (964) has nroduced as a new approach owards a heory of robus esmaon. In (973), he exended he dea of M-esmaor for regresson by mnmzng a symmerc smooh funcon of he resduals over he parameer esmaes, whch can presened as follows: n n n n ˆ ˆ y x β ρ( r) ρ( y x = β) or ρ( r) = ρ, () = = = = ˆ σ
New M-esmaor objecve funcon 9 Where, ˆ σ he esmaed scale of resduals, ρ s a symmerc, posve defne funcon wh a unque mnmum a zero, and s chosen o be less ncreasng han square. Le, β = ( β,... β p ) s a vecor of regresson model parameers ncludng he nercep erm, The M-esmaor of β based on he funcon ρ ( r ) s he vecor β whch s he soluon of he followng p equaons; n r ψ ( r ) =, =,,..., n, j =,,..., p, () = β j Where, ψ ( r ) s called Ps funcon. Also, s called he nfluence funcon, whch measures he nfluence of a daum on he value of he parameer esmae. The M- esmaor for β, based on he funcon ρ and he daa s he value βˆ n whch mnmzes ψ ( y x β ), βˆ s deermned by solvng he se of p = smulaneous equaons as n ψ ( y x β ) x =, = ψ ( r ) By defnng he weghed funcon w ( r) =, and le, w = w ( r), hence he p r smulaneous equaons n becomes: n ( y x β ) w x =, (4) = Equaon (4) can be combned no he followng sngle marx equaon X WXβ = X Wy And hence = ( X WX β ) X Wy, (5) Equaon (5) can be solved as weghed leas squares (WLS). Bhar (). I s noed ha weghs depend upon he resduals, he resduals depend upon he esmaed coeffcens, and a he same me he esmaed coeffcens depend upon he weghs, so an erave soluon, whch s called eravely reweghed leas-squares ( IRLS) s requred n hs case as:. Selec nal esmaes such as he leas-squares esmaes. ( ). A each eraon, calculae resduals r and assocaed weghs ( ) ( ) w = w r from he prevous eraon. 3. Solve WLS esmaes as: ( ) ( ) ˆ ( ) β = X W X X W y, ( ) where X s he marx of he model, and W ( ) = dag { w } s he curren weghed marx.
Ahmed H. Youssef e al. 4. Seps. and 3 are repeaed unl he esmaed coeffcens converge. Fox ( ) () connued unl consecuve esmaes ˆ β ˆ( ) and β are suffcenly close o one anoher. 3- The SLS Esmaon Consder he srucural equaon model as y = Y γ + x β + u, =,,... T, (6) where y s he dependen varable, Y s a vecor (G ) of endogenous varables, x s a vecor ( K ) of exogenous random varables and u s he error erm. Km and Muller (7) have neresed n esmang he srucural parameers α = ( γ, β ), hey denoed x he vecor ( K ) of exogenous random varables ha are absen from equaon (6). Assumng ha, Y = ( Y, Y,... YG ), hen he reduced form can be represened for each Y j as j =,... G, Y j = x Π j + V j, (7) =,,... T Where x = ( x, x ), Π j s a vecor ( K ) of unknown parameers wh K = K+ K, and V j s he error erm of he reduced form. The SLS esmaor s obaned basng on LS regresson of y on Y and x. Thus, he SLS name arses from he wo regressons n he procedure:. Sage. Oban he leas squares predcons from regresson of Y on x.. Sage. Esmae α by leas squares regresson of y on Y ˆ and x. Green (8) 4-Two-Sage Huber Esmaon The frs sage of M- esmaon ha yelds Π as esmae of Π j n (7) would be obaned as he soluon of T ρ( Y j x Π j ) (8) = The second sage of Huber esmaon ha yelds ˆα as esmae of α = ( γ, β ) n (6) would be obaned as he soluon of T mn ρ( y zˆ α) (9) = where zˆ ˆ = Y j, x. Huber funcon has o be modfed usng some scale esmaor of he error erm so, Flavn (999), has been used a sandardzed MAD o oban he requred scale esmaors as follows:
New M-esmaor objecve funcon - ˆ σ =.483MAD,where MAD = medan{ r medan{ r }} for equaon (8) j - ˆ σ =.483MAD, where MAD = medan{ r medan{ r}} for equaon (9) where r j are he resduals obaned from he LS regresson of Y j on x and r are he resduals from he LS regresson of y on z ˆ. a- Huber Objecve Funcon Huber M- esmaor ha we consdered s represened as r r for ˆ c σ ˆ σ ρ ( ˆ H r σ) = =,..., n () r r c c for > c ˆ σ ˆ σ where ρ a smooh and symmerc funcon, c s a unng value, we use c = H b- Huber Wegh Funcon The Huber wegh funcon of he resduals for equaon () accordng o Flavn (999) s defned as r for c ˆ σ w H ( r) = =,,..., n () c ˆ σ r for > c r ˆ σ where r s a general concep for resduals wheher n he frs sage or n he second sage n SLS. j j 5- Tukey M- Esmaor In addon o Huber M- esmaor, Tukey M- esmaor s consdered as one of he mos famous M- esmaor. Tukey objecve funcon s defned as 3 k r for r k, 6 ρ ( ) k = T r =,..., n () k for r > k, 6 where k s a unng value, k = 4.685, and he Tukey wegh funcon wt ( r ) s defned as:
Ahmed H. Youssef e al. r for r k wt ( r ) =, =,..., k n for r > k where w ( r ) denoes Tukey resduals wegh funcon. Fox () T 6- Suggesed M - Esmaor (S-M) In hs secon, suggesed M objecve funcon (S-M) has been nroduced, whch combned beween LS objecve funcon and anoher logsc funcon ha approaches he dea of Cauchy funcon ha has he form, c r ρch = log + ( ), =,..., n (4) c Where c s a unng value, he 95% asympoc effcency on he sandard normal dsrbuon s obaned wh he unng consan c=.3849. Bhar (). Accordng o LS, and Cauchy funcon n (4), he suggesed funcon can be defned as r f r < k ρs M ( r) = =,,..., n (5) ln( r + ) f r k where k s a unng value and ρ S- M( r ) sasfes he same properes of ρ above. The frs dervave of (5), whch s called nfluence funcon, r f r < k ψ S M ( r) = r =,,..., n (6) f r k ( r + ) ψ S M ( r) Snce, weghed funcon w S M s defned as so, r f r < k w S M ( r) = =,,..., n (7) f r k r + We graph he ρ-funcon, Ψ-funcon and he wegh funcon n Fg..
New M-esmaor objecve funcon 3 4 3.5 3.5.5.5 6 5 4 3 - - -3-4 -5-6 Resduals 3 - - -3 6 5 4 3 - - -3-4 -5-6 Resduals (A) (B)..8.6.4. 6 5 4 3 - - -3-4 -5-6 Resduals (C) Fgure : Graph of S-M funcon: (A) Objecve funcon (B) Ψ-funcon (C) Wegh funcon. To deermne he value of k whch makes he esmaor more effcen, a smulaon sudy has been execued by consderng he wegh funcon n equaon (7) a dfferen values of k (,.5,,.5, and 3) for dfferen samples szes (5, 3, 5, ) when he errors follow Ch-square dsrbuon wh degrees of freedom, (Chsq (n,)) as a heavy aled dsrbuon. The resuls of he smulaon are shown n ables () and (), whch dsplay he properes of he S-M funcon a dfferen values of k. These ables show he esmaes of he hree parameers (γ, β, and β ), bas and mean square errors \
4 Ahmed H. Youssef e al. (MSE) compared wh SLS. I can be seen from ables () and () when k = 3 he funcon has leas MSE for mos dfferen sample szed excep for n = 3 bu a he same me has he less bas. So, we wll use S-M wh k = 3 (S-M ) n our comparson o nvesgae s performance compared wh oher M- esmaor funcons based on Bas and MSE. Table (): Esmaed parameers, Bas and MSE: (n = 5, 3) n=5 n=3 S-M- Esmaor Crera SLS k= k=.5 k= k=.5 k=3 ˆ γ.4.49.49.499.497.56 Bas.54.8...3.6 MSE.36.49.47.39..3 7.9.58.647.634.644.563 Bas 6.9.58.647.634.644.563 MSE 44.8 64.367 64.6 64.63 63.39 63.336 ˆβ.496.88..77.74.7 Bas.96...3.6.3 MSE.99.35.3.367.34.35 ˆ γ.99.468.467.449.466.48 Bas..3.33.5.34.8 MSE.4.66.68.687.689.646 4.53.659.683.838.74.573 Bas 3.53.659.683.838.74.573 MSE 6.8 35.73 37.68 38.934 38.489 35.695 ˆβ.4.56.59.63.45.55 Bas..56.59.63.45.55 MSE.6.764.794.788.8.736
New M-esmaor objecve funcon 5 Table (): Esmaed parameers, Bas and MSE: (n = 5,) n=5 n= Crera SLS S-M- Esmaor k= k=.5 k= k=.5 k=3 ˆ γ.8.484.484.473.474.467 Bas.3.6.6.7.6.33 MSE 4.69.83.9.36.3.78-7.78.57.576.655.6.679 Bas 8.78.57.576.655.6.679 MSE 54.53 7.83 8.43.45.93 8.47 ˆβ.649.46.4.3.3.4 Bas.449.54.6.68.68.58 MSE.76.4.394.49.48.369 ˆ γ.55.5.499.49.57.54 Bas.5...8.7 -.4 MSE.78.94.7.9.3.73 3.73.397.43.44.3.349 Bas.73.397.43.44.3.349 MSE 5.83.644.435 3.73.3.35 ˆβ.3.8.6..97.4 Bas.7.8.6..3.4 MSE.57.55.65.78.69.4 7. Smulaon Sudy for comparng Dfferen M- esmaors The Smulaon Sudy was desgned o nvesgae he properes of some M-esmaors compared wh SLS, when here are oulers n exogenous varables (vercal oulers). Oulers n exogenous varables were refleced as a heavy aled errors dsrbuon. a. Smulaons Model The Smulaon Sudy was execued consderng wo smulaneous equaons:
6 Ahmed H. Youssef e al. y = Y γ + (, x ) β + u =.5Y + +.x + u (8) where γ = B =.5, β = ( β, β) = ( Γ, Γ ) = (,.), x s he second elemen of x, and u s he frs elemen of U and he second equaon s, Y =.7y + +.4x +.5x 3 + ε (9) where x and x 3 are, respecvely, he hrd and he fourh elemens of x, and ε s he second elemen of U.The compac form for he model of equaons (8) and (9) can be represened as: y B +Γ x = U Y () By consderng he model (), he reduced form of equaons (8) and (9) was conduced as: [ y Y ] = X [ π Π ] + [ υ V ] () Where, [ π Π ] = Γ ( B ) and [ υ V ] = U( B ). b. Smulaon Frame Work Smulaons for wo smulaneous equaons model as Km and Muller (7) have been execued by consderng he equaons (8) o () as follows: -[ υ V ] n equaon () were drown from he sandard normal N (, ), he Lognormal wh log-mean = and log-sandard devaon =, denoed by LN (, ), he Suden- wh 4 degrees of freedom, denoed by (4), and Chsq (). - The second o he fourh columns n X are drawn from he normal dsrbuon wh mean (.5,,.), varances equal o. X s fxed a each replcaon. 3-[ y Y ] were generaed usng he reduced form parameers basng on he generaed exogenous varables X, and he generaed errors as equaon (). 4- replcaons were used for dfferen sample szes 5, 3, 5, and. For each replcaon, he value of he parameers ( γ, β, β ) has been esmaed. 5- The followng mehods have been compared: a- Two Sage Leas Square (SLS) b-two sage Huber M- esmaor wh a unng value =, whch s denoed as SH(). c- Two sage Tukey M-esmaors wh a unng value =4.685, ST(4.685). d- Suggesed M-esmaor wh unng value =3, S-M. 6-The comparson among hese M- esmaors based on Bas, MSE, and relave Var of SLS effcency (RE) ha s defned as Var of he robus M - esmaor 7- Esmang he parameer usng M- esmaors based on erave reweghed leas square (IRLS), whch depends on wegh funcons as n equaons (),, and (7).
New M-esmaor objecve funcon 7 c. The Smulaon Resuls - Normal Errors: In able, when n =5, s noed ha SH () and S-M gave he bes resuls respecvely whle a n = 3, SLS resuls are beer han he oher mehods of esmaon. In able (4), we can conclude generally, ha SLS s he mos effcen esmaor a large samples szes, and s bas decreases as sample sze ncreases, where has he leas MES a large sample szes. The esmaed values of robus mehods converges each oher. - Log Normal Errors In able, when n= 5, SLS and SH () gave he bes resuls compared wh he oher wo mehods. When n= 3, can be seen generally ha ST (4.685) has a beer resuls among he oher mehods of esmaon. In Table (4), we can see ha he behavor of S-M and ST(4.685) s he bes compared wh he oher wo mehods. 3-Suden- Errors In able (5), when n = 5 or 3, can be seen generally ha SH () has a beer resuls among he oher mehod of esmaon. In able (6), when n = 5, can be noed ha SH () and S-M gave he bes resuls respecvely whle a n=, SLS and SH () gave he bes resuls respecvely compared wh he oher wo mehods. 4- Ch- Square Errors In able (5), when n = 5, SLS s he mos effcen mehod compared o oher esmaors whle a n = 3, SLS and SH () gave he bes resuls respecvely compared wh he oher wo mehods. In able (6), when n = 5, SH () and S-M gave he bes resuls respecvely whle a n =, s noed ha SH () gave he bes resuls compared wh he oher mehods.
8 Ahmed H. Youssef e al. Table Esmaed parameers, Bas, MSE, and RE: N (, ), and LN (, ): (n = 5, 3) n Crera SLS SH() N (, ) LN (, ) ST (4.685) S-M SLS SH () ST (4.685) S-M ˆ γ.684.49.475.485.3.5.496.483 Bas.84..5.5.377..4.7 MSE.758.49.355.86.63.44.69.579 5 3 RE..99.43.533..74.9.9.58.3.64.39 5.77.79.9.34 Bas.47.3.64.39 4.77.79.9.34 MSE 5.488.64.9.387 4.759.88 5.55 4.8 RE..55.8.7..35.68.8.436..8..366.75.47.75 Bas.36..8..66.5.53.5 MSE.67.36.48.35.9.5.73.67 RE..69.44.6..98.45.478 ˆ γ.4.493.53.5.9.5.58.498 Bas.99.67.8.38..8. MSE.9.3.89.55.73.34.44.49 RE..65.43.56..333.3.33.56..996. 4.538.373.4.54 Bas.56..4. 3.538.373.4.54 MSE.797.99.687.49 9.3.49 9.587.68 RE..44.86.343..63.787.77.5.8..3.56.94.4.36 Bas.75.78..3.36.6.4.36 MSE.7.5.8.9.99.99.97.33 RE..3.86.98..66.65.58
New M-esmaor objecve funcon 9 Table (4) Esmaed parameers, Bas, MSE, and RE: N (, ), and LN (, ): (n = 5, ) Crera n N (, ) LN (, ) 5 SLS SH() ST (4.685) S-M SLS SH () ST (4.685) S-M ˆ γ.469.56.5.57.43.5.495.497 Bas.3.6..7.357.5.3 MSE.5.5.78.68.53..73.74 RE..96.68.7..55.356.35.3.99.6.986 5.4.436.43.93 Bas.3..6.4 4.4.436.43.93 MSE.5.544.847.747 9.7 7.4 4.7 4.369 RE..84.59.6..37.68.97 ˆβ.84.96.98..69.99.9.88 Bas.6.4..469..9. MSE.66.67.5.87.76.39.8.7 RE..985.69.759. 3.899 4.596 4.638 ˆ γ.437.56.53.57.78.54.53.59 Bas.63.6.3.7.8.4.3.9 MSE.37.44.63.57.586.73.49.47 RE..75.55.579. 7.37.936.38.3.983.958.976.466.399.3.8 Bas.3.7.4.4.466.399.3.8 MSE.38.45.646.577 4.4 5.46.75.99 RE..78.59.569. 6.897 4.9 3.749 ˆβ.64.3.5.7.3.97.99.99 Bas.36.3.5.7.3.3.. MSE.34.3.48.4.54.64.45.44 RE..3.688.787..46 3.4 3.5
Ahmed H. Youssef e al. Table (5) Esmaed parameers, Bas, MSE, and RE: (4), and Chsq (): (n = 5, 3) Crera n (4) Chsq () 5 3 SLS SH() ST (4.685) S-M SLS SH () ST (4.685) ˆ γ.7.477.454.46 -.4.49.5.56 Bas.48.3.6.39.54.8..6 MSE.48.368.596.444.36.8.443.3 S-M RE..64.397.53..7.6.77.98.5.8.78 7.9.788.83.563 Bas.98.5.8.78 6.9.788.83.563 MSE 4.84 3.3 4.785 3.653 44.8 48.59 77.745 63.336 RE..983.64.87..5.89. ˆβ.47..4.36.496.88.6.7 Bas.7..4.36.96..39.3 MSE.536.479.77.574.99.95.635.35 RE..965.6.87..99.55.635 ˆ γ.34.57.495.5.99.487.478.48 Bas.6.7.5...3..8 MSE.877.86.99.8.4.384.787.646 RE. 4.576.846 3.94..547.67.35.58.983.5.97 4.53.97.484.573 Bas.58.7.5.9 3.53.97.484.573 MSE 5.55.695.689.45 6.8 8 39.595 35.7 RE. 3.75.938.55..594.383.43 ˆβ.68....4.5.54.55 Bas.3...5.54.55 MSE.389.48.38.89.6.468.895.736 RE..565.6.343..5.64.78
New M-esmaor objecve funcon Table (6) esmaed parameers, Bas, MSE, and RE: (4), and Chsq (): (n = 5, ) Crera n (4) Chsq () SLS SH() ST (4.685) S-M SLS SH () ST (4.685) S-M ˆ γ.53.499.499.5.8.495.495.467 Bas.3...3.5.5.33 MSE.4.97.7.5 4.69.68.9.78 5 RE..78.976.8. 7.497.66.64.89.3.998-7.78.94.36.679 Bas.9.3. 8.78.94.36.679 MSE.45.48.4.7 54.5 5.9 6.77 8.47 RE..339.995.98. 6.43.554.999.47.4.98..649.79.74.4 Bas.7.4...449..6.58 MSE.4.35.8.5.76.3.43.369 RE..48.5.67..77.53.378 ˆ γ.4.49.496.496.55.49.499.54 Bas.99..4.4.5.9..4 MSE.67.7.96.88.78.9.77.73 RE..84.594.648..497.6.9.39.3.. 3.73.945.33.349 Bas.39.3...73.945.33.349 MSE.69.73.978.9 5.83.574 9.63.35 RE..86.595.645..44.43.38.49.98.4..3.4..4 Bas.5..4..7.4..4 MSE..55.75.64.57.9.9.4 RE..436.54.34..395.78.78
Ahmed H. Youssef e al. Conclusons For normal errors, he esmaed values of he compared M- esmaors for hree parameers converge. As sample sze ncreases he performance of SLS mproves so, s he mos effcen esmaor for large sample sze. Also, SH() and S-M are he mos effcen M- esmaors among oher M esmaors bu for Lognormal errors, he S-M s more effcen han SH () and SLS has he leas based for he nercep erm whle for Suden- errors, SH () s he mos effcen esmaor, followed by S-M and he bas of SLS decreases as sample sze ncreases and for Ch-square errors dsrbuon, SLS s he mos based esmaors compared wh dfferen M-esmaors. Also, MSE of compared M- esmaors decreases as sample szes ncrease. Generally, here s no one of he compared mehods can elmnae he ohers. However, he properes of each one dffer accordng o sample sze and he ype of errors dsrbuon. References - Bhar, L. Robus regresson, Advances n Daa Analycal Technques.(eds.) R. Parsad, VK Gupa, LM Bhar, VK Bhaa. Indan Agrculural Sascs Research Insue, (), III69-III78. - Brkes, D. and Dodge, Y. Alernave Mehods of Regresson. Wley Seres n Probables and Sascs (993). 3- Flavn, M., Robus esmaon of he jon consumpon/asse demand decson. NBER, Cambrdge, Workng Paper No. 7, (999). 4- Fox, J. Robus Regresson. hp://cran.r-projec.org (). 5- Green, W.H. Economerc Analyss. Pearsone Educaon, Inc(8) 6- Huber, P.J. Robus esmaon of a locaon parameer, The Annals of Mahemacal Sascs, 35 (964), 73-. 7- Huber, P.J. Robus regresson: Asympocs, conjecures and Mone Carlo, The Annals of Sascs, (5) (973), 799-8. 8- Km,TH and Muller,C. Two-sage Huber esmaon. Journal of Sascal Plannng and Inference, 37 (7), 45 48 Receved: Aprl 5, 3