Monte Carlo Study of Some Classification-Based Ridge Parameter Estimators

Transcription

1 Journal of Modern Aled Statstcal Volume 6 Issue Artcle Monte Carlo Study of Some Classfcaton-Based Rdge Parameter Estmators Adewale Folaranm Lukman Ladoke Akntola Unversty of Technology, wale3005@yahoo.com Kayode Aynde Ladoke Akntola Unversty of Technology, kaynde@lautech.edu.ng Adegoke S. Ajboye Federal Unversty of Technology Follow ths and addtonal works at: htt://dgtalcommons.wayne.edu/jmasm Part of the Aled Statstcs Commons, Socal and Behavoral Scences Commons, and the Statstcal Theory Commons Recommended Ctaton Lukman, A. F., Aynde, K., & Ajboye, A. S. (07). Monte Carlo study of some classfcaton-based rdge arameter estmators. Journal of Modern Aled Statstcal, 6(), do: 0.37/jmasm/ Ths Regular Artcle s brought to you for free and oen access by the Oen Access Journals at gtalcommons@waynestate. It has been acceted for ncluson n Journal of Modern Aled Statstcal by an authorzed edtor of gtalcommons@waynestate.

2 Journal of Modern Aled Statstcal May 07, Vol. 6, No., do: 0.37/jmasm/ Coyrght 07 JMASM, Inc. ISSN Monte Carlo Study of Some Classfcaton- Based Rdge Parameter Estmators A. F. Lukman Ladoke Akntola Unv. of Technology Ogbomosho, Ngera K. Aynde Ladoke Akntola Unv. of Technology Ogbomosho, Ngera A. S. Ajboye Federal Unversty of Technology Akure, Ngera Rdge estmator n lnear regresson model requres a rdge arameter, K, of whch many have been roosed. In ths study, estmators based on orugade (04) and Adnan et al. (04) were classfed nto dfferent forms and varous tyes usng the dea of Lukman and Aynde (05). Some new rdge estmators were roosed. Results shows that the roosed estmators based on Adnan et al. (04) erform generally better than the exstng ones. Keywords: error lnear regresson model, multcollnearty, rdge estmator, mean square Introducton The arameter estmates obtaned through the use of the Ordnary Least Squares (OLS) estmator have otmal erformance when there s no volaton of any of the assumtons of the classcal lnear regresson model. One of the most basc of these assumtons s that exlanatory varables are ndeendent. Multcollnearty refers to the resence of strong or erfect lnear relatonshs among the exlanatory varables. Multcollnearty s an nherent henomenon n most economc relatonshs due to the nature of economc magntude (Koutsoyanns, 003). When there s a erfect relatonsh among the exlanatory varables, the regresson coeffcents of the OLS estmator are ndetermnate, and the standard error of the estmates becomes very large. Also, when there are strong relatonshs among the exlanatory varables, the regresson estmates are determnate but ossesses large standard error (Koutsoyanns, 003). Adewale Folaranm Lukman s a teachng assstant n the eartment of Statstcs. Emal at wale3005@yahoo.com. Prof. Kayode Aynde s a lecturer n the eartment of Statstcs. Emal at kaynde@lautech.edu.ng. r. Ajboye S. Adegoke s a lecturer n the eartment of Statstcs. 48

3 LUKMAN & AYINE Generally, the erformance of OLS estmator s unsatsfactory when there s multcollnearty (Koutsoyanns, 003). Several technques have been suggested n the lterature to handle ths roblem. Massy (965) ntroduced the rncal comonent regresson to elmnate the model nstablty and reduce the varances of the regresson coeffcents. Wold (966) develoed the artal least square to deal wth the roblem of multcollnearty. Hoerl and Kennard (970) roosed the rdge estmator for dealng wth multcollnearty n a regresson model, whch modfes the OLS to allow based estmaton of the regresson coeffcents. Ths study s lmted to the alcaton of the rdge regresson estmator n handlng the roblem of multcollnearty. Rdge estmator s defned as: KI R X X X Y () where K s a non-negatve constant known as rdge arameter and I denotes an dentty matrx. When K equals zero, () returns to OLS estmator; ths s defned as follows: OLS X X X Y () The corresondng mean square error (MSE) of () and () are defned resectvely as: MSE R K (3) MSE K K OLS (4) where λ, λ,, λ are the egenvalues of X'X, s the estmator of the rdge arameter K and s the th element of the vector. Although ths estmator s based, t gves a smaller mean squared error when comared to the OLS estmator for a ostve value of K (Hoerl and Kennard, 970). The use of the estmator deends largely on the rdge arameter, K. Several methods for estmatng ths rdge arameter have been roosed by dfferent authors, as follows: Hoerl and Kennard (970); Mconald and 49

4 SOME CLASSIFICATION-BASE RIGE PARAMETERS Galarneau (975); Lawless and Wang (976); Hockng et al. (976); Wchern and Churchll (978); Gbbons (98); Nordberg (98); Kbra (003), Khalaf and Shukur (005), Alkhams et al. (006), Munz and Kbra (009), Mansson et al. (00), orugade (04) and recently, Lukman and Aynde (05). The urose of ths study s to classfy the rdge arameters roosed by orugade (04) and Adnan et al. (04) nto dfferent forms and varous tyes. A smulaton study s conducted and the erformances of the estmators s examned va mean square error (MSE). Model and Estmators A lnear regresson model can be exressed n matrx form as: Y X + U (5) where X s an n matrx wth full rank, Y s a n vector of deendent varable, β s a vector of unknown arameters, and U s the error term such that E(U) = 0 and E(UU') = σ I n. The Ordnary Least Square (OLS) estmator of β s defned n (): Model (5) can be wrtten n canoncal form. Suose there exsts an orthogonal matrx Q such that X'QX = Ʌ, where Ʌ = dag(λ, λ,, λ ) and λ, λ,, λ are the egenvalues of X'X. Substtutng α = Q'β, model (5) can be wrtten as: Y Z + U (6) where Z'Z = Ʌ. Therefore, the rdge estmator of α can be defned as: KI R Z Z ZY (7) The corresondng mean square error (MSE) s defned as: MSE K R (8) K K 430

5 LUKMAN & AYINE where s the th element of the vector α = Q'β. Hoerl and Kennard (970) defned the value of the rdge arameter K that mnmzes the mean square error as: n e, where (9) n Hoerl and Kennard (970) roosed. HK They suggested estmatng rdge arameter by takng the maxmum (Fxed Maxmum) of such that the estmator of K s: k (0) FM HK max Hoerl et al. (975) roosed a dfferent estmator of K by takng the Harmonc Mean of the rdge arameter K HK. Ths estmator s gven as: HM HK () Kbra (003) roosed some new estmators of K by takng the geometrc mean, arthmetc mean and medan ( 3) of the rdge arameter K HK. These estmators are resectvely defned as: GM HK () AM HK (3) 43

6 SOME CLASSIFICATION-BASE RIGE PARAMETERS M HK Medan (4) Furthermore, Munz and Kbra (009) roosed some estmators of K n the form of the square root of the geometrc mean of K HK and ts recrocal, the medan of the square root of K HK and ts recrocal, and varyng maxmum of the square root of K HK and ts recrocal. These estmators are resectvely defned as: GMSR HK (5) GMRSR HK (6) MSR HK Medan (7) MRSR HK Medan (8) VMSR HK max (9) VMRSR HK max (0) 43

7 LUKMAN & AYINE orugade (04) suggested the modfcaton of the generalzed rdge arameter n (9) by multlyng the denomnator wth λ max /. The estmator s defned as: k () max where λ max s the maxmum egenvalue of X'X. Followng Kbra (003), orugade (04) suggested the followng ordnary rdge regresson for the rdge arameter n (). HM max () M Medan max (3) GM max (4) (5) AM HK max Followng orugade (04), Adnan et al. (04) roosed some rdge arameters: HM N 5 max (6) HM N max (7) 433

8 SOME CLASSIFICATION-BASE RIGE PARAMETERS HM N 3 4 (8) HM N 4 (9) The roosed rdge estmators by orugade (04) and Adnan et al. (04) are classfed nto dfferent forms and varous tyes. Rdge Parameter Proosed by orugade (04) orugade (04) roosed the rdge arameter. max Its estmators n the lght of dfferent forms and varous tyes are summarzed n Table. Table. Summary of fferent Forms and Varous Tyes for K / max Forms Fxed Maxmum Varyng Maxmum Arthmetc Mean Harmonc Mean Geometrc Mean Medan Tyes of K Orgnal Recrocal Square Root Recrocal Square Root FMO FMR HK max max FMO FMRSR K HK K FMO VMO max max VMR max VMSR K max K AMO * R max AMO K AMSR AMO K HMO * HMR max HMO K HMSR HMO K GMO max * MO MR medan max * GMR GMO K GMSR GMO K medan Notes: * orugade (04); all others are roosed estmators VMRSR MSR K medan K MRSR max AMRSR HMRSR GMRSR medan AMO HMO GMO 434

9 LUKMAN & AYINE Rdge Parameter Proosed by Adnan et al. (04) Adnan et al. (04) roosed the rdge arameter. Its estmators n the lght of dfferent forms and varous tyes are summarzed n Table. Table. Summary of fferent Forms and Varous Tyes for Forms Fxed K / = Tyes of K Orgnal Recrocal Square Root Recrocal Square Root FMO FMR FMO max FMRSR K K FMO Maxmum Varyng Maxmum Arthmetc Mean Harmonc Mean Geometrc VMO max AMO HMO * GMO Mean Medan MO medan VMR max VMSR K max K AMR AMSR AMO AMO K K HMR HMO K HMSR HMO K GMR GMO K GMSR GMO K MR medan Notes: * Adnan et al. (04); all others are roosed estmators MSR K medan K VMRSR MRSR max AMRSR HMRSR GMRSR medan AMO HMO GMO The rdge arameter estmators n Table and were examned and evaluated n ths study. Monte Carlo Smulaton The consdered regresson model s of the form: Y X X B X U (30) t 0 t 435

10 SOME CLASSIFICATION-BASE RIGE PARAMETERS where t =,,, n; = 3, 7. The error term U t was generated to be normally dstrbuted wth mean zero and varance σ, U t ~ N(0, σ ). In ths study, σ were taken to be 0.5, and 5. β 0 was taken to be dentcally zero. When = 3, the values of β were chosen to be β = (0.8, 0., 0.6)'. When = 7, the values of β were chosen to be β = (0.4, 0., 0.6, 0., 0.5, 0.3, 0.53)'. The arameter values were chosen such that β'β = whch s a common restrcton n smulaton studes of ths tye (Munz and Kbra, 009). We vared the samle szes between 0, 0, 30, 40 and 50. Followng Mconald and Galarneau (975), Wchern and Churchll (978), Gbbons (98), Kbra (003), Munz and Kbra (009), Lukman and Aynde (05), the exlanatory varables were generated usng the followng equaton: X Z Z,,,3,..., n, j,,...,. (3) j j where Z j s ndeendent standard normal dstrbuton wth mean zero and unt varance, ρ s the correlaton between any two exlanatory varables and s the number of exlanatory varables. The number of exlanatory varable () s taken to be three (3) and seven (7). The value of ρ s taken as 0.95, 0.99 resectvely. Three dfferent values of σ, 0.5, and 5, were also used. The exerment s relcated,000 tmes. The rdge arameter estmators are evaluated usng mean square error (MSE). Results The results of the smulaton are resented n Table 3 and 4. These tables rovde the results of the estmated mean square error of the rdge arameter when the number of regressors s three (3) and seven (7) resectvely. The mean square error ncreases as the multcollnearty level ncreases. Across each multcollnearty level, the mean square error decreases as the samle szes ncrease from 0 to 50, whle ncreasng the number of regressors ncreases the estmated MSE. However, t s observed that the rdge estmators based on K erformed consstently better than. Occasonally, ths method erforms better than K. For nstance, estmators and VMSR erform consstently well AMSR over estmators based on K esecally when the number of regressors ncreases to seven (7), and when the number of regressors s three (3), esecally when n 0. Ths can be seen n Fgure and. The followng rdge arameter 436

11 LUKMAN & AYINE FMSR HMO FMO estmators based on K :,,, erformed best when comared to others. All but GMSR HMSR, GMO MO,, and are roosed n FMSR ths study. When = 3, erforms better than the exstng rdge arameter HMO HMO FMSR, whle erforms better than when = 7. The estmators consdered best n ths study have the least MSE when comared to others. The roosed estmators erform better than the exstng estmators based on. HMO Fgure. Grahcal Illustraton when n = 0, σ = 0.5, = 3 Fgure. Grahcal Illustraton when n = 50, σ = 0.5, = 7 437

12 SOME CLASSIFICATION-BASE RIGE PARAMETERS Table 3. Estmated Mean Square Error of rdge arameter when = 3 K = 3, σ = 0.5, ρ = 0.95 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 438

13 LUKMAN & AYINE Table 3, contnued. K = 3, σ = 0.5, ρ = 0.99 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 439

14 SOME CLASSIFICATION-BASE RIGE PARAMETERS Table 3, contnued. K = 3, σ =, ρ = 0.95 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 440

15 LUKMAN & AYINE Table 3, contnued. K = 3, σ =, ρ = 0.99 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 44

16 SOME CLASSIFICATION-BASE RIGE PARAMETERS Table 3, contnued. K = 3, σ = 5, ρ = 0.95 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 44

17 LUKMAN & AYINE Table 3, contnued. K = 3, σ = 5, ρ = 0.99 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 443

18 SOME CLASSIFICATION-BASE RIGE PARAMETERS Table 4. Estmated Mean Square Error of rdge arameter when = 7 K = 7, σ = 0.5, ρ = 0.95 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 444

19 LUKMAN & AYINE Table 4, contnued. K = 7, σ = 0.5, ρ = 0.99 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 445

20 SOME CLASSIFICATION-BASE RIGE PARAMETERS Table 4, contnued. K = 7, σ =, ρ = 0.95 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 446

21 LUKMAN & AYINE Table 4, contnued. K = 7, σ =, ρ = 0.99 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 447

22 SOME CLASSIFICATION-BASE RIGE PARAMETERS Table 4, contnued. K = 7, σ = 5, ρ = 0.95 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K 448

23 LUKMAN & AYINE Table 4, contnued. K = 7, σ = 5, ρ = 0.99 n = 0 n = 0 n = 30 n = 40 n = 50 FMO FMR FMSR FMRSR VMO VMR VMSR VMRSR AMO AMR AMSR AMRSR HMO HMR HMSR HMRSR GMO GMR GMSR GMRSR MO MR MSR MRSR K Concluson In ths study, rdge arameters roosed by orugade (04) and Adnan et al. (04) are classfed nto dfferent forms and varous tyes followng the dea of Lukman and Aynde (05), and some new rdge arameters are roosed. The erformances of these estmators are evaluated through Monte Carlo Smulaton, where levels of multcollnearty, samle szes, number of regressors and error varances have been vared. The erformance evaluaton was done usng the mean square error. The roosed estmators generally have the least mnmum square error when comared to others. References Adnan, K., Yasn, A. & Asr, G. (04). Some new modfcatons of Kbra s and orugade s methods: An alcaton to Turksh GP data. Journal 449

24 SOME CLASSIFICATION-BASE RIGE PARAMETERS of the Assocaton of Arab Unverstes for Basc and Aled Scences. 0, do: 0.06/j.jaubas Alkhams, M., Khalaf, G. & Shukur, G. (006). Some modfcatons for choosng rdge arameters. Communcatons n Statstcs- Theory and, 35(), do: 0.080/ Alkhams, M. & Shukur, G. (007). A Monte Carlo study of recent rdge arameters. Communcatons n Statstcs- Smulaton and Comutaton, 36(3), do: 0.080/ orugade, A. V. (04). New rdge arameters for rdge regresson. Journal of the Assocaton of Arab Unverstes for Basc and Aled Scences, 5, do: 0.06/j.jaubas Gbbons,. G. (98). A smulaton study of some rdge estmators. Journal of the Amercan Statstcal Assocaton, 76(373), do: 0.080/ Hockng, R., Seed, F. M. & Lynn, M. J. (976). A class of based estmators n lnear regresson. Technometrcs, 8(4), do: 0.080/ Hoerl, A. E. & Kennard, R. W. (970). Rdge regresson: based estmaton for non-orthogonal roblems. Technometrcs, (), do: 0.080/ Hoerl, A. E., Kennard, R. W. and Baldwn, K. F. (975). Rdge regresson: Some smulaton. Communcatons n Statstcs Smulaton and Comutaton, 4(), do: 0.080/ Khalaf, G. & Shukur, G. (005). Choosng rdge arameters for regresson roblems. Communcatons n Statstcs- Theory and, 34(5), do: 0.08/sta Kbra, B. M. G. (003). Performance of some new rdge regresson estmators. Communcatons n Statstcs - Smulaton and Comutaton, 3(), do: 0.08/sac Koutsoyanns, A. (003). Theory of Econometrcs ( nd Ed). Basngstoke, UK: Palgrave. Lawless, J. F. & Wang, P. (976). A smulaton study of rdge and other regresson estmators. Communcatons n Statstcs - Theory and, 5(4), do: 0.080/

25 LUKMAN & AYINE Lukman, A. F. & Aynde, K. (05). Revew and classfcaton of the Rdge Parameter Estmaton Technques. Hacettee Journal of Mathematcs and Statstcs, 46(3), -. do: 0.567/hjms Mansson, K., Shukur, G. & Kbra, B. M. G. (00). A smulaton study of some rdge regresson estmators under dfferent dstrbutonal assumtons. Communcatons n Statstcs-Smulatons and Comutatons, 39(8), do: 0.080/ Massy, W. F. (965). Prncal Comonents Regresson n exloratory statstcal research. Journal of the Amercan Statstcal Assocaton, 60(309), do: 0.080/ Mconald, G. C. & Galarneau,. I. (975). A Monte Carlo evaluaton of some rdge-tye estmators. Journal of the Amercan Statstcal Assocaton, 70(350), do: 0.080/ Munz, G. & Kbra, B. M. G. (009). On some rdge regresson estmators: An emrcal comarson. Communcatons n Statstcs-Smulaton and Comutaton, 38(3), do: 0.080/ Munz, G., Kbra, B. M. G., Mansson, K. & Shukur, G. (0). On evelong Rdge Regresson Parameters: A Grahcal Investgaton. SORT. 36(), Nordberg, L. (98). A rocedure for determnaton of a good rdge arameter n lnear regresson. Communcatons n Statstcs - Smulaton and Comutaton, (3), do: 0.080/ Wchern,. & Churchll, G. (978). A comarson of rdge estmators. Technometrcs, 0(3), do: 0.080/ Wold, H. (966). Estmaton of rncal comonents and related models by teratve least squares. In P.R. Krshnaaah (Ed.). Multvarate Analyss. (.39-40) New York: Academc Press. 45