PRE-TEST ESTIMATORS OF THE INTERCEPT FOR A REGRESSION MODEL WITH MULTIVARIATE STUDENT-t ERRORS

Size: px

Start display at page:

Download "PRE-TEST ESTIMATORS OF THE INTERCEPT FOR A REGRESSION MODEL WITH MULTIVARIATE STUDENT-t ERRORS"

Augustus Morris
5 years ago
Views:

1 PRE-TEST ESTIMATORS OF THE INTERCEPT FOR A REGRESSION MODEL WITH MULTIVARIATE STUDENT-t ERRORS Shahjaha Kha Departmet of Mathematics & Computig Uiversity of Souther Queeslad Toowoomba, Australia ad A.K.Md. Ehsaes Saleh Departmet of Mathematics & Statistics Carleto Uiversity Ottawa, Caada ABSTRACT I presece of a ucertai prior iformatio about the slope parameter, the estimatio of the itercept of a simple regressio model with a multivariate Studett error distributio is ivestigated. The urestricted, restricted ad prelimiary test maximum likelihood estimators are defied. The expressios for the bias ad the mea square error of the three estimators are provided ad the relative efficiecies are aalysed. A maximi criterio is established, ad graphs ad tables are costructed for differet umber of degrees of freedom D.F. as well as sample sizes. These tables of relative efficiecies ca be used to determie a proper choice of the sigificace level of the prelimiary test which i tur determies the choice of the estimator. Key Words ad Phrases: Regressio model, ucertai prior iformatio, prelimiary test estimator, multivariate t-distributio, iverted Gamma ad o-cetral F distributios, icomplete Beta distributio, urestricted ad restricted maximum likelihood estimators, mea square errors, relative efficiecy, maximi rule. AMS 1991 Subject Classificatio: Primary 6F30 ad Secodary 6J05. 1

2 1. INTRODUCTION Ofte i liear regressio aalysis, the distributio of the errors is assumed to be ormal, ad idepedet. I this paper a broader assumptio is made, amely the errors are assumed to follow a family of Studet-t distributios. Thus the joit distributio of the error compoets associated with the resposes is a multivariate Studet-t. The assumptio of a t-model violates two basic assumptios of the ormal model, amely, the ormal distributio ad the idepedece of the sample resposes as uder the t-model the sample resposes are ucorrelated but ot idepedet. The problem at had falls i the realm of the statistical iferece problem with ucertai prior iformatio. I the process of solvig this kid of problems the ucertai prior iformatio that appears i the form of a costrait is treated as the uisace parameter ad removed by usig Fisher s Recipe of testig it out. Such a problem was first addressed by Bacroft 1944 ad the resultig estimator has bee kow i the literature as the prelimiary test PT estimator. Sice the the PT estimator has bee studied by a host of authors, otably, Mosteller 1948, Kitagawa 1963, Ha ad Bacroft 1968, Saleh ad Se 1978, Saleh ad Se 1984, Saleh ad Ha 1990, ad more recetly, Saleh ad Kibria 1993 ad Wa 1994 to metio a few, uder the ormal theory. The icreasig criticism of the ormal theory with its ofte urealistic assumptios of idepedece ad ideticity as well as beig o-robust has led researchers to fid a better alterative amog the class of symmetrical distributios. I may cases, the theoretical advatages ad mathematical coveieces are egligible compared to the price paid i terms of loss of efficiecy ad precisio uder the ormal theory. The cocer was voiced by Fisher 1956, p.133 i the followig words, It is a oteworthy peculiarity of iductive iferece that comparatively slight differeces i the mathematical specificatio of a problem may have logically importat effects o the ifereces possible. Not surprisigly Fisher 1960, p.37 aalysed Darwi s data uder ormal theory ad later p.46 assumig a symmetrical distributio. Fraser ad Fick 1975 aalysed the same data usig a family of Studet-t distributios. Obviously, the family of t-distributios represets a spectrum of symmetric desities ragig from the ormal as the degrees of freedom parameter, ν 0, dow to the Cauchy whe ν 0 = 1, ad to eve thicker tailed sub-cauchy distributios for 0 < ν 0 < 1. Zeller 1976 revealed the fact that depedet but ucorrelated resposes ca be aalysed by a Studet-t model. He discussed the differeces as well as the similarities of the results i both classical ad Bayesia cotext for the ormal ad Studet-t based models.

3 Fraser 1979, p.37 emphasised that the ormal distributio is extremely short tailed ad thus urealistic as a sole distributio for variatio. He also demostrated the robustess of the Studet-t family as opposed to the ormal distributio based o empirical studies ad aalyses. His fidigs suggest that a Studet-t model based aalysis works reasoably well both for the ormal ad the Studet-t model resposes, but the same does ot hold for the commoly ormal based aalysis cf. Fraser, 1979, p.41. I justifyig the appropriateess ad essece of the Studet-t model Prucha ad Kalajia 1984 have poited out that the ormal model based aalysis i is geerally very sesitive to deviatios from its assumptio, ii places too much weight o outliers, iii fails to utilize sample iformatio beyod the first two momets, ad iv appeals to a cetral limit theorem at most approximately, ot exactly, ormal. I this paper, we cosider a liear regressio model, y j = θ + βx j + e j ; j = 1,,, 1.1 where y j is the j th value of the depedet variable correspodig to x j, a give fixed value of the idepedet variable; e j is the error compoet associated with the respose y j ; ad θ ad β are the itercept ad slope parameters respectively of the model. It is assumed that the vector of the errors e = e 1, e,, e is distributed accordig to the multivariate Studet-t law with Ee = o ad Eee = σ e I where σ 1 e is the commo variace of e j s, j = 1,,, ad I is the idetity matrix of order. The class of Studet-t distributios with varyig degrees of freedom ca be expressed as a variace mixture of ormal distributios as follows: fe; σ, ν 0 = 0 f N efτdτ 1. where f N e is the p.d.f. of e whe e N0, τ I ; ad fτ is the p.d.f. of a Iverted Gamma IG distributio with scale parameter σ ad degrees of freedom parameter ν 0, that is, τ IGσ, ν 0. Therefore, we obtia the joit desity of e 1, e,, e as with hν 0 = ν ν0 0 Γ ν 0 + fe; σ, ν 0 = hν 0 σ [ ν σ π Γ ν 0 ad < θ <. e j j=1 ] ν , as the ormalizig costat, σ e = ν 0 ν 0 σ > 0, ν 0 3, It is well kow that the above desity fuctio approaches to the ormal form as ν 0, ad whe ν 0 = 1, it becomes Cauchy. Furthermore, the margial 3

4 distributio of each compoet of e is uivariate Studet-t. Also, for smaller values of ν 0, this distributio has thicker tails tha that uder ormal distributio. Our choice of the Studet-t family thus icludes a class of symmetrical distributios with variable tail thickess. Ulike the ormal distributio, it is capable of hadlig outliers as well as depedet but ucorrelated resposes. Further support for the applicatio of the Studet-t model may be foud i Haq ad Kha 1990, ad Kha ad Haq I this paper, we cosider the problem of estimatig the itercept parameter, θ based o the observed sample observatios y 1, y,, y as specified i the model 1.1 whe a ucertai prior iformatio i the form of a ull hypothesis H 0 : β = β 0 is available. We defie three differet estimators, amely, urestricted, shrikage restricted, ad shrikage prelimiary test estimators of θ ad study their properties based o both the ubiasedess ad mea square error m.s.e. criterio. I the presece of a ucertai prior iformatio the usual procedure is to pre-test H 0 before the actual estimatio of the parameter. The problem of estimatio as well as the samplig properties of the estimators for the liear regressio model whe a ucertai prior iformatio exists have bee widely ivestigated i the literature [see for istace, Judge ad Bock 1978, Griffiths et al. 199 ad the refereces there i] uder the ormal theory. As discussed above, there is a icreasig evidece i the literature that i may cases the set of data may have bee geerated by processes whose distributios have higher kurtosis, that is, heavier tails tha the ormal distributios. Giles 1991 cited a umber of refereces from the commodity ad fiacial markets study where the uderlyig distributios are symmetrical but ot ormal. Several of those authors revealed the fact that i may real life data the Studet-t distributio model fits far better tha the ormal based model. For example, see Praetz 197, ad Blattberg ad Goedes The implicatios of these applied studies resulted i the use of the Studet-t distributio of the error terms i the regressio model by may authors. Recetly, statistical aalyses of the liear regressio model based o the Studet-t distributio have bee pursued by Giles 1991, Kha ad Haq 1994, ad Kha ad Saleh 1995 to metio a few. I the ext sectio we defie three differet estimators of θ. Some useful theorems for the computatio of the m.s.e. of the estimators are give i sectio 3. The expressios for the bias ad the m.s.e. are obtaied i sectio 4, while the relative efficiecy for the estimators are derived i sectio 5. Comparisos amog the estimators ad recommedatios are made i sectio 6. Some cocludig remarks are icluded i sectio 7. 4

5 . Estimators of the Itercept I this sectio, our objective is to defie various estimators of θ based o the sample observatios y 1, y,, y havig the joit desity as give i 1.3 whe it is suspected that the ull hypothesis H 0 : β = β 0 may hold, but ot sure. It is well kow cf. Zeller, 1976 that the maximum likelihood estimators m.l.e. of β, θ ad σ for the simple regressio model with errors havig multivariate Studet-t distributio are j=1 β = x j xy j ȳ j=1 x, j x θ =ȳ β x, σ =s = 1 ad y j θ βx j j=1.1 respectively, where x = 1 j=1 x j ad ȳ = 1 j=1 y j. The estimator θ, as defied above, is a fuctio of the maximum likelihood estimator of β ad will be called the urestricted estimator of θ. If the ull hypothesis is true, the the atural estimator of β is β 0. However, sice we are ot sure about the teability of the ull hypothesis, we make use of the available sample iformatio i the estimatio of θ. Thus takig a covex combiatio of β 0 ad β we defie the followig shrikage restricted sr estimator of β as ˆβ sr = cβ c β, 0 c 1. where we call c as the degree of cofidece i the ull hypothesis. The estimator ˆβ sr is kow as the shrikage restricted estimator of β. Thus for every possible value of c the estimator i. will produce a differet estimator. Obviously, our estimator of β would be 1 β 0 + β if c = 1. Clearly, the shrikage restricted estimator i. is a compromise betwee the two extreme estimators, oe based o the ull hypothesis, totally igorig the sample iformatio, ad the other based o the sample observatios aloe, disregardig the ucertai prior iformatio i the form of the ull hypothesis. Thus we defie shrikage restricted estimator of θ as sr = ȳ [cβ c β ] x..3 I applicatios, usually the ull hypothesis is suspicious. I such a case we follow the Fisher s Recipe to remove the suspicio by testig H 0 out through a appropriate test statistic. Usig Zeller s 1976 idea we cosider the test statistic F = s x β β 0 s e.4 5

6 to test H 0 : β = β 0, where Uder H 0, m =. s x = s e = x j x ad j=1 { y j ȳ β }.5. x j x j=1 F i.4 follows a cetral F -distributio with 1, m D.F. where Thus, for the estimatio of θ whe H 0 : β = β 0 is suspicious, we carry out a F -test ad choose θ sr if H 0 is ot teable ad if H 0 is accepted at a prescribed level of sigificace, say, α 0 < α < 1. The, we have the shrikage prelimiary test spt estimator of θ as follows: spt sr = IF < F α + θ IF F α = θ sr θ IF < F α = θ + c β β 0 xif < F α.6 where I is the idicator fuctio which assumes values either 0 or 1, F α is the 1 α th quatile of a cetral F -distributio with 1, m D.F. If c = 1, we obtai the ordiary prelimiary test pt estimator: pt = θ + β β 0 xif < F α..7 I the foregoig sectio, we have defied three estimators of θ, amely, θ, the urestricted m.l.e.; sr spt, the shrikage restricted m.l.e.; ad, the shrikage prelimiary test m.l.e. To determie the power fuctio of the test statistic uder the alterative hypothesis, H 1 : β β 0 we have some prelimiaries i sectio Some Prelimiaries Theorem 3.1. Suppose Y 1, Y,, Y are idetically ad idepedetly distributed as Nθ, τ with θ = EY j, ad τ follows a Iverted Gamma IG distributio with parameters ν 0, σ give by { }{ fτ ν 0, σ ν0 σ = Γ ν 0 The the joit distributio of Y 1, Y,, Y fy; θ, σ, ν 0 = hν 0 σ [ ν σ 6 } ν 0 τ ν 0+1 e ν 0 σ τ. 3.1 is give by j=1 y j θ ] ν+. 3.

7 Proof: Completig the followig itegratio we get πτ e 1 τ j=1 y j θ fτ ν 0, σdτ 3.3 Theorem 3.. The distributio of g = m F, where F is give by.4 uder H 1 : β β 0 follows the distributio defied by the followig desity fg = { Γr + ν 0 ν 0 r ν 0 Γr + 1Γ ν 0 1 r 0 Γr + Γr Γ gr g 1 + ν 0 } r+ 1 ν 0+r 3.4 where = δ σ i which δ = β β 0 ad σ = ν 0 ν 0 σ. Theorem 3.3. The distributio fuctio d.f. of g is give by G 1 1,m F 0; = { Γr + ν 0 ν 0 Γr + 1Γ ν 0 1 r 0 I u r + 1 ; m } 1 + ν 0 ν 0 r 1 ν 0+r 3.5 where I u r + 1 ; m is the icomplete Beta fuctio with u = F 1,mα m+f 1,m α ad F 0 is the value of the F 1,m variable such that 1 α 100 percet area uder the cetral F 1,m distributio curve is to the left of F 0 for give values of α ad m. The proof follows from the straight forward expectatio of the o-cetral F -distributio with respect to the Iverted Gamma distributio with parameters ν 0, σ. Now observe that lim P F F 0 = G 1,m F 0 ; 3.6 ν 0 which is the distributio fuctio d.f. of a o-cetral F -distributio with 1, m D.F. ad o-cetrality parameter = δ σ { Γr + ν 0 lim ν 0 Γr + 1Γ ν 0 1 ad lim ν 0 σ = σ. sice 1 + ν 0 ν 0 r 1 ν 0+r } = r e Γr Therefore, the results obtaied i this paper will remai valid for the ormal based model as a special case whe ν 0. I the ext sectio, the expressios for the bias ad the mea square error of the three estimators, defied i the previous sectio, are provided. 7

8 4. Bias ad M.S.E. of Estimators The bias ad the m.s.e. of the urestricted estimator of θ are B 1 θ = E{ θ θ} = 0, ad M 1 θ = E{ θ θ} = ν 0 ν 0 σ 1 + x s x 4.1 respectively. Similarly, for the shrikage restricted estimator of θ the expressios for the bias ad the m.s.e. are B sr = E{ sr θ} = cδ, ad M sr = E{ sr θ} = ν 0 ν 0 σ[ c x s x + c x ] 4. respectively, where = ν δ 0 i which δ = β β 0. ν 0 σ Fially, the expressios of the bias ad the m.s.e. of the shrikage prelimiary test spt estimator of θ are B 3 spt = E{ spt θ} = cδg 1 3,m l α;, ad M 3 spt = E{ spt θ} = ν 0 ν 0 σ[ 1 + x x + c s x s x x c c s x G 3,m l α; { G 1 3,m l α; cg 1 5,m l α; }] 4.3 respectively, where l α = 1 3 F 1,mα, l α = 1 5 F 1,mα, ad G 1 3,m l α; = ν 0 i which u 1 = 3l α m+3l α, G 1 5,m l α; = ν 0 i which u = 5l α m+5l α, ad G 3,m l α; = ν 0 4 { Γr + ν 0 r 0 Γr + 1Γ ν 0 1 I u1 r + 3 ; m } { Γr + ν 0 r 0 Γr + 1Γ ν 0 1 I u r + 5 ; m } { Γr 1 + ν 0 r 0 Γr + 1Γ ν 0 I u1 r + 3 ; m } 1 + ν 0 ν 0 r 1 + ν 0 1 ν 0+r ν 0 r 1 + ν 0 1 ν 0+r ν 0 r 1 ν 0+r

9 i which u 1 = 3l α m+3l α. With the results obtaied i this sectio, we provide the relative efficiecies of the estimators i the ext sectio. The relative efficiecy R.E. of expressio 5. Relative Efficiecy Expressios sr with respect to θ is give by the followig E 1 c, = 1 + [1 x + 1 c x + c x ] 1, 0 c 1, 5.1 s x ad that of spt with respect to θ by where φ = 1 + x s x 1 [ c x c E c, α, = s x x s x s x [ 1 + φ ] 1 5. { G 1 3,m l α; G 1 5,m l α; } ] c cg 3,m l α; sr Fially, the expressio for the relative efficiecy R.E. of is foud to be spt with respect to E 3 c, α, = [1 + 1 c x + c x ] 1 + x where φ is the same as defied i 5.3. s x s x 1 [ 1 + φ ] Based o the relative efficiecy of the estimators, comparisos ad recommedatios are made i the followig subsectio. 6. Comparisos ad Recommedatios First, we preset the domiace picture of the three estimators uder the ull hypothesis i the followig theorem. Theorem 6.1. Uder the H 0 : β = β 0 the domiace picture of the estimators is sr spt θ. 6.1 Proof: Cosider the m.s.e. differeces 9

10 i ii iii M 1 M = ν 0 ν 0 σ x [ s 1 1 c ] > 0 6. x that is, sr is better tha θ uder H 0. M 1 M 3 = ν 0 ν 0 σ x [ ] s c cg 3,m l α; 0 > x that is, spt is better tha θ uder H 0. that is, sr Compariso M 3 M = ν 0 ν 0 σ x [ { }] s c c 1 G 3,m l α; 0 > x is better tha Betwee sr spt uder H 0. ad θ First, ote that for a give set of x -values, both x ad s x are kow ad fixed. Cosider the relative efficiecy expressio E 1 c,. For a fixed c, it is a decreasig fuctio of. At = 0, E 1 c, 0 = 1 + [1 x + 1 c x ] 1, 0 c 1, 6.5 s x which becomes 1 whe c = 0 ad 1 + x s x s x [ 1 + 1, E 1 c, 0 for c 0. However, for 0 c c better tha θ. For c = 1, c c tha θ. If c = 1, the rage of rage tha the ordiary PT estimator, pt θ performs better. Compariso Betwee spt x s x = 3, thus for 0 3, sr is 0 to 1. Thus ad θ sr ] 1 whe c = 1. As, sr performs performs better has a wider performace for small c values. Outside this rage Oce agai, cosider the R.E. expressio, E c, α, as give i 5.. For a fixed c, it is a fuctio of α,. The fuctio has its maximum at = 0 with the value Ec, α, 0 = E give by [ E = 1 c c 1 + x s x 1G 3,m l α; 0] 1 > The fuctio E c, α, decreases as icreases crossig the lie E c, α, = 1 to a maximum E 0 c, α, at 0 =, the icreases towards 1 as. Now, for = 0 ad α varyig, we have max E c, α, 0 = E c, 0, 0 = [1 c c] α 1 10

11 The value of E c, α, 0 decreases as α icreases. O the other had, whe α = 0 ad varies, the the curves E c, 0, ad E c, 1, = 1 itersect at = c c. For geeral α, the fuctios E c, α 1, ad E c, α, will always itersect i the iterval 0 c c. The value of at the itersectio decreases as α icreases. Therefore, for two differet α, say α 1 ad α, the fuctios E c, α 1, ad E c, α, will ever itersect above E c, 1, = 1. Some graphs are show to illustrate the pheomeo. I order to choose a estimator with maximum relative efficiecy, we adopt the followig rule: If it is kow that [0, c sr c ], the estimator is chose sice E c, α, is largest i this iterval. However, is geerally ukow ad i which case there is o way of choosig a uiformly best estimator. Thus we preassig a prescribed R.E.-value, say, E 0 which is tolerable. The cosider the set A = {α E c, α, E 0 } ad try to choose a estimator which maximises E c, α, over all α A ad all. That is, we solve for α such that max α A mi E c, α, = E Hece we have a maximi rule for the optimum choice of the level of sigificace for the shrikage prelimiary test estimator, spt. Tables of R.E., both maximum E ad miimum E 0, ad the value of at which the miimum occurs 0 for a give α are reported at the ed of this sectio. Discuss a example from the table with ν 0 = 3, 6, 9, 1, CONCLUDING REMARKS I this paper, we have studied the properties of the urestricted, shrikage restricted ad prelimiary test estimators of the itercept parameter of the liear regressio model with a class of Studet-t errors. Sice the Studet-t model with ν 0 D.F. ecompasses a class of symmetrical distributios which icludes the ormal as ν 0 as well as other wider tailed distributios so the proposed estimators i this paper are robust i this class of regressio models. I this study, we fid the behaviour of the estimators similar to those i the case of the ormal model. The differeces i the relative efficiecies of the estimators are show by the graphs for varied values of ν 0. The decisio rule for selectig a optimal α, the level of sigificace is also show usig tabular values of maximum E ad miimum E 0 relative efficiecies for a rage of ν 0 -values. It is expected that the results obtaied i this paper will be preferable to ormal theory results by the practitioers as it icludes the ormal based results as a special case ad may others i the 11

12 family of elliptically symmetric distributios. Moreover, the results obtaied uder the Studet-t model are valid for the ormal model but ot vice-versa. REFERENCES Bacroft, T.A O biases i estimatio due to the use of prelimiary tests of sigificace. A. Math. Statist., 15, Blattberg, R.C. ad Goedes, N.J A compariso of the stable ad Studet distributio as statistical model for stock prices. Joural of Busiess, 47, Fisher, R.A Statistical Methods Scietific Iferece. Oliver ad Boyd, Lodo. Fisher, R.A The Desig of Experimets. 7 th ed. Hafer, New York. Fraser, D.A.S Iferece ad Liear Models. McGrow Hill, New York. Fraser D.A.S. ad Fick, G.H Necessary aalysis ad its implemetatio. Proc. Symposium o Statistics ad Related Topics, A.K.Md.E. Saleh ed., Carleto Uiversity, Caada, Giles, J.A Pre-testig for liear restrictios i a regressio model with spherically symmetric disturbaces. Joural of Ecoometrics, 50, Griffiths, W.E., Lutkepohl, H., ad Bock, M.E Readigs i Ecoometric Theory ad Practice: a volume i hoor of George Judge. ed. North- Hollad, New York. Ha, C.P. ad Bacroft, T.A O poolig meas whe variace is ukow. Jou. Amer. Statist. Assoc., 63, Haq, M.S. ad Kha, S Predictio distributio for liear model with Multivariate Studet-t errors distributio. Commuicatios i Statistics, Theory & Methods, 19, Judge, G.G. ad Bock, M.E Statistical Implicatios of Pre-test ad Stei-rule Estimators i Ecoometrics. North-Hollad, Amsterdam. Kha, S. ad Haq, M.S Predictive iferece for the multiliear model with errors havig multivariate t-distributio ad first order auto-correlatio structure. Sakhya, B, 56, Kha, S. ad Saleh, A.K.Md.E Prelimiary test estimators of the mea for samplig from some Studet-t populatios. Faculty of Scieces Wrokig Paper Series, Uiversity of Souther Queeslad, Australia. Kitagawa, T, Estimatio after prelimiary test of sigificace. Uiversity of Califoria Publicatios i Statistics, Vol. B, Lobato, I. N Testig That a Depedet Process Is Ucorrelated, Joural of the America Statistical Associatio, Volume 96455, Mosteller, F O poolig data. Jou. Amer. Statist. Assoc., 43, Praetz, P.D The distributio of share price chages. Joural of Busiess, 45, Prucha, I.R. ad Kelejia, H.H The structure of simultaeous equatio estimators: A geeralizatio towards o-ormal disturbaces. C, 5,

13 Saleh, A.K.Md.E. ad Se, P.K No-parametric estimatio of locatio parameter after a prelimiary test o regressio. A. Statist., 6, Saleh, A.K.Md.E. ad Kibria, B.M.G Performace of some ew prelimiary test ridge regressio estimators ad their properties. Commuicatios i Statistics, Theory & Methods,, Saleh, A.K.Md.E. ad Ha, C.P Shrikage estimatio i regressio aalysis. Estadistica, 4, Wa, A.T.K The samplig preformace of iequality restricted ad pre-test estimators i a mis-specified liear model. Australia Joural of Statistics, 363, Zeller, A Bayesia ad o-bayesia aalysis of the regressio model with multivariate Studet-t error term. Jou. Amer. Statist. Assoc. 60,

1 Inferential Methods for Correlation and Regression Analysis

1 Inferential Methods for Correlation and Regression Analysis 1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet