Econoetrcs orkng Paper EP7 ISSN 485-644 Departent of Econocs ON EIGTED ESTIMATION IN LINEAR REGRESSION IN TE PRESENCE OF PARAMETER UNCERTAINTY udth A Clarke Departent of Econocs, Unversty of Vctora Vctora, BC, Canada V8 Y Septeber, 7 Abstract e exane the proble of estatng the lnear regresson odel s coeffcents when there are uncertan lnear restrctons about these paraeters Theores are provded that generalze results obtaned by Magnus and Durbn (999) and Danlov and Magnus (4) Keywords: Mean squared error; eghted estators; Lnear restrctons EL classfcaton: C; C3; C; C5 Author Contact: Table : Unforty Tests for Growth Cyc udth A Clarke, Tel: + 5 7 854; fax: + 5 7 64 On the Robustness of Racal Dscrnaton Fndngs n Eal address: jaclarke@uvcca
Introducton Consder estatng the coeffcents of the odel y = Xβ + Zγ + u, u N(, σ I n n ) () where y(n ) s a vector of observatons, X(n k) and Z(n ) are nonrando regressor atrces, β(k ) and γ( ) are paraeter vectors th [X : Z] (of full colun-rank K = (k+)) and φ [β γ], odel () s y = φ + u () Suppose, n addton, lnear uncertan belefs exst about φ expressed as Rφ=r; r( ) and R( K) are nonstochastc wth R of full row-rank, <K Ths fraework extends that of Magnus and Durbn (999) and Danlov and Magnus (4) who suppose that X contans requred varables, whereas Z conssts of doubtful regressors, ncluded to perhaps provde a better estator of β; e, ther pror belefs regard excluson restrctons See also Magnus (999, ), Danlov (5) and Zou et al (7) Gven the uncertanty about γ, these authors consder the followng estator of β b w, = λb u + (-λ)b r (3) where b u s the unrestrcted least squares (OLS) estator, b r s the restrcted least squares (RLS) estator obtaned under γ =, λ λ(g u, ) s a rando weght functon, s the OLS resdual vector and g u s the OLS estator of γ ; the dependence of λ on s usually va an error varance estator Usng an F-test to choose between b u and b r results when λ = I [,c] (F), where F s the F-test statstc, c s the crtcal value fro an F dstrbuton wth and n-k degrees of freedo, and I [,c] (F) = when F [,c], otherwse Then, b w, s the tradtonal pretest estator (eg, udge and Bock, 978; Gles and Gles 993), nadssble and never preferred, usng rsk under quadratc loss or ean squared error, to ether of ts coponent estators Indeed, the rsk of the pretest estator can be greater than ether of ts coponent estators, an unattractve feature for a strategy adopted to prove knowledge on β As the a s to obtan a preferred estator of β, as opposed to undertakng a test about whether γ=, t akes sense to allow λ to be a contnuous functon (wth λ
) Ths gves rse to any possble cobnaton estators, ncludng Magnus and Durbn s (999) weghted-average least squares (ALS) estator, the shrnkage estators of udge and Bock (978, pp4-4), whch extend the aes and Sten (96) estators, and the aes-sten type estator of K and hte () The goal s to optally x the coponent estators based on a chosen crteron; eg, ean squared error (MSE) or rsk under quadratc loss Frst glance suggests that ths task wll depend on the odel s features: β, γ, X, Z and σ Indeed, ths has led researchers to assue orthonoral regressors or to explore rsk of the predcton vector (E(y X,Z)) rather than the coeffcent vector No longer s ths needed wth Magnus and Durbn s elegant Equvalence Theore Ths theore shows that deternng λ to nze the MSE of b w, reduces to ascertanng λ such that the MSE of λˆ s nzed where ˆ / = (Z MZ) gu N(, σ I ) wth M = I n n X(XX) - X e need only deterne λ such that λˆ s a preferred estator of - the ean vector of an -varate noral dstrbuton; a task that s ndependent of specfc regresson detals Mxng just b r and b u s lkely restrctve all of the regressors n Z are ether n or out Researchers ay exane partally restrcted odels that contan soe of Z s coluns There are odels to choose between wth auxlary regressors; let M be the odel that poses that none, one, soe or all of the eleents of γ are zero, b the subsequent LS estator of β, g the restrcted estator of γ and the assocated resdual vector, =,, ; e, = y Xb Zg = y p The cobnaton estator that weghts all possble estators of β s exaned by Danlov and Magnus (4): b w, = = λb (4) wth weghts that satsfy λ, λ = = and λ λ ( ˆ, ) Note: b = b u ( = b r ) when all (none) of Z s coluns are regressors and b w, collapses to b w, when only b u and b r are cobned Danlov and Magnus extend the equvalence theore to cover the weghted estator defned by expresson (4) 3
These sgnfcantly useful fndngs are shown for estatng β when there exsts uncertanty about whch auxlary regressors to nclude n the specfcaton In ths note, we generalze these equvalence theores to the estaton of φ, so coverng estaton of γ n addton to β, when the pror belefs relate to any lnear cobnaton of the coponents of φ rather than sply excluson restrctons The optal (n ters of MSE) cobnaton estator s deterned solely by ascertanng the optal estator of the ean of a noral rando varate wth unknown varance, whch has nothng to do wth the regresson odel s structure nor the specfc for of the pror lnear belefs Setup Our focus s on estatng the full coeffcent vector φ wth uncertan belefs Rφ=r Allowng for the unrestrcted odel, the fully restrcted odel and all possble partally restrcted odels that ncorporate soe of the restrctons, there are odels to consder; let M be the th odel (=,, ) Let A be a a selecton atrx of rank a (e, A = [I ] or a colun-perutaton thereof) such that a a odel M corresponds to odel () subject to A Rϕ = A r The atrx R s of full row rank, whch ples that A R also has full row rank As before, we denote p as the LS estator of φ assocated wth odel M wth resdual vector Defne S =, p u = S - y, = y p p r = p u S - R[RS - R] - (Rp u -r), p = p u S - RA [A RS - RA ] - A (Rp u -r), = [RS - R] -/ (Rφ-r), ˆ / = + [RS R ] RS u, P = [RS - R] / A [A RS - RA ] - A [RS - R] /, M = I n n S -, = S - R[RS - R] -/, = M u, = + P ˆ, = S - -, = S - P, B = I P, = P 4
5 The OLS estator of φ s p u =p wth A =P = and the RLS estator, obtaned fro posng all restrctons, s p r =p wth A =P =I Consder the cobnaton estator: p w, = λp u + (-λ)p r, (5) where the weght λ satsfes λ wth ) ˆ, λ( λ e also consder a ore general weghted estator:, p w p = λ = (6) where the weghts satsfy λ, = λ = and ) ˆ, ( λ λ The estators p w, and p w, generalze, respectvely, the weghted estators of Magnus and Durbn (999) and Danlov and Magnus (4) to all coeffcents when there s uncertanty about lnear cobnatons of these paraeters 3 Generalzed equvalence theores As p r =p u -ˆ and p =p r +B ˆ we have + σ ϕ ϕ ϕ r u M M M M B B I S, P N p p ˆ p (7) Usng (7) enables us to establsh the followng theores Theore Denote p w, = λp u + (-λ)p r, where λ λ(, ˆ ), u N(, σ I n n ) and ) I, N( ˆ σ Then, the MSE of p w, s MSE(p w, ) = σ + [MSE( )] (8) where = λ ˆ Proof As p u = p r + ˆ, p w, = p r + and usng that pr s ndependent of ˆ and, we have r w, ) E(p ˆ,) (p E + = So
E(pw, ) = ϕ + [E( )] (9) In addton, var( pw, ˆ, ) = var(pr ) = σ, whch ples var( pw, ) = σ + [var( )] () Cobnng expressons (9) and () copletes the proof Theore Denote w, = λp = p where λ λ( ˆ, ), u N(, σ I n n ) and ˆ N(, σ I ) Then, the MSE of p w, s MSE(p w, ) = σ + [MSE( )] () where = Bˆ wth B = λb = Proof Let B = λ B = As p = p r + B ˆ, p w = p r + B ˆ then, snce p r s ndependent of ˆ and, the condtonal ean s E(pw, ˆ, ) = E(pr ) + wth = Bˆ The uncondtonal ean s then (p ) = ϕ + [E( )] () E w, In addton, var( pw, ˆ, ) = var(pr ) = σ, whch ples var( pw, ) = σ + [var( )] (3) The result then follows by cobnng () and (3) Theores and show that the MSE propertes of the weghted estators of φ crucally depend on the MSE propertes of the weghted estator of, the ean 6
vector of a noral rando varate Underlyng s the copatblty of the pror nforaton wth the coeffcent vector: Rφ-r Ths ples the rrelevance of the specfc regresson data n deternng how best to wegh the varous estators of φ Specal cases of Theore and are, respectvely, Theore of Magnus and Durbn (999) and Theore of Danlov and Magnus (4) For both, let =, R( K) = [ I ], r =, p = [(b ˆg ) g ], pr = [b r g r ], k u r u u p = [b g ], b r = (XX) - Xy, g u = (ZMZ) - ZMy, g r =, M = I n n X(XX) - X, / ˆ = (Z MZ) g, / = ( Z MZ) γ, = [-(ZMZ) -/ ZX(XX) - (ZMZ) -/ ] = [ ], (X X) = and P u / / (Z MZ) A[A (Z MZ) A ] A (Z MZ) = Defne b w, = λb u + (- bw, = λb and g w, = λ)b r, g w, = λg u + (-λ)g r, = j=,: MSE(b w,j ) = σ (XX) - + MSE( j) λg = Usng our results wth (4) & MSE(g w,j ) = MSE( j) (5) Expresson (4) corresponds to that presented by Magnus and Durbn (999) for j= (e, b w, ) and to that reported by Danlov and Magnus (4) for j= (e, b w, ) Collapsng the results n ths fashon hghlghts that the sae weght functon that provdes an optal estator of gves an optal estator of both γ and β, not just β as stressed by Magnus, Durbn and Danlov 3 Concludng Rearks e have consdered cobnng estators of the coeffcent vector of a lnear regresson odel n the presence of uncertanty about lnear restrctons Ths setup 7
generalzes that of Magnus and Durbn (999) and Danlov and Magnus (4), and also related work by Magnus (999, ), Danlov (5) and Zou et al (7), all of who lt attenton to estatng soe of the coeffcents when others are not of nterest e generalze ther results to our fraework: the optal MSE weghted estator s deterned by optally estatng the ean vector of an uncorrelated, hooskedastc dstrbuton Ths fndng ples, for nstance, that the extensve lterature on how best to estate the ean of a noral dstrbuton carres over to the lnear regresson odel wth uncertan paraeter belefs Acknowledgeents I a grateful to Davd Gles for hs encourageent and helpful coents, to Nlanjana Roy, Graha Voss and other partcpants at a Unversty of Vctora senar for useful coents, and to a referee for helpful suggestons References Danlov, D, 5 Estaton of the ean of a unvarate noral dstrbuton when the varance s not known Econoetrcs ournal 8, 77-9 Danlov, D, Magnus, R, 4 On the har that gnorng pretestng can cause ournal of Econoetrcs, 7-46 Gles, A, Gles, DEA, 993 Pre-test estaton and testng n econoetrcs: Recent developents ournal of Econoc Surveys 7, 45-97 aes,, Sten, C, 96 Estaton wth quadratc loss Proceedngs of the Fourth Berkeley Syposu on Matheatcal Statstcs and Probablty, 36-379 udge, GG, Bock, ME, 978 The Statstcal Iplcatons of Pre-Test and Sten- Rule Estators n Econoetrcs North-olland, Asterda K, T, hte,, aes-sten-type estators n large saples wth applcaton to the least absolute devatons estator ournal of the Aercan Statstcal Assocaton 96, 697-75 Magnus, R, 999 The tradtonal pretest estator Theory of Probablty and ts Applcatons 44, 93-38 8
Magnus, R, Estaton of the ean of a unvarate noral dstrbuton wth known varance Econoetrcs ournal 5, 5-36 Magnus, R, Durbn,, 999 Estaton of regresson coeffcents of nterest when other regresson coeffcents are of no nterest Econoetrca 67, 639-643 Zou, G, an, ATK, u, X, Chen, T, 7 Estaton of regresson coeffcents of nterest when other regresson coeffcents are of no nterest: The case of nonnoral errors Statstcs & Probablty Letters/jspl69 9