ANOMALIES OF THE MAGNITUDE OF THE BIAS OF THE MAXIMUM LIKELIHOOD ESTIMATOR OF THE REGRESSION SLOPE

Transcription

1 P a g e ANOMALIES OF THE MAGNITUDE OF THE BIAS OF THE MAXIMUM LIKELIHOOD ESTIMATOR OF THE REGRESSION SLOPE Darmud O Drscoll ¹, Donald E. Ramrez ² ¹ Head of Department of Mathematcs and Computer Studes Mary Immaculate College Ireland ² Department of Mathematcs Unsty of Vrgna USA Correspondng Aut: darmud.odrscoll@mc.ul.e Abstract: The slope of the best-ft lne y h( x) 0 x from mnmzng a functon of the squared tcal and zontal errors s the root of a polynomal of degree four whch has exactly two real roots, one postve and one negatve, wth the global mnmum beng the root correspondng to the sgn of the correlaton coeffcent. We solve second order and fourth order moment equatons to estmate the varances of the errors n the measurement error model. Usng these solutons as an estmate of the error rato n the maxmum lkelhood estmator, we ntroduce a new estmator whch relates. We create a functon to the oblque parameter, used n the parameterzaton of the lne from ( x, h( x)) to ( h ( y), y), to ntroduce an oblque estmator. A Monte Carlo smulaton study shows mprovement n bas and mean squared error of each of these two new estmators o the ordnary least squares estmator. In O Drscoll and Ramrez (0), t was noted that the bas of the MLE estmator of the slope s monotone decreasng as the estmated varances error rato approaches the true varances error rato. Howe for a fxed estmated varances error rato, t was noted that the bas s not monotone decreasng as the true error rato κ approaches. Ths pa shows ths anomaly by showng that as κ approaches a fxed, the bas of the MLE estmator of the slope s also dependent on the magntude of. Keywords: Maxmum lkelhood estmaton, Measurement errors, Moment estmatng equatons, Oblque estmators

2 P a g e Introducton Wth ordnary least squares OLS ( y x) regresson we have data ( x, Y X x ),...,( xn, Yn X n xn ) and we mnmze the sum of the squared tcal errors to fnd the best-ft lne y h( x) 0 x where t s assumed that the ndependent or causal varable X s measured wthout error. The measurement error model does not assume that X s measured wthout error, has wde nterest wth many applcatons and has been studed n depth by many, for example, Carroll et al. (006) and Fuller (987). As n the regresson procedure of Demng (943) to account for both sets of errors and, we determne a ft so that a functon of both the squared tcal and the squared zontal errors wll be mnmzed. In Secton, we outlne the Oblque Error Method and the measurement error model and ntroduce second order and fourth order equatons to estmate n the maxmum lkelhood estmator. We also ntroduce two new estmators and and descrbe our Monte Carlo smulatons. We report on our fndngs n Secton 3 and conclude that that our estmators and MSE assocated wth the ordnary least squares estmator. Methodology. Mnmzng Squared Oblque Errors greatly reduce the Bas and From the data pont x, y ) to the ftted lne h( x) x, defne the ( y 0 tcal length v y 0 x from whch t follows that the sum of the squares of the oblque lengths from x, y ) to ( h ( y ) ( x h ( y )), y ( ( h( x ) y )) s SSEo ( 0,, ) ( ) v / v. () In a comprehensve pa by Rggs et al. (978), the auts state that: It s a poor method ndeed whose results depend upon the partcular unts chosen for measurng the varables. As n O Drscoll and Ramrez (0), so that our equaton s dmensonally correct we consder a standardzed weghted model where SSEo ( 0,, ) ( ) s v / s v, and. The soluton of = 0 s gven by x and the solutons of 0 y = 0 are the roots of the fourth degree polynomal, P ), 4 ( ( s / ).5 4 / 3 ( ) ( ) ( / ) 0.5 s s s s s. ()

3 For example, f graph of s P a g e 3, the From O Drscoll et al. (008), the Complete Dscrmnaton System { D,..., D n } of Yang (999) s a set of explct expressons that determne the number (and multplcty) of roots of a polynomal. Ths system s used to show that the fourth order polynomal P 4 ( ) has exactly two real roots, one postve and one negatve wth the global mnmum beng the postve (respectvely negatve) root correspondng to the sgn of s xy. For the graph of P ) s 4 ( Wth λ = we reco the mnmum squared tcal errors wth estmated slope and wth λ = 0 we reco the mnmum squared zontal errors wth estmated slope. The geometrc mean estmator s / s has the fxed oblque parameter λ = 0.5 and for the measurement error model, when both the tcal and zontal models are reasonable, a compromse

4 P a g e 4 estmator such as s wdely used and s hoped to have mproved effcency. Howe, Lndley and El-Saad (968) proved that the expected value of s based unless.. Measurement Error Model; Second and Fourth Moment Estmaton We now consder the measurement error model as follows. In ths pa t s assumed that X and Y are random varables wth respectve fnte varances and Y, fnte fourth moments and have the lnear functonal relatonshp Y. The observed data { x, y ), n } are subject to error by 0 X x where t s also assumed that ( X and y Y X and j. It s well known, n a measurement error model, that the expected value for ( OLS ( y x) ) s attenuated to zero by the attenuatng factor X /( X ) called the relablty rato by Fuller (987); and smlarly the expected value for ( OLS ( x y) ) s amplfed to nfnty by the amplfyng factor ( Y ) / Y. From Gllard and Iles (009), second moment equatons are and fourth moment equatons are (4) These equatons yeld the estmators ~ ~ (5) the Frsch hybola of Van Montfort (987) and the fourth order equaton ( s xy ( s ~ )( ~ s ) sxy (3) (6) ~ ~ 3 )( ) ( ) ( 3 ~ s s s s s ). (7) xy xy We use equatons (6) and (7) to solve for ~ and ~ mposng sutable restrctons on the possble solutons; frstly the varances must be postve; secondly the kurtoss of the underlyng dstrbuton must be sgnfcantly dfferent from the kurtoss of the normal dstrbuton to assure the valdty of Equaton (4) and thrdly the sample szes must be adequately large. We then use these solutons as estmates for the rato n the maxmum lkelhood estmator as descrbed n Secton.3. A typcal graph of equatons (6) and (7) xy xy

5 P a g e 5 s True Rato : Soluton.3 The Maxmum Lkelhood Estmator If the rato of the error varances / s assumed fnte, then Madansky (959), among others, showed that the maxmum lkelhood estmator for the slope s mle ( s s ) ( s s ) 4 s s ˆ ( ). (8) s s For fnte t also follows that the moment estmator agrees wth the MLE. If = n Equaton (8) then the MLE (often called the Demng Regresson estmator) s equvalent to the pendcular estmator,, frst ntroduced by Adcock (878). In the partcular case where fxed λ value of 0.5. If the researcher knows the true error rato then then mle has a ( ˆ ( ) (9) and there are no bas problems. We wll dscuss the more realstc stuaton when κ s an unknown parameter and must be estmated by.

6 P a g e 6 3. Monte Carlo Smulaton 3. The Bas of the MLE for an ncorrect choce of κ. We set the estmated rato of the error varances The X data was generated from a unform dstrbuton on to set The lnear regresson model had slope and sample sze n = 50. For the measurement error model, we used normal errors wth mean equal to zero and varances varyng o {,,3,4,5,9}. We used Mntab for our smulaton study settng the number of runs N = The results for the bas are recorded n Table. Table wth The rows of Table are sorted n ascendng order of the theoretcal bas,, dsplayed n Column 5. We make the followng observatons. Frstly, wth, the rankng for the bas concurs wth the rankng of the dfferences n the error varances but does not concur wth the rankng for n terms of ts closeness to. The value for κ = n Row s closer to the assumed value than the value for κ = n Row 3 s. Howe the absolute value for the bas 0.00 n Row s approxmately double the absolute value for the bas n Row 3; that s, the magntude of the bas for the MLE estmator s not monotone n κ. Secondly, for equal = 3/ n Row 7 and κ = 9/3 n Row 0, the respectve bases and are approxmately proportonal to the respectve dfferences of the error varances and.

7 P a g e 7 3. The effcency of dfferent slope estmators Usng the solutons ~ and ~ from equatons (6) and (7) as estmates for mle n, we ntroduce a new estmator whch forms y well n our Monte Carlo smulaton. 3.3 Relaton between pa and bda Wth estmated as n Secton., the ntble functon :[0, ] [0,] defned by ( ) c /( c ), c s / s, creates a new estmator. Ths proposed oblque estmator also forms y well n our Monte Carlo smulaton. Snce the range of κ ncludes nfnty, we do not compute ts aage value n our smulaton. Instead, we compute the aage λ value for, and use ) ( _ as the effectve aage ~ for κ. To determne the effcency of the sx estmators {,,,,, }, we conducted a set of Monte Carlo smulatons for varyng values of the true slope. We report n Tables -5 the MSE, the Bas, the assocated parameter and the assocated oblque angle for each of the sx estmators above. The orentaton for s chosen such that for <. X s UD(0,0), =.0, 0, 0 < < Table = 0, N =000, n =00, =, and for = 3 3 MSE 0 %Bas λ Table 3 X s UD(0,0), =.5, 0 = 0, N = 000, n = 00, =, 3 MSE 0 %Bas λ = ,

8 In the cases represented by Tables and 3 we can see that and make sgnfcant mprovement n (MSE, Bas) o the estmator P a g e 8 and each of the compromse estmators and. Of course forms well n each of these cases but ts use would have been based on pror knowledge that. Table 4 X s UD(0,0), =.0, 0 = 0, N = 000, n =00, =, = 3 MSE 0 %Bas λ X s UD(0,0), = 0.75, 0 Table 5 = 0, N =000, n =00, =, = 3 MSE 0 %Bas λ In the cases represented by Tables 4 and 5 we agan see that and make sgnfcant mprovement n (MSE, Bas) o the estmators and. Wth, formed y well n each case as expected snce. The condton of Lndley and El-Saad (968) of s satsfed n the case represented by Table 4 but not by Table 5 and hence formed y well n Table 4 but not as well n Table 5. Rggs et al. (978) state that no one method of estmatng the true slope s the best method under all crcumstances. Tables -5 show that and form well n all of the above four cases where no pror knowledge of the errors s assumed.

9 P a g e 9 Table 6 reports the effectve aage for ~, as descrbed n Secton 3.3, for (, ) {,4,9} {,4,9 }. Table 6 Effectve ~ aage; X s UD(0,0), =, 0 = 0, N =000, n = 00 = = 4 = 9 = = = Concluson and References Our smulaton study n 3. llustrates that the bas of the MLE estmator of the regresson slope s dependent on the magntude of, the varance of the errors n x. Our smulaton studes n 3.3 support the clam that our estmators and, under the condtons outlned n., greatly reduce the Bas and MSE assocated wth the ordnary least squares estmator. Adcock, R. J. (878). 'A problem n least-squares', The Analyst, 5: Carroll, R. J., Rupt, D., Stefansk, L. A., Cranceanu, C. M. (006). Measurement Error n Nonlnear Models - A Modern Perspectve, Boca Raton: Chapman & Hall/CRC, Second Edton. Demng, W. E. (943). Statstcal Adjustment of Data, New York: Wley. Fuller, W.A. (987). Measurement Error Models, New York: Wley. Gllard, J., Iles T. (009). 'Methods of fttng straght lnes where both varables are subject to measurement error', Current Clncal Pharmacology, 4: Lndley, D., El-Saad, M. (968). 'The Bayesan estmaton of a lnear functonal relatonshp', Journal of the Royal Statstcal Socety Seres B (Methodologcal,), 30: Madansky, A. (957). 'The fttng of straght lnes when both varables are subject to error', Journal of Amercan Statstcal Assocaton, 54:

10 P a g e 0 O'Drscoll, D., Ramrez, D., Schmtz, R. (008). 'Mnmzng oblque errors for robust estmaton', Irsh. Math. Soc. Bulletn, 6: O'Drscoll, D., Ramrez, D. (0). 'Geometrc Vew of Measurement Errors', Communcatons n Statstcs-Smulaton and Computaton, 40: Rggs, D., Guarner, J., Addelman, S. (978). 'Fttng straght lnes when both varables are subject to error', Lfe Scences, : Van Montfort, K., Moojaart, A., and de Leeuw, J. (987). 'Regresson wth errors n varables', Statst Neerlandca, 4: Yang L. (999). 'Recent advances on determnng the number of real roots of parametrc polynomals', J. Symbolc Computaton, 8: 5-4.