CALCULATION METHOD FOR NONLINEAR DYNAMIC LEAST-ABSOLUTE DEVIATIONS ESTIMATOR

Transcription

1 J. Japan Statist. Soc. Vol. 3 No CALCULAION MEHOD FOR NONLINEAR DYNAMIC LEAS-ABSOLUE DEVIAIONS ESIMAOR Kohtaro Hitomi * and Masato Kagihara ** In a nonlinear dynamic model, the consistency and asymptotic normality of the Nonlinear Least-Absolute Deviations (NLAD) estimator were proved by Weiss (99), even though they are difficult to compute in practice. Overcoming this difficulty will be critical if the NLAD estimator is to become practical. We propose an approximated NLAD estimator with the same asymptotic properties as the original with the exception that ours is easier to use. Key words and phrases: Calculation method, Least absolute deviations estimator, Nonlinear dynamic model.. Introduction In parametric regression models, the Least-Squares (LS) estimator is usually used for parameter estimates. If the error term is distributed as normal, then the LS estimator is a Maximum Likelihood Estimator (MLE) and attains minimum variance within unbiased estimators. At the same time, however, it is well known that only one outlier may cause a large error in an LS estimator. his occurs in the case of fatter tail distributions of the error term. In such a case, more robust estimators are desirable. One is the Least-Absolute Deviations (LAD) estimator. In a linear regression model, a linear programming method is available as a calculation method for the LAD estimator. On the other hand, in the nonlinear dynamic model, no computational method is proposed, although it has been shown theoretically by Weiss (99) that a Nonlinear LAD (NLAD) estimator is consistent and asymptotically normal. herefore, we seek an approximate estimator of the NLAD estimator that is practically computable. In a linear case, Hitomi (997) proposed an estimator of this kind called the Smoothed LAD (SLAD). In order to obtain an SLAD estimator, he approximated the non-differentiable original objective function by the smoothed function which is differentiable. hat is, he replaced x with x 2 + d 2 where d>0is the distance from the origin Received August 29, Revised February 9, 200. Accepted March 7, 200. *Department of Architecture and Design, Kyoto Institute of echnology; hitomi@hiei.kit.ac.jp. **Graduate School of Economics, Kyoto University; e60x0032@ip.media.kyoto-u.ac.jp. Horowitz (998) also proposed another SLAD estimator, which has different purposes than ours. In his article, the main target is not a calculation method of the LAD estimator. In practice, his SLAD estimator seems to be difficult to compute, as he implied, because it involves an integral of kernel for nonparametric density estimation.

2 40 J. JAPAN SAIS. SOC. Vol.3No.200 Figure. 6 y=abs(x) y=sqrt(x^2+) Figure Standard Normal distribution Laplace distribution

3 CALCULAION MEHOD FOR NONLINEAR DYNAMIC LAD ESIMAOR 4 (see Figure ). hen the distance between the original and smoothed objective functions is equal to or smaller than d for all x. Hitomi (997) controlled the parameter d connected with the sample size so that the SLAD estimator has the same asymptotic properties as the original LAD estimator. In this article, we extend this method to the nonlinear dynamic model. In conclusion, we obtained the general calculation method of the NLAD estimator. We call it the Nonlinear SLAD (NSLAD) estimator. We introduce a nonlinear dynamic model in Section 2, which was investigated by Weiss (99). In Section 3, we prove that the NSLAD estimator has the same asymptotic properties as the original NLAD estimator under the nonlinear dynamic model and Section 4 presents the results of the Monte Carlo study. Concludingremarks are given in Section 5. Finally, our assumptions are described in Appendix A and the sketch of proof that our model in Section 4 meets assumptions is given in Appendix B. 2. Model Weiss (99) considered the followingnonlinear dynamic model. (2.) y t = g(x t,β 0 )+u t g : known function x t =(y t,...,y t p,z t ) z t : vector of exogenous variables β 0 :(k ) vector of unknown parameter u t : unobserved error term which satisfies Median(u t I t )=0 I t : σ-algebra (information set at period t) generated by {x t i } (i 0) and {u t i } (i ). hen the NLAD estimator is defined as the solution of the followingproblem: (2.2) min Q (β) min β β y t g(x t,β). In these basic settings, Weiss (99) proved that the NLAD estimator ˆβ was consistent and asymptotically normal under the set of assumptions in Appendix A. Under the assumptions described in Ap- heorem (Weiss(99)) pendix A, (2.3) ˆβ p β 0 and ( ˆβ β0 ) d N(0,V),

4 42 J. JAPAN SAIS. SOC. Vol.3No.200 where (2.4) (2.5) V = D AD, A = lim E D = 2 lim E [ gt β (β 0) ] β (β 0) [ f t (0 I t ) β (β 0) β (β 0), ], f t ( I t ) : density function of u t conditional on I t. However, no computational method was proposed by Weiss (99). his is critical, and we solved this problem. FollowingHitomi (997), y g(x, β) is approximated by (y g(x, β))2 + d 2 (d > 0). he NSLAD estimator is defined as the solution of the followingproblem which minimizes the smoothed objective function: 2 (2.6) min Q s (β) min β β (yt g(x t,β)) 2 + d 2. First and second derivatives are (where g t (β) :=g(x t,β)) (2.7) and (2.8) Q s β = 2 Q s β β = y t g t (β) (yt g t (β)) 2 + d 2 β (β) { d 2 (β) (β) ((y t g t (β)) 2 + d 2 ) 3/2 β(β) β } y t g t (β) 2 g t (β). (yt g t (β)) 2 + d 2 β β 3. Asymptotic properties 3.. Consistency he difference between the original and smoothed objective functions defined in (2.2) and (2.6), respectively, is (3.) 0 Q s (β) Q (β) d x, y and β. herefore, if we control the parameter d such that d 0as, (3.2) sup{q (β) Q (β)} 0 as. β 2In appropriate conditions, Q converges in probability to a smooth function as to which Q s also converges in probability if d 0as. herefore it is understandable that the original NLAD and NSLAD estimator have the same asymptotic properties.

5 CALCULAION MEHOD FOR NONLINEAR DYNAMIC LAD ESIMAOR 43 his implies the followingtheorem. heorem. Assume the original objective function Q (β) converges in probability uniformly in β to a function that is uniquely minimized at β 0. 3 If d 0 as,then the NSLAD estimator is consistent Asymptotic normality he asymptotic normality of the NLAD estimator ˆβ is mainly based on the followingasymptotic first order condition: (3.3) sgn(y t g t ( ˆβ)) β ( ˆβ) =o p (). hus, if the NSLAD estimator also satisfies (3.3), it is asymptotically normal with the same asymptotic covariance matrix of the original NLAD, accordingto Weiss (99) results. herefore, we have proved that the NSLAD estimator ˆβ s satisfies the condition (3.3) in the followings. Now, note the NSLAD estimator ˆβ s satisfies the first order condition: (3.4) y t g t ( ˆβ s ) g t (y t g t ( ˆβ s )) 2 + d 2 β ( ˆβ s )=0. hen, all we have to do is show that the followingequation is satisfied: A := sgn(y t g t ( ˆβ s y t g t ( )) ˆβ s ) g t (y t g t ( ˆβ s )) 2 + d 2 β ( ˆβ (3.5) s ) = o p (). Let δ be a positive number. Divide the data set into two groups such that D = {t : y t g t ( ˆβ s ) >δ } and D 2 = {t : y t g t ( ˆβ s ) δ }. hen the followinginequality holds. (3.6) (3.7) t D sgn(y t g t ( ˆβ s y t g t ( )) ˆβ s ) (y t g t ( ˆβ s )) 2 + d 2 β ( ˆβ s ) + t D 2 sgn(y t g t ( ˆβ s y t g t ( )) ˆβ s ) (y t g t ( ˆβ s )) 2 + d 2 β ( ˆβ s ) ( ) < +d 2 /δ 2 β ( ˆβ s ) + t D β ( ˆβ s ) t D 2 =: A + A 2. A 3Weiss (99) demonstrated this fact in his nonlinear dynamic model described in Appendix A.

6 44 J. JAPAN SAIS. SOC. Vol.3No.200 First, we focus on the term A. By the aylor expansion, it follows that = ( ) ( 2 ( ) ) 2 d d (3.8) + o. +d 2 /δ 2 2 δ δ Hence, ( ) 2 ( d ( ) ) A < 2 δ β ( ˆβ 2 s ) d (3.9) + o. δ hus, A = o p () when ( ) 2 d 0( ) and δ β ( ˆβ (3.0) s ) p c : a finite constant. Next, consider the term A 2, which we regard as a function of β, A 2 = A 2 ( ˆβ s )= ( y t g t ( ˆβ s ) δ ) β ( ˆβ (3.) s ) = w t ( ˆβ s ) where w t (β) :=( δ + {g t (β) g t (β 0 )} u t δ + {g t (β) g t (β 0 )}) β (β) and ( ) represents the indicator function. Here, assume that the conditional density function of u t on I t is bounded from above, (3.2) f t (u I t ) <M t, I t and the next condition is satisfied in an open neighborhood B 0 of β 0, lim E β (β) (3.3) <M 2 β B 0 for some real values 0 <M,M 2 <. hen, for β B 0,weget (3.4) E(w t (β)) = E[E(w t (β) I t )] [ <E β (β) ( δ + {g t (β) g t (β 0 )} u t ] δ + {g t (β) g t (β 0 )})f(u t I t )du t < 2M δ E β (β).

7 CALCULAION MEHOD FOR NONLINEAR DYNAMIC LAD ESIMAOR 45 hus, by the Markov inequality, (3.5) P ( A 2 (β) ɛ) E A 2(β) ɛ < 2 ɛ M δ E β (β). herefore, when δ 0as, β B 0 A 2 (β) 0. Hence, invokingthe consistency of ˆβ s (heorem ), we can conclude that A 2 ( ˆβ s ) p 0. From the discussions presented above, A = o p (), when the convergence rate of d satisfies p (3.6) ( d δ ) 2 0 and δ 0( ) and (3.0), (3.2) and (3.3) are satisfied. heorem 2. Suppose the following conditions are satisfied: (i) y t = g(x t,β 0 )+u t, and the conditional median of u t on I t is zero. (ii) he conditional density of u t on I t is bounded from above for all t, I t, thus, M such that for all t, I t f t (u I t ) <M. (iii) In an open neighborhood B 0 of β 0 for a finite constant M 2, lim E β (β) <M 2 β B 0. and β ( ˆβ s ) E β (β 0) p 0 Furthermore, assume the conditions described in Appendix A, then ( ˆβ β 0 ) d N(0,V) for the NLAD estimator ˆβ (See Section 2). If the smoothing parameter d satisfies that 3/4 d 0,then the NSLAD estimator ˆβ s is asymptotically normal. ( ˆβs β 0 ) d N(0,V). Proof. By Weiss (99) proof method of the asymptotic normality of the NLAD estimator, 4 we can see that all we have to do is show (3.3) for the NSLAD estimator because the other conditions are implied by nonlinear dynamic model described in Appendix A. Hence, we have already completed our proof in the above discussions. 4. Monte Carlo experiments In this section, we present some examples of the NSLAD estimator and compare the performance of the NSLAD and nonlinear least-squares (NLS) 4See Weiss (99) pp.6-63, proof of heorem 3, especially p.62.

8 46 J. JAPAN SAIS. SOC. Vol.3No.200 able. = 500 u Normal λ(= 0.5) β (= ) β 2 (= ) φ(= 0.5) Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) u Laplace Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) estimators. hese experiments are conducted in the followingnonlinear dynamic model, which satisfies the assumptions given in Appendix A. 5 We set β = β 2 =,λ= φ =0.5 and y 0 =0. zt λ (4.) y t = φy t + β + β 2 + u t. λ We generated u t from two distributions. One is the standard normal distribution, where the NLS estimator becomes MLE, and the other is the Laplace distribution, where the NLAD estimator is MLE. he density of the Laplace distribution, whose variance is adjusted to, is f u (u) = exp( 2 u )/ 2. Median(u) =0, V(u) =, where V ( ) indicates a variance. Next, let z be distributed as a uniform distribution: z U(0, 9/2). Note: this makes V ((zt λ )/λ) ==V (u). he experiments are examined under the followingconditions: () he number of replications is,000. (2) he sample sizes ( ) are 50, 00, 200, 5See Appendix B.

9 CALCULAION MEHOD FOR NONLINEAR DYNAMIC LAD ESIMAOR 47 able 2. = 200 u Normal λ β β 2 φ Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) u Laplace Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) and 500. (3) he smoothingparameter is, d =, which satisfies the conditions of heorems and 2. he results are reported in the followingtables. Each table is reported in the same format. he upper block of this table shows a case with an error term distribution that is standard normal, and the lower block shows a case with the Laplace distribution. In each block, for the NLS and NSLAD estimators, the sample mean, standard deviation, median, and st and 3rd quartiles are reported. he bias becomes negligible as the sample size increases, and there are no differences between the NSLAD and NLS estimators if attention is focused exclusively on this point. In terms of standard deviations, the NLS estimator has a smaller standard deviation when the error term s distribution is standard normal, and the NSLAD estimator has a smaller standard deviation when the error term s distribution is the Laplace distribution. When the error term is distributed as standard normal, standard deviations of the NSLAD estimators are about 20% larger than those of the NLS estimators and at most 28.3% larger in the estimate of λ as = 200. When

10 48 J. JAPAN SAIS. SOC. Vol.3No.200 able 3. = 00 u Normal λ β β 2 φ Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) u Laplace Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) the error term is distributed as the Laplace distribution, standard deviations of the NLS estimators are 0% 30% larger than those of the NSLAD estimators, and the largest difference is 65.7% larger in the estimate of λ as = 50. he second largest is 29.8% larger in the estimate of β as = 500. With respect to median and quartiles, there are no differences between the NLS and NSLAD estimators regarding their performances. Finally, we compare the computingtime of the NLS and NSLAD estimators. he average computing time of the NLS and NSLAD estimators is 0.35 and 0.47 seconds, respectively, when = 500 with the standard normal error, and also 0.35 and 0.42 seconds when = 500 with the Laplace error. In the same manner, the average computing time of the NLS and NSLAD estimators is 0.28 and 0.37 seconds, respectively, when = 200 with standard normal, and also 0.28 and 0.33 seconds when = 200 with the Laplace distribution. Similarly, when = 00 and 50, the NSLAD estimate takes about 30% as much time as the NLS estimate with standard normal, and takes about 20% as much time as the NLS estimate with the Laplace distribution. We use GAUSS for Windows (32 bit), Version , and GAUSS

11 CALCULAION MEHOD FOR NONLINEAR DYNAMIC LAD ESIMAOR 49 able 4. =50 u Normal λ β β 2 φ Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) u Laplace Mean (SLAD) (NLS) Standard Deviation (SLAD) (NLS) Median (SLAD) (NLS) st Quartile (SLAD) rd Quartile (NLS) Applications Optimization to conduct these experiments in an environment with a CPU that has a PentiumII, 400MHz, and 64MB memory. From these results, we can conclude that the NSLAD estimator performed very well and equal to the NLS estimator. 5. Concluding remarks he aim of this article is on the actual usage of the NLAD estimator, which has attractive properties such as robustness. he problem lies in computational difficulty. herefore, we proposed an NSLAD estimator, a generalization of Hitomi s(997) SLAD estimator, which is practically computable and has the same asymptotic properties as the NLAD estimator in Weiss (99) nonlinear dynamic model. he Monte Carlo experiment implies a good performance of the NSLAD estimator, at least equal to the NLS estimator. Consequently we obtained a computable approximate to the NLAD estimator, which we called the NSLAD estimator, and the NLAD and NSLAD estimators have the same asymptotic properties.

12 50 J. JAPAN SAIS. SOC. Vol.3No.200 Appendix A: Set of Assumptions In this section, we described the set of assumptions under which Weiss (99) demonstrated the consistency and asymptotic normality of the NLAD estimator. he basic model has already been introduced in Section 2. Assumption. In the model (2.), (i) (Probability space) (Υ, F, P) is a complete probability space and {u t,x t } are random vectors on this space. (ii) (Parameter space) Let B be a compact subset of Euclidean space R k, and β 0 is interior to B. (iii) (Regression function) g(x, β) is measurable in x for each β B and is A-smooth in β with variables A 0t and function ρ. 6 i g t (β) isasmooth with variables A it and functions ρ i,i =,...,k. In addition, max i ρ i (s) s for s>0 small enough. 7 (iv) (Conditional distribution of u t ) f t ( I t ) is Lipschitz continuous uniformly in t, and for each t, u, f t (u I t ) is continuous in its parameter. Furthermore there exist h>0 such that for all t, and some constant p > 0,P(f t (0 I t ) h) >p. 8 (v) (Unconditional density) An unconditional density f t of (u t,x t ) is continuous in its parameter at every point for each t. (vi) here exists < and r > 2 such that α(m) m λ for some λ < 2r/(r 2), where α( ) stands the usual measure of dependence for α-mixingrandom variables on (Υ, F, P) with respect to the σ- algebras generated by {u t,x t }. (vii) (Dominance conditions) 9 For all t, i, E sup β A it r <. here exist measurable functions a t,a 2t such that i g t (β) a t,i=,..., k, f t a 2t and for all t, a 2t du t dx t < and a 3 ta 2t du t dx t <. (viii) (Asymptotic stationarity) here exists a matrix A such that a+ t=a+ E[ g t(β 0 ) g t (β 0 )] A as, uniformly in a. (ix) (Identification condition) here exists δ>0 such that for all β B and sufficiently large, det( E[ g t (β) g t (β) f t (0 I t ) >h]) >δ. Appendix B: ConfirmingAssumptions in Our Model In this section we briefly sketch a proof that our model in Section 4 meets the assumptions described in Appendix A 6g(x t,β)isa-smooth with variables A t and function ρ if, for each β B, there is a constant τ>0 such that β β τ implies that g t( β) g t(β) A t(x t)ρ( β β ) for all t a.s., where A t and ρ are nonrandom functions such that lim sup E[A t(x t)] <,ρ(y) > 0 for y>0,ρ(y) 0as y 0. 7 i := / β i (i =,...,k), := / β. 8f t( I t) is a density function of u t conditional on I t. 9For a consistent result, these conditions are replaced by the following: here exist measurable functions a 0t,a t such that g t(β) a 0t, i g t(β) a t,i=,...,k, where E a jt r j <,j =0, for some r 0 >,r > 2. he other conditions can be relaxed.

13 CALCULAION MEHOD FOR NONLINEAR DYNAMIC LAD ESIMAOR 5 First, we begin with Assumption 2. We set +c φ c, c λ c (0 < c < ) and C β,β 2 C (0 < C < ). Note that the parameter space and the support of z are bounded. For Assumption 3, we use the aylor expansions, and for Assumption 4, we can show f(u 2 ) f(u ) < u 2 u and f(u) >his trivial. As to Assumption 6, we transformed our model as follows: yt = 2 y t + η t = i=0 2 η i t i s.t. yt = y t 2(2 2 ), η t = u t +2( z t 2) i.i.d. o this transformed model, we apply heorem 4.9 in Davidson (994) p.29. For Assumption 7, note an unconditional density f t = f because y t is stationary. We can see that E[ g t (β) g t (β)] < for Assumption 8, and E[ g t (β) g t (β)] is nonsingular in Assumption 9. References Buchinsky, M. (995). Quantile regression, Box-Cox transformation model, and the U.S. wage structure, Journal of Econometrics, 65, Davidson, J. (994). Stochastic Limit heory. Oxford University Press. Hitomi, K. (997). Smoothed least absolute deviations estimator and smoothed censored least absolute deviations estimator. mimeo. Horowitz, J. L.(998). Bootstrap methods for median regression models. Econometrica, 6, Newey, W. K. and McFadden, D. (994). Large sample estimation and hypothesis testing. Handbook of Econometrics (ed. Engle, R. F. and McFadden, D. L.), volume 4. Elsevier. Pollard, D. (985). New ways to prove central limit theorems. Econometric heory,, Pollard, D. (99). Asymptotics for least absolute deviation regression estimators. Econometric heory, 7, Portnoy, S. and Koenker, R. (997). he Gaussian hare and the Laplacian tortoise: Computability of squared-error versus absolute-error estimators. Statistical Science, 2, Sposito, V. A. (990). Some properties of l p estimation. Robust Regression: Analysis and Applications (ed. Lawrence, K. D. and Arthur, J. L.). Marcel Dekker, Inc. Weiss, A. A. (99). Estimating nonlinear dynamic models using least absolute error estimation. Econometric heory, 7,