1 Introduction. 18 October 2004

Transcription

1 wo sided analysis of variance with a latent time series Lars Hougaard Hansen University of Copenhagen and Codan forsikring Bent Nielsen Nuffield College, University of Oxford Jens Perch Nielsen Codan forsikring 18 October 2004 Abstract: Many real life regression problems exhibit some kind of calender time dependency and it is often of interest to predict the behavior of the regression function along this calender time direction. his can be formulated as a regression model with an added latent time series and the task is to be able to analyse this series. In this paper we engage this through a two step procedure, firstlywetreatthetimedependent elements as parameters and estimate them in the two-sided analysis of variance setup, secondly we use the estimated time series as predictor of the latent time series. An application to risk theory is discussed. Key Words: regression, time series, risk theory. 1 Introduction We start by defining the standard linear regression model with common slope and different intercept in each group, where the groups correspond to changing calender years. his model is well studied and estimation can be carried out by Ordinary Least Squares. If we reformulate the model such that the intercept term in each year is stochastic, we can use the intercept estimates from the linear model as predictions for the latent time series. We wish to draw inference about the parameters in the model for the time series, and the theory of the estimator for the first model gives us a starting point for this task. he obvious way to predict the latent time series would be to use conditional expectations in some form but these are often difficult to calculate under general assumptions and, more importantly, they require a fixed model for the time series. Our method enables separating the time series analysis from the prediction of the series, and gives the opportunity to wait deciding on the distribution of the time series until one has the predicted series. Suppose an unbalanced two-sided array of data is observed with a dependent variable Y ti and a q-dimensional covariate X ti so t =1,... is a time index while the number of individuals i =1,...,n t vary over time. A standard two-sided analysis of variance model can be formulated as Y ti = α t + Xtiγ 0 + e ti, 1

2 where the error terms, e ti, are independent over individuals, i, and time t, and identically distributed with mean zero and variance η 2. he least squares estimators for α t and γ are then found by minimising the sum of squared deviations P Pn t n Yti 0 o 2 P Y t Xti X t γ + n t ³Y t X 0 tγ α t 2, where Y t and X t represent averages over individuals with a common time index t. his gives the estimators ½ ¾ P Pn t 1 0 P Pn t ˆγ = Xti X t Xti X t Xti X t Yti, ˆα t = Y t X 0 tˆγ. he total number of observations is denoted n = P n t. We will now formulate a latent time series regression model for the same array of data. his is formulated as Y ti = µ t + Xtiβ 0 + ε ti, (1) where µ 1,...,µ is the latent time series. As in the two-sided analysis of variance model we estimate the regression coefficient β and the latent time series µ t by ½ ¾ P Pn t 1 0 P Pn t ˆβ = Xti X t Xti X t Xti X t Yti, ˆµ t = Y t X 0 tˆβ. Imagining a situation where is large and n t even larger we will show three types of results: (i) he regression function can be estimated and analysed in the same way as if the time component had been deterministic. (ii) he latent time series can be estimated very accurately. (iii) Oracle efficiency: the estimated time series can be analysed as the time series itself, had it be given by an oracle. 2 Analysis of the latent time series regression model he three types of results for the latent time series regression model are now discussed in detail. o formulate the conditions precisely, let k k be the spectral norm so kak 2 equals the maximal eigenvalue of A 0 A for a matrix A and reduces to A 0 A for a vector A. 2.1 he regression estimator he first result shows that the regression estimator ˆβ is asymptotically normal in similar way to what would arise in a two-sided analysis of variance model. he result is formulated for large sample length of the time series,, in terms of a Liapounov Central Limit heorem. 2

3 he essence of the conditions is that the innovations ε ti, given the regressors, have a standardised distribution, while the regressors X ti are allowed to vary over both indices as long as the information is spread across the individuals and over time. Importantly, the limiting distribution of ˆβ does not depend on neither the nature of the latent time series nor is the number of individuals at each time point, n t, required to increase with. heorem 1 Suppose that for some sequence a depending on, it holds that (i) the variables satisfy (a) the arrays (ε ti,x ti, for i =1,...,n t ) are independent over t, (b) the pairs (ε ti,x ti ) are independent over i, (ii) the innovations satisfy, for some k<, (a) E(ε ti X ti )=0, (b) Var(ε ti X ti )=σ 2, (c) max t,i E(kε ti k 4 X ti ) <k. (iii) the regressors satisfy, for a positive definite matrix Σ, (a) a 2 E P P nt 0 Xti X t Xti X t Σ, (b) a 2 P P nt 0 P Xti X t Xti X t Σ, (c) a 4 E P n P nt t X ti X t 4 0. hen, for, a ³ˆβ β P N 0,σ 2 Σ Prediction of the time series We will seek to apply time series analysis to the estimated time series ˆµ 1,...,ˆµ with a view to predicting future values of µ t. his will be possible when the estimated time series ³ ˆµ t = Y t X 0 tˆβ = µ t + X 0 t β ˆβ + ε t (2) is close to latent time series µ t uniformly over time. his issue is investigated in heorem 2below. Notingthatˆβ β is invariant to changes in the value of β the predicted time series ˆµ t inherits this property. In addition to the assumptions of heorem 1 the heorem 2 concerning ˆµ t requires that the number of individuals n t at each point in time growths faster than. heorem 2 Suppose the assumptions of heorem 1 are satisfied, and that, (iv) max t (n 1 t )=o( 1 ), (v) a 2 E P n 1 t P nt kx tik

4 hen, for, P (ˆµ t µ t ) 2 =o P (1). 2.3 Analysis of the predicted time series As an example of a time series model for µ t consider the first order autoregression µ t = ρµ t 1 + ξ t (t =2,...,) (3) conditional on µ 1, where the innovations ξ t constitute a martingale difference sequence with variance ω 2. Had the latent time series µ t been observed statistical analysis would typically be based on the standardised least squares estimator t = P ξ tµ t 1. (4) 1/2 ³ P µ2 t 1 he asymptotic properties of this statistic are well known: when ρ < 1 it is normal distributed due to a Central Limit heorem argument for martingale differences, see Hall and Heyde (1980, p.172), when ρ =1it has a non-standard distribution that can be represented using Brownian motions, see Dickey-Fuller (1979), whereas when ρ > 1 and the innovations are independent and normal it is asymptotically normal distributed, see Anderson (1959). he parameters ρ, ω 2 can be estimated by least squares estimators based on the estimated series ˆµ t given as P ˆρ = ˆµ tˆµ t 1 P, ˆω 2 = 1 P ˆµ2 1 t 1 ˆµ2 t ˆµ tˆµ t 1 2 P ˆµ2 t 1. ³ P he following result shows that for very weak assumptions to the innovation process the estimators ˆρ, ˆω 2 will have properties similar to what could have been achieved had the latent time series been given to us by an oracle. In particular the series ˆµ t can be analysed well regardless of whether the latent time series is stationary, or non-stable. heorem 3 Suppose, as stated in heorem 2, that P (ˆµ t µ t ) 2 =o P (1). Suppose, further, that (ξ t ) is a martingale difference sequence with respect to a filtration (F t ) satisfying E(ξ 2 t F t 1 )=ω 2 and max t E( ξ t 3 F t 1 ) <. hen, for any ρ R and 4

5 it holds P ˆµ2 t 1 = P µ2 t 1 {1+o P (1)}, ³ P P 1/2 ˆt = ˆµ2 t 1 (ˆρ ρ) = ξ tµ t 1 {1+o 1/2 P (1)} +o P (1), 3 Risk heory ³ P µ2 t 1 ˆω 2 = 1 P ξ2 t +o P (1). he model presented above has an application to Risk heory. In the standard setting of Risk heory, see Bühlmann (1970), Beard, Pentikainen and Pesonen (1984) or Norberg (1990), the focus concentrates on the investigation of the claim process, XN τ C τ = Z i, (τ 0), where N τ is a counting process representing the number of claims occurred in a time interval [0; τ] and Z i is the size of the ith claim. In practice, it is of interest to study the growth of claim sizes in order to predict future liabilities. It is therefore natural to include a time dependency in the model for Z i to describe this pattern, and this is where the latent time series regression model proves its worth. Grouping the claims in years, gives us the opportunity to use the setup from the previous section, C τ = τx Xn t Y ti, (τ N 0 ), where n t is the number of claims in year t and, Y ti is the size of the ith claim in the tth year, following the model defined by (1) and (3). A slight modification has to be made to heorems 1 and 2 when using the latent time series regression model in Risk heory, to allow for stochastic behavior of n t.he following holds. heorem 4 Suppose that the Assumptions (i, a), (iii), (v) hold, while Assumptions (i, b), (ii) are modified to hold conditionally on n t, and (iv) is replaced by (iv 0 )max t E(n 1 t n t > 0) = o( 1 ). hen, for, a ³ˆβ β P N 0,σ 2 Σ and P (ˆµ t µ t ) 2 =o P (1). 5

6 he assumptions of heorem 4 can be shown to be valid in many specific situations. In many Risk heory models it is assumed that the number of individuals is Poisson(λ t )- distributed. Assuming that max λ 1 t =o( 1 ) it is clear that Assumption (iv 0 ) is satisfied. o see this, use Jensen s inequality, E n 1 t n t > 0 {E (n t n t > 0)} 1 = 1 exp ( λ t) λ t 1 λ t =o 1. 4 Illustration he asymptotic theory is now illustrated by a simulation, where the covariates X ti are chosen to be trending over time. We will assume that =20periods are considered and the numbers of individuals, n t, are independently Poisson (λ)-distributed where λ =100, so it is not unreasonable to assume that λ 1 =o( 1 ). he regressors X ti are assumed to be Γ (δ t, 1)-distributed where the shape parameter δ t =1+t/10 growths linearly over time and the regression coefficient is chosen to be β =0due to the invariance with respect to β. he conditional distribution of the errors ε ti given regressors is assumed to be N(0, 3 2 ), while the innovations ξ t are N(0, 1 2 ) and thus have much smaller variation than the errors, ε ti. Finally, assume an autoregressive coefficient of unity, ρ =1. In this situation the Assumptions (i), (ii), (iv 0 ) of heorem 4 are trivially met. Choosing a 2 = λ 2 and P 2 δ t Σ =1/20 some tedious calculations presented in the Appendix show that Assumptions (iii) and (v) are met. able 1. Simulation results λ value be P (ˆµ t µ t ) be P ε2 t be(bt t) able 1 reports the results of the simulation study. For each of five different values of the Poisson parameter λ the expectation of three statistics were simulated using 5000 repetitions. For the smallest value, λ =10, it happened 6 times that n t took the value 0 which was then replaced by 1. he first row of able 1 investigates the convergence P (ˆµ t µ t ) 2 =o P (1) stated in heorem 4. he expectation of this term appear to be of order λ 1. Even for a modest value of λ it is small compared to the expected sum of squared innovations of the latent time series, which is E P ξ2 t =20. he proof of the convergence result uses the decomposition ˆµ t µ t = X 0 t(β ˆβ)+ε t, see (2). While both terms vanish a comparison of the firsttworowsofable1indicatesthatthetermε t is dominating in accordance with the proof. 6

7 In the third row of able 1 the convergence of heorem 3 is investigated. he mean square error b E(bt t) 2 is simulated and reported in able 1. his statistic also appears to be vanishing fast. Once again, it is relatively small compared to the expectation Et 2, in that the limiting distribution of t 2 has expectation of about 1.142, see Nielsen (1997). When λ is larger than about 40 and thus somewhat larger than this mean square error is therefore of relatively minor importance. 5 Discussion We have introduced a latent time series regression model and extended it to a specific risk theoretical setting. While the concept of a latent time series is new in risk theory, it has some history in the area of mortality estimation and in panel data analysis. It was introduced in mortality estimation in a widely cited paper by Lee and Carter (1992). First, they estimate how the mortality depends of three non-parametrically specified functions, two depending on age and one depending on calendar time. his, their first step, is rather similar to our first step estimating our two components from a standard analyses of variance. Afterwards, they analyse their non-parametric function depending on calendar time as a simple autoregressive function much like we investigate our calendar effect as a time series. he major contribution of this paper is to consider the latent time series to be part of the original model formulation and to formulate standard theorems of mathematical statistics to investigate the properties of such a model. One interesting future line of research could be to treat the important mortality model of Lee and Carter in a similar way. Another interesting future direction of research could be to investigate the possibility of including such a latent time series in risk theoretic models of outstanding insurance liabilities based on aggregated data, see England and Verrall (2000) for a review of such models. he proposed model is also related to some recent latent time series models for balanced panel data, so n t is constant over t. Forni, Hallin, Lippi and Reichlin (2000) consider a model of the form Y ti = γ i (L) u t + ε ti, so γ i (L) u t is a moving average process where the lag polynomials γ i (L) have coefficients depending on i while the innovations u t are common for all individuals. hey propose an estimator for the latent component γ i (L) u t. Inthesimplesituationwhereγ i (L) =1 this is done in a similar way to our formulation of ˆµ t by estimating u t by Y t. Pesaran (2003) looks at the model Y ti = α i + β i Y t 1,i + γ i f t + ε ti. and analyses the regression function α i + β i Y t 1,i withaviewtotestingthejointunit root hypothesis β i =1for all i. his is done in a similar way to our model in that the latent time series γ i f t is replaced by c i +d i Y t 1 where c i and d i are auxiliary parameters. he results presented in this paper could possibly inspire the formulation of theoretical properties of these more complicated models. 7

8 6 Acknowledgements Discussion with D.R. Cox is gratefully acknowledged. All simulations has been performed using R A Proofs Proof of heorem 1. Consider the triangular array y t = a 1 P nt Xti X t εti and show that the Liapounov conditions of Davidson (1994, p. 373) are satisfied: (1) By Assumption (i, a) then y t is independent over t. (2) By Assumptions (i, b), (ii, a) then E (y t )=0. (3) By Assumptions (i, b), (ii, b), (iii, a) then s 2 = Var (y t ) σ 2 Σ. (4) Note first, that with z ti = X ti X t it holds Ã! a 4 E ky,t k 4 n t = E P 4 2 P z ti ε ti = E ε ti ε tj ztjz 0 ti. Conditioning on the regressors and n t and using Assumption (ii) this is bounded by a 4 E ky,t k 4 P k n t kz ti k 4 +2σ 2 P kz ti k 2 kz tj k 2. i6=j By Jensen s inequality this in turn is bounded, for some K<, by a 4 E ky,t k 4 Pn t Kn t kz ti k 4. hus the Liapounov condition P E ky,tk 4 0 is met under Assumption (iii, c). Proofofheorem2. Due to the expression (2) and Markov s inequality it suffices to argue that P ε2 t and (β ˆβ) P 0 X tx 0 t(β ˆβ) vanish. o show that P ε2 t vanish use Chebychev s inequality to see that ³ P P ε2 t >δ δ 1 E P ε2 t = δ 1 P Eε2 t. It therefore suffices to show Eε 2 t =o( 1 ). Using first Assumption (i, b) andthenthe bound to n t in Assumption (iv) it holds Eε 2 t = Eσ 2 /n t =o 1. It follows from heorem 1 that (ˆβ β) = O P a 1 so it suffices to show that a 2 P X tx 0 t vanishes. Using first Chebychev s inequality, then twice the triangle inequality, and Jensen s inequality ³ P δp X tx 0 t >δ E P X tx 0 t E P X t X 0 t = P E X t 2 i,j P E µ 1 n t P nt kx t,ik 2 P E 1 n t P nt kx t,ik 2. 8

9 his is of order o(a 2 ) as desired due to Assumption (v). Proofofheorem3. he key to this result is to show µ P 1 µ 2 t =O P ( 1 ) and P ξ 2 t =O P ( ). (5) For the stated assumptions to ξ this holds for any ρ R according to Lai and Wei (1983, heorems 1 and 3). It is first shown that P ˆµ2 t 1 = P µ2 t 1 {1+o P (1)}. Let d t =ˆµ t µ t and use Cauchy-Schwarz s inequality to see P ˆµ2 t 1 P ³ P P 1/2 µ2 t 1 +2 P µ2 t 1 d2 t 1 + d2 t 1 heorem 2 and the first property in (5) give the desired result. urning to the first expression of interest the numerator satisfies P ˆµt ρˆµ t 1 ˆµt 1 = P (ξ t + d t ρd t 1 ) P µ t 1 + d t 1 = ξ tµ t 1 + R, where R is some remainder term. By the triangle inequality and Cauchy-Schwarz s inequality R is found to be of the order ³ P P ξ2 t d2 t 1 1/2 +(1+ ρ ) ³ P µ2 t 1 P d2 t 1/2 +(1+ ρ ) P d2 t. heorem 2 and (5) then show R =o P ( P µ2 t ) 1/2. ogether with the result for denominator shown above this implies the desired result. he estimator ˆω 2 satisfies ( 1)ˆω 2 = P ˆµt ρˆµ t 1 2 n P ˆµt ρˆµ t 1 ˆµt 1 o 2 P ˆµ2 t 1 = P 2 ˆµt ρˆµ t 1 (ˆρ ρ) 2 P ˆµ2 t 1. he first term equals P (ξ t + d t ρd t 1 ) 2 whichinturnequals P ξ2 t +o P ( ) by an argument as that for the numerator of the first expression. he second term is asymptotically equivalent to the statistic (4) due to the firstpartofthisheorem, whichinturnisofordero P ( ) according to Nielsen (2001, Lemma A3). Proofofheorem4. heproofsofheorems1,2canbeadapted: Asymptotic distribution of ˆβ. (a) It suffices to argue that the bound to Eε 2 t holds. By iterated expectations Eε 2 t = E ε 2 t n t > 0 P (n t > 0) E ε 2 t n t > 0 = E E ε 2 t n t > 0,n t nt > 0 ª. 9

10 his equals σ 2 E n 1 t n t > 0 ª due to Assumptions (i 0 ), (ii 0,b). his is in turn bounded by (1+a) accordingtoassumption(iv 0 ). (b) his argument can be done in the same way as (a) by conditioning on n t. (c) Same argument as in the proof of heorem 2. (d) he argument concerning the bound to EX 2 t has to be adapted along the lines of (a). Finally it is argued that the Assumptions (iii) and (v) aremetintheillustrationof Section 4. First note that En t = λ, En 2 t = λ 2 + λ, E (X ti n t )=δ t, E Xti n 2 t = δ 2 t + δ t, E (X ti δ t ) 2 ª n t = δt, κ t = E (X ti δ t ) 4 ª n t =3δ 2 t +6δ t. Noting that n t Xti X t =(nt 1) (X ti δ t ) P k6=i (X tk δ t ) it is seen that n Xti o E X t 2 nt n Xti o E X t 4 nt ª = δ t 1+O n 1 t, = κ t 1+O n 1 t ª + δ 2 t O n 1 t while using that X ti X t =(Xti δ t ) X t δ t it is seen that, for i 6= j, n Xti 2 o E X t Xtj X t 2 nt = δ 2 t 1+O n 1 t urning to the Assumptions it then follows for (iii, a) that = a 2 1 λ 2 P P P E nt n Xti o E X t 2 nt δ t En t {1+o(1)} = 1 2 Next, for (iii, b) it suffices to prove that where E a 2 P ½ nt, ª + κt O n 2 t P δ t {1+o(1)} Σ. ¾ 2 P 2 Xti X t λδt = I 1 + I 2 0,. I 1 = I 2 = ½ 1 P nt ¾ P 2 2 λ 2 E Xti X 4 t λδt = 1 P λ 2 κλ {1+o(1)} 0, 4 ½ 1 P nt ¾ ½ P 2 ns ¾ P 2 λ 2 E Xti X 4 t λδt E Xsi X s λδs =0. s6=t 10

11 For (iii, c) and (iv) it holds a 4 P Pn t En t a 2 P n Xti o E X t 4 nt = 1 λ 2 4 En 1 t Pn t E Xti 2 n t = 1 λ 2 P κλ 2 {1+o(1)} 0, P δ 2 t {1+o(1)} 0. B References Anderson,.W. (1959). On asymptotic distributions of estimates of parameters of stochastic difference equations. Annals of Mathematical Statistics 30, Beard, R.E., Pentikainen,., Pesonen, E. (1984) Risk heory, Chapmann and Hall, London, 3rd edition Bühlmann, H. (1970) Mathematical methods in risk theory, Springer-Verlag, Heidelberg. Davidson, J.(1994) Stochastic limit theory, Oxford University Press. Dickey, D.A. and Fuller, W.A. (1979) Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association 74, England, P. and Verrall, R.J. (2002). Stochastic claims reserving in general insurance. British Actuarial Journal 8, Forni, M., Hallin, M., Lippi, M. and Reichlin, L. (2002) he generalized dynamicfactor model: identification and estimation. Review of Economics and Statistics 82, Hall, P., and Heyde, C.C. (1980). Academic Press, San Diego. Martingale Limit heory and Its Applications. Lai,.L. and Wei, C.Z. (1983b) Asymptotic properties of general autoregressive models and strong consistency of least-squares estimates of their parameters. Journal of Multivariate Analysis 13, Lee, R. and Carter, L. (1992) Modelling and forecasting U.S. mortality. Journal of the American Statistical Association 87, Nielsen, B. (1997) Bartlett correction of the unit root test in autoregressive models. Biometrika 84, Nielsen, B. (2001) he asymptotic distribution of unit root tests of unstable autoregressive processes. Econometrica 69,

12 Norberg, R. (1990) Risk theory and its statistics environment. Statistics 21, Pesaran, M.H. (2003) A simple panel unit root test in the presence of cross section dependence. Cambridge University DAE Working Paper Sundt, B. (1993) An introduction to non-life insurance mathematics, Veröf. des Inst. für Versich., Universität Mannheim, 28, Karlsruhe, 3rd edition 12