Integer Valued AR Processes with Explanatory Variables

Transcription

1 Integer Valued AR Processes with Explanatory Variables V. Enciso-Mora, P. Neal & T. Subba Rao First version: 2 December 2008 Research Report No. 20, 2008, Probability and Statistics Group School of Mathematics, The University of Manchester

2 Integer valued AR processes with explanatory variables Víctor Enciso-Mora (University of Manchester, Peter Neal (University of Manchester and T. Subba Rao (University of Manchester November 7, 2008 Address: School of Mathematics, University of Manchester, Alan Turing Building, Oxford Rd, Manchester, M3 9PL, United Kingdom. Abstract Integer valued AR (INAR processes are perfectly suited for modelling count data. We consider the inclusion of explanatory variables into the INAR model to extend the applicability of INAR models and give an alternative to Poisson regression models. An efficient MCMC algorithm is constructed to analyze the model and incorporates both explanatory variable and order selection. The methodology is illustrated by analyzing monthly polio incidences in the USA and claims from the logging industry to the British Columbia Workers Compensation Board Keywords: Integer valued time-series, Reversible jump MCMC, count data, explanatory variables. AMS Primary: 62M0; AMS Secondary: 65C40. Introduction There has been considerable recent interest in developing models and statistical inference for integer valued time series. The modeling has primarily been divided into two approaches, Poisson regression models, see for example Davis et al. (2003 and Davis et al. (2005 and integer valued ARMA models, see for example McCabe and Martin (2005 and Jung and Tremayne (2006. The author would like to thank CONACYT (Consejo Nacional de Ciencia y Tecnología, México for funding this research. corresponding author

3 Poisson regression models assume that an integer valued time series {X t } given explanatory variables w t is generated by X t Po(exp(wt T β, where β is a vector of parameters. (Throughout this paper all vectors are taken to be column vectors. This has been extended in Davis et al. (2000 to incorporate a more explicit dependence between the {X t } with X t W t Po(exp(wt T β + W t, where {W t } is a real valued stationary process with E[exp(W t ] =. This model is very flexible and statistical inference is usually relatively straightforward. In contrast, integer valued AR (INAR models are discrete analogues of the (standard, real valued AR model. The INAR(p model is given by p X t = α j X t j + Z t, (. j= where the {Z t } are independent and identically distributed Poisson random variables and denotes a binomial operator. That is, for any 0 γ, and nonnegative integer random variable Y, γ Y = D Bin(Y, γ. Statistical inference for INAR models is generally more complicated than for the Poisson regression models and most attention has been devoted to the INAR( model. However, recently in Neal and Subba Rao (2007 an efficient MCMC algorithm has been developed for INARMA processes of known AR and MA orders with extensions to unknown AR and MA orders given in Enciso-Mora et al. (2008. The main benefit of the INARMA models over the Poisson regression models is their similarity to standard ARMA models. In fact, the stationary and invertibility conditions of ARMA models hold for INARMA models, see Latour (997 and Latour (998. The aim of this paper is to extend the analysis of INAR models to incorporate explanatory variables. That is, we look to increase the flexibility of INAR models whilst maintaining the AR structure of the model. Thus seeking an alternative to Poisson regression models for integer valued time series. Moreover, the two approaches are complimentary and are appropriate to different types of data with the Poisson regression models being multiplicative in structure through e Wt, whilst the INAR model has additive structure through the binomial operator. As with our earlier work, Neal and Subba Rao (2007 and Enciso-Mora et al. (2008, an MCMC algorithm will be introduced to perform statistical inference for the model. The only previous work, that we are aware of, which incorporates explanatory variables into an INAR model is Brännäs (995. The paper is structured as follows. The model is introduced in Section 2. In Section 3, the MCMC algorithm is constructed. The algorithm includes explanatory variable and AR order selection which enables us to determine the most parsimonious model. Finally, in Section 4 the MCMC algorithm is applied to two real-life data sets, the total number of monthly cases of polio in the USA from January 2

4 970 to December 983 and the total number of monthly claims from the logging industry to the British Columbia Workers Compensation Board from January 985 to December Model Suppose that for each time point there are r explanatory variables. For t Z and i =,2,...,r, let w t,i denote the value of the i th explanatory variable at time t and let w t = (w t,0,w t,,...,w t,r, where w t,0 = for all t Z. Let p denote the maximum AR order of the model and for j =,2,...p let δ j = (δ j,0,δ j,,...,δ j,r. Let γ = (γ 0,γ,...,γ r. Then for t Z, the INAR(p model with explanatory variables is given by X t = where Z t Po(λ t, p α t,j X t j + Z t, (2. j= α t,j = { + exp(w T t δ j } (2.2 and λ t = exp(w T t γ. (2.3 The special case where p = was considered in Brännäs (995. In Brännäs (995, separate explanatory variables were used for α t, and λ t and is covered by the current set up by fixing some of the components of δ j and γ equal to 0. The binomial operator could be replaced by some generalised Steutel and van Harn operator (see, Steutel and van Harn (979 and Latour (997. Also alternative forms for α t,j and λ t can be used so long as for all t Z and j =,2,...,p, 0 α t,j and λ t 0. However, for ease of exposition we restrict attention to the model defined by (2., (2.2 and (2.3. The model defined by (2. is the full model. It will often be the case that a simpler model which does not include all the explanatory variables or all the AR terms will suffice. Therefore we assume that there exists R {,2,...,r} and A {,2,...,p} such that for i A, α t,i = 0 and for j R, δ i,j = 0 and γ j = 0, where A and R are unknown and are parameters in the model to be estimated. Finally, it is important to note that the explanatory variables can be used to model a linear trend or periodicity. Unlike standard AR processes for INAR processes trends and periodicity can not easily be removed by transforming the original time series since any transformation would need to preserve 3

5 the integer nature of the data. Therefore if there are trends and periodicity in the data these can be incorporated through explanatory variables. Suppose that α t,j 0 for all t Z and j =,2,...,p. Then X t Po(exp(wt T γ and there is no dependence between successive time points. This is equivalent to the Poisson regression model with e Wt for all t Z. Thus the Poisson regression model and the INAR model reduce to the same model in the absence of temporal dependence. Therefore the differences in modelling arise from the ways in which temporal dependence are included in the model, as a latent stationary process in Poisson regression models and as an observation driven binomial operator in INAR models. 3 MCMC algorithm 3. Likelihood of INAR(p model with explanatory variables For n, let x = (x p,x 2 p,...,x n denote observations from the integer-valued stochastic process {X t }. As in Neal and Subba Rao (2007 and Enciso-Mora et al. (2008, statistical inference is greatly facilitated by the use of data augmentation. For t Z and j =,2,...,p, let Y t,j α t,j X t,j. For t, let y t = (y t,,y t,2,...,y t,p, y = (y,y 2,...,y n and z t = (z,z 2,...,z t. Note that z t = x t p j= y t,j. For t =,2,...,n, let x t = (x p,x 2 p,...,x t and let w = (w p,w 2 p,...,w n. Note that each of Y t,,y t,2,...,y t,p and Z t are independent given (γ,δ,x t. Thus for t =,2,...,n, λ z t t z f t (y t,z t x,w,δ,γ t! e λt { p (xt i } y i= y t,i α t,i t,i ( α t,i xt i yt,i z t + p i= y t,i = x t, (3. 0 otherwise. Therefore f(y,z n x,w,δ,γ = n f t (y t,z t x,w,δ,γ t= n t= λ zt t z t! e λt p i= {( xt i y t,i α yt,i t,i ( α t,i xt i yt,i }, (3.2 subject to the constraint z t + p i= y t,i = x t for t =,2,...,n. For i = 0,,...,r, γ i and δ j,i (j =,2,...,p are real-valued. Therefore in the absence of conjugate priors, we assign independent Gaussian priors to each of γ i (i = 0,,...,r and δ j,i (j =,2,...,p; i = 0,,...,r. For simplicity of exposition we take π(γ i,π(δ j,i N(µ,σ 2 P and throughout this paper take µ = 0 and σ P =. As mentioned in Section 2, we assume that there is a maximum (fixed model order p and a (unknown subset A {,2,...,p} such that for j A, α t,j 0 and for j A, α t,j = 0, and hence, y t,j = 0. Therefore we 4

6 shall make inference about A with similar analysis for real-valued AR processes considered in Troughton and Godsill (997. The prior chosen for A is π(a n A A {,2,...,p}. (3.3 The prior on A penalises models with larger values of A and is motivated by the BIC-penalisation, Schwarz (978, which has been used effectively in Enciso-Mora et al. (2008. Furthermore, we will not assume that all of the explanatory variables are necessary and will perform explanatory variable selection. That is, we assume that there exists R {,2,...,r} such that for i R, γ i 0 and δ j,i 0 (j A and for i R, γ i = 0 and δ j,i = 0 (j A. Finally, we impose a BIC-penalisation prior on R, π(r n R R {,2,...,r}. (3.4 Note that we assume that there is an intercept term, so γ 0 0 and δ j,0 0 (j A. 3.2 Details of the MCMC algorithm The MCMC algorithm naturally separates into four parts, parameter updates γ and δ, updating (y,z n, updating R and updating A. We shall describe each part of the MCMC algorithm in turn with the latter two parts requiring reversible jump MCMC, Green (995. The algorithm is then run for the desired number of iterations to obtain a sample from the posterior distribution of (A, R,γ,δ. For j A and i R, we update, one at a time, each γ i and δ j,i using random walk Metropolis. For example, we propose a new value for δ j,i, δ j,i N(δ j,i,σδ;j,i 2, where σ δ;j,i is chosen based upon pilot runs of the algorithm such that approximately 40% of proposed moves are accepted, see Gelman et al. (995. The acceptance probability is straightforward to calculate using (3.2 and the prior for δ j,i. For t =,2,...,n, we update one set of components (y t,z t at a time. We use an efficient independent sampler which is essentically identical to that introduced in Neal and Subba Rao (2007. For j A, sample y t,j Bin(x t j,α t,j, resampling if x t j A y t,j. Set z t = x t j A y t,j and accept the proposed values (y t,z t with probability { min, z t! z t! λz t t zt }. Explanatory variable selection is performed as follows. Select K uniformly at random from {,2,...,r} and propose to change the inclusion/exclusion status of explanatory variable K. That is, if K R, we propose that K R and if K R, we propose that K R. We shall describe the inclusion of a 5

7 explanatory variable move in detail with the exclusion of a explanatory variable move simply being the reverse procedure. In order to use reversible jump MCMC it is necessary to define a bijection between the parameters in the current model and the parameters in the proposed model. This can be done by extending the state space of one or both of the models to include auxiliary variables. Therefore for j A, let S j N(0,σ 2 R and let S N(0,σ 2 R, where σ R is chosen to produce efficient between model moves and σ R = is found to be more than adequate for the examples considered in this paper. We set δ j,k = S j (j A and γ K = S with all other parameters remaining unchanged. Therefore the Jacobian for the transformation from (δ,γ,s, S to (δ,γ is. The proposed move affects {α t,j } and {λ t }, so the acceptance probability for the proposed move is the minimum of and n ( λ zt ( t α yt,j ( e λt λ t t,j α xt j y t,j t,j λ t= t α t,j α t,j j A ( σ R (δ j,k 2 exp n σ P 2σ 2 (δ j,k 2 j A R 2σP 2 σ ( R (γ exp K 2 σ P 2σR 2 (γ K 2 2σP 2. (3.5 The latter term is the ratio of the proposal and prior for the parameters added into the model as a result of including the explanatory variable and the multiplier / n comes from the prior on R. For the case σ R = σ P = the latter term is identical to. Order determination for INAR models has been considered in detail in Enciso-Mora et al. (2008. The procedure used here is based upon Enciso-Mora et al. (2008, Section 3 adjusted to take account of the inclusion of explanatory variables and the form of A. It was noted that from equations (3.2 and (3.3 of Enciso-Mora et al. (2008 a certain function of the parameters, corresponding to E[X t ] was well estimated from the data and this was used to devise an efficient order switching reversible jump MCMC algorithm. This idea is the motivation for the proposed moves in the intercept terms given below. Select L uniformly at random from {,2,...,p} and propose to change the inclusion/exclusion status of the lag L term in the AR processes. That is, if L A, we propose that L A and if L A, we propose that L A. We shall describe the inclusion of the lag L term move in detail with the exclusion of the lag L term move simply being the reverse procedure. Select M uniformly from A, if A = an alternative scheme is required which we will describe later. We propose to split the AR terms, so that the AR contribution from the M lag term in the current model is divided between the L lag and M lag terms in the proposed model. In Enciso-Mora et al. (2008, this is done by drawing U U[0,] and setting α L = Uα M and α M = ( Uα M. We propose δ L,0 and δ M,0 based upon the approach used in Enciso-Mora et al. (

8 Specifically, draw U U[0,] and set + exp(δ L,0 = U + exp(δ M,0 (3.6 + exp(δ M,0 = ( U + exp(δ M,0. (3.7 For j R, let V j N(0,σO 2, where σ O can be tuned to produce efficient between model moves (σ O = was found to be adequate for the examples in this paper and propose δ L,j = V j and δ M,j = δ M,j V j. The above defines the proposed changes in parameter values which necessitates changes to the augmented data. For t =,2,...,n, we propose ( y t,m Bin α t,m y t,m, α t,m + α t,l (3.8 y t,l = y t,m y t,m, (3.9 which splits y t,m based upon the proposed parameter values. This scheme is again similar to Enciso- Mora et al. (2008, Section 3.2. The transformation form (δ,y,u,v to (δ,y factorises with only the transformation given by (3.6 and (3.7 having a Jacobian other than. The Jacobian, J, for the transformation from (δ M,0,U to (δ M,0,δ L,0 is given by J = ( UU exp(δ M,0 { + exp(δ M,0 } {U + exp(δ M,0 }{ U + exp(δ M,0 }. (3.0 Increasing the AR order of the model by leads to the ratio of the priors for the two models being / n. Therefore the acceptance probability for the proposed move is the minimum of and J ( σ O (δ j,l 2 ( exp n σ P 2σ 2 (δ j,l 2 j A O 2σP 2 exp 2σP 2 (δj,m 2 (δ j,m 2 n t= ( xt M y t,m (α t,m y t,m ( α t,m xt M y t,m ( xt L y t,l ( xt M y t,m (αt,m yt,m ( α t,m xt M yt,m ( y t,m y t,m (α t,l y t,l ( α t,l xt L y t,l (ωt y t,m ( ωt yt,m y t,m, (3. where ω t = α t,m /(α t,m + α t,l. The reverse move where the lag L term is excluded from the AR model is dealt with by reversing the above procedure. The inclusion of an AR lag term when A = requires a slightly different procedure. In this case we adjust the noise term to compensate for the inclusion of an AR term, see Enciso-Mora et al. (2008. Let U U[0,] and set /(+exp(δ L,0 = U and exp(γ 0 = ( Uexp(γ 0. For j R, let V j N(0,σ 2 O and propose δ L,j = V j. Then change the data augmentation by y t,l Bin(x t x t,α t,l and z t = x t y t,l. Finally, note that the algorithm described above generally performs well. However, the performance of the algorithm can be improved by normalizing the explanatory variables, so that for i =,2,...,r, 7

9 n n i= w t,i = 0 and n n i= w2 t,i =. For the examples, considered in Section 4 normalization was not necessary. Since normalization is a linear operation the algorithm can be used to obtain parameter estimates for the normalized explanatory variables which can then be transformed into parameter estimates associated with the original explanatory variables. 4 Data analysis We apply the MCMC algorithm of Section 3 to two real-life data sets. The first data set is taken from Zeger (988 and consists of the monthly total number of polio cases in the USA from January 970 to December 983. The second data set is taken from Zhu and Joe (2006 and consists of the monthly number of claims of short-term disability benefits made by injured logging industry workers to the British Columbia Workers Compensation Board from January 985 to December Polio data The polio data has been studied by Zeger (988 and Davis et al. (2000 using Poisson regression models and therefore provides an ideal opportunity to compare INAR models with Poisson regression models. Therefore we follow these earlier papers in taking w t = ( t, 000,cos ( 2πt 2,sin ( 2πt 2,cos ( 2πt 6,sin ( 2πt where t = t 73. Thus there is assumed to be in a linear trend (with intercept January 976 and two periods, one of 6 months and the other of 2 months. Furthermore, for a full comparison with Davis et al. (2000 we assume that the model is INAR( model and all the explanatory variables are included in 6, the model. Therefore for this data we do not consider either order or model selection. We ran the MCMC algorithm to obtain a sample of size 00,000 from the posterior distribution of the parameters following a burn-in of 0,000 iterations. The results are presented in Table below. It should be noted that the estimates of γ are similar to those obtained for the Poisson regression model in Davis et al. (2000 and in even closer agreement to those of Zeger (988. This is to be expected given the high value of the intercept for δ,0 which implies α t, is small for most t. That is, the observations are driven 8

10 by the explanatory variables and the Poisson innovations rather than the AR dependence. Explanatory variable γ δ Intercept 0.079( (0.776 Trend 3.242( (2.09 cos ( 2πt ( (0.846 sin ( 2πt ( (0.907 cos ( 2πt ( (0.850 sin ( 2πt ( (0.864 Table. Means (standard deviations of parameter coefficients for the INAR( model. Finally, to test the models appropriateness we studied the predictive capabilities of the model. At each iteration of the MCMC algorithm we simulated a realization for the number of monthly cases for each month in 983 given the parameter estimates and the true number of cases for the previous month. We then compared the sample means from the posterior distributions of the number of monthly cases for each month in 983 with the actual monthly figures. For the Poisson regression model Monte Carlo estimation of {W t } was performed using the parameter estimates obtained in Davis et al. (2000 since the data is informative about the process {W t }. Since the driving force is the explanatory variables rather than autoregressive dependence, the results are unsurprisingly rather similar for the two models. Jan Feb March April May June July Aug Sept Oct Nov Dec Actual IN AR( Poisson reg Table 2. Actual monthly total number of cases for 983 compared with mean predictions. 4.2 Benefit claims data Exploratory data analysis gives support for an annual period being included in the model. Further analysis reveals that July has the highest number of claims and the summer months (May to November produce significantly more claims than the winter months (December to April. Finally, there is evidence to suggest that the number of claims might be decreasing over time. Therefore we take the following explanatory variable vector w t = (, t 20,cos ( 2πt,sin 2 ( 2πt,s t, (4. 2 9

11 where s t = for the summer months (May to November inclusive and s t = 0 otherwise. The explanatory variables are normalized before applying the MCMC algorithm. Since the data consists of only 20 timepoints we assume a maximum AR order of 3. The MCMC algorithm was run to obtain a sample of size 00,000 from the posterior distribution of the parameters following a burn-in of 0,000 iterations. Every sample from the posterior distribution contained a summer effect and only 5 samples did not contain the first order AR term. There was also some support for the inclusion of the second order AR term (49,869 samples and the linear trend (26,575 samples. Only 8,357 samples contained the third order AR term and less than 2,000 samples contained either the cosine or sine term. Therefore there is no evidence to support a periodic term in the model in addition to the summer effect. Only six of the models were obtained in more than,000 samples and their details are given in the table below. AR orders Explanatory variables Number of samples,2 S t 35,3 S t 3,46 t,s t 4,32,2 t,s t 9,34,2,3 S t 3,973,3 S t 2,562 Table 3. Models with the highest posterior probabilities. The contribution from the second and third order AR terms are small even when they are included in the model. Therefore we focus upon the parameter estimates for the noise term and the first order AR term only. Explanatory variable γ δ Intercept ( ( Trend ( ( Summer ( ( Table 4. Mean (standard deviation of parameter coefficients. Thus the summer effect is only really present in the Poisson innovation term. This is confirmed by allowing the explanatory variables to be present in some terms but not in others. Interestingly, the linear trend is decreasing in the noise term, whilst increasing in the AR( term. This leads to similar size observations over the length of the period but with higher correlation between later observations. 0

12 References [] Brännäs, K. (995 Estimation and testing in integer-valued AR( models. Umeå Economic Studies. 38. [2] Davis, R. A., Dunsmuir, W. T. and Streett, S. B. (2003 Observation-driven models for Poisson counts. Biometrika 90, [3] Davis, R. A., Dunsmuir, W. T. and Streett, S. (2005 Maximum Likelihood Estimation for an Observation Driven Model for Poisson Counts. Methodology and Computing in Applied Probability 7, [4] Davis, R. A., Dunsmuir, W. T. and Wang, Y. (2000 On autocorrelation in a Poisson regression model. Biometrika 87, [5] Enciso-Mora, V., Neal, P. and Subba Rao, T. (2008 Efficient order selection algorithms for integer valued ARMA processes. To appear in Journal of Time Series Analysis. [6] Gelman, A.,Roberts, G.O. and Gilks, W.R. (995 Efficient Metropolis Jumping Rules. Bayesian Statistics V (eds. J.M. Bernardo et al. Oxford University Press pp [7] Green P.J. (995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, [8] Jung, R.C. and Tremayne, A.R. (2006 Coherent forecasting in integer time series models. International Journal of Forecasting 22, [9] Latour, A. (997 The multivariate GINAR(p process. Advances in Applied Probability 29, [0] Latour, A. (998 Existence and stochastic structure of a non-negative integer-valued autoregressive process. Journal of Time Series Analysis 9, [] McCabe, B. P. M. and Martin, G. M. (2005 Bayesian predictions of low count time series. International Journal of Forecasting 22, [2] Neal, P. and Subba Rao, T. (2007 MCMC for integer-valued ARMA processes. Journal of Time Series Analysis 28, [3] Schwarz, G. (978 Estimating the Dimension of a Model. The Annals of Statistics. 6,

13 [4] Steutel F. and van Harn, K. (979 Discrete analogues of self-decomposability and stability. Annals of Probability 7, [5] Troughton, P. T. and Godsill, S. J. (997 Bayesian model selection for time series using Markov chain Monte Carlo. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing 997, vol. V. pp [6] Zeger, S. (988 A regression model for time series of counts. Biometrika 75, [7] Zhu R. and Joe H. (2006 Modelling count data time series with Markov processes based on binomial thinning. Journal of Time Series Analysis 27,