Stationary Gaussian Markov processes as limits of stationary autoregressive time series

Transcription

1 Stationary Gaussian Markov processes as liits of stationary autoregressive tie series Pilip A. rnst 1,, Lawrence D. Brown 2,, Larry Sepp 3,, Robert L. Wolpert 4, Abstract We consider te class, C p, of all zero ean stationary Gaussian processes, Y t : t (, )} wit p derivatives, for wic te vector valued process (Y (0) t Markov process, were Y (0) t,..., Y (p) t ) : t 0} is a p+1-vector = Y (t). We provide a rigorous description and treatent of tese stationary Gaussian processes as liits of stationary AR(p) tie series. Keywords: Continuous autoregressive processes, stationary Gaussian Markov processes, stocastic differential equations 2010 MSC: Priary: 60G10, Secondary: 60G15 1. Introduction In any data-driven applications in bot te natural sciences and in finance, tie series data are often discretized prior to analysis and are ten forulated using autoregressive odels. Te teoretical and applied properties of te convergence of discrete autoregressive ( AR ) processes to teir continuous analogs (continuous autoregressive or CAR processes) as been well studied by any ateaticians, statisticians, and econoists; see, e.g., [1, 2, 4, 8, 12]. For references on stocastic differential equations, wic underlie te teory of CAR processes, we refer to [6, 7, 14, 15]. Subitted June 26, 2016 Corresponding autor. -ail: pilip.ernst@rice.edu -ail: lbrown@warton.upenn.edu -ail: sepp@warton.upenn.edu -ail: wolpert@stat.duke.edu 1 Departent of Statistics, Rice University, Houston, TX 77005, USA 2 Departent of Statistics, University of Pennsylvania, Piladelpia, PA 19104, USA 3 Deceased April 23, Departent of Statistical Science, Duke University, Dura, NC 27710, USA Preprint subitted to Journal of Multivariate Analysis Deceber 23, 2016

2 A special class of autoregressive processes are te discrete-tie zero-ean stationary Gaussian Markovian processes on R. Te continuous tie analogs of tese processes are docuented in Capter 10 of [11] and in pages of [13]. For processes in tis class, te saple pats possess p 1 derivatives at eac value of t, and te evolution of te process following t depends in a linear way only on te values of tese derivatives at t. Notationally, we ter suc a process as a eber of te class C p. For convenience, we will use te notation CAR(p) = C p. Te standard Ornstein Ulenbeck process is of course a eber of C 1, and ence CAR(p) processes can be described as a generalization of te Ornstein Ulenbeck process. It is well understood tat te Ornstein Ulenbeck process is related to te usual Gaussian AR(1) process on a discrete-tie index, and tat an Ornstein Ulenbeck process can be described as a liit of appropriately cosen AR(1) processes; see [7]. In an analogous fasion we sow tat processes in C p are related to AR(p) processes and can be described as liits of an appropriately cosen sequence of AR(p) processes. Of course, tere is also extensive literature on te weak convergence of discrete-tie tie series processes, particularly tat of ARMA and GARCH processes. For exaple, Duan [5] considers te diffusion liit of an augented GARCH process and Lorenz [9] discusses (in Capter 3) liits of ARMA processes. However, to te best of our knowledge, none of tese references (nor siplifications of teir results) discuss ow to correctly approxiate C p by discrete AR(p) processes, and tus tis is te goal of our paper. Section 2 begins by reviewing te literature on CAR(p) processes, recalling tree equivalent definitions of te processes in C p. Section 3 discusses ow to correctly approxiate C p by discrete AR(p) processes. Appendix A contains te proof of our ain result, Teore quivalent descriptions of te class C p Tere are tree distinct descriptions of processes coprising te class C p, wic are docuented on p. 212 of [13] but in different notation. On pp , Rasussen and Willias [13] prove tat tese descriptions are equivalent ways of describing te sae class of processes. Te first description atces te euristic description given in te introduction. Te reaining descriptions provide ore explicit descriptions tat can be useful in construction and interpretation of tese 2

3 processes. In all te descriptions Y = Y (t) : t [0, )} sybolizes a zero-ean Gaussian process on [0, ). In te present paper we use te first of te tree equivalent descriptions in [13], as follows. Let Y be stationary. Te saple pats are continuous and are p 1 ties differentiable, a.e., at eac t [0, ) (Te derivatives at t = 0 are defined only fro te rigt. At all oter values of t, te derivatives can be coputed fro eiter te left or te rigt, and bot rigt and left derivatives are equal). For eac i 1,..., p 1}, we denote te it derivative at t by Y (i) (t). At any t 0 (0, ), te conditional evolution of te process Y (t) : t [0, t 0 ]} depends in a linear way only on te set of values Y (i) (t 0 ) : i 0,..., p 1}}. Te above can be foralized as follows: let (Y (0) t,..., Y (p 1) t ) : t 0} denote te values of a ean zero Itô vector diffusion process defined by te syste of equations dy (i 1) t = Y (i) t dt, t > 0, i = 1,..., p 1 dy (p 1) p 1 (2.1) t = a i+1 Y (i) t dt + σdw t i=0 for all t > 0, were W t is te Wiener process, σ > 0. Ten let Y (t) = Y (0) t Caracterization of stationarity via (2.1) Te syste in (2.1) is linear. Stationarity of vector-valued processes described in suc a way as been studied elsewere; see in particular Teore on p. 357 in [7]. Te coefficients in (2.1) tat yield stationarity can be caracterized via te caracteristic polynoial of te atrix Λ, were Λ λi is λ p a p λ p 1 a 2 λ a 1 = 0. (2.2) Te process is stationary if and only if all te roots of q. (2.2) ave strictly negative real parts. In order to discover weter te coefficients in (2.1) yield a stationary process it is tus necessary and sufficient to ceck weter all te roots of q. (2.2) ave strictly negative real parts. In te case of C 2 te condition for stationarity is quite siple, naely tat a 1, a 2 sould lie in te quadrant a 1 < 0, a 2 < 0. Te covariance functions for C 2 can be found in [11], p For iger 3

4 order processes te conditions for stationarity are not so easily described. Indeed, for C 3 it is necessary tat a 1 < 0, a 2 < 0, a 3 < 0 siultaneously, but te set of values for wic stationarity olds is not te entire octant. For larger p one needs to study te solutions of te iger order polynoial in q. (2.2). 3. Weak convergence of te -AR(2) process to CAR(2) process 3.1. Discrete tie analogs of te CAR processes We now turn our focus to describing te discrete tie analogs of te CAR processes and te expression of te CAR processes as liits of tese discrete tie processes. In tis section, we discuss te situation for p = 2. Define te -AR(2) processes on te discrete tie doain doain 0,, 2,...} via X t = b 1X t + b 2X t 2 + ς Z t, (3.1) wit Z t IID N (0, 1) for all t = 2, 3,... Te goal is to establis conditions on te coefficients b 1, b 2 and ς so tat tese AR(2) processes converge to te continuous tie CAR(2) process as in te syste of equations given in (2.1). We ten discuss soe furter features of tese processes. To see te siilarity of te -AR(2) process in (3.1) wit te CAR(2) process of (2.1), we introduce te corresponding -VAR(2) processes 0;t, 1;t defined via 0;t 0;t = 1;t, (3.2) 1;t 1;t = (c 1 0;t + c 2 1;t ) + ξ Z t wit Z t IID N (0, 1) for all t =, 2,... and Pr 0;0, 1;0) Γ} = ν 2 (Γ) for all Γ B(R 2 ), (3.3) were ν2 is a probability easure on (R2, B(R 2 )) for te Borel σ-algebra B(R 2 ). We assue also tat as 0, 0;0, 1;0 ) converges in distribution to a rando variable pair 0 0;0, 0 1;0 ) wit probability easure ν2 0 on (R2, B(R 2 )). Te above assuptions for te initial distribution follow Assuption 3 of [10] as well as te discretized process defined by (2.22) (2.24) terein. 4

5 Fro (3.2) we see tat 0;t = 0;t + 1;t = 0;t + 1;t + (c 1 0;t + c 2 1;t )2 + ξ Z t = (2 + c c 2) 0;t (1 + c 2) 0;t 2 + ξ Z t. Tis sows tat te -AR(2) process of (3.1) is equivalent to te -VAR(2) in (3.2) wit b 1 c c 2 + 2, b 2 c 2 1, ς ξ, (3.4) or, equivalently, c 1 2 (b 1 + b 2 1), c 2 1 ( 1 b 2), ξ 1 ς. (3.5) In Teores 3.1 and 3.2, we consider weak convergence in te sae sense as tat of [10]; see footnote 7 on p. 13 terein. Teore 3.1. Consider a sequence of -AR(2) processes of (3.1) wit coefficients given by (3.4), were c j a j 0 for j = 1, 2 and ξ / σ as 0. We also assue tat as 0, 0;0, 1;0 ) converges in distribution to a rando variable pair 0 0;0, 0 1;0 ) wit probability easure ν 0 2 on (R2, B(R 2 )). Ten te sequence of -AR(2) processes of (3.1) converges in distribution to te CAR(2) process of (2.1), were W t : t 0} is a one-diensional Brownian otion, independent of ( Y 0, Y (1) ) 0, and Pr(Y0, Y (1) 0 ) Γ} = ν2 0(Γ) for any Γ B(R2 ). Proof. Tis proof is a special case of Teore 3.2 and is tus oitted. Reark 3.1. Teores 2.1 and 2.2 in [10] are explicitly stated for real-valued processes but apply to vector-valued processes as well. One only needs to explicitly allow te processes to be vector-valued, and to write te regularity conditions to allow for te full cross-covariance of te vector-valued observations, rater tan just ordinary covariance functions. Our processes are uc better beaved tan te ost general type of process considered in [10] since our error variance is constant (depending only on ) and our distributions are Gaussian, and ence very ligt tailed. Tus bot of Teores 2.1 and 2.2 in [10] apply. 5

6 Reark 3.2. Teore 3.1 otivates te following question: wat appens wen a given CAR(2) process is sapled? Tis question is resolved in [3], wic found tat a sapled CAR(2) process gives rise to an ARMA (2,1) process Weak convergence of -AR(p) process to CAR(p) process We now consider te AR(p) process on te discrete tie doain 0,, 2,...}, given as X t = b 1X t + + b px t p + ς Z t, (3.6) were Z t IID N (0, 1) for all t = p, (p + 1) and sow tat subject to suitable conditions on te coefficients b 1,..., b p and ς, tis converges as 0 to its continuous tie CAR(p) process of te for Y (p) p 1 t = a i+1 Y (i) t + σw t, t > 0 (3.7) i=0 were a j 0 for all j 1,..., p} and σ 2 > 0. Define te coefficients c j : j = 1,..., p} and ζ troug te equations b i ( 1) i 1 (p i ) + k=i Te following teore is proven in te Appendix. ( ) } k 1 p k+1 c k, ς p 1 ξ. (3.8) i 1 Teore 3.2. Consider te -AR(p) process of (3.6) wit coefficients given by (3.8), were c j a j 0 for all j 1,..., p} and ξ / σ as 0, We furter assue tat as 0, 0;0,..., p 1;0 ) converges in distribution to 0 0;0,..., 0 p 1;0 ) wit probability easure ν 0 p on (R p, B(R p )). Ten te -AR(p) process of (3.6) converges in distribution to te CAR(p) process of (3.7), were W t : t 0} is a one-diensional Brownian otion, independent of (Y 0, Y (1) 0,..., Y (p 1) 0 ), and Pr(Y 0, Y (1) 0,..., Y (p 1) 0 ) Γ} = ν 0 p(γ) for any Γ B(R p ). It is of interest to note te scaling for te Gaussian variable Z t in (3.6). In order to ave te desired convergence, one ust ave ζ /(σ ) 1 and via (3.8) tis entails ζ /(σ p 1/2 ) 1. 6

7 Acknowledgents We greatly tank two anonyous referees for teir careful reading of te paper and for teir constructive feedback, wic greatly elped to iprove te quality of te paper. We are also very grateful for bot te very elpful input as well as te generosity of tie fro te ditor-in-cief, Prof. Cristian Genest. Appendix A Tis Appendix proves Teore 3.2. To do so, we first study te siilarity of te -AR(p) process in (3.6) wit te CAR(p) process (3.7). We begin by introducing te corresponding - VAR(p) process 0;t 0;t = 1;t,..., p 2;t p 2;t = p 1;t, (A.1) fro wic one gets, troug iteration, p 1 p 1;t p 1;t = c i+1 i;t + ξ Z t, wit Z t IID N (0, 1) for all t =, 2,... and i=0 Pr 0;0,..., p 1;0) Γ} = ν p (Γ) for all Γ B(R p ), (A.2) νp being a probability easure on (R p, B(R p )) for te Borel σ-algebra B(R p ). For i = 1, te process of (A.1) iediately yields ( ) ( ) 1 1 1;t = 0;t 0;t 2 = ( 1) 0 0;t 0 + ( 1) 1 0;t 2 1. For i = 2, te process of (A.1) iediately yields 2 2;t = 1;t 1;t 2 ( ) ) ( ) ( ) = ( 1) 0 0;t 0 + ( 1) 1 0;t ( 1) 2 0;t 3 2. Te process of (A.1) generalizes as follows: for all i 1,..., p 1}, 7

8 i+1 i i;t = ( ) i ( 1) k 1 0;t k k 1. k=1 (A.3) We now prove (A.3) via ateatical induction. For i = 1 te relationsip (A.3) olds trivially. Let (A.3) old for i =. It is now straigtforward to sow tat (A.3) also olds for i = + 1. Furterore, for i = 1, it follows fro (A.1) tat 0;t = 0;t + 1;t. For i = 2, it follows fro (A.1) tat 0;t = 0;t + 1;t + 2;t) = 0;t + 1;t + 2 2;t = = i i;t + p 1 p 1;t p 2 ( ) p 2 p 1 = i i;t + p 1 p 1;t + c i+1 i;t + ξ Z t = i=0 (p ) ( 1) k 1 + k k=1 i=k i=0 i=0 ( ) i 1 p i+1 c i k 1 } 0;t k + p 1 ξ Z t, (A.4) wic, wen copared wit te -AR(p) process of (3.6), yields te relationsips b i ( 1) i 1 (p i ) + k=i ( ) } k 1 p k+1 c k, ς p 1 ξ (A.5) i 1 for all i 1,..., p}. Fro above, te -AR(p) process of (3.6) wit coefficients given by (3.8) is equivalent to te -VAR(p) of (A.1). We sall next find te coefficients c 1,..., c p in ters of te coefficients b 1,..., b p. In particular, we ave tat, fro (3.8), wen i = p, tat ( ) p b p = ( 1) p 1 + p ( ) } p 1 c p, c p = 1 ( ) p 1 } ( 1) p 1 b p 1. p 1 p 1 8

9 Wen i = p 1, ( ) p ( ) k 1 b p 1 = ( 1) p 2 + p k+1 c k p 1 p 2, k=p 1 [ ( ) ( ) } ] p 2 p 1 c p 1 = 2 ( 1) p 2 b p 1 + b p 1. p 2 p 2 Tis leads to te following general forula. Proposition A.1. For all i 1,..., p}, one as ( } k 1 c i = p+i 1 ( 1) )b i 1 k i 1 1. (A.6) k=i Proof. We prove Proposition A.1 by backward induction. (i) For i = p te relationsip (A.6) olds trivially. (ii) Let (A.6) old for every i p 1, p 2,..., + 1}. (iii) We sall sow tat (A.6) also olds for i =. For i =, one as ( ) p b = ( 1) 1 + and ence it follows fro (i) and (ii) tat ( 1) 1 b = ( ) p + p +1 c + ( 1) 1 b i=+1 i= ( ) i 1 ( 1) i 1 ( ) } i 1 p i+1 c i k=i ( ) k 1 b k i 1 i=+1 ( ) i 1. Intercanging te order of suation in te double su, te forer index bounds i k p and + 1 i p ave now becoe + 1 i k and + 1 k p. We furter ave i=+1 ( ) i 1 ( 1) i 1 k=i ( ) k 1 b k i 1 = k=+1 b k = ( 1) ( ) k k 1 ( ) k ( 1) j+ 1 j j=1 ( ) k 1 b k, k=+1 9

10 as well as i=+1 Te last relationsip yields ( ) i 1 = i=+1 ( ) i ( ) p ( 1) 1 b = + p +1 c + ( 1) ( ) k 1 p +1 c = ( 1) 1 b k 1, k= ( )} i 1 = k=+1 b k ( ) p 1. ( ) k 1 ( ) } p 1, concluding part (iii) and tus te proof. Finally, for t > 0 we know tat dy t = Y (1) t dt, dy (1) t = Y (2) t dt,..., dy (p 2) t = Y (p 1) t dt, (A.7) and fro (3.7) we ave also tat dy (p 1) t = a 1 Y t + a 2 Y (1) t + + a p Y (p 1) t }dt + σdw t. (A.8) Tus te CAR(p) process of (3.7) is equivalent fro (A.7) and (A.8) to te syste of stocastic differential equations in (2.1). We are now ready to prove Teore 3.2. Proof. We prove Teore 3.2 by proving tat it suffices to sow tat te -VAR(p) process of (A.1) converges to te SDs syste of (2.1). We eploy te fraework of Teores 2.1 and 2.2 of [10]. Let M t be te σ-algebra generated by i;0, i;, i;2,..., i;t, for i = 0,..., p 2, and p 1;0, p 1;, p 1;2,..., p 1;t for t =, 2,... Te -VAR(p) process of (A.1) is clearly Markovian of order 1, since we ay construct 0;t, 1;t, 2;t,..., p 1;t fro 0;t, 1;t, 2;t,..., p 1;t by constructing first p 1;t fro te last equation of (A.1), p 2;t fro (A.1) for i = p 1, and so fort, and ten finally 0;t fro (A.1) for i = 1. Tis also establises tat te set i;t : i 0,..., p 2}} is M t-adapted. Tus te corresponding 10

11 drifts per unit of tie conditioned on inforation at tie t are given, for all i 1,..., p 1}, by i 1;t } i 1;t M i 1;t + i;t t = } i 1;t M t = i;t (A.9) and p 1;t+ } p 1;t M t = c 1 0;t + + c p p 1;t, (A.10) were te second inequality olds by te last equation of (A.1) Furterore, te variances and covariances per unit of tie are respectively given, for all i 1,..., p 1}, by i 1;t } i 1;t )2 M t = ( 2, i;t) (A.11) and p 1;t+ } ( p 1;t )2 ξ M t = (c 1 0;t + + c p p 1;t) 2 ) 2 +, (A.12) were te last equality assues tat Z t+ IID N (0, 1). By te sae logic, one as, for all i, j 1,..., p 1} wit i j, i 1;t i 1;t ) j 1;t j 1;t ) } M t = i;t j;t, (A.13) and i 1;t i 1;t ) p 1;t+ p 1;t ) } M t = i;t(c 1 0;t + + c p p 1;t). (A.14) Terefore, te relationsips of (A.11) (A.14) are suc tat, for all i, j 1,..., p 1} wit i j, 11

12 i 1;t } i 1;t )2 M t = o(1), (A.15) p 1;t+ } p 1;t )2 M t = ( ξ ) 2 + o(1), (A.16) i 1;t i 1;t ) j 1;t j 1;t ) } M t = o(1) (A.17) and i 1;t i 1;t ) p 1;t+ p 1;t ) } M t = o(1), (A.18) were te o(1) ters vanis uniforly on copact sets. We can ten define te continuous tie version of te -VAR(p) process of (A.1) by i;t i;kfor k t < (k+1) and all i 0,..., p 1}. Tus, according to Teore 2.2 in [10], te relationsips (A.9) (A.10) and (A.15) (A.18) provide te weak liit diffusion. Tis is precisely te linear SD syste of (2.1) and it as a unique solution wit te asserted initial distribution. References [1] Brockwell, P.J., Davis, R. & Yang, Y. (2007). Continuous-tie Gaussian autoregression. Statistica Sinica, 17, [2] Brockwell, P.J., Ferrazzano, V. & Klüppelberg, C. (2013). Hig-frequency sapling and kernel estiation for continuous-tie oving average processes. Journal of Tie Series Analysis, 34, [3] Brockwell, P.J. & Lindner, A. (2009). xistence and uniqueness of stationary Lévy-driven CARMA processes. Stocastic Processes and teir Applications, 119, [4] Brockwell, P.J. & Lindner, A. (2010). Strictly stationary solutions of autoregressive oving average equations. Bioetrika, 97,

13 [5] Duan, J.-C. (1997). Augented GARCH (p, q) process and its diffusion liit. Journal of conoetrics, 79, [6] Durrett, R. (1996). Stocastic Calculus: A Practical Introduction. CRC Press, Boca Raton, FL. [7] Karatzas, I. & Sreve, S. (2012). Brownian Motion and Stocastic Calculus. Springer, New York. [8] Kirsner, H., Unser, M. & Ward, J.P. (2014). On te unique identification of continuoustie autoregressive odels fro sapled data. I Transactions on Signal Processing, 62, [9] Lorenz, R. (2006). Weak Approxiation of Stocastic Delay: Differential quations wit Bounded Meory by Discrete Tie Series. Doctoral Dissertation, Huboldt Universität zu Berlin, Gerany. [10] Nelson, D. B. (1990). ARCH odels as diffusion approxiations. Journal of conoetrics, 45, [11] Papoulis, A. (1984). Probability, Rando Variables and Stocastic Processes. McGraw-Hill, New York. [12] Pa, D.-T. (2000). stiation of continuous-tie autoregressive odel fro finely sapled data. I Transactions on Signal Processing, 48, [13] Rasussen, C.. & Willias, C. (2006). Gaussian Processes for Macine Learning. MIT Press, Cabridge, MA. [14] Resnick, S.I. (2013). Adventures in Stocastic Processes. Springer, New York. [15] Rogers, L.C.G. & Willias, D. (2000). Diffusions, Markov Processes, and Martingales, Volue I. Cabridge University Press, Cabridge. 13