Chapter 13: Functional Autoregressive Models

Transcription

1 Chapter 13: Functional Autoregressive Models Jakub Černý Department of Probability and Mathematical Statistics Stochastic Modelling in Economics and Finance December 9, / 25

2 Contents 1 Introduction / 25

3 Causal model Definition Let {ε n, n Z} be a white noise and let {c j, j N 0 } be a sequence of constants such that j=0 c j <. A random sequence {X n, n Z} defined as X n = c j ε n j, j=0 n Z is called causal linear process. Above definition briefly says that causal linear processes depend only on the present and past errors, not the future ones. 3 / 25

4 AR model Definition Random sequence {X n, n Z} is called autoregressive model of order p, AR(p),if p X n = φ i X n i + ε n i=1 where φ 1,..., φ p are real constants, φ p 0 and {ε n, n Z} is a white noise. In general, the definition of the AR model is based on the white noise, but [Horváth, Kokoszka (2011)] assume that {ε n, n Z} is a sequence of iid mean zero random variables which is a narrower class of processes. 4 / 25

5 Causal AR model If c j = ψ j and ψ < 1, AR(1) model {X n, n Z} is causal and has a unique solution in form X n = ψ j ε n j j=0 Our goal is to use AR(1) model condition ψ < 1 and establish analogous condition for the functional model. 5 / 25

6 FAR(1) model To lighten the notation we set L =. Definition We say that a sequence {X n, n Z} of mean zero elements of L 2 follows a functional AR(1) (FAR(1)) model if X n = Ψ(X n 1 ) + ε n (1.1) where Ψ L and {ε n, n Z} is a sequence of iid mean zero errors in L 2 satisfying E ε n 2 <. Following theory is developed under the assumption of iid errors. General case can be found in [Bosq (2000)]. 6 / 25

7 Existence - Lemma Lemma For any Ψ L, the following two conditions are equivalent: C1 There exists an integer j 0 0 such that Ψ j 0 < 1. C2 There exists a > 0 and 0 < b < 1 such that for every j 0, Ψ j ab j. Proof. The condition C2 clearly implies C1. So we must show only that C1 implies C2. We set j = qj 0 + r where q 0 and 0 r < j 0. Therefore, Ψ j = Ψ qj 0+r Ψ j 0 q Ψ r. If Ψ j 0 = 0, then C2 holds for any a > 0 and 0 < b < 1, so we assume in the following that Ψ j 0 > 0. 7 / 25

8 Existence - Lemma cont. Since q > j/j 0 1 and Ψ j 0 < 1, we get Ψ j Ψ j 0 j/j 0 1 Ψ r ( Ψ j 0 1/j 0) j Ψ j 0 1 max 0 r<j 0 Ψ r, so C2 holds with a = Ψ j 0 1 max Ψ r, b = Ψ j 0 1/j 0. 0 r<j 0 Note that condition C1 is weaker than the condition Ψ < 1. Nevertheless, C2 is a suffciently strong condition to ensure the convergence of the series j Ψj (ε n j ) and the existence of a stationary causal solution to FAR(1) equations, as stated in the following theorem. 8 / 25

9 Existence and uniqueness - Theorem Theorem If condition C1 holds, then there is a unique strictly stationary causal solution to 1.1. This solution is given by X n = Ψ j (ε n j ) (2.1) j=0 The series converges almost surely, and in the L 2 norm, i.e E X n m Ψ j (ε n j ) j=0 2 0, m. 9 / 25

10 Existence and uniqueness - Proof Proof. Let us remind that we work with L 2 (Ω, L 2 ([0, 1])) which is a Hilbert space with the inner product E X, Y, X, Y L 2 ([0, 1]). To show that the sequence X (m) n = m j=0 Ψj (ε n j ) has a limit in L 2 (Ω, L 2 ([0, 1])) it suffices to check that it is a Cauchy sequence in m for every fixed n. E m j=m+1 Ψ j (ε n j ) 2 = = m j,k=m+1 m j=m+1 E Ψ j (ε n j ), Ψ k (ε n k ) E Ψ j (ε n j ) / 25

11 Existence and uniqueness - Proof cont. Therefore by the previous lemma m 2 E Ψ j m m (ε n j ) Ψ j 2 E ε 0 2 E ε 0 2 a 2 j=m+1 Thus X (m) n By the condition C2 E Ψ j ε n j j=0 j=m+1 j=m+1 converges in L 2 (Ω, L 2 ([0, 1])), it is enough to verify that Ψ j (ε n j ) < a.s. j=0 2 b 2j. Ψ j Ψ k E ε 0 2 E ε 0 2 ab j j,k=0 which is finite and so j=0 Ψj ε n j < a.s. j= / 25

12 Existence and uniqueness - Proof cont. 2 The series X n is strictly stationary, and it satisfies (1.1). Suppose {X n} is another strictly stationary casual sequence satisfying (1.1). Then, iterating 1.1, we obtain, for any m 1, Therefore X n = m Ψ j (ε n j ) + Ψ m+1 (X n m+1). j=1 E X n X (m) n Ψ m+1 E X n m+1 E X 0 ab m+1. Hence X n is equal a.s. to the limit of X (m) n, i.e. to X n. 12 / 25

13 Brief example Consider an integral Hilbert-Schmidt operator on L 2 defined by Ψ(x)(t) = ψ(t, s)x(s)ds, x L 2, which satisfies ψ 2 (t, s)dtds < 1. (2.2) From the introduction ([Horváth, Kokoszka (2011), chapter 2.2]) we know that the left-hand side of above inequality is equal to Ψ 2 S. Since Ψ Ψ S, we see that (2.2) implies condition C1 with j 0 = / 25

14 Ψ - Idea First, we will focus on the AR(1) model X n = ψx n 1 + ε n. We assume that ψ < 1 so there is a stationary solution. Then, multiplying previous equation by X n 1 and taking the expected value, we get γ 1 = ψγ 0, where γ k = Cov(X n, X n k ) = E(X n X n k ). The usual estimator of ψ is ˆψ = ˆγ 1 /ˆγ 0 where ˆγ k are sample autocovariances. This approach is known as the Yule-Walker estimation. 14 / 25

15 Ψ We want to apply this technique to the functional model. By 1.1 and under condition C1 E [ X n, x X n 1 ] = E [ Ψ(X n 1 ), x X n 1 ], x L 2. The lag-1 autocovariance operator is defined by C 1 (x) = E [ X n, x X n+1 ]. and denote T the adjoint operator, then C T 1 = CΨT, i.e. C 1 = ΨC The above identity is analogous to the AR(1) process, so we would like to obtain an estimate of Ψ by using a finite sample version of the following relation Ψ = C 1 C / 25

16 Ψ - operator C However, the operator C does not have a bounded inverse on the whole H. Recall that C 1 (C(x)) = x, where C 1 (y) = j=1 λ 1 j y, v j v j. The operator C 1 is defined if all λ j are positive. Since C 1 (v n ) = λ 1 n, as n it is unbounded. This makes the estimation of the bounded operator Ψ using the relation Ψ = C 1 C 1 difficult. Note that if λ 1 λ 2 λ p > λ p+k = 0 for all k = 1, 2,... then {X n } is in the space spanned by {v 1,..., v p }. On this subspace we define C 1 (y) = p j=1 λ 1 j y, v j v j. 16 / 25

17 Ψ - EFPC A practical solution is to use only p most important EFPC s ˆv j, and to define p ÎC p (x) = ˆλ 1 j x, ˆv j ˆv j j=1 The operator ÎC p is defined on the whole of L 2, and it is bounded if ˆλ j > 0 for j p. By reasonable choice of p we can find a balance between retaining the relevant information in the sample, and the danger of working with the small eigenvalues. 17 / 25

18 Ψ - Estimator To derive a computable estimator of Ψ, we will use an empirical version of C 1 = ΨC. C 1 is estimated by we get, for any x L 2, p Ĉ 1 ÎC p (x) = Ĉ1 Ĉ 1 (x) = 1 N 1 X k, x X k+1, N 1 = = 1 N 1 j=1 ˆλ 1 j N 1 k=1 N 1 1 N 1 k=1 k=1 x, ˆv j ˆv j X k, p j=1 p j=1 ˆλ 1 j x, ˆv j ˆv j X k+1 ˆλ 1 j x, ˆv j X k, ˆv j X k / 25

19 Ψ - Estimator cont. The estimator Ĉ1ÎC p can be used in principle, but typically an additional smoothing step is introduced by using the approximation From this follows Ψ p (x) = 1 N 1 N 1 k=1 X k+1 p p X k+1, ˆv i ˆv i. i=1 p j=1 i=1 ˆλ 1 j x, ˆv j X k, ˆv j X k+1, ˆv i ˆv i. The estimator is consistent if p = p N is a function of the sample size N. Sufficient conditions for Ψ p Ψ 0 can be found in [Bosq (2000)]. 19 / 25

20 Ψ - Estimator cont. 2 The estimator is a kernel operator with the kernel ˆψ p such that Ψ p (x)(t) = ˆψ p (t, s)x(s)ds from which follows that ˆψ p (t, s) = 1 N 1 N 1 k=1 p p j=1 i=1 ˆλ 1 j X k, ˆv j X k+1, ˆv i ˆv j (s)ˆv i (t). The kernel estimation can be obtained using R function pca.fd which is contained in a fda package (Functional Data Analysis). More about this package can be found at 20 / 25

21 Ψ - Example Figure : The kernel surface ψ(t, s) (top left) and its esimates ˆψ(t, s) for p = 2, 3, 4 (Source: [Horváth, Kokoszka (2011), p. 241]) 21 / 25

22 Ψ - Example cont. In previous figure are illustrated Gaussian kernel ψ(t, s) = α exp{ (t 2 + s 2 )/2} with α chosen so that the Hilbert-Schmidt norm is 1/2, and three estimates (series length N = 100) for p = 2, 3, 4 and ε n were generated as Brownian bridges. We may see very large discrepancies in magnitude and in shape (which were observed for other kernels and innovations). Moreover, by any reasonable distance measure between two surfaces, the distance between ψ and ˆψ p increases as p increases. This is counterintuitive because by using more EFPC s ˆv j, we would expect the approximation to improve. 22 / 25

23 Ψ - Example cont. 2 The sums p j=1 ˆλ j explain 74, 83 and 87 % of the variance for p = 2, 3, 4, but the absolute deviation distances between ψ and ˆψ p are 0.4, 0.44 and The same pattern was obtained for the RMSE distance ˆψ ψ S. As N increases, these distances decrease, but their tendency to increase with p remains. This problem is partially due to the fact that for many FAR(1) models, the estimated eigenvalues ˆλ j are very small, except ˆλ 1 and ˆλ 2, and small error in their estimation means large error in the rest ˆλ 1 j. Partial solution to this problem can be found in [Kokoszka, Zhang (2010)] by adding a positive baseline to the ˆλ j. However, as we shall further see, precise estimation of the kernel ψ is not necessary to obtain satisfactory predictions. 23 / 25

24 Bosq, D.: Linear Processes in Function Spaces, Springer, New York, Horváth L., Kokoszka P.: Inference for Functional Data with Applications, Preprint, Kokoszka, P., Zhang, X.: Improved estimation of the kernel of the functional autoregressive process, Technical Report, Utah State University, Prášková Z.: Základy náhodných procesů II, Karolinum, Praha, / 25

25 Thank you for your attention! Jakub Černý 25 / 25