Robust Testing and Variable Selection for High-Dimensional Time Series

Transcription

1 Robust Testing and Variable Selection for High-Dimensional Time Series Ruey S. Tsay Booth School of Business, University of Chicago May, 2017 Ruey S. Tsay HTS 1 / 36

2 Outline 1 Focus on high-dimensional linear time series 2 Basic theory: Extreme value theory in the domain of Gumble distribution 3 Testing for serial correlations 4 Finite-sample adjustments and simulations 5 A simple example 6 Variable selection: statistics and procedure 7 Simulation and comparison Ruey S. Tsay HTS 2 / 36

3 Objectives Part of an ongoing project on analysis of big dependent data 1 Develop statistical methods for analysis of high-dimensional time series 2 Focus on simple methods if possible 3 Make analysis of big dependent data easy, including statistical inference. Ruey S. Tsay HTS 3 / 36

4 Basic Theory Extreme Value Theory: (Maximum and minimum) Let x 1,..., x k be a random sample (iid) from a standard Gaussian distribution. Define the order statistics x k,k x k 1,k x 2,k x 1,k. Also, define the norming constants c k = 1 2 ln(k), d k = ln(4π) + ln(ln(k)) 2 ln(k) 2. 2 ln(k) Then, as k T 1,k = x 1,k d k c k d X, T k,k = x k,k d k c k d X, and T 1,k and T k,k are asymptotically independent, where the CDF of X is Λ(x) = exp( e x ). See, for instance, the classical book by Embrechts, et al. (2001). Ruey S. Tsay HTS 4 / 36

5 Basic tool and its property Spearman s rank correlation: Consider two random variables X and Y with continuous marginal distributions. Let {(x i, y i )} be a sample of size n from (X, Y ). Let ri x be the rank of x i and r y i be the rank of y i in the sample. The Spearman s rank correlation is defined as ˆρ = n i=1 (r i x r)(r y i r) n(n 2, 1)/12 where r = (n + 1)/2. Limiting property: nˆρ d N(0, 1) as n, if X and Y are independent. Ruey S. Tsay HTS 5 / 36

6 High-dimensional time series Let X t = (x 1t,..., x pt ) be a p-dimensional stationary time series. Let {x 1,..., x n } be a realization of n observations of X t. Let r t = (r 1t,..., r pt ) be the corresponding tth observation of ranks. That is, r it denotes the rank series of x it. The lag-l Spearman s rank cross-correlation matrix is defined by 12 n Γ l = [ Γ l,ij ] = n(n 2 (r t r)(r t l r), 1) t=l+1 where r denotes the constant vector of (n + 1)/2. The robustness of the rank correlation has been widely studied in the literature. Ruey S. Tsay HTS 6 / 36

7 A fundamenal question Are there serial correlations in a given high-dimensional time series? H 0 : Γ 1 = = Γ m = 0 vs H 1 : Γ i 0 for some i, where, for simplicity, Γ l = E[ Γ l ]. The question also applies to the residuals of a fitted model, e.g. Is the model adequate for the data? The well-known Ljung-Box Q(m) statistic is not useful as the dimension p increases. Some work available, e.g. Chang, Yao and Zhou (2016) and Li et al. (2016). Ruey S. Tsay HTS 7 / 36

8 Basic properties of rank correlations: A theorem Conditions: Continuous random varaibles (no moment requirements) 1 X t is a white noise (no serial correlations) 2 Components are independent: X it and X jt are independent for i j. Ruey S. Tsay HTS 8 / 36

9 Theorem continued The rank cross-correlation matrix Γ l satisfies (a) E( Γ l,ii ) = n l n(n 1), 1 l n 1, (b) E( Γ l,ij ) = 0, i j, (c) Var( Γ l,ii ) = 5(n l) 4 5n 2 + O(n 3 ), (d) Cov( Γ l,ii, Γ h,ii ) = 2 n 2 + O(n 3 ), 1 l < h n 1, (e) Var( Γ l,ij ) = 1 n + O(n 2 ), i j, (f) Cov( Γ l,ij, Γ l,uv ) = 0 + O(n 2 ), (i, j) (u, v), where 1 i, j p. (a), (c), and (d) are shown by Dufour and Roy (1986). Others by independence condition. Ruey S. Tsay HTS 9 / 36

10 Summary of the theorem Let vec(a) be the column-stacking vector of matrix A. Then, n vec( Γl ) d N(0, I), n, where N(0, I) denotes the p 2 -dimensional standard Gaussian distribution, provided that the conditions hold. Ruey S. Tsay HTS 10 / 36

11 Testing: single cross-correlation matrix H 0 : Γ l = 0 versus H 1 : Γ l 0. Define T l,max = n max{ Γ l } T l,min = n min{ Γ l }. Let c p and d p be the norming constants with k = p 2. Under H 0, T l,max = T l,max d p c p d X, T l,min = T l,min d p c p d X, where X denotes the Gumble distribution, if n and p. For type-i error α, H 0 is rejected if either T l,max v α or T l,min v α, where v α = ln( ln(1 α 2 )). Ruey S. Tsay HTS 11 / 36

12 Single test statistic Define T l = n max{t l,max, T l,min } = n max Γ l. Then, under H 0, (T l d p )/c p d max{x 1, X 2 } as n and p, where X 1 and X 2 are two independent Gumble random variates. The limiting distribution of T l is available. Ruey S. Tsay HTS 12 / 36

13 Joint test for multiple lags H 0 : Γ 1 = = Γ m = 0 versus H 1 : Γ i 0 for some 1 i m. Define T max (m) = max{t l,max l = 1,..., m} T min (m) = max{t l,min l = 1,..., m} and the norming constants c p,m and d p,m as before with k = mp 2. Then, T max d m,p c m,p d X, T min d m,p c m,p d X. Ruey S. Tsay HTS 13 / 36

14 Empirical sizes: 10,000 realizations Test p = 300, n = 3000 p = 300, n = 5000 p = 500, n = 5000 Statistic 10% 5% 1% 10% 5% 1% 10% 5% 1% (a) Standard Gaussian distribution T T T T T T T (5) T (10) Ruey S. Tsay HTS 14 / 36

15 Empirical sizes continued Test p = 300, n = 3000 p = 300, n = 5000 p = 500, n = 5000 Statistic 10% 5% 1% 10% 5% 1% 10% 5% 1% (b) Cauchy distribution, i.e. t 1 T T T T T T T (5) T (10) Ruey S. Tsay HTS 15 / 36

16 Finite-sample adjustments Adjust for actual data used: Γ l = 12 (n l)[(n l) 2 1] n (r t r)(r t l r), t=l+1 where r is a p-dimensional constant vector of (n l + 1)/2, r t is the rank matrix of X[(l + 1) : n, ] and r t l is the rank matrix of X[1 : (n l), ]. Bias adjustment of auto-correlations Γa l,ii = Γ l,ii + n l n(n 1), i = 1,..., p Ruey S. Tsay HTS 16 / 36

17 Finite-sample adjustments continued Variance adjustment 5n 2 /[5(n l) 4] vec( Γ a l ) d N(0, I), n. The test statistic for a single lag matrix T a l = 5n 2 /[5(n l) 4] max{ Γ a l,ij }. Ruey S. Tsay HTS 17 / 36

18 Block size adjustment: for n < 3000 Adjustmented norming constants: c a p = [2 ln(p 2 ξ)] 1/2, d a p = where ξ = where η = min{5, (n + p)/n}. 2 ln(p 2 ξ) ln(4π) + ln ln(p2 ξ) 2[2 ln(p 2 ξ)] 1/2, { 0.78 η if n < 3000, 1 otherwise, Ruey S. Tsay HTS 18 / 36

19 Empircal sizes: individual lag, 30,000 realizations n = 100 n = 300 n = 500 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (a) Lag (b) Lag Ruey S. Tsay HTS 19 / 36

20 Empircal sizes: individual lag continued n = 100 n = 300 n = 500 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (c) Lag (d) Lag Ruey S. Tsay HTS 20 / 36

21 Number of lags adjustment: for p < 300 Effective lags used { m m 0.9 (n+p)/n if p < 300, = m otherwise, and adjust the scale and location parameters as cp,m a = [2 ln(p 2 ξm )] 1/2 dp,m a = 2 ln(p 2 ξm ) ln(4π) + ln ln(p2 ξm ) 2[2 ln(p 2 ξm )] 1/2. Ruey S. Tsay HTS 21 / 36

22 Empirical sizes: joint test, 30,000 realizations n = 100 n = 300 n = 500 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (a) m = 5, the first 5 lags (b) m = 10, the first 10 lags Ruey S. Tsay HTS 22 / 36

23 General covariance matrix 1 p < n: Apply PCA to x t. Apply tests to the principal component series 2 p n: Select p = 0.75n series for testing Selection: H 0 : Γ 1 = = Γ m = 0. For each Γ l, define a p-dimensional weight vector w l = (w l,1,..., w l,p ) such that w l,i = max 1 j p Γ l,i. The weights for each component x it is w i = m w l,i, i = 1,..., p. l=1 Select the p sub-series with the highest weights. Ruey S. Tsay HTS 23 / 36

24 Correlated series x t = A 1/2 ɛ t where A is a symmetric matrix with A ij = 0.9 i j and elements of ɛ t are iid t 3 random variates. The results are based on 10,000 realizations. Ruey S. Tsay HTS 24 / 36

25 General covariance matrix: size simulation n = 100 n = 300 n = 500 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (a) Lag (b) Lag Ruey S. Tsay HTS 25 / 36

26 General covariance matrix: size simulation continued n = 100 n = 300 n = 500 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (c) Lag (d) Lag Ruey S. Tsay HTS 26 / 36

27 Joint tests n = 100 n = 300 n = 500 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (a) m = 5, the first 5 lags (b) m = 10, the first 10 lags Ruey S. Tsay HTS 27 / 36

28 Power study Two data generating processes 1 VAR(1): x t = Φx t 1 + e t 2 VMA(1): x t = e t Θe t 1 where e t are iid N(0, I), x 0 = 0. For VAR(1) models, let ζ is the sparsity parameter. Non-zero elements of Φ is N = p 2 ζ. For each realization, Φ is obtained using Random sample N from 1 : p 2 without replacement Draw N uniform random variates from [ 0.95, 0.95] Assign (ii) to the N positions in vec(φ). Ruey S. Tsay HTS 28 / 36

29 Power of VAR(1) processes, 10,000 realizations n = 100 n = 300 n = 500 p ζ 10% 5% ζ 10% 5% ζ 10% 5% (a) Individual test: T1 a (b) Individual test: T2 a Ruey S. Tsay HTS 29 / 36

30 Power study: Joint statistics n = 100 n = 300 n = 500 p ζ 10% 5% ζ 10% 5% ζ 10% 5% (c) Joint test: T a (5) (d) Joint test: T a (10) Ruey S. Tsay HTS 30 / 36

31 VMA(1) models: power study Θ = [Θ ij ] matrix is defined as { 0 if i j > 1 Θ ij = Otherwise g ij with { g ij = 0 with prob 1 ω U[ 0.95, 0.95] with prob ω Ruey S. Tsay HTS 31 / 36

32 n = 100, ω = 0.3 n = 300, ω = 0.2 n = 500, ω = 0.1 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (a) Indivudal test: T1 a (b) Individual test: T2 a Ruey S. Tsay HTS 32 / 36

33 Joint test: power n = 100, ω = 0.3 n = 300, ω = 0.2 n = 500, ω = 0.1 p 10% 5% 1% 10% 5% 1% 10% 5% 1% (c) Joint test: T a (5) (d) Joint test: T a (10) Ruey S. Tsay HTS 33 / 36

34 An example Daily returns, in percentages, of 92 component series of the S&P 100 index from January 2, 2004 to December 31, Sample size n = 2517 Ruey S. Tsay HTS 34 / 36

35 rtn year Ruey S. Tsay HTS 35 / 36

36 Individual Test Joint test T1 a T2 a T3 a T4 a T5 a T a (5) T a (10) (a) Original return series (b) Residuals of the VAR(1) model critical values: (10%,5%,1%) 1 Individual lag: 4.34, 4.51, Joint 5 lags: 4.65, 4.81, Joint 10 lags: 4.79, 4.94, Ruey S. Tsay HTS 36 / 36