Forecasting Exchange Rates with Generalized Principal Components

Transcription

1 Forecasting Exchange Rates with Generalized Principal Components Karo Solat and Kwok Ping Tsang Abstract We extract principal components from a panel of 17 exchange rates and use the deviations from the components to forecast future exchange rate movements, following the idea in Engel, Mark, and West (2015). Instead of using the standard method, we apply a generalized principal components analysis that captures temporal and crosssectional variation and covariation among the exchange rates. We find that the method dominates forecasts by existing standard methods and random walk, with or without including macroeconomic fundamentals. Keywords: Factor model, principal component analysis, exchange rates, out-of-sample forecasting. JEL Classification: C53, F31, F47. Department of Economics, Virginia Tech, Pamplin Hall (3053), Blacksburg, VA 24061; karos@vt.edu. Corresponding author. Department of Economics, Virginia Tech, Pamplin Hall (0316), Blacksburg, VA 24061; byront@vt.edu. 1

2 1 Introduction The random walk model is hard to beat in forecasting exchange rates, and this finding has more or less survived the numerous studies since Meese and Rogoff (1983a) and Meese and Rogoff (1983b). The model essentially forecasts that log level of exchange rate remains the same in the future, and this seemingly simple model beats well-founded, sophisticated models of exchange rates that make use of economic fundamentals like output, interest rates, or inflation rates. It is a well-established finding for horizons from 1 quarter to 3 years, while the results are more ambiguous for longer horizons. 1 Instead of looking for some new fundamentals or econometric methods to beat the random walk, some recent papers look for predictability from the exchange rates themselves. In particular, factors are extracted from a panel of exchange rates, and the deviations of the exchange rates from the factors are used to forecast their future changes. 2 Engel, Mark, and West (2015) first propose this new direction and find mixed results. They extract three principal components from a panel of 17 exchange rates (with the US dollar as the base currency), and they find that the factors improve the random walk only for long horizons during the more recent period (1999 to 2007). Wang and Wu (2015) adopt the method of independent component factors 1 Engel and West (2005) shows that random walk dominates when the discount factor is near one and the fundamentals are persistent. For a recent survey on the empirical findings, see Rossi (2013) and Maasoumi and Bulut (2013). 2 For simplicity, we abuse the usage and refer to principal component as factor in this paper. 2

3 that is robust to fat tails, and, using a longer sample, they are able to beat the random walk at medium and long horizons regardless of the sample periods. This paper follows this line of research and extracts factors in a simple and intuitive way. While traditional principal components analysis (PCA) exploits the contemporaneous correlations across variables, we adopt a more general approach and make use of across-time variations for each variable and across variables as well. Though we are agnostic on what the factors represent, we believe that we are better at capturing unobserved fundamentals that make exchange rates persistent and correlated through time. Indeed, we find that relaxing the traditional PCA substantially improves the forecasting performance of the factors in Engel, Mark, and West (2015) beating the random walk in most horizons and sample periods. We also show that our more transparent method improves upon that proposed by Wang and Wu (2015). We begin by describing the generalized PCA method and explaining how it is different from the traditional one (the technical details are in the Appendix). Next we compare our forecasting performance with that in Engel, Mark, and West (2015) and Wang and Wu (2015), using the same data analyzed in each paper. 3

4 2 The Method of Factorization: Generalized Principal Component Analysis (GPCA) The idea behind all methods of factorization is to capture the most variations in the data through a few factors with some properties of interest, e.g. independence or orthogonality. One of the most popular methods of factorization is the PCA. The PCA extracts factors by rotating the coordinate system to a new system with axes equal to the (normalized) PCs. Suppose we have a set of m random variables (2.1) X t = X 1t X 2t. X mt N m (µ, Σ) then we can use the eigenvalue decomposition method to decompose the covariance matrix and construct 1) new principal components are orthogonal and 2) new coordinate axes which are (normalized) principal components constructed in a way that they fit the data in a lower dimension ellipsoid. To do so, we rotate the coordinate system to a more appropriate coordinate system so that we can drop the axes that cover a very small portion of the variation in data. Table 1 presents the underlying assumptions imposed to the joint distribution of PCs. The unbiased maximum likelihood estimators (MLEs) of parameters in 4

5 the PCA relies on the structure of temporal dependency (Φ) in the data. Assume the sampling matrix of order m T from the same population as X t = (X 1t,..., X mt ) N m (µ, Σ) can be shown as (2.2) X = (X 1,..., X T ) N m,t (µe T 1, Σ Φ) where e T 1 = (1,..., 1). To retain an operational model for 2.2 we need a set of assumptions over Φ. Suppose Φ is known, the MLEs of µ and Σ (say ˆµ and ˆΣ) are of the form: (see Anderson (2003) and Gupta, Varga, and Bodnar (2013)) (2.3) ˆµ = X Φ 1 e T 1 e T 1 Φ 1 e T 1 (2.4) ˆΣ = 1 n X(Φ 1 Φ 1 e T 1 e T 1 Φ 1 e T 1 Φ 1 e T 1 )X The MLEs ˆµ and ˆΣ assert that to have an operational model we need to know exactly the form of temporal dependency (Φ) in the data. The classical PCA assumes Normality and temporal independency (Φ = I T ). Gupta and Varga (1994) argue that a vector-variate elliptically contoured distribution (v.e.c.d.), consisting of m variables and T observations, can be viewed as a matrix-variate elliptically contoured distribution (m.e.c.d.) of order m T. In fact, in the the existence of dependence and heterogeneity, it is more appropriate to use the general form of sampling covariance matrix Σ Φ. The form of the covariance matrix of any m.e.c.d. captures two sources 5

6 of variation in the data: the variation and covariation between individual variables across time and the variation and covariation between two points of time among all variables 3. More formally, Definition 2.1 Assume we have a random matrix X of order m T. We say X has a matrix variate normal distribution,i.e. x 11 x 12 x 1T x 21 x 22 x 2T (2.5) X (m T ) = N m,t (M, Σ Φ) x m1 x m2 x mt Where M = E(X) : m T, Σ = E((X M)(X M) ) 0 : m m, Φ = E((X M) (X M)) 0 : T T, Cov(X) = Σ Φ. The characteristic function is of the form (2.6) φ X (S) = etr(is M 1 2 S ΣSΦ), where S : m T. Also, the probability density function is of the form (2.7) f(x) = (2π) mt 2 Σ T 2 Φ m 2 etr( 1 2 (X M) Σ 1 (X M)Φ 1 ). Now that we have a more generalized covariance matrix we can extend the the PCA to the matrix form to capture both sources of variation in the 3 This could be applied to the multivariate normal (Gaussian) distribution since it is a member of the Elliptically Contoured distribution family. 6

7 data by decomposing both Σ and Φ. We call this extension the generalized principal component analysis (GPCA) (see Solat and Spanos (2017)). In the GPCA, we use the eigenvalue decomposition method on both covariance matrices Σ and Φ to extract p GPCs (p < m). The fact that any symmetric positive semi-definite matrix (such as Σ and Φ) has non-negative real eigenvalues and is diagonalizable enables us to decompose Σ and Φ as follow: (2.8) Σ = A Λ m A, Φ = B Γ T B. where Λ m and Γ T are diagonal matrices that contain non-negative eigenvalues of Σ and Φ, respectively. Also, A and B are matrices that consist of the eigenvectors of Σ and Φ, respectively. 4 Define A p as an m p matrix containing the first p columns of A (i.e. the first p eigenvectors of Σ). The first p GPCs are Y = A p(x M)B N p T (0 p T, (A pσa p ) (B ΦB)). Also, because A p and B are orthonormal matrices, (2.9) Y = A p(x M)B N p T (0 p T, Λ p Γ T ). 4 Note that the order of eigenvalues and respective eigenvectors are based on descending order of eigenvalues. 7

8 The proof to the claim that these components will capture the most variation in the data is presented in the Appendix. The matrix of GPCs, Y, represents the sampling matrix of p individual GPCs with T observations. The joint distribution is as follow: (2.10) Y = A p(x µe T 1)B N p T (0 p T, Λ p Γ T ) where µ = (µ 1,..., µ m ) is the mean vector of X t = (X 1t,..., X mt ). What is the essential difference between the PCA and the GPCA? The main difference when we deal with the sample matrix is that, in the PCA, the variation between two points of time across individual variables has been ignored through an implicit assumption about Φ (i.e. Φ = I T ) but the GPCA model considers other forms of temporal covariance matrix (Φ) and the only assumption imposed is that Φ is non-singular matrix. In fact, this assumption makes the PCA model a special case of the GPCA model. Table 2 summarizes the assumptions imposed to the joint distribution of GPCs. 3 Empirical Results 3.1 Data We use quarterly data of nominal exchange rates for currencies of 17 OECD countries from 1973:1 to 2007:4. The data is the end of quarter value of 17 currencies based on the US dollar, obtained from International Financial 8

9 Statistics. The countries are Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Japan, Italy, Korea, Netherlands, Norway, Spain, Sweden, Switzerland, and the United Kingdom. Table 3 presents the descriptive statistic summary of the data. 3.2 Models of exchange rates The purpose of this section is to show that the GPCA can perform better than other methods of factorization in the context of exchange rate forecasting. We will focus on the discussions and arguments that have been presented by Engel, Mark, and West (2015), because it includes a comprehensive analysis of factor model specifications and auxiliary macro-variables along with the results from different factor models adopted to conduct out-ofsample forecasting of exchange rates. We want to examine if there is any improvement in the context of out-of-sample forecasting by replacing their factorization method with the GPCA method. Although in some cases the PCA method for British Pound, as a base currency, has some improvement compare to the factor analysis (FA) method, Engel, Mark, and West (2015) conclude that overall results for the FA method are better than the PCA method. We compare the out-of-sample forecastability power of the GPCA method to the FA method adopted by Engel, Mark, and West (2015). We derive three sets of out-of-sample forecasting: First, for the 9 non-euro currencies (Australia, Canada, Denmark, Japan, Korea, Norway, Sweden, Switzerland, and the United Kingdom) called long sample 9

10 forecasting for the time period 1987:1 to 2007:4. Second, for the 17 currencies ( Australia, Austria, Belgium, Canada, Denmark, Finland, France, Germany, Japan, Italy, Korea, Netherlands, Norway, Spain, Sweden, Switzerland, and the United Kingdom) called early sample forecasting for the time period 1987:1 to 1998:4 (before Euro). Finally, for the 10 currencies (countries included in long sample plus Euro) called late sample for the time period 1999:1 to 2007:4. To determine the number of GPCs we use the eigenvalue test which gives the percentage of variation explained through the retained GPCs. Three components, will explain 96% of the variation in the data (similar to the PCA). By the method explained in section 2 we derive GPCs and estimate the coefficients based on the following model: (3.1) s it = const. + δ 1i gpc 1t + δ 2i gpc 2t + δ 3i gpc 3t + u it = const. + GP C it + u it, u it N.i.i.d.(0, σ 2 u) where s it, (i = 1,..., 17), is the log of nominal exchange rates of currency i based on the US dollar, the derived GPCs are gpc 1t, gpc 2t & gpc 3t. We aim to use GP C it = δ 1i gpc 1t + δ 2i gpc 2t + δ 3i gpc 3t to forecast s it. The rest of the model specifications that we take into account is similar to what has been proposed by Engel, Mark, and West (2015). First we assume that GP C it s it is stationary and can be useful to capture the stationary regularity of future values of s it through s it+h s it where h = 1, 4, 8, 12 is quarterly horizons of forecasting. 10

11 Let ĜP C it = δ 1i ĝpc 1t + δ 2i ĝpc 2t + δ 3i ĝpc 3t for currencies i = 1,..., 17. We use it as a central tendency to estimate the coefficients of following regression: (3.2) s it+h s it = α i + β(ĝp C it s it ) + ɛ it+h, ɛ it+h N.i.i.d.(0, σ 2 ɛ i ) where α i is the individual effect of currency i. The estimated coefficients α i and β can be used to predict the future value of the nominal exchange rates. As an example, the following Figure illustrates the procedure for quarterly horizon h = 4 in the long sample forecasting: We use data from 1973:1 to 1986:4 to estimate ĜP C it and then estimate the panel regression (3.3) s it+4 s it = α i + β(ĝp C it s it ) + ɛ it+4, t {1973 : 1,..., 1985 : 4}. By the estimated coefficients α i and β from the regression (3.3), we calculate the predicted value of s i,1987:4 s i,1986:4 using the following equation (3.4) s i,1987:4 s i,1986:4 = α i + β(ĝp C i,1986:4 s i,1986:4 ). 11

12 We repeat this process by adding another observation to the end of the sample to produce predictions by a recursive method. 5 Also, the forecast evaluation is based on the method and measurement presented in Engel, Mark, and West (2015) which is root mean squared prediction error (RMSPE). We compute Theil s U-statistic (Theil (1971)) that is equal to a ratio by dividing RMSPE of factor model (GPCA or FA) to the RMSPE of the random walk model. The U-statistic less than one means that the factor model has a better performance than random walk model. 3.3 Discussion of results In the PCA, most of the variation among individual variables has been retained by the first three factors. But the GPCA, in addition to what has been captured by the PCA, also captures the variation and covariation between different points of time across all variables. That is the reason why factors are converging to the same pattern despite some differences at the beginning. Figure 1 depicts the time plot of three GPCs for the whole sample 1973:1 to 2007:4. Table 4 presents the median Theil s U-statistics for early, late and long samples regarding to the following model: 6 The model that uses the GPCA to extract factors (ĜP C it s it ), and 5 We need to centralize data to extract the factors, and, to make sure that the forecasts are truly out-of-sample, data are centralized only using in-sample data. 6 For this paper, the U-statistic is defined as the ratio of RMSP E Model to RMSP E RandomW alk. Results for individual countries are available upon request. 12

13 The model that uses FA to extract factors ( F it s it ). 7 First column indicates the factor model that has been used in the forecasting evaluation. Second column lists the type of sample and number of currencies in that sample. Third column presents the measurement method that has been used to evaluate the forecastability power of the model which is the median U-statistic. Also, It reports the number of currencies in the sample that have the U-statistic value less than one 8. The last four columns are reporting the median U-statistic for different horizons (h = 1, 4, 8, 12) and samples. The results presented in Table 4 shows that the GPCA is outperforming both FA and the driftless random walk model in all cases. The FA used by Engel, Mark, and West (2015) beats the random walk model only in 5 cases, and in all the 5 cases the FA is dominated by the GPCA. Engel, Mark, and West (2015) use three sets of auxiliary macro variables as a measure of central tendency to improve the factor model in a way that captures more regularity pattern in the exchange rates to forecast more accurate. These auxiliary macro variables are monetary model (Mark (1995)), Taylor rule (Molodtsova and Papell (2009)) and PPP (Engel et al. (2007)). Table 5 compares the results that has been obtained by the GPCA method without any auxiliary macro variables with the FA method presented 7 Although the results for three factors model have not been reported in Engel, Mark, and West (2015), fortunately, they have made their codes available on their website ( cengel/data/factor/factordata.htm) for replication. 8 U-statistic less than one means that the model has smaller RMSPE compare to the random walk. 13

14 in Engel, Mark, and West (2015) using auxiliary macro variables. Based on the results presented in Table 5, the GPCA itself is outperforming the FA method with auxiliary macro variables. 3.4 Comparing with an alternative method Another study related to the predictability of exchange rates that has been published recently is a paper by Wang and Wu (2015). To summarize, they argue that the information related to the third moment can be useful to improve the forecastability power of the exchange rates forecasting models. They apply the denoising source separation (DSS) algorithm (Särelä and Valpola (2005)) on the normalized nominal exchange rates s n it = s it µ si σ si extract independent components (IC) and mixing coefficients that can be to used to construct an IC-based fundamental exchange rate (Êit). Ê it s n it can be used to predict s it+h s it. The rest of the model is similar to the model presented by Engel, Mark, and West (2015). The number of factors has been determined by three different criteria, the cumulative percentage of total variance (CP V ) (Jackson (1993)), Bayesian information criterion (BIC 3 ) and IC p2 (Bai and Ng (2004)). They conclude that the IC-based model can perform better than the PCA in the context of out-of-sample forecasting of exchange rates. Table 6 compares the results based on the GPCA method and the ICbased model, using the data provided by Wang and Wu (2015). For most horizons the GPCA forecasts better than the IC-based model, the only 14

15 exception being horizon h = 12 for the early sample. 4 Conclusion In this paper we show that using factors that incorporate both contemporaneous and across-time correlations substantially improves out-of-sample forecasting performance. Exchange rates are found to converge to such factors, while the convergence is not as clear when traditional methods of extracting factors are used (with or without including macroeconomic fundamentals). What do these factors represent? What are the underlying economic forces? Clearly these questions go beyond the econometric method we have employed, and we leave them for future research. 15

16 References Anderson, Theodore Wilbur An introduction to multivariate statistical analysis. Wiley-Interscience. Bai, Jushan Estimating cross-section common stochastic trends in nonstationary panel data. Journal of Econometrics 122 (1): Bai, Jushan and Serena Ng A PANIC attack on unit roots and cointegration. Econometrica 72 (4): Engel, Charles, Nelson C Mark, and Kenneth D West Factor model forecasts of exchange rates. Econometric Reviews 34 (1-2): Engel, Charles, Nelson C Mark, Kenneth D West, Kenneth Rogoff, and Barbara Rossi Exchange Rate Models Are Not as Bad as You Think [with Comments and Discussion]. NBER Macroeconomics annual 22: Engel, Charles and Kenneth D West Exchange rates and fundamentals. Journal of political Economy 113 (3): Gupta, AK and T Varga A new class of matrix variate elliptically contoured distributions. Statistical Methods Applications 3 (2): Gupta, Arjun K and Daya K Nagar Matrix variate distributions, vol CRC Press. Gupta, Arjun K, Tamas Varga, and Taras Bodnar contoured models in statistics and portfolio theory. Springer. Elliptically Jackson, Donald A Stopping rules in principal components analysis: a comparison of heuristical and statistical approaches. Ecology 74 (8): Maasoumi, Esfandiar and Levent Bulut Predictability and Specification in Models of Exchange Rate Determination. Springer, Mark, Nelson C Exchange rates and fundamentals: Evidence on long-horizon predictability. The American Economic Review :

17 Meese, Richard and Kenneth Rogoff. 1983a. The out-of-sample failure of empirical exchange rate models: sampling error or misspecification? University of Chicago Press, Meese, Richard A and Kenneth Rogoff. 1983b. Empirical exchange rate models of the seventies: Do they fit out of sample? Journal of international economics 14 (1-2):3 24. Molodtsova, Tanya and David H Papell Out-of-sample exchange rate predictability with Taylor rule fundamentals. Journal of international economics 77 (2): Rossi, Barbara Exchange rate predictability. Journal of economic literature 51 (4): Solat, Karo and Aris Spanos Generalized Principal Component Analysis: An Extension to the PCA. Virginia Tech, Working paper. Stock, James H and Mark W Watson Forecasting using principal components from a large number of predictors. Journal of the American statistical association 97 (460): Särelä, Jaakko and Harri Valpola Denoising source separation. Journal of machine learning research 6 (Mar): Theil, Henri Applied economic forecasting.. Wang, Yi-chiuan and Jyh-lin Wu Fundamentals and Exchange Rate Prediction Revisited. Journal of Money, Credit and Banking 47 (8):

18 Figures and Tables Figure 1: Generalized Principal Components t-plot Note that the boxes are the plots of GPCs from 10 th observation to the end of data set 18

19 Table 1: Normal/Principal Components Statistical GM Y t = A p(x t µ) + ɛ t t N [1] Normality Y t N (.,.) t N [2] Linearity E(Y t ) = A p(x t µ) t N [3] Constant covariance Cov(Y t ) = Λ p t N [4] Independence {Y t, t N} is an independent process [5] t-invariance Θ := (µ, A p, Λ p ) is not changing with t Note: Y t is the matrix of PCs; A p is a matrix consist of eigenvectors of Σ corresponding to the first p(< m) eigenvalues (sorted in a descending order) and Λ p = diag(λ 1,..., λ p ) where λ i is an eigenvalue of Σ, i {1,..., p} Table 2: Normal/Generalized Principal Components Statistical GM Y = A p(x µe T 1 )B + ɛ [1] Normality Y N (.,.) [2] Linearity E(Y ) = A p(x µe T 1 )B [3] Constant covariance Cov(Y ) = Λ p Γ T [4] Independence {Y t, t N} is an independent process [5] t-invariance Θ := (µ, A p, Λ p ) is not changing with t Note: In order to have an operational model, Φ has to be known so does B and Γ T ; Thus, they are not parameters and depend on the structure of Φ and Γ T we can adjust the factors to avoid time-varying factors (Solat and Spanos (2017)). 19

20 Table 3: Summary Statistic Country N Mean St. Dev. Min Max Australia Canada Denmark United Kingdom Japan Korea Norway Sweden Switzerland Austria Belgium France Germany Spain Italy Finland Netherlands Table 4: Forecast evaluation: GPCA vs FA (Engel, Mark and West(2015)) Horizon h Model Sample(# Currencies) Measurement h=1 h=4 h=8 h=12 ĜP C it s it Long sample (9) Median U-statistic (#U < 1) 0.996(7) 0.960(8) 0.910(8) 0.921(7) F it s it Long sample (9) Median U-statistic (#U < 1) 1.003(3) 0.996(5) 0.996(5) 1.038(4) ĜP C it s it Early sample (17) Median U-statistic (#U < 1) 0.993(15) 0.957(14) 0.919(16) 0.973(9) F it s it Early sample (17) Median U-statistic (#U < 1) 1.000(10) 0.995(9) 1.000(9) 1.130(3) ĜP C it s it Late sample (10) Median U-statistic (#U < 1) 0.993(7) 0.964(9) 0.887(10) 0.769(9) F it s it Late sample (10) Median U-statistic (#U < 1) 1.008(3) 1.020(3) 0.953(8) 0.822(8) Note: ĜP C it s it and F it s it represent deviations from factors produced by the GPCA and the FA, respectively. 20

21 Table 5: Forecast evaluation: GPCA vs FA+Auxiliary Macro variables (Engel, Mark, and West (2015)) Horizon h Model Sample(# Currencies) Measurement h=1 h=4 h=8 h=12 ĜP C it s it Long sample (9) Median U-statistic (#U < 1) 0.996(7) 0.960(8) 0.910(8) 0.921(7) F it s it + T aylor Long sample (9) Median U-statistic (#U < 1) 1.008(1) 1.035(0) 1.068(1) 1.052(3) F it s it + Monetary Long sample (9) Median U-statistic (#U < 1) 1.008(3) 1.064(3) 1.200(4) 1.456(4) F it s it + P P P Long sample (9) Median U-statistic (#U < 1) 1.002(3) 0.993(6) 0.942(7) 0.903(5) ĜP C it s it Early sample (17) Median U-statistic (#U < 1) 0.993(15) 0.957(14) 0.919(16) 0.973(9) F it s it + T aylor Early sample (17) Median U-statistic (#U < 1) 1.010(3) 1.041(2) 1.103(4) 1.156(3) F it s it + Monetary Early sample (17) Median U-statistic (#U < 1) 0.995(10) 0.997(9) 1.115(7) 1.190(7) F it s it + P P P Early sample (17) Median U-statistic (#U < 1) 0.998(7) 0.972(14) 1.015(8) 1.098(3) ĜP C it s it Late sample (10) Median U-statistic (#U < 1) 0.993(7) 0.964(9) 0.887(10) 0.769(9) F it s it + T aylor Late sample (10) Median U-statistic (#U < 1) 1.009(2) 1.036(2) 1.004(4) 0.828(8) F it s it + Monetary Late sample (10) Median U-statistic (#U < 1) 1.013(3) 1.033(4) 0.977(6) 1.126(5) F it s it + P P P Late sample (10) Median U-statistic (#U < 1) 1.005(4) 0.999(5) 0.900(8) 0.727(9) Note: ĜP C it s it and F it s it represent deviations from factors produced by the GPCA and the FA, respectively. Table 6: Forecast evaluation: GPCA vs ICA (Wang and Wu (2015)) Horizon h Model Sample(# Currencies) Measurement h=1 h=4 h=8 h=12 ĜP C it s it Long sample (9) Median U-statistic (#U < 1) 0.995(7) 0.963(8) 0.937(8) 0.937(8) Ê it s n it(criterion: CPV) Long sample (9) Median U-statistic (#U < 1) 1.000(4) 0.986(7) 0.956(7) 0.946(9) Ê it s n it(criterion: BIC 3 ) Long sample (9) Median U-statistic (#U < 1) 1.000(4) 0.991(8) 0.955(7) 0.941(9) Ê it s n it(criterion: IC p2 ) Long sample (9) Median U-statistic (#U < 1) 1.001(3) 1.002(3) 0.974(7) 0.951(9) ĜP C it s it Early sample (17) Median U-statistic (#U < 1) 0.993(15) 0.957(14) 0.919(16) 0.973(9) Ê it s n it(criterion: CPV) Early sample (17) Median U-statistic (#U < 1) 0.999(11) 0.991(13) 0.950(13) 0.965(10) Ê it s n it(criterion: BIC 3 ) Early sample (17) Median U-statistic (#U < 1) 0.999(10) 0.994(13) 0.956(14) 0.964(10) Ê it s n it(criterion: IC p2 ) Early sample (17) Median U-statistic (#U < 1) 1.000(7) 0.999(9) 0.976(14) 0.976(10) Note: ĜP C it s it and Êit s n it represent deviations from factors produced by the GPCA and the ICA, respectively. 21

22 Appendix Proof of equation 2.9: To prove the claim that equation 2.9 is the most appropriate derivation to capture the highest variation in the data, we can extend the traditional proof of the classical vector-variate PCA. The first element of matrix Y can be found by the following optimization problem: Let v and u be vectors of order m 1 and T 1, respectively. Assume v and u are optimizing V ar(v Xu) subject to the restrictions v = 1 and u = 1. The Lagrangian function is: (4.1) L (v, u, ξ v, ξ u ) = (v Σv u Φu) ξ v (v v 1) ξ u (u u 1). The first-order conditions are: (4.2) Σv ξ v v = 0 = Σv = ξ v v and (4.3) Φu ξ u u = 0 = Φu = ξ u u. So, ξ v (ξ u ) is an eigenvalue for Σ (Φ) and v (u) is the corresponding eigenvector. In fact, because ξ v (ξ u ) is optimizing the variance function, so it is the highest eigenvalue for Σ (Φ). By repeating the same procedure, for kl th element of Y we solve the following optimization problem k 1 l 1 (4.4) V ar(v [X v ixu j ]u). i=1 j=1 22