Supplementary Materials for Tensor Envelope Partial Least Squares Regression

Transcription

1 Supplementary Materials for Tensor Envelope Partial Least Squares Regression Xin Zhang and Lexin Li Florida State University and University of California, Bereley 1 Proofs and Technical Details Proof of Lemma 1 Proof. From the vectorized linear model 3, we can see that B m+1 is the regression coefficient matrix for the multivariate linear regression of Y on vecx. Therefore B m+1 = cov 1 {vecx}cov{vecx, Y } = Σ 1 m Σ 1 1 Cm+1. It follows from the basic property of Tucer operator that B = C; Σ 1 1,..., Σ 1 m, I r. Proof of Lemma 2 Proof. Recall that in the tensor PLS regression, EY X = B m+1 vecx = EY T = Ψ m+1 vect, where T = X; W T 1,..., W T m. Therefore, B m+1 vecx = Ψ m+1 vect = Ψ m+1 vec X; W T 1,..., W T m = Ψ m+1 W T m W T 1 vecx = Ψ; W 1,..., W m, I r m+1 vecx. This implies that B = Ψ; W 1,..., W m, I r under the PLS regression assumption, and hence B PLS = Ψ; Ŵ1,..., Ŵm, I r. The rest of the proof follows from the fact that cov{vect } = WmΣ T m W m W1 T Σ 1 W 1 and then applying Lemma 1 on the regression of Y on T and 1

2 Proof of Proposition 1 Proof. From Lemma 2, we have B PLS = Ψ; Ŵ1,..., Ŵm, I r = ĈT ; Ŵ 1 T Σ 1 Ŵ 1 1,..., Ŵ m T Σ m Ŵ m 1, I r ; Ŵ1,..., Ŵm, I r = ĈT ; Ŵ1Ŵ 1 T Σ 1 Ŵ 1 1,..., ŴmŴ m T Σ m Ŵ m 1, I r = Ĉ; Ŵ1Ŵ 1 T Σ 1 Ŵ 1 1 Ŵ1 T,..., ŴmŴ m T Σ m Ŵ m 1 Ŵm, T I r, where the last equality follows from ĈT = Ĉ; Ŵ T 1,..., Ŵ T m, I r. The conclusion then follows from Ŵ1Ŵ T 1 Σ 1 Ŵ 1 1 Ŵ T 1 = PŴ Σ 1 Σ and B OLS = 1 1 Ĉ; Σ 1,..., Σ m, I r. Proof of Proposition 2 Proof. First, X Q X P implies 0 = cov{vecx Q, vecx P } = Σ m Q Σ P Σ 1. It holds if and only if Q Σ P = 0. By the definition of a reducing subspace Coo et al., 2010, Q Σ P = 0 if and only if spanp = spanγ is a reducing subspace of Σ, i.e., Σ = Γ Ω Γ T + Γ 0Ω 0 Γ T 0. Second, we substitute X = X P + X Q into 3 and get Y = B m+1 vecx P + B m+1 vecx Q + ε = B P m+1 vecx P + B Q m+1 vecx Q + ε. Therefore, Y X Q X P implies B Q = 0, which is equivalent to B = B P = B Γ Γ T, which gives the parametrization B = Θ; Γ 1,, Γ m, I r. Proof of Proposition 3 Proof. We first show T ΣX B E Σm Bm EΣ1 B1. From Definition 2, this means we need to show that a E Σm Bm EΣ1 B1 is a reducing subspace of Σ X, and b it contains spanb m+1. It is straightforward to see a from Σ X = Σ m Σ 1 and that E Σ B is a reducing subspace of Σ for each. To show b, we can write B = P B, where P is the projection on to E Σ B, because 2

3 spanb E Σ B by definition. This further implies that B = B P, = 1,..., m, and that B = B 1 P 1 2 m P m. Taing mode-m + 1 matricization on both sides of the last equation, we have b. We next show T ΣX B E Σm Bm EΣ1 B1. By definition, we can write T ΣX B = E m E 1 for some E IR p, = 1,..., m. It remains to show that E E Σ B. We achieve this by showing c E is a reducing subspace of Σ, d E contains spanb, and then by noticing E Σ B is the smallest subspace satisfying c and d. Note that d can be directly obtained from Proposition 2. To get c, we recall that T ΣX B is a reducing subspace of Σ X = Σ m Σ 1 and thus P Em E 1 Σ m Σ 1 Q Em E 1 = 0, where P Em E 1 = P Em P E1 equation we get, and Q Em E 1 = I P Em E 1. Expand the above P Em Σ m P E1 Σ 1 P Em Σ m P Em P E1 Σ 1 P E1 = 0, which implies the following equality by right-multiplying P Em P E2 I p1 : P Em Σ m P Em P E2 Σ 2 Q E2 P E1 Σ 1 Q E1 = 0, which implies E 1 reduces Σ 1. Similarly we get E reduces Σ for all = 1,..., m. This completes the proof. Proof of Lemma 3 Proof. From Algorithm 4, we now that w s is the dominant eigenvector of C T s 1 C s 1, T which equals to Q s 1 C0 C 0 Q s 1. The conclusion follows from noticing C = T C 0 C 0. Proof of Theorem 1 Proof. From Lemma 3, we see that w 1 is the first eigenvector of C. Then for s > 1, we have w s is the first eigenvector of Q s 1 C Q s 1 where Q s 1 is the projection 3

4 onto the orthogonal complement of span Σ w 1,..., w s 1. Following the proof of Proposition 4.1 in Coo et al. 2013, we have W 0... W u = E Σ C = W u +1 =... = W p, where E Σ C is the Σ -envelope of span C. We next need to show E Σ B = E Σ C. From Lemma 3, we see that span C = spanc. From Lemma 1, we see that spanb = spanσ 1 C. Finally, from Proposition 2.4 of Coo et al. 2010, we have E Σ B = EΣ Σ 1 B and thus E Σ B = EΣ C = E Σ C. Proof of Theorem 2 Proof. From Theorem 1, if d is chosen as u, the population value spanw = W u = E Σ B. Since the sample version of Algorithm 4 is based on eigen-decomposition and n-consistent sample covariance matrices Ĉ, Σ Y, and Σ, = 1,..., m, it is clear that Ŵ is n-consistent for W. Hence Ŵ is n-consistent for the envelope E Σ B. For d u, we have W d = W u = E Σ B, therefore the projection PŴ Σ is n-consistent for P W Σ. Since W is a semi-orthogonal basis for the envelope E Σ B, it is a reducing subspace of Σ and hence P W Σ = P W by definition. Therefore, PŴ Σ is n-consistent estimator for the projection onto the envelope E Σ B. Then from Proposition 1, we recall that BPLS = B OLS 1 PŴ1 Σ 1 2 m PŴm Σ m. The n-consistency of BPLS then follows from the n-consistency of BOLS and PŴ Σ. 2 Additional simulations We consider an additional simulation example, with a univariate response and a 3-way tensor predictor with higher dimension and ran. Specifically, the simulation setup is similar to that in Section 5.2, with two main changes. The first is that the response is now a scalar, which allows a more direct comparison with the CP method of Zhou et al that was designed for a univariate response. The second is that we now consider both the original setup of a predictor tensor X IR with a core tensor Θ IR p = 20, u = 2, and a new setup of a predictor tensor X IR with 4

5 p, u 20, 2 40, 5 p, u 20, 2 40, 5 Model Prediction OLS CP TEPLS TEPLS-CV I II III I II III Model Estimation OLS CP TEPLS TEPLS-CV I > 10 4 > II III > 10 6 > I > 10 5 > II III > 10 7 > Table S1: Univariate response and 3-way predictor. Performance under various scenarios and comparison of estimators. OLS, CP, tensor envelope PLS with true and estimated envelope dimensions. Reported are the average and standard error in parenthesis of the prediction mean squared error evaluated on an independent testing data, and the estimation error, all based on 100 data replications. a core tensor Θ IR p = 40, u = 5. The latter has a higher predictor dimension and ran, and is comparable to the dimension of the ADHD real data example in Section 6.2. The sample size for training and testing data is still fixed at n = 200. Table S1 summarizes the prediction and estimation results based on 100 data replications. It is again clearly seen that the proposed tensor envelope PLS method is more competitive than the alternative solutions in terms of both prediction and estimation accuracy across all model scenarios. References Coo, R. D., Helland, I. S., and Su, Z Envelopes and partial least squares regression. J. R. Stat. Soc. Ser. B. Stat. Methodol., 755:

6 Coo, R. D., Li, B., and Chiaromonte, F Envelope models for parsimonious and efficient multivariate linear regression. Statist. Sinica, 203: Zhou, H., Li, L., and Zhu, H Tensor regression with applications in neuroimaging data analysis. Journal of the American Statistical Association, :