Projected Principal Component Analysis. Yuan Liao

Size: px

Start display at page:

Download "Projected Principal Component Analysis. Yuan Liao"

Justin Collins
6 years ago
Views:

1 Projected Princial Comonent Analysis Yuan Liao University of Maryland with Jianqing Fan and Weichen Wang January 3, 2015

2 High dimensional factor analysis and PCA Factor analysis and PCA are useful tools for dimension reductions. In high. dim. models, they are asymtotically equivalent. In articular, PCA is often used for estimating factor models. However, require relatively large samle size, not suitable for high-dimension-low-samle-size alications.

3 Conventional factor models y it = λ i f t + u it, i = 1,...,N, t = 1,...,T, Financial asset returns: Over 3,000 assets are being traded in U.S. market Monthly data over three consecutive years contain merely 36 observations However, consistent estimation of loadings and factors (if unknown) requires T.

4 Semi-arametric factor model: Connor et al. (2012): y it = g(x i ) f t + u it, i = 1,...,n, t = 1,...,T, Loadings are modeled using observed characteristics X i In financial al., X i can be firm secific var.: market caitalization, rice-earning ratio, etc In health studies, X i can be individual char.: weight, genetic information, etc. g( ) is a nonarametric function vector. The models assumes loading is fully exlained by X i. can be restrictive in many alications.

5 A more ractical model: λ i = g(x i ) + γ i γ i : loading comonent that cannot be exlained by X i, orthogonal to X i, E(γ i X i ) = 0. Loadings deend on obser. characteristics, but not fully exlained.

6 This aer: Projected-PCA Model in matrix form Y = {G(X) + Γ}F + U Projected-PCA Proose Projected-PCA to estimate factors and loadings First roject data Y on the sace sanned by X, then run PCA. If rojection is genuine, achieves faster convergence rate. Projection ste smooths away U and Γ. Remark Assume G( ) additive, estimate it by sieve aroximation.

7 Overview of the results Consistency is granted even if T is finite. If T, achieve faster convergence rate if G( ) is smooth. Secification tests on the loading orthogonal decomosition: Λ = G(X) + Γ, E(Γ X) = 0 H 0 : G(X) = 0: chara. have exlaining ower on loadings. H 0 : Γ = 0: chara. fully exlain loadings. (rojection-based) Eigen-value ratio for selecting num. factors.

8 Projected-PCA

9 Main Idea Y = ΛF + U, E(U X) = 0. Normalization condition: Proj. oerator P ( ) = E( X). F F/T = I K, Projected-PCA: Λ P Λ is diagonal with district entries. Oerate P on both sides: P Y = P ΛF, Projected data matrix: Y P Y/T = FΛ P ΛF /T, 1 T Y P YF = 1 T FΛ P ΛF F = FΛ P Λ Columns of F are first K eigenvectors of Y P Y/T. 1 T P YF = 1 T P ΛF F = P Λ Both F and P Λ can be recovered.

10 The Estimator Observe X = {X i } i i.i.d. data, each X i is d-dim. r.v. Let Φ(X) be (Jd) matrix of basis functions, e.g., B-sline transformations of X. J is the number of chosen basis functions. Let P = Φ(X)(Φ(X) Φ(X)) 1 Φ(X) be Projection matrix. Projected-PC estimators: The columns of F/ T are the first K eigenvectors of Y PY. Ĝ(X) = 1 T PY F, Λ = 1 T Y F, Least squares on F, Γ = Λ 1 Ĝ(X) = (I P)Y F. T

11 Asymtotic Proerties

12 Main Conditions Identification: 1 T F F = I K, Λ PΛ is diagonal with distinct entries Genuine rojection and ervasive factors: c 1 < λ min (Λ PΛ/) < λ max (Λ PΛ/) < c 2 Imlications: Projection is genuine: X has exlaining ower on Λ Factors are ervasive: most elements of Λ are significant. Can extend to semi-weak factors.

13 Theorem In the conventional factor model Y = ΛF + U, 1 T F F 2 F = O P ( J ), 1 Ĝ(X) PΛ 2 F = O P ( J T + J2 2 ). In the semi-arametric factor model Λ = G(X) + Γ, ( 1 T F F 2 1 F = O P + 1 ) J κ, 1 Ĝ(X) G(X) 2 F = O P ( J + J 2 T + J J κ + Jν ). κ is the smoothing arameter, the smoother G( ), the larger κ. ν = var(γ i ).

14 Remarks Consistency only requires, and T can be bounded. When,T, faster convergence rate is achieved. For instance, when Γ = 0, otimal rate is ( ) 1 T F F 2 1 F = O P, ( 1 Ĝ(X) Λ 2 F = O P 1 (T ) 1 1/κ /κ ). Comared with rates of conventional factor models: ( 1 T F F 2 1 F = O P + 1 ) ( 1 1, T 2 Λ Λ 2 F = O P + 1 T ).

15 Loading Secification Tests

16 Two secification tests Λ = G(X) + Γ, E(Γ X) = 0 H0 1 : G(X) = 0 tests whether X has exlaining ower on loadings. diagnostic tool as whether or not to emloy rojected-pca. H0 2 : Γ = 0 tests whether X fully exlains loadings. tests adequacy of semi-ara. factor models in the literature.

17 H0 1 : G(X) = 0 As PΛ G(X), equivalent to testing H 0 : PΛ = 0. rejects if S G is large, where S G = 1 tr(w 1 Λ P Λ), W 1 = ( 1 Λ Λ) 1. Λ is traditional PC estimator, as the rojected-pca is inconsistent under H 1 0. Under H0 1, as,t,j, S G JdK 2JdK d N(0,1),

18 H 2 0 : Γ = 0 As Γ Λ PΛ, equivalent to testing H 0 : PΛ = Λ. rejects if S Γ is large, where S Γ = tr( Λ(I P)Σ 1 u (I P) Λ) Assume Σ u to be diagonal. Relace with a diagonal estimator. Can extend to non-diagonal but sarse Σ u. Under H0 2, as,t,j, TS Γ K 2K d N(0,1).

19 Testing on real data Data: daily ex. returns of S&P 500 constituents Four characteristics X as in Connor et al. 12: size, value, momentum and volatility. Tests are based on rolling windows. Results: Strong evidence of exlaining owers of X on loadings; roviding theoretical basis for rojected-pca. For a majority of eriod, Γ = 0 is rejected; assets characteristics cannot fully exlain markets betas.

20 Figure: Normalized statistics. Uer: S G ; Lower: S Γ Testing semi ara factor model for G(X) Normalized statistic T=10 T=21 T=63 T= date Testing semi ara factor model for Gamma Normalized statistic T=10 T=21 T=63 T= date

21 Estimating the Number of Factors

22 Strengths of factors are affected by eigenvalues of G(X)G(X). G(X)G(X) has K very siked eigenvalues. P Y = G(X)F imlies 1 T P Y(P Y) = G(X)G(X) Eigenvalues of G(X)G(X) are the same as those of YP Y /T. Hence, emloy eigenvalue-ratio method (Ahn and Horenstein 13 and Lam and Yao 12) on the rojected data PY: K = arg max 1 k [min{,t }/2] λ k (Y PY) λ k+1 (Y PY).

23 Allow semi-weak factors: c < λ min ( α G(X) G(X)) < λ max ( α G(X) G(X)) < C for some α (0,1]. Conventionally, α = 1, strong factors. Theorem Under regularity conditions, as,t,j, P( K = K ) 1.

24 Numerical Studies

25 Design 1 Data of 337 stocks from S&P 500 are collected. Use four char. and three factors. True values of GDP are calibrated from the data.

26 Figure: P-PCA of Ĝ G, P-PCA of Γ Γ, and regular PCA of Λ Λ Maximal Error PPC:G(X) T=10 PPC:Gamma PC:Lambda Maximal Error PPC:G(X) T=50 PPC:Gamma PC:Lambda Frobenius Error PPC:G(X) T=10 PPC:Gamma PC:Lambda Frobenius Error PPC:G(X) T=50 PPC:Gamma PC:Lambda

27 Figure: P-PCA of F F, and regular PCA of F F Maxmimal Error Frobenius Error PPC:F PC:F T= PPC:F PC:F T= Maxmimal Error Frobenius Error PPC:F PC:F T= PPC:F PC:F T=

28 Design 2 Simulate one characteristic and three factors. g = (x,x 2 1,x 3 2x), Γ = 0. Comare three methods for estimating loadings: Projected-PCA, PCA, and least squares w/ known factors (SLS). Comare two methods for estimating K : on rojected data and on non-rojected data. Results: Projected-PCA erforms: significantly better than regular PCA. as well as if the factors are known when is large. more accurately in estimating K.

29 Figure: Ĝ(X) G(X) of Projected-PCA, PCA, and SLS Maximal Error Frobenius Error PPC:G(X) PC:Lambda SLS:G(X) T= PPC:G(X) PC:Lambda SLS:G(X) T= Maximal Error PPC:G(X) T=50 PC:Lambda SLS:G(X) Frobenius Error PPC:G(X) T=50 PC:Lambda SLS:G(X)

30 Figure: Estimating K = 3 using Projected-PCA with Ahn and Horenstein 13 (non-rojected data), 50 relications mean standard deviation Estimating K by AH and Projected PC (mean) Estimating K by AH and Projected PC (SD) Number of factors K PPC T=50 PPC T=10 AH T=50 AH T=10 Number of factors K PPC T=50 PPC T=10 AH T=50 AH T=

31 Summary Semi-arametric Factor model Loadings deend on observed characteristics. More flexible than existing model secifications. Verified by emirical studies. Projected-PCA: Aly PCA on rojected data. Consistency is granted even under finite samle size. Faster rate of convergence Loading Secification Tests: Tests whether char. have exlaining owers on loadings Tests whether char. fully exlain loadings.

By Jianqing Fan, Yuan Liao and Weichen Wang Princeton University and University of Maryland

Submitted to the Annals of Statistics SUPPLEMENARY APPENDIX O HE PAPER PROJECED PRINCIPAL COMPONEN ANALYSIS IN FACOR MODELS By Jianqing Fan, Yuan Liao and Weichen Wang Princeton University and University