Modeling Multi-Way Functional Data With Weak Separability

Transcription

1 Modeling Multi-Way Functional Data With Weak Separability Kehui Chen Department of Statistics University of Pittsburgh, Seville, Spain December 09, 2016

2 Outline Introduction. Multi-way FPCA, marginal FPCA and product FPCA. Weak separability. Data examples.

3 Based on joint work with Brian Lynch (Pittsburgh), Pedro Delicado (Barcelona) and Hans-Georg Müller (Davis). K. Chen and H.G. Müller (2012), Modeling repeated functional observations, JASA. K. Chen, P. Delicado and H.G. Müller (2016), Modeling functional-valued stochastic processes, with applications to fertility dynamics, JRSSB. B. Lynch and K. Chen (2016+), Weak Separability: Concepts and Inference. In Preparation. Partially supported by NSF

4 Functional Data Functional data consist of fully or partially observed samples of random functions [0, T] R q, usually q = 1. As infinite-dimensional objects, functional data require dimension reduction. Such data are ubiquitous longitudinal studies; tracking and monitoring. Any data observed over a continuum for many individuals.

5 Function-Valued Stochastic Processes Traditional: T X(t) R, X L 2 (T ), X is smooth Now the value of the process at each t T is a random function X(, t) on a domain S: with T X(, t) L 2 (S), X L 2 (S T ), E(X(s, t)) = µ(s, t), C(s, t; u, v) = cov(x(s, t), X(u, v)).

6 Fertility Data ASFR(.,t) for USA ASFR(s,t) for USA ASFR s= t=1960 t=1980 t=2000 s= t=

7 Other Examples of Multi-Way Functional Data Multi-way functional data: the time index t of the stochastic process and the argument s of the observed functions; or the other way around. X(s, t) L 2 (S T ). Data obtained from tracking apps where animal s/people s 24-hour profiles of activities are recorded every day. EEG/fMRI data with spatial and/or temporal index, or even with longitudinal follow-ups over years. Different from traditional spatial-temporal data, as we consider a sample of n realizations.

8 Karhunen-Loève (KL) Representation X(s, t) = µ(s, t) + Z r γ r (s, t), s S, t T. (1) r=1 Here {γ r : r 1} are the eigenfunctions of the linear operator with kernel C(s, t; u, v) = cov(x(s, t), X(u, v)). {Z r = γ r (s, t)x c (s, t)dsdt : r 1} are the (uncorrelated) functional principal components (FPCs) with E(Z r ) = 0. Multi-Way FPCA: Modeling the variability in the random process and dimension reduction. Serve as building blocks for further modeling and regularization.

9 Challenges with Multi-Way FPCA A non-parametric modeling of C(s, t; u, v) = cov(x(s, t), X(u, v)) is very difficult. For dense regular design with p 1 grids in T and p 2 grids in S, a nonparametric estimation of C(s, t; u, v) based on sample covariance has a huge dimension p 2 1 p2 2. Covariance estimation for sparse designs requires 4-dimensional smoothing. The effects of s and t are hard to separate and to visualize.

10 Marginal Kernel Define the the marginal covariance functions t C(s, t; u, t)dt C S (s, u) = s,t C(s, t; s, t)dsdt, C T (t, v) = C(s, t; s, v)ds. C(s, t; s, t)dsdt s s,t The marginal kernels are normalized to have s C S(s, s)ds = 1 and t C T (t, t)dt = 1. The eigen decompositions are C S (s, u) = j=1 λ jψ j (s)ψ j (u) and C T (t, v) = k=1 γ kφ k (t)φ k (v), where λ 1 λ 2..., and γ 1 γ 2..., are the marginal eigenvalues.

11 A Tensor Product Representation The product of marginal eigen functions φ k ψ j forms an orthogonal basis on L 2 (S T ). We have the representation of X(s, t), Product FPCA X(s, t) = µ(s, t) + X(s, t) µ(s, t) + j=1 k=1 P j=1 k=1 χ jk φ k (t)ψ j (s). K χ jk φ k (t)ψ j (s).

12 Loss bound on product FPCA For P 1 and K 1, consider the following loss minimization P K E {X(s, t) X, φ k ψ j φ k (t)ψ j (s)} 2 dsdt. S,T j=1 k=1 Let Q be the minimum unexplained variance that can be achieved using any basis f k (t)g j (s), we have Q < Q +ae X 2 where a = min(a T, a S ), with (1 a T ) denoting the fraction of variance explained by K terms for G T (t, v) and analogously for a S.

13 The product FPCA under strong separability Strong separability: C(s, t; u, v) = ac S (s, u)c T (t, v), with s C S(s, s)ds = 1 and t C T (t, t)dt = 1. We can show that C S and C T are the same as the marginal kernels. Under strong separability, the eigenfunctions of C(s, t; u, v) is in the form of the product of marginal eigenfunctions.

14 Strong separability Under strong separability: X(s, t) = µ(s, t) + j=1 k=1 is the same as the Multi-Way KL expansion. The scores χ jk are mutually uncorrelated. χ jk φ k (t)ψ j (s), However, the strong separability assumption of a covariance is often too restrictive.

15 Brian & Chen (2016+) Concept of weak separability For any orthogonal bases {f j, j 1} in L 2 (S) and {g k, k 1} in L 2 (T ), we can have X(s, t) = µ(s, t) + j=1 k=1 χ jk f j (s)g k (t). X(s, t) is weakly separable if there exist orthogonal bases {f j, j 1} and {g k, k 1} such that cov( χ jk, χ j k ) = 0 for j j or k k., i.e., the scores { χ jk, j 1, k 1} are uncorrelated to each other.

16 Unique bases for weak separability Brian & Chen (2016+) If X is weakly separable, the pair of bases {f j, j 1} and {g k, k 1} that satisfies weak separability is unique: f j (s) ψ j (s), g k (t) φ k (t), where ψ j (s) and φ k (t) are the eigenfunctions of the marginal kernels. Weak separability is testable.

17 Weak separabile covariance structure Under weak separability, the covariance becomes C(s, t; u, v) = η jk ψ j (s)ψ j (u)φ k (t)φ k (v), (2) j=1 k=1 where η jk = var(χ jk ). Strong separability is a special case of weak separability: Strong separability is equivalent to the condition that η jk = aλ j γ k.

18 Weak separable covariance structure cont. Define the two-way array V = (η jk, j 1, k 1), where η jk = var(χ jk ). rank + (V) = min{l : V = V V l, V i 0, rank(v i ) = 1, i}, where V i 0 means that V i is entrywise nonnegative. If V has rank + (V) = L, then V has nonnegative decomposition V = L l=1 al Λ l (Γ l ) T, where Λ l = (λ l j, j 1) and Γ l = (γk l, k 1) are all nonnegative for l = 1,..., L.

19 Weak separable covariance structure cont. Brian & Chen (2016+): a notion of L-separability, and the strong separability corresponds to 1-separability. We have C(s, t; u, v) = L a l CS l (s, u)cl T (t, v), l=1 where C l S (s, u) = j λl j ψ j(s)ψ j (u) and C l T (t, v) = k γl k φ k(t)φ k (v).

20 A variation: Marginal FPCA Using the marginal eigenfunctions {ψ j : j 1} of L 2 (S), X(s, t) = µ(s, t) + ξ j (t)ψ j (s) with random coefficient functions {ξ j : j 1}. Applying the KL representations of ξ j (t), ξ j (t) = k=1 χ jkφ jk (t), with eigenfunctions φ jk and FPCs χ jk, leads to X(s, t) = µ(s, t) + j=1 j=1 k=1 χ jk φ jk (t)ψ j (s).

21 Fertility Data -Specific Fertility Rate (AFSR) for 17 countries, 1951 to s of mothers s range from 12 to 55 years old. (Human Fertility Database 2013 (HFD-2013)) Data: 17 independent units (countries), corresponding to a realization of the function valued stochastic process ASFR(, t) at each year t. Observation grid (age, calendar-year) has equidistant points. AFSR for age s (expressed in years) and calendar year t: Births during the year t to women of age s Person-years lived for the year t by women of age s.

22 AUT BGR CAN CZE FIN FRA HUN JPN NLD PRT SVK SWE

23 PRT SVK SWE CHE GBRTENW GBR_SCO USA ESP

24 Mean function µ(s, t) ASFR sample mean ASFR sample mean

25 Comparing 2-d FPCA with Marginal FPCA and Product FPCA (1) As expected, standard FPCA based on the two-dimensional Karhunen-Loève expansion needs less components to explain a certain amount of variance, as 4 eigenfunctions lead to a FVE of 89.74%, while marginal FPCA representation achieves a FVE of 87.51% with 6 terms, and product FPCA needs 7 terms to explain 87.42%.

26 Table: Fraction of Variance Explained (FVE) of ASFR(s, t) for the leading terms in the proposed marginal FPCA, product FPCA and 2d FPCA. Number of terms are selected to have accumulative FVE more than 85%. marginalfpca FVE productfpca FVE 2d FPCA FVE Six Seven Four ˆφ 11 (t) ˆψ 1 (s) ˆφ1 (t) ˆψ 1 (s) 53.7 ˆγ 1 (s, t) ˆφ 21 (t) ˆψ 2 (s) ˆφ2 (t) ˆψ 2 (s) 8.18 ˆγ 2 (s, t) ˆφ 22 (t) ˆψ 2 (s) 6.88 ˆφ1 (t) ˆψ 2 (s) 8.06 ˆγ 3 (s, t) ˆφ 12 (t) ˆψ 1 (s) 4.63 ˆφ3 (t) ˆψ 2 (s) 5.54 ˆγ 4 (s, t) 6.05 ˆφ 23 (t) ˆψ 2 (s) 4.41 ˆφ2 (t) ˆψ 1 (s) 4.4 ˆφ 31 (t) ˆψ 3 (s) 4.22 ˆφ4 (t) ˆψ 2 (s) 3.86 ˆφ 1 (t) ˆψ 3 (s) 3.68

27 Comparison Cont. (2) Product FPCA and Marginal FPCA represent the functional data as a sum of terms that are products of two functions, each depending on only one argument. This provides for much better interpretability and feature discovery. (3) Marginal FPCA makes it much easier than standard FPCA to analyze the time dynamics of the fertility process.

28 Eigen Functions and Score Functions Eigenfunction 1 (FVE: 61.16%) Eigenfunction 2 (FVE: 27.72%) Eigenfunction 3 (FVE: 6.93%) Eigenfunction Eigenfunction Eigenfunction Functional scores at eigenfunction 1 Functional scores at eigenfunction 2 Functional scores at eigenfunction 3 Scores at eigenfunction USA BGR CZE HUN SVK AUT CAN FRA SWE GBRTENW FIN GBR_SCO PRT CHE JPN ESP NLD BGR USA SVK GBRTENW GBR_SCO HUN CZE PRT AUT CAN ESP CHE JPN FRA FIN SWE NLD Scores at eigenfunction SVK JPN CAN USA FIN PRT NLD CZE FRA BGR GBR_SCO HUN CHE ESP SWE GBRTENW AUT USA FRA GBRTENW SWE FIN NLD GBR_SCO CAN CHE AUT BGR HUN CZE JPN SVK ESP PRT Scores at eigenfunction NLD PRT ESP AUT FIN SWE CAN CHE HUN GBR_SCO SVK USA BGR FRA GBRTENW CZE JPN ESP GBRTENW GBR_SCO SWE CHE NLD CAN PRT JPN FIN HUN USA FRA AUT SVK CZE BGR

29 Track Plots For Selected Countries Scores at eigenfunctions 2 vs. 1 Scores at eigenfunction JPN 1966.PRT 1966.ESP 1966.NLD 1951.PRT 1951.NLD 1981.PRT 2006.NLD 1981.ESP 1981.JPN 1996.NLD 1996.JPN 1996.PRT 1981.USA 1981.NLD 2006.CZE 2006.JPN 2006.ESP 2006.PRT 1951.ESP 1996.ESP 1966.JPN 2006.USA 1951.USA 1996.USA 1981.CZE 1966.USA 1951.CZE 1996.CZE 1966.CZE Scores at eigenfunction 1 Scores at eigenfunctions 3 vs NLD 2006.ESP Scores at eigenfunction PRT 1996.ESP 1996.NLD 1966.PRT 1951.ESP 1981.USA 2006.NLD 1981.PRT 1966.ESP 1981.ESP 2006.PRT 1996.JPN 2006.JPN 1996.PRT 1966.NLD 1981.NLD 1951.JPN 2006.CZE 1966.JPN 1981.JPN 1996.USA 1966.USA 1981.CZE 1966.CZE 2006.USA 1951.USA 1951.CZE 1996.CZE Scores at eigenfunction 1

30 Summary Observe a sample of X i (s, t), we want to model the variability in the random process and perform dimension reduction. Tensor product representation: allows for analyzing the separate (possibly asymmetric) effects of s and t. Optimal representation based on marginal kernels: Marginal FPCA and product FPCA. The new concept of weak separability: much more flexible than strong separable covariance assumption.

31 Thank You! Questions?