Modeling Repeated Functional Observations

Transcription

1 Modeling Repeated Functional Observations Kehui Chen Department of Statistics, University of Pittsburgh, Hans-Georg Müller Department of Statistics, University of California, Davis Supplemental Material Supplement A: Auxiliary Results and Proofs We first state two auxiliary results that are useful for the theoretical arguments. One of these is a Bernstein-type concentration inequality (Del Barrio et al. 7; Geer 6): Let X,..., X n be independent real-valued random variables with expectation zero. Suppose that for all i, Then for any a >, P ( E X i l l! Cl B, l =, 3,... n a X i a) exp[ ]. (6) (ac + nb ) i= A second auxiliary result relates uniform convergence of eigenfunctions to the uniform convergence of corresponding covariance functions. Lemma. Under assumption L.3, if sup t,t T,s S Ĝ(t, t s) G(t, t s) = O(a n ), then sup ˆφ k (t s) φ k (t s) = O(a n ), a.s. t T,s S a.s., Let m n = n n i= m i, L n = min i,j {L ij } and h T, h S denote the smoothing bandwidths to estimate µ(s, t), and b T, b S the smoothing bandwidths for G(t, t s) when smoothing is used. For the second FPCA step, let b k denote the smoothing bandwidth for R k (s, s ), where R k (s, s ) = cov(ξ k (s ), ξ k (s )), k. In the following proofs, we do not consider measurement errors for dense designs, as then pre-smoothing can be used to construct continuously observed functions with greatly reduced errors. Proof of Theorem. Define G = (t l, s j ) for l L and j m. Under the assumptions max l (t l t l ) = O(n ) and max j (s j s j ) = O(n ), we only need to consider the case L = n and m = n.

2 Since T S is compact, and X i (t s) is Lipschitz continuous by (L.), and the values between grid points are obtained by linear interpolation, we have sup ˆµ(t s) µ(t s) = sup ˆµ(t s) µ(t s) + O( t T,s S (t,s) G n ). Now considering a fixed (t, s) G, using condition (L..a), by (6), for any a >, ( n P (X i (t s) µ(t s)) ( ) ( ) log n) /a n log na exp = n B, n n ((n log n) / ac + nb ) i= where B = a /[(( log n n )/ ac + B )]. Thus, P ( sup ˆµ(t s) µ(t s) ( log n) /a) n n B = n B. (7) (t,s) G n We can find large enough a such that B > for large n. Then, sup ˆµ(t s) µ(t s) = O((log n/n) / ), t T,s S a.s. Defining Z i (t, t s) = X i (t s)x i (t s) on T S, and using condition (L..b), a similar argument gives the result for Ĝ(t, t s), which completes the proof. Proof of Theorem. Write δ n = {[ + (h T L n ) + (h S m n ) + (h S m n h T L n ) ] log n/n} /, δ n = {[+(b T L n ) +(b T L n ) +(b S m n ) +(b S m n b T L n ) +(b S m n b T L n) ] log n/n} /. We note that the local linear smoothing estimators defined in (8) and (9) are slightly different from the one used in Li and Hsing (), and modify m n accordingly. The strong law of large numbers for weighted samples is needed to obtain the same results as those in Lemma in Li and Hsing () (details for this step are provided in a working paper of Zhang and Wang ()). Minor modifications of the proofs of Theorem 3. and Theorem 3.3, with correspondingly modified conditions (A.) - (A.6), yield sup ˆµ(t s) µ(t s) = O(h T + h S + h S h T + δ n ) a.s. t T,s S sup, t s) G(t, t s) = O(h T + h S + h S h T + δ n + b T + b S + b T b S + δ n ) a.s. t,t,s Theorem (a) and (b) now follow by using the respective conditions.

3 Proof of Lemma. Define G(, s) = { G(t, t s) dt dt } /, the Hilbert-Schmidt norm of G(, s) for any s, and ψ( s) = { ψ(t s) dt} /. Then one has sup n) a.s. (8) s S For each s and for a fixed k, Lemma 4.3 in Bosq () implies that ˆλ k (s) λ k (s) Ĝ(, s) G(, s), where δ k is defined in (5). Combining (8) and (9) yields sup s S ˆφ k ( s) φ k ( s) δ Ĝ(, s) G(, s), k ˆλ k (s) λ k (s) = O(a n ) a.s., sup ˆφ k ( s) φ k ( s) = O(a n ) a.s. s S Note that for any k K, λ k (s)φ k (t s) = G(t, t s)φ k (t s)dt, and therefore sup λ k (s) ˆφ k (t s) λ k (s)φ k (t s) s S,t T sup s,t ˆλ k (s) ˆφ k (t s) λ k (s)φ k (t s) + sup ˆλ k (s) ˆφ k (t s) λ k (s) ˆφ k (t s), s,t where the second term is O(a n ) a.s. and the first term is bounded by sup s,t (Ĝ(t, t s) G(t, t s)) ˆφ k (t s)dt + sup s,t G(t, t s)( ˆφ k (t s) φ k (t s))dt = O( sup t,t,s = O(a n ) a.s., Ĝ(t, t s) G(t, t s) sup ˆφ k (s t) + sup s,t t,t s (9) G(t, t s) sup ˆφ k ( s) φ k ( s) ) s using (L.3). Further noting that λ k = inf s λ k (s) is a positive constant, Lemma follows. Proof of Theorem 3. Define θ in = ( sup t,s X i (t s) + sup t,s µ(t s) + sup φ k (t s) + ) (log n/n) /. t,s As L n = O(n ), we can neglect the error in the numerical integration, so may consider ˆξ ik (s ij ) = ( X i (t s ij ) ˆµ(t s ij ) ) ˆφk (t s ij )dt. Noting the target is (3) ξ ik (s ij ) = (Xi (t s ij ) µ(t s ij ) ) φ k (t s ij )dt, 3

4 one finds, for any k, using (), () and Lemma, sup ˆξ ik (s ij ) ξ ik (s ij ) sup (X i (t s ij ) µ(t s ij ))(φ(t s ij ) ˆφ(t s ij ))dt j m i j m i + sup (µ(t s ij ) ˆµ(t s ij ))φ(t s ij )dt j m i + sup (ˆµ(t s ij ) µ(t s ij ))(φ(t s ij ) ˆφ(t s ij ))dt = O(θ in ) a.s., (3) j m i where O( ) is uniform over i. Then (L.3) and the strong law of large numbers imply (5). For each k, if the target working processes ξ ik (s) are used, one can easily derive that sup ˆR k (s, s ) R k (s, s ) = O((log n/n) / ) a.s. s,s S as in Yao et al. (5), Hall et al. (6), and Li and Hsing (). Scrutinizing the estimating procedure for R(s, s ), we find that if the empirical working data are used, then sup ˆR(s, s ) R(s, s ) = O((log n/n) / + θ n ) a.s., s,s S where θ n = n n θ in; see also Yao and Lee (6). By (5), θ n = O(log n/n) / ) a.s.. This means that the rate of convergence for ˆR k (s, s ) and { ˆψ kp (s), p =,..., P } remains the same as for the true targets. Proof of Theorem 4. For the case of a design that is sparse in s, if we were to observe X i (t ijl s ij ), the proof of part (a) would be the same as that of Theorem 3. However, the additional measurement errors for the sparse case cannot be alleviated by individual curve smoothing. Starting with noisy observations U ijl = X i (t ijl, s ij ) + ɛ ijl, we decompose ˆξ ik (s ij ) = ξ ik (s ij ) + R ik (s ij ) + ɛ ijk, L ij ξ ik (s ij ) = {X ij (t ijl s ij ) ˆµ(t ijl s ij )} ˆφ k (t ijl s ij )(t ijl t ij,l ), l= L ij R ik (s ij ) = ɛ ijl { ˆφ k (t ijl s ij ) φ k (t ijl s ij )}(t ijl t ij,l ), l= L ij ɛ ijk = ɛ ijl φ k (t ijl s ij )(t ijl t ij,l ). (3) l= As L n = O(n ), one may neglect the numerical integration error in ξ ik (s ij ) and consider ξ ik (s ij ) = ( X i (t s ij ) ˆµ(t s ij ) ) ˆφk (t s ij )dt. For the targets ξ ik (s ij ) = ( X i (t s ij ) 4

5 µ(t s ij ) ) φ k (t s ij )dt, for any k, using (3), (4) and Lemma, max j m i ξ ik (s ij ) ξ ik (s ij ) = O( ( sup t,s X i (t s) + sup t,s µ(t s) + sup φ k (t s) + ) (b s + log n/nb s ) / )) a.s., (33) t s where O( ) is uniform over i. Then n n i= max j m i ξ ik (s ij ) ξ ik (s ij ) = O(b s + log n/nb s ) / ) a.s., using (L.3). For the remainders R ik (s ij ), by (4) and Lemma, m i max R ik (s ij ) = O((b s + log n/nb s ) / [ j m i L j= ij L ij ɛ ijl ]) a.s., (34) l= Noting that max i n m i M for finite M, and as all ɛ ijl are i.i.d variables with finite mean, we have n n i= max j m i R ik (s ij ) = O((b s + log n/nb s ) / ), a.s. For ɛ ijk, we know that E( ɛ ijk ) = and E ɛ ijk var( ɛ ijk ) / = O(L / n ). Since m i M for finite M, we have E sup j mi ɛ ijk = O(Ln / ) = O((log n/n) / ), where O( ) is uniform over i for fixed k. Then by the strong law of large numbers, n n sup ɛ ijk = O((log n/n) / ) a.s. (35) j m j i= Combining (33), (34) and (35), (9) follows. For each k, if the true target processes ξ ik (s) were used, then by the Corollary 3.5 in Li and Hsing (), sup ˆR k (s, s ) R k (s, s ) = O( b k + [log n/(n b k)] / ) a.s. s,s S Results () () and () then follow by the same arguments as used for Theorem 3. Proofs of Corollary and Corollary. Considering the case of a dense design in s, P for each fixed k and p, the proof for Theorem 3 implies that sup s S ˆξ ik (s) ξ ik (s), and also sup s S ˆψ P kp (s) ψ kp (s). Part (a) directly follows. For part (b), by the Karhunen-Loève expansion, for fixed s and t, X K i (t s) = µ(t, s) + and also for any fixed k and s, K P ξ ik (s)φ k (t s) X i (t s) as K, k= ξ P ik(s) = P P ζ ikp ψ kp (s) ξ ik (s) as P k. p= 5

6 Define the truncated version X K,P i (t s) = µ(t s) + K P k= p= ζ ikpφ kp (s)ψ k (t s). For any integer m, and δ >, one can find K(m) and P (m), such that P ( X K,P i (t s) X i (t s) > δ/) m. Note that ˆX i (t s) as in () is obtained by plugging in the estimates of φ k (t s), ψ kp (s) and ζ ikp in X K,P i (t s). Using the consistency results for ˆφ k (t s), ˆψkp (s) and ˆζ ikp, one can find large enough n(m) such that P ( ˆX i (t s) X K,P i (t s) > δ/) < m. So for any m and δ >, there are large n(m), K(n(m)), and P (n(m)) such that P ( ˆX i (t s) X i (t s) > δ) <, which implies ˆX P m i (t s) X i (t s). For the case of a sparse design in s, note that ζ ikp = γ kp ψt ikp R ik ξ ik, (36) where ψ ikp = (ψ kp (s i ),..., ψ kp (s i,mi )), Rik is a m i by m i matrix with (j, l)-th element R k (s ij, s il ), ξ ik is defined as (ξ ik (s i ),..., ξ ik (s i,mi )), and γ kp is the pth largest eigenvalue of the covariance R k (s, s ); see also Theorem 3 in Yao et al. (5). Estimates ˆζ ikp are obtained from (36), by substituting estimates of γ kp, ψ ikp, R ik and ξ ik, leading to ˆζ ikp = ˆγ kp ˆ ψt ˆ R ikp ik ˆ ξ ik. The uniform convergence of ψ kp (s), R k (s, s ) and ˆξ ik (s ij ) then implies ˆζ P ikp ζ ikp. Part (b) then follows by similar arguments as used in the cases of a dense design in s. REFERENCES Bosq, D. (), Linear Processes in Function Spaces: Theory and Applications, New York: Springer-Verlag. Del Barrio, E., Deheuvels, P., and Geer, S. (7), Lectures on empirical processes: theory and statistical applications., European Mathematical Society. van de Geer, S. A. (6), Empirical Process Theory and Applications, handout WS 6, ETH Zürich. Hall, P., Müller, H.-G., and Wang, J.-L. (6), Properties of principal component methods for functional and longitudinal data analysis, Annals of Statistics, 34,

7 Li, Y. and Hsing, T. (), Uniform convergence rates for nonparametric regression and principal component analysis in functional/longitudinal data, Annals of Statistics, 38, Yao, F. and Lee, T. C. M. (6), Penalized Spline Models for Functional Principal Component Analysis, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68, 3 5. Yao, F., Müller, H.-G., and Wang, J.-L. (5), Functional data analysis for sparse longitudinal data, Journal of the American Statistical Association,, Zhang, X. and Wang, J.L. (), Nonparametric Estimation for Longitudinal and Functional Data: A Unified Theory. Supplement B: Comparisons With the Karhunen-Loève Expansion As an alternative approach to the proposed two-step FPCA, we also implemented and fitted the two-dimensional Karhunen-Loève expansion as in Eq. (4). This implementation is illustrated with the mortality data that have been introduced and discussed in Section 6. These data are regular and quite dense, which means that one can use the sample covariance as an estimate of the four-dimensional covariance function G(t, t, s, s ) = cov(x(t, s ), X(t, s )), where the covariance function is a necessary ingredient for the Karhunen-Loève implementation. Since the data contain measurement errors, the eigenfunctions ρ k (t, s) resulting from this approach are then smoothed, with the resulting estimates shown in Figure 7. Alternatively, one may estimate the covariance function G(t, t, s, s ) by four-dimensional local linear smoothing. The eigenfunction estimates resulting from this smoothing approach are in Figure 8. We find that in the dense regular case, these two estimating methods for the covariance function G(t, t, s, s ) lead to almost identical estimates of the eigenfunctions ρ k (t, s). Not surprisingly, the fourdimensional smoothing method is computationally much slower. Note that the first two eigenfunction estimates ρ (t, s) and ρ (t, s) are quite similar to the corresponding first two surfaces of the proposed model, ϕ (t s) and ϕ (t s), which are shown in Figure 5. We note that we consider here a scenario where t and s have inherently different meanings in that s is a longitudinal time and t a functional time, and therefore the components φ k and ξ k (s) obtained from the first step in our proposed two-step FPCA approach are of interest in themselves (see Figure 4). These components are however not available when one uses the Karhunen-Loève approach. 7

8 Even if one were to ignore the roles played by t and s, the proposed method runs much faster compared to the Karhunen-Loève expansion with a two-dimensional argument as in equation (4), especially when the data are not regular and dense. This is due to the fact that then a four-dimensional smoothing step for the covariance G(s, t, s, t ) is needed to implement the Karhunen-Loève expansion. For further illustration, we sparsified the mortality data, retaining only one third of the measurements available per trajectory, where the measurements retained are randomly selected. Then we applied the proposed method, which involves a three-dimensional smoothing step for G(t, t s), as well as the Karhunen-Loève expansion with a four-dimensional smoothing step for G(t, t, s, s ). We found that the proposed method yields estimates of the model components that are very close to those obtained when observing the entire data set without missings (Figure 9). In contrast, the more complex Karhunen-Loève approach (4) proved to be extremely time consuming (it is 5 times slower than the proposed method) and the estimates resulting from the case of sparse data were rather poor (Figure ). 8

9 μ ρ ρ ρ Figure 7: The estimated mean function and first three eigenfunctions ˆρ k (t, s) for the Karhunen-Loève decomposition (4) for the mortality data, using sample covariances for estimation. 9

10 μ ρ ρ ρ Figure 8: The estimated mean function and first three eigenfunctions ˆρ k (t, s) for the Karhunen-Loève decomposition (4) for the mortality data, using four-dimensional smoothing for estimation.

11 μ ϕ ϕ ϕ Figure 9: The estimated mean and first three principal surfaces ˆϕ, ˆϕ, ˆϕ for the proposed approach () for the sparsified mortality data (with one third of the available measurements randomly deleted).

12 μ ρ ρ ρ Figure : The estimated mean and first three eigenfunctions ˆρ k (t, s) for the Karhunen- Loève decomposition (4) for the sparsified mortality data (one third of the available data), using four-dimensional smoothing for the covariance surface.

13 Supplement C: List of Countries Included in Mortality Data Analysis The 3 countries are: Australia, Austria, Belarus, Belgium, Bulgaria, Canada, Czech Republic, Denmark, Estonia, Finland, France, Hungary, Iceland, Ireland, Italy, Japan, Latvia, Lithuania, Luxembourg, Netherlands, New Zealand, Norway, Poland, Portugal, Russia, Slovakia, Spain, Sweden, Switzerland, United Kingdom, Ukraine, USA. Supplement D: Additional Simulations An additional simulation study was conducted specifically to assess the performance of the proposed method for a situation where eigenvalues cross at several locations s, with specifications as proposed by an anonymous reviewer. We generate data as in model (3) X i (t s) = µ(t s) + ξ ik (s)φ k (t s), i =,..., n, s [, ], t [, ], k= where φ (t s) = sin{π(t s)}, φ (t s) = cos{π(t s)} and sample size n = 4. The random functions ξ i (s) and ξ i (s) are generated as zero mean Gaussian processes with covariance structures R (s, s ) = 4 cos(4πs ) cos(4πs ) + sin(4πs ) sin(4πs ) and R (s, s ) = 6 sin(πs ) sin(πs ) + cos(πs ) cos(πs ). The grid for t consists of equi-spaced points on [,], and the grid for s of 5 equi-spaced points on [,]. One of the challenges of this simulated data is that the two eigenvalue functions λ (s) = var(ξ i (s)) and λ (s) = var(ξ i (s)) cross four times. Following the method described in Section 3, we first estimate the functions µ(t, s) and G(t, t s) by their empirical estimators. For s =, we determine φ ( s) to be the eigenfunction associated with the larger eigenvalue and φ ( s) to be the eigenfunction associated with the smaller eigenvalue. The average value of ϑ, chosen by the method described in Section 3, for simulation runs was 3.. The gaps from omitting some values of s as described in the method were small and easily filled by smoothing the available values ˆφ k ( s j ) across s. Figure demonstrates nearly perfect recovery of the true basis functions φ k (t s) and the eigenvalues λ k (s), for k =, obtained in the first step FPCA. Figure demonstrates fairly good performance of the second FPCA, applied to the working processes ˆξ ik (s), where ψ kp (s) are the eigenfunctions of ξ ik (s). To quantify the quality of the estimates of φ(t s), we use the relative squared error RSE = φ(t s) ˆφ(t s) φ(t s), (37) 3

14 where φ(t s) = φ(t s) dsdt, analogously for ˆψ kp (s). The boxplots of the relative squared errors over simulation runs as reported in Figure 3 are seen to be reasonably small for all φ k (t s) and ψ kp (s), except that there about outliers, which occurred because the crossing of the eigenvalue functions was not correctly identified. We note that the number of outliers shrinks quickly with increasing sample size. 4

15 φ (t s) φ (t s) ˆφ (t s) ˆφ (t s) 6.5 λ (s) λ (s) Figure : True and estimated φ k (t s) and eigenvalue functions λ k (s) for k = (left) and k = (right) from one simulation run, as described in Supplement D. 5

16 5 ˆξ i 5 ˆξ i ψ (s) ψ (s) ψ (s).5 ψ (s).5 Figure : The estimated random functions ξ ik (s) for k =,, i =,..., 5, and true and estimated eigenfunctions ψ kp (s) from one simulation run, as described in Supplement D. 6

17 Rrelative Squared Error phi_ phi_ psi_ psi_ psi_ psi_ Figure 3: The relative squared errors for the eigenfunctions φ k (t s), k =,, and ψ kp (s), k =,, p =,, obtained from simulation runs, as described in Supplement D. 7

18 Supplement E: Eigenanalysis of the random functions ξ (s) and ξ (s) for the mortality data The functions ξ (s) and ξ (s) are obtained in the first stage of the proposed double FPCA method, as in the basic model in (). For the mortality data application, these functions are plotted in Figure 4 and their characteristic features have been discussed in Section 6. Here we provide additional details on the second stage of the double FPCA method in the context of the mortality data analysis. The second stage consists of the eigenanalysis of the functions ξ (s) and ξ (s) that are shown in the right panels of Figure 4 and yields the eigenfunctions ψ p and ψ p of the covariance operators of these random processes, where p =,, as described in Section 6. The eigenfunctions ψ kp are key components of the principal surfaces ϕ k (t s) = ψ kp (s)φ k (t s) and the corresponding FPC scores ζ ikp = ξ ik (s)ψ kp (s) ds serve as random scores to represent the repeatedly observed functions X i (t s), see eq. (5). We plot the eigenfunctions ψ, ψ, explaining 83. % and 3.4%, respectively, of the variation of random functions ξ (s), in the upper panel of Figure 4, and eigenfunctions ψ, ψ explaining 73. % and 7.%, respectively, of the variation of random functions ξ (s) in the lower panel. We find that the first eigenfunction ψ for processes ξ (s) nicely reflects the main variance increase around 98-99, in accordance with the shapes in the top right panel of Figure 4. Observe here that the sign of the eigenfunctions is arbitrary. The second eigenfunction ψ indicates an additional increase in the variation of processes across countries with increasing calendar year. As processes ξ (s) are tied to the basic ageincrease in mortality, as seen in the top left panel of Figure 4, this implies an ongoing differentiation into higher- and lower-mortality countries. Processes ξ (s) are associated with the contrast between old and oldest-old mortality, as can be seen in the lower left panel of Figure 4. The first eigenfunction ψ reflects the increase in the variation across countries with increasing calendar date, including a recent slight acceleration of this increase, in accordance with the function shapes depicted in the lower right panel of Figure 4. The second eigenfunction ψ reflects a contrast between pre- and post-98, indicating that there is a tendency for a reversal between the pre-98 and the post-98 old to oldest-old mortality differential. Indeed, taking a closer look at the right lower panel of Figure 4 shows that some countries exhibit a reversal in the shapes of ξ (s). To summarize, this analysis demonstrates the usefulness of the results of the second stage FPCA that is provided by the proposed method. 8

19 . ψ (s).3 ψ (s) ψ (s). ψ (s) Figure 4: The estimated eigenfunctions ψ kp (s), k =,, p =,, for ξ (s) (upper panels) and ξ (s) (lower panels), obtained from the mortality data, as described in Supplement E. 9