Asymptotic distributions of nonparametric regression estimators for longitudinal or functional data

Transcription

1 Journal of Multivariate Analysis 98 (2007) Asymptotic distriutions of nonparametric regression estimators for longitudinal or functional data Fang Yao Department of Statistics, Colorado State University, Fort Collins, CO 80523, USA Received 28 January 2005 Availale online 28 Septemer 2006 Astract The estimation of a regression function y kernel method for longitudinal or functional data is considered. In the context of longitudinal data analysis, a random function typically represents a suject that is often oserved at a small numer of time points, while in the studies of functional data the random realization is usually measured on a dense grid. However, essentially the same methods can e applied to oth sampling plans, as well as in a numer of settings lying etween them. In this paper general results are derived for the asymptotic distriutions of real-valued functions with arguments which are functionals formed y weighted averages of longitudinal or functional data. Asymptotic distriutions for the estimators of the mean and covariance functions otained from noisy oservations with the presence of within-suject correlation are studied. These asymptotic normality results are comparale to those standard rates otained from independent data, which is illustrated in a simulation study. Besides, this paper discusses the conditions associated with sampling plans, which are required for the validity of local properties of kernel-ased estimators for longitudinal or functional data Elsevier Inc. All rights reserved. AMS 99 suject classification: 62G08 Keywords: Asymptotic distriution; Covariance; Functional data; Longitudinal data; Regression; Within-suject correlation. Introduction Modern technology and advanced computing environments have facilitated the collection and analysis of high-dimensional data, or data that are repeatedly measured for a sample of sujects. The repeated measurements are often recorded over a period of time, say on an closed and ounded interval T. It also could e a spacial variale, such as in image or geoscience applications. addresses: fyao@utstat.toronto.edu, fyao@stat.colostate.edu X/$ - see front matter 2006 Elsevier Inc. All rights reserved. doi:0.06/j.jmva

2 F. Yao / Journal of Multivariate Analysis 98 (2007) When the data are recorded densely over time, often y machine, they are typically termed functional or curve data with one oserved curve or function per suject, while in longitudinal studies the repeated measurements usually take place on a few scattered oservational time points for each suject. A significant intrinsic difference etween two settings lies in the perception that functional data are oserved in the continuum without noise [2,3], whereas longitudinal data are oserved at sparsely distriuted time points and are often suject to experimental error [4]. However, in practice functional data are analyzed after smoothing noisy oservations [0], which indicates that the differences etween two data types related to the way in which a prolem is perceived are argualy more conceptual than actual. Therefore in this paper, kernel-ased regression estimators otained from oservations at discrete time points contaminated with measurement errors, rather than oservations in the continuum, are considered for these realistic reasons. In the context of kernelased nonparametric regression, the effects of sampling plans on the statistical estimators are also investigated. A vast literature has een developed in the past decade on the kernel-ased regression for independent and identically distriuted data, for summary, see Fan and Gijels [5]. There has een sustantial recent interest in extending the existing asymptotic results to functional or longitudinal data [8,,4,3,9]. The issues caused y the within-suject correlation are rigorously addressed in this paper. Hart and Wehrly [8] studied the Gasser Müller estimator of the mean function for repeated measurements oserved on a regular grid y assuming stationary correlation structure, and showed that the influence of the within-suject correlation on the asymptotic variance is of smaller order compared to the standard rate otained from independent data and will disappear when the correlation function is differentiale at zero. Our asymptotic distriution result is in fact consistent with that in Hart and Wehrly [8] and applicale for general covariance structure without stationary assumption. This prolem was also discussed y Staniswalis and Lee [2] and Lin and Carroll [9], where they used the heuristic arguments of the local property of local polynomial estimation and intuitively ignored the within-suject correlation while deriving the asymptotic variances. This paper derives appropriate conditions that are required for the validity of the local property of kernel type estimators otained from longitudinal or functional data. These conditions also provide practical guidelines for various sampling procedures. The contriution of this paper is the derivation of general asymptotic distriution results in oth one-dimensional and two-dimensional smoothing context for real-valued functions with arguments which are functionals formed y weighted averages of longitudinal or functional data. These asymptotic normality results are comparale to those otained for identically distriuted and independent data. These results are applied to the kernel-ased estimators of the mean and covariance functions, which yields asymptotic normal distriutions of these estimators. In particular, to the est of our knowledge, no asymptotic distriution results are availale up to date for nonparametric estimation of covariance functions otained from longitudinal or functional data contaminated with measurement error. By comparison, Hall et al. [6,7] investigated asymptotic properties of nonparametric kernel estimators of autocovariance, where the measurements were only oserved from a single stationary stochastic process or random field. Although the asymptotic distriutions are derived for random design in this paper, the arguments can e extended to fixed design and other sampling plans with appropriate modifications, and asymptotic ias and variance terms can also e otained in similar manner. This will provide theoretical asis and practical guidance for the nonparametric analysis of functional or longitudinal data with important potential applications which are ased on the asymptotic distriutions. Typical examples include the construction of asymptotic confidence ands for regression functions and confidence regions

3 42 F. Yao / Journal of Multivariate Analysis 98 (2007) for covariance surfaces, and also fast selection of andwidth for covariance surface estimation ased on asymptotic mean squared error. Other applications in the context of smoothing independent data can e explored for the smoothing of longitudinal or functional data using kernel-ased estimators. The remainder of the paper is organized as follow. In Section 2 we derive the general asymptotic distriutions of one- and two-dimensional smoothers otained from longitudinal or functional data for random design. These general asymptotic results are applied to commonly used kernel-type estimators of the mean curve and covariance surface in Section 3. Extension to fixed design is discussed in Section 4. A simulation study is presented to evaluate the derived asymptotic results for correlated data in Section 5, while discussions, including potential applications of the resulting asymptotic normality, are offered in Section General results of asymptotic distriutions for random design In this section we will define general functionals that are kernel-weighted averages of the data for one-dimensional and two-dimensional smoothing. The introduced general functionals include the most commonly used types of kernel-ased estimators as special cases, such as Gasser Müller estimator, Nadaraya Waston estimator, local polynomial estimator, etc. Since Nadaraya Waston and local polynomial estimators are mostly used in practice, their asymptotic ehaviors in terms of ias and variance for independent data have een thoroughly studied in existing literature. However, for longitudinal or functional data, particularly in regard to covariance surface estimators, the asymptotic ehaviors of ias and variance of these two commonly used estimators are still largely unknown. Therefore in Section 3, the general asymptotic results developed in this section are applied to Nadaraya Waston and local polynomial estimators in oth one-dimensional and two-dimensional smoothing settings. In particular, the lack of asymptotic results for the covariance surface estimators of longitudinal or functional data is an additional motivation for the definition of the two-dimensional general functional that can e applied to develop the asymptotic distriutions for these estimators. We first consider random design while extension to other sampling plans is deferred to Section 4. In classical longitudinal studies, measurements are often intended to e on a regular time grid. However, since individuals may miss scheduled visits, the resulting data usually ecome sparse, where only few oservations are otained for most sujects, with unequal numers of repeated measurements per suject and different measurement times T ij per individual. This sampling plan motivates the following assumptions which are applicale for a numer of longitudinal data encountered in practice. Let X ={X(t),t [0, T ]}, T <, e a continuous-time stochastic process defined on a proaility space (Ω, A,P) with finite variance, E( T 0 X2 (t) dt) <, and X i e independently and identically distriuted (i.i.d.) realizations of X which can e modelled as follows in general. The trajectories X i have the mean function μ(t) and uncorrelated random coefficients ξ ik with mean zero, variances λ k satisfying k= λ k < and corresponding asis functions k (t), where the within-suject covariance function is determined as C(s,t) = cov(x i (s), X i (t)) = k= λ k k (s) k (t). A typical example is the Karhunen Loève representation of stochastic processes, where λ k and k correspond to eigenvalues and eigenfunctions. Suppose that one has data {(T ij,y ij ), i n, j N i }, where Y ij are oservations drawn from the process X i at time T ij, incorporating the uncorrelated measurement errors ε ij that have mean zero and a constant

4 F. Yao / Journal of Multivariate Analysis 98 (2007) variance σ 2, Y ij = X i (T ij ) + ε ij = μ(t ij ) + ξ ik k (T ij ) + ε ij, T ij T, () k= where Eε ij = 0, var(ε ij ) = σ 2, and the numer of oservations, N i (n) depending on the sample size n, are considered random. We make the following assumptions, (A.) The numer of oservations N i (n) made for the ith suject or cluster, i =,...,n,isar.v. with N i (n) i.i.d N(n), where N(n) > 0 is a positive integer-valued random variale with lim sup n (n) 2 /[(n)] 2 < and lim sup n (n) 4 /(n)(n) 3 <. In the sequel the dependence of N i (n) and N(n) on the sample size n is suppressed for simplicity; i.e., N i = N i (n) and N(n) = N. The oservation times and measurements are assumed to e independent of the numer of measurements, i.e., for any suset J i {,...,N i } and for all i =,...,n, (A.2) ({T ij : j J i }, {Y ij : j J i }) is independent of N i. Writing T i = (T i,...,t ini ) T and Y i = (Y i,...,y ini ) T, it is easy to see that the triples {T i, Y i,n i } are i.i.d Asymptotic normality of one-dimensional smoother To assume appropriate regularity conditions that are used to derive asymptotic properties, we define a new type of continuity that differs from those which are commonly used. We say that a real function f(x,y) :R p+q Ris continuous on x A R p uniformly in y R q, provided that for any x A and ε > 0, there exists a neighorhood of x not depending on y, saying U(x) R p, such that f(x,y) f(x,y) < ε for all x U(x) and y R q. For random design, (T ij,y ij ) are assumed to have the identical distriution as (T, Y ) with joint density g(t, y). Assume that the oservation times T ij are i.i.d. with the marginal density f(t), ut dependence is allowed among Y ij and Y ik that are oservations made for the same suject or cluster. Also denote the joint density of (T j,t k,y j,y k ) y g 2 (t,t 2,y,y 2 ), where j = k. Let ν,ke given integers, with 0 ν <k. We assume regularity conditions for the marginal and joint densities, f(t), g(t, y), g 2 (t,t 2,y,y 2 ) and the mean function of the underlying process X(t), i.e., E[X(t)]=μ(t), with respect to a neighorhood of a interior point t T, assuming that there exists a neighorhood U(t) of t such that: d (B.) k f (u) exists and is continuous on u U(t), and f (u) > 0 for u U(t); du k (B.2) g(u, y) is continuous on u U(t)uniformly in y R; dk g(u, y) exists and is continuous du on u U(t) uniformly in y R; k (B.3) g 2 (u, v, y,y 2 ) is continuous on (u, v) U(t) 2 uniformly in (y,y 2 ) R 2 ; d (B.4) k μ(u) exists and is continuous on u U(t). du k Let K ( ) e nonnegative univariate kernel functions in one-dimensional smoothing. The assumptions for kernels K :R Rare as follows. We say that a univariate kernel function K is of order (ν,k),if 0, 0 l <k, l = ν, u l K (u) du = ( ) ν ν!, l = ν, (2) = 0, l = k,

5 44 F. Yao / Journal of Multivariate Analysis 98 (2007) (B2.) K is compactly supported, K 2 = K 2 (u) du < ; (B2.2) K is a kernel function of order (ν,l). Let = (n) e a sequences of andwidths that are used in one-dimensional smoothing. We develop asymptotics as n, and require (B3) 0, n() ν+, () 0, and n() 2k+ d 2 for some d with 0 d <. One could see in the proof of Theorem that the assumptions (B3) comined with (A.) provide the condition such that the local property of kernel-type estimators holds for longitudinal or functional data with the presence of within-suject correlation. Let {ψ λ } λ=,...,l e a collection of real functions ψ λ :R 2 R, which satisfy: (B4.) ψ λ (t, y) are continuous on {t} uniformly in y R; d (B4.2) k ψ dt k λ (t, y) exists for all arguments (t, y) and are continuous on {t} uniformly in y R. Then we define the general weighted averages Ψ λn = n ν+ n N i i= j= ψ λ (T ij,y ij )K ( t Tij ), λ =,...,l. and μ λ = μ λ (t) = dν dt ν ψ λ (t, y)g(t, y) dy, λ =,...,l. Let σ κλ = σ κλ (t) = ψ κ (t, y)ψ λ (t, y)g(t, y) dy K 2, λ, κ l, and H :R l Re a function with continuous first order derivatives. We denote the gradient vector (( H/ x )(v),...,( H/ x l )(v)) T y DH (v) and N = n i= N i /n. Theorem. If the assumptions (A.), (A.2) and (B.) (B4.2) hold, then n N 2ν+ D [H(Ψ n,...,ψ ln ) H(μ,...,μ l )] N (β, [DH(μ,...,μ l )] T Σ[DH(μ,...,μ l )]), (3) where β = ( )k d k! u k K (u) du l λ= H μ λ {(μ,...,μ l ) T } dk ν dt k ν μ λ(t), Σ = (σ κλ ) κ,λ l. Proof. It is seen that N can e replaced with y Slutsky Theorem under (A.). We now show that n() 2ν+ [H(EΨ n,...,eψ ln ) H(μ,...,μ l )] β. (4)

6 F. Yao / Journal of Multivariate Analysis 98 (2007) Since (A.) and (A.2) hold, and K is of order (ν,k), using Taylor expansion to order k, one otains EΨ λn = n n ν+ E N i ( ) t Tij ψ λ (T ij,y ij )K i= N = ν+ E j= j= = { ( )} t T ν+ E ψ λ (T, Y )K = μ λ + ( )k k! [ ( ) t Tj E ψ λ (T j,y j )K N] u k K (u) du dk ν dt k ν μ λ(t) k ν + o( k ν ). (5) Then (4) follows from an l-dimensional Taylor expansion of H of order around (μ,...,μ l ) T, coupled with (5). If we can show n() 2ν+ [(Ψ n,...,ψ ln ) T (EΨ,...,EΨ ln ) T ] D N (0, Σ), (6) in analogy to Bhattacharya and Müller [], and continuity of DH at (μ,...,μ l ) T and applying similar arguments used in (5), we find DH(EΨ n,...,eψ ln ) DH(μ,...,μ l ). Then Carmér Wold device yields n() 2ν+ D [H(Ψ n,...,ψ ln ) H(EΨ,...,EΨ ln )] N (0,DH(μ,...,μ l ) T ΣDH(μ,...,μ l )), (7) comined with (4), leading to (3). It remains to show (6). Oserving (A.) and (A.2), one has n() 2ν+ cov(ψ λn, Ψ κn ) = E N j= E N [ N E I I 2. k= j= ψ λ (T j,y j )K ( t Tj ψ λ (T j,y j )K ( t Tj ( ) ] t Tk ψ κ (T k,y k )K ) [ N ) k= ( ) ] t Tk ψ κ (T k,y k )K

7 46 F. Yao / Journal of Multivariate Analysis 98 (2007) It is ovious that I 2 = O() = o() from the derivation of (5). For I, it can e written as I = E N ( ) t ψ λ (T j,y j )ψ κ (T j,y j )K 2 Tj j= + E j =k N ψ λ (T j,y j )ψ κ (T k,y k )K ( t Tj Q + Q 2. Applying (A.) and (A.2), one has Q = E N [ ( ) t E ψ λ (T j,y j )ψ κ (T j,y j )K 2 Tj N] j= ) ( ) t Yk K = ( )] t Y [ψ E λ (T, Y )ψ κ (T, Y )K 2 = σ λκ + o(). Then (4) will hold, oserving (A.) and the following argument that guarantees the local property of the kernel-ased estimators with the presence of within-suject correlation in longitudinal or functional data, Q 2 = N [ ( ) ( ) t E Tj t Tk E ψ λ (T j,y j )ψ κ (T k,y k )K K N] = = (N ) E (N ) j =k N [ ( t T ψ λ (T,Y )ψ κ (T 2,Y 2 )K )] ( ) t T2 K R 4 ψ λ (t u, y )ψ κ (t v, y 2 )K (u)k 2 (v) g 2 (t u, t v, y,y 2 )dudvdy dy 2 (N ) = ψ λ (t, y )ψ κ (t, y 2 )g 2 (t,t,y,y 2 )dy dy 2 + o() = o(), R 2 i.e., the within-suject correlation can e ignored while deriving the asymptotic variance Asymptotic normality of two-dimensional smoother The general asymptotic result can e extended to two-dimensional smoothing. Let (ν, k) denote the multi-indices ν = (ν, ν 2 ) and k = (k,k 2 ), where ν =ν + ν 2 and k =k + k 2. In two-dimensional smoothing, more regularity assumptions are needed for joint densities. Let f 2 (s, t) e the joint density of (T j,t k ), and g 4 (s,t,s,t,y,y 2,y,y 2 ) the joint density of (T j,t k,t j,t k,y j,y k,y j,y k ) where j = k, (j, k) = (j,k ). Denote the covariance surface y C(s,t) = cov(x(t j ), X(T k ) T j = s, T k = t). The following regularity conditions are assumed, where U(s,t) is some neighorhood of {(s, t)}, (C.) d k du k dv k 2 f 2(u, v) exists and is continuous on (u, v) U(s,t), and f 2 (u, v) > 0 for (u, v) U(s,t);

8 F. Yao / Journal of Multivariate Analysis 98 (2007) (C.2) g 2 (u, v, y,y 2 ) is continuous on (u, v) U(s,t) uniformly in (y,y 2 ) R 2 d ; k du k dv k 2 g 2 (u, v, y,y 2 ) exits and is continuous on (u, v) U(s,t) uniformly in (y,y 2 ) R 2 ; (C.3) g 4 (u, v, u,v,y,y 2,y,y 2 ) is continuous on (u, v, u,v ) U(s,t) 2 uniformly in (y,y 2,y,y 2 ) R4 ; d (C.4) k C(u, v) exists and is continuous on (u, v) U(s,t). du k dv k 2 Let K 2 e nonnegative ivariate kernel functions used in the two-dimensional smoothing. The assumptions for kernels K 2 are as follows, (C2.) K 2 is compacted supported with K 2 2 = R 2 K2 2 (u,v)dudv <, and is symmetric with respect to coordinates u and v. (C2.2) K 2 is a kernel function of order ( ν, k ), i.e., l +l 2 = l 0, 0 l < k, l = ν, u l v l 2 K 2 (u,v)dudv = ( ) ν ν!, l = ν, R 2 = 0, l = k. (8) Let h = h(n) e a sequence of andwidths used in two-dimensional smoothing, while it is possile that the andwidths used for two arguments may e different. Since we will focus on the estimator of the covariance surface that is symmetric aout the diagonal, it is sufficient to consider the identical andwidths for the two arguments. The asymptotics is developed as n as follows: (C3) h 0, n 2 h ν +2, h 3 0, and ne[n(n )]h 2 k +2 e 2 for some 0 e <. Similar to the one-dimensional smoothing case, assumptions (C3) and (A.) guarantee the local property of the ivariate kernel-ased estimators with the presence of within-suject correlation. Let { λ } λ=,...,l e a collection of real functions λ :R 4 R, λ =,...,l, satisfying (C4.) λ (s,t,y,y 2 ) are continuous on {(s, t)} uniformly in (y,y 2 ) R 2 ; d (C4.2) k ds k dt k 2 λ(s,t,y,y 2 ) exit for all arguments (s,t,y,y 2 ) and are continuous on {(s, t)} uniformly in (y,y 2 ) R 2. Then the general weighted averages of two-dimensional smoothing are defined y, for λ l, n Φ λn = Φ λn (t, s) = ne[n(n )]h ν +2 λ (T ij,t ik,y ij,y ik ) i= j =k N i Let K 2 ( s Tij h m λ = m λ (s, t) =, t T ) ik. h ν +ν 2 = ν d ν ds ν dt ν 2 R 2 λ (s,t,y,y 2 )g 2 (s,t,y,y 2 )dy dy 2, λ l,

9 48 F. Yao / Journal of Multivariate Analysis 98 (2007) and ω κλ = ω κλ (s, t) = κ (s,t,y,y 2 ) λ (s,t,y,y 2 )g 2 (s,t,y,y 2 )dy dy 2 K 2 2, R 2 κ, λ l, and H :R l Ris a function with continuous first order derivatives as previously defined. Theorem 2. If assumptions (A.), (A.2) and (C.) (C4.2) hold, then n N( N )h 2 ν +2 [H(Φ n,...,φ ln ) H(m,...,m l )] where D N (γ, [DH(m,...,m l )] T Ω[DH(m,...,m l )]), (9) γ = ( ) k e k! l λ= k +k 2 = k d k u k v k 2 K 2 (u,v)dudv R 2 ds k dt k 2 λ (s,t,y,y 2 )g 2 (s,t,y,y 2 )dy dy 2 R 2 { } H (m,...,m l ) T, m λ Ω = (ω κλ ) κ l. The proof of Theorem 2 essentially follows that of Theorem with appropriate modifications which are required for two-dimensional smoothing. 3. Applications to nonparametric regression estimators for functional or longitudinal data Although various versions of kernel-ased estimators have een introduced in literature, Nadaraya Waston and local polynomial, especially local linear estimators, are the most commonly used non-parametric smoothing techniques in longitudinal or functional data analysis. Due to within-suject correlation, the asymptotic ehavior in terms of ias and variance of these estimators for noisily oserved longitudinal or functional data has yet een as well understood as for i.i.d. data. Especially, asymptotic results for covariance estimators do not exist. Therefore in this section, we apply the asymptotic results developed for general functionals to Nadaraya Waston and local linear estimators of regression function and covariance surface to otain their asymptotic distriutions. 3.. Asymptotic distriutions of mean estimators We apply Theorem to the local asymptotic distriutions of the commonly used Nadaraya Waston kernel estimator ˆμ N (t) and local linear estimator ˆμ L (t) for functional/longitudinal

10 F. Yao / Journal of Multivariate Analysis 98 (2007) data: n ˆμ N (t) = N i i= j= ˆμ L (t) =ˆα 0 (t) =arg min (α 0, α ) K ( t Tij n ) N i i= j= Y ij / n K ( t Tij N i i= j= K ( t Tij ), (0) ) [Y ij (α 0 + α (T ij t))] 2. () Corollary. If assumptions (A.), (A.2), and (B.) (B3) hold with ν = 0 and k = 2, then ( ) D d μ n N[ˆμ (2) (t)f (t)+2μ () (t)f () (t) N (t) μ(t)] N σ 2 K 2 f(t), var(y T =t) K 2, f(t) (2) where d is as in (B3), σ 2 K = u 2 K (u) du and K 2 = K 2 (u) du. One can see that the variance function var(y T = t) can e easily derived from model () as var(y T = t) = k= λ k 2 k (t) + σ2, where λ k and k are the variances of the uncorrelated random coefficients and corresponding asis functions, and σ 2 is the variance of the measurement error. Proof. Choose ν = 0, k = 2, ψ (u, v) = v, ψ 2 (u, v) and H(x,x 2 ) = x /x 2 in Theorem, then ˆμ N (t) = H(Ψ n, Ψ 2n ). To compute β, use DH(μ, μ 2 ) = (/μ 2, μ /μ 2 2 ), and note μ (t) = tg(t,v)dv = f(t)μ(t). It is easy to show that (d 2 /dt 2 )μ (t) =[f (2) μ+2f () μ () +f μ (2) ](t), and (d 2 /dt 2 )μ 2 (t) = f (2) (t), leading to the ias term in (2). For the asymptotic variance, note that σ = K 2 v 2 g(t, v) dv = K 2 E(Y 2 T = t)f(t), σ 2 = σ 2 = K 2 μ(t)f (t), σ 22 = K 2 f(t), and DH(μ, μ 2 ) = (/μ 2, μ /μ 2 2 ), yielding the variance term in (2). Concerning the local linear estimator ˆμ L (t), one otains Corollary 2. If assumptions (A.), (A.2), and (B.) (B3) hold with ν = 0 and k = 2, then ( ) D d n N[ˆμ L (t) μ(t)] N 2 μ(2) (t)σ 2 var(y T = t) K, K 2, (3) f(t) where d is as in (B3), σ 2 K = u 2 K (u) du and K 2 = K 2 (u) du. Proof. The local linear estimator ˆμ L (t) of the mean function μ(t) can e explicitly written as i ˆμ L (t) =ˆα 0 (t) = j w ij Y ij i i j w j w ij (T ij t) ij i j w â (t), (4) ij where ˆα (t) = i j w ij (T ij t)y ij ( i i j w ij (T ij t) 2 ( i j w ij (T ij t) i j w ij (T ij t)) 2 /( i j w ij Y ij )/( i j w ij ) j w ij ). (5)

11 50 F. Yao / Journal of Multivariate Analysis 98 (2007) Here w ij = K ((t T ij )/)/(n), where K is a kernel function of order (0, 2), satisfying (B2.) and (B2.2), and ˆα (t) is an estimator for the first derivative μ (t) of μ at t. Oserving that Corollary implies ˆμ N (t) p μ(t), let f(t) ˆ = i j w ij /N i, it is easy to p p show f(t) ˆ f(t) in analogy to Corollary. We proceed to show â (t) μ (t). Denote σ 2 K = u 2 K (u) du, the kernel function K (t) = tk (t)/σ 2 K, and define Ψ λn, λ 3 y ψ (u, y) = y, ψ 2 (u, y), ψ 3 (u, y) = u t. Oserve that K is of order (, 3), f(t) ˆ p f(t), and define H(x,x 2,x 3 ) = x x 2 ˆμ N (t) x 3 x2 2/ f(t) ˆ σ 2 and H(x,x 2,x 3 ) = x x 2 μ(t). x K 3 Then ˆα (t) = H(Ψ n, Ψ 2n, Ψ 3n ) [ = H(Ψ n, Ψ 2n, Ψ 3n ) + Ψ ] 2n(μ(t) ˆμ N (t)) Ψ 3n Ψ 3n Ψ 3n + 2 Ψ 2 2n / f(t) ˆ σ 2. K Note that μ = (μ f +mf )(t), μ 2 = f (t), and μ 3 = f(t), implying Ψ λn μ λ = O p (/ n N 3 ), for λ =, 2, 3, y Theorem. Using Slutsky s Theorem, H(Ψ n, Ψ 2n, Ψ 3n ) μ (t) = O p (/ n N 3 ) follows. For the asymptotic distriution of ˆμ L, note that ˆμ L (t) = i j w ij Y ij i j w ij (T ij t)â (t) i j w. ij j w ij (T ij t) = n Nσ 2 K 2 Ψ 2n. Since K is of order (, 3), Considering n N i Theorem implies Ψ 2n = f (t)+o p (/ n N 3 ), which yields n Nσ 2 K 2 Ψ 2n = n N 5 σ 2 K f (t) + σ 2 K O p () = o p () y oserving n N 5 d 2 for 0 d <. Since f(t) ˆ p f(t)and ˆα (t) μ (t) =O p (/ n N 3 ) = o p (),wefind lim n N[ˆμ n L (t) μ(t)] = D lim n N n { i j w ij Y ij μ (t) i j w ij T ij + tμ (t) i i j w ij j w ij } μ(t). Using the kernel K of order (0, 2), we re-define Ψ λn, λ 3, through ψ (u, y) = y, ψ 2 (u, y) = u and ψ 3 (u, y), setting ν = 0, k = 2, l = 3 and H(x,x 2,x 3 ) =[x μ (t)x 2 + tμ (t)x 3 ]/x 3. Then (3) follows y applying Theorem Asymptotic distriutions of covariance estimators Note that in model (), cov(y ij,y ik T ij,t ik ) = cov(x(t ij ), X(T ik )) + σ 2 δ jk, where δ jl isif j = k and 0 otherwise. Let C ij k = (Y ij ˆμ(T ij ))(Y ik ˆμ(T ik )) e the raw covariances, where ˆμ(t) is the estimated mean function otained from the previous step, for instance, ˆμ(t) =ˆμ N (t) or ˆμ(t) =ˆμ L (t). It is easy to see that E[C ij k T ij,t ik ] cov(x(t ij ), X(T ik )) + σ 2 δ jk. Therefore,

12 F. Yao / Journal of Multivariate Analysis 98 (2007) the diagonal of the raw covariances should e removed, i.e., only C ij k, j = k, should e included as input data for the covariance surface smoothing step, as previously oserved in Staniswalis and Lee [2] and Yao et al. [5]. Commonly used nonparametric regression estimators of the covariance surface, C(s,t) = E{[X(T ) μ(t )][X(T 2 ) μ(t 2 ) T = s, T 2 = t]}, are the two-dimensional Nadaraya Waston estimator and local linear estimator defined as follows: / n Ĉ N (s, t) = i= j =k K 2 ( s Tij h, t T ik h n ( s Tij K 2 h i= j =k n Ĉ L (s, t) = ˆβ 0 (s, t) =arg min β ) C ij k, t T ) ik, (6) h i= j =k K 2 ( s Tij h, t T ) ik h [C ij k f(β,(s,t),(t ij,t ik ))] 2, (7) where β = (β 0, β, β 2 ) and f(β,(s,t),(t ij,t ik )) = β 0 + β (T ij s) + β 2 (T ik t). Applying Theorem 2 to the aove estimators yields the asymptotic distriutions. Corollary 3. If assumptions (A.), (A.2), and (C.) (C3) hold with ν =0and k =2, then n N( N )h 2 D [Ĉ N (s, t) C(s,t)] N (γ N (s, t), v(s, t) K 2 2 ), f 2 (s, t) (8) where e is as in (C3), γ N (s, t) = e { d 2 C(s,t) 4 σ2 K 2 ds 2 f 2 (s, t) + d2 C(s,t) dt 2 f 2 (s, t) [ df2 (s, t) dc(s, t) +2 + df ]}/ 2(s, t) dc(s, t) f 2 (s, t), dt dt ds ds v(s, t) = var{(y μ(t ))(Y 2 μ(t 2 )) T = s, T 2 = t}, σ 2 K 2 = (u 2 + v 2 )K 2 (u,v)dudv, R 2 K 2 2 = K 2 R 2 2 (u,v)dudv. Proof. For two-dimensional Nadaraya Waston estimator Ĉ N (s, t) of the covariance C(s,t), one uses the raw oservations, C ij k = (Y ij ˆμ(T ij ))(Y ik ˆμ(T ik )). Let C ij k = (Y ij μ(t ij ))(Y ik μ(t ik )). Note that C ij k = C ij k + (Y ij μ(t ij ))(μ(t ik ) ˆμ(T ik )) + (Y ik μ(t ik ))(μ(t ij ) ˆμ(T ij )) +(μ(t ij ) ˆμ(T ij ))(μ(t ik ) ˆμ(T ik )). Applying Lemma in Yao et al. [6], one has the weak uniform convergence rate sup t T ˆμ(t) μ(t) =O p (/( n)) for oth possile choice of ˆμ(t) = ˆμ N (t) and ˆμ(t) = ˆμ L (t). Letting

13 52 F. Yao / Journal of Multivariate Analysis 98 (2007) (t,t 2,y,y 2 ) = (y μ(t ))(y 2 μ(t 2 )), 2 (t,t 2,y,y 2 ) = y μ(t ), and 3 (t,t 2,y,y 2 ), then sup t,s T Φ pn =O p (), for p =, 2, 3, y Lemma of Yao et al. [6]. This implies that sup t,s T Φ 2n O p (/( n)) = O p (/( n)) and sup t,s T Φ 3n O p (/( n)) = O p (/( n)). Since sup t T ˆμ(t) μ(t) 2 = O p (/(n)) are negligile compared to Φ n, the Nadaraya Waston estimator Ĉ N (s, t),ofc(s,t) otained from C ij k is asymptotically equivalent to that otained from C ij k, denoted y C N (t, s). Therefore, it is sufficient to show that the asymptotic distriution of C N (s, t) follows (8). Choose ν = (0, 0), k =2, (s,t,y,y 2 ) = (y μ(s))(y 2 μ(t)), 2 (s,t,y,y 2 ) and H(x,x 2 ) = x /x 2 in Theorem 2, then C N (s, t) = H(Ψ n, Ψ 2n ). To compute γ N (s, t), use DH(m,m 2 ) = (/m 2, m /m 2 2 ), and note m (s, t) = R 2 (y μ(s))(y 2 μ(t))g 2 (s,t,y,y 2 ) dy dy 2 = f 2 (s, t)c(s, t) and m 2 (s, t) = f 2 (s, t). One has (d 2 /dt 2 )m (s, t) =[(d 2 f 2 /dt 2 )C + 2(df 2 /dt)(dc/dt) + f 2 (d 2 C/dt 2 )](s, t), (d 2 /d 2 t)m 2 (s, t) = d 2 f 2 (s, t)/dt 2 and similar derivatives with respect to the argument s leading to the ias term in (2). For the asymptotic variance, note that ω = K 2 2 R 2 (y μ(s)) 2 (y 2 μ(t)) 2 g 2 (s,t,y,y 2 )dy dy 2 = E[(Y μ(t )) 2 (Y 2 μ(t 2 )) 2 T = s, T 2 = t)f 2 (s, t) K 2 2, ω 2 = ω 2 = K 2 2 f 2 (s, t)c(s, t), ω 22 = K 2 2 f 2 (s, t), and DH(m,m 2 ) = (/m 2, m /m 2 2 ), yielding the variance term in (2). Corollary 4. If the assumptions (A.), (A.2), and (C.) (C3) hold with ν =0 and k =2, then n N( N )h 2 [Ĉ L (s, t) C(s,t)] ( D e N 4 σ2 K 2 [d 2 C(s, t)/ds 2 + d 2 C(s, t)/dt 2 ], v(s, t) K 2 2 ), (9) f 2 (s, t) where e is as in (C3), v(s, t) = var{(y μ(t ))(Y 2 μ(t 2 )) T = s, T 2 = t), σ 2 K 2 = R 2 (u 2 + v 2 )K 2 (u,v)dudv, K 2 2 = R 2 K2 2(u,v)dudv. Proof. In analogy to the proof of Corollary 3, the local linear estimator Ĉ L (s, t) otained from C ij k is asymptotically equivalent to that otained from C ij k, denoted y C L (t, s). Also denote the solution to (7), after sustituting C ij k for C ij k,y β(s, t) = ( β 0 (s, t), β (s, t), β 2 (s, t)), and in fact β 0 (s, t) = C L (s, t). For simplicity, let W ij k = K 2 ((s T ij )/h, (t T ik )/h)/(nh 2 ) and i,j =k " is areviation of n i= j =k ". Algera calculations yield that i,j =k C C ij k W ij k β i,j =k L = W ij kt ij + β i,j =k W ij ks β 2 i,j =k W ij kt ik + β 2 i,j =k W ij kt i,j =k W, ij k β = R 00(S 0 S 02 S 0 S ) + R 0 (S 00 S 02 S 0 S 20 ) R 0 (S 00 S S 0 S 02 ) S 00 S 20 S 02 S 00 S 2 S2 0 S 02 + S 0 S 0 S + S 20 S 0 S S 0 S20 2, β 2 = R 00(S 0 S S 0 S 02 ) R 0 (S 00 S S 0 S 20 ) + R 0 (S 00 S 20 S0 2 ) S 00 S 20 S 02 S 00 S 2 S2 0 S 02 + S 0 S 0 S + S 20 S 0 S S 0 S20 2, where R pq = W ij k (T ij s) p (T ik t) q C ij k, i,j =k S pq = W ij k (T ij s) p (T ik t) q. i,j =k

14 F. Yao / Journal of Multivariate Analysis 98 (2007) Note that β and β 2 are local linear estimators of the partial derivatives of C(s,t), dc(s, t)/ds and dc(s, t)/dt, respectively. In analogy to the proof of Corollary 2, it can e shown that β (s, t) dc(s, t)/ds =O p (/ n(n )h 4 ) and β 2 (s, t) dc(s, t)/dt =O p (/ n N( N )h 4 ) y applying Theorem 2. Then one can sustitute dc(s, t)/ds, dc(s, t)/dt for β (s, t), β 2 (s, t) in C L (s, t), and denote the resulting estimator y CL (s, t). It is easy to see that lim n N( N )h 2 [C L (s, t) C(s,t)] = D lim n n n N( N )h 2 [CL (s, t) C(s,t)]. We define Φ λn, λ 4, through (s,t,y,y 2 ) = (y μ(s))(y 2 μ(t)), 2 (s,t,y,y 2 ) = s, 3 (s,t,y,y 2 ) and 4 (s,t,y,y 2 ) = t. Put ν = (0, 0), k =2and H(x,x 2,x 3,x 4 ) = [x x 2 dc(s, t)/ds x 4 dc(s, t)/dt]/x 3 + s dc(s, t)/ds + t dc(s, t)/dt. Then result (9) follows y applying Theorem Extension to fixed design In the studies of functional data, it is often the case that the repeated measurements are recorded densely and regularly over time y machine. For realistic reason, the measurements are assumed to e contaminated with experimental error. The asymptotic distriution results developed for random design that is often encountered in longitudinal data can e extended to fixed equally spaced design for typical functional data with slight modifications. Now the density functions g(t, y) is defined as the density of Y(t) at the fixed time t, g 2 (s,t,y,y 2 ) is the joint density of (Y (s), Y (t)) at (s, t), and g 4 (s,t,s,t,y,y 2,y,y 2 ) is the joint density of (Y (s), Y (t), Y (s ), Y (t )) at (s,t,s,t ), while f(t) for the random oservation time T is not needed. The fixed equally spaced design considered here is as follows: (A ) N i (n) = N(n), T i,j+ T i,j = T i,j + T i,j for j,j N, and T ij = T i j for i, i n and j N. The extension that will e discussed in this section is also applicale to the case that the vector of oservation times T i are independent and identically distriuted with N i = N and T i,j+ T i,j = T i,j + T i,j, which is in fact a case that lies etween random unalanced and fixed equally spaced designs. For the general result in Theorem, the weighted averages Ψ λn should e re-defined y Ψ λn = nn ν+ n N i= j= ψ λ (T ij,y ij )K ( t T ij Assumption (B3) now ecomes (B3 ) 0, nn ν+, N 0 and nn 2k+ d 2 with 0 d <. Then in analogy to the proof of Theorem with appropriate modifications, Theorem is still valid if the assumptions (A ), (B.) (B2.2), (B3 ), and (B4.) (B4.2) hold. For the general result of two-dimensional smoothing in Theorem 2, one modifies assumption (C3) as follows: (C3 ) h 0, nn 2 h ν +2 0, hn 3 0, and nn(n )h 2 k +2 e 2 for 0 e <. Then Theorem 2 holds under assumptions (A ), (C.) (B2.2), (C3 ), and (C4.) (C4.2). Therefore, Theorems and 2 can e applied to otain asymptotic distriutions of kernel-ased estimators for the mean and covariance functions under fixed equally spaced design. For instance, the asymptotic normal distriutions of the local linear estimators ˆμ L (t) and Ĉ L (s, t) are similar to ).

15 54 F. Yao / Journal of Multivariate Analysis 98 (2007) those in Corollaries 3 and 4, with f(t)replaced y / T and f(s,t)replaced y / T 2, where T is the length of the interval. 5. Simulation study A numerical study is conducted to evaluate the derived asymptotic properties. The key finding in this paper is that the asymptotic results for functional or longitudinal are comparale to those otained from independent data, i.e., the influence of within-suject covariance does not play significant role in determining the asymptotic ias and variance. For simplicity, we focus on the local polynomial mean estimators which are often superior to the Nadaraya Waston estimators. We first generated M = 200 samples consisting of n = 50 i.i.d. random trajectories each. Following model (), the simulated process has a mean function μ(t) = (t /2) 2,0 t which has a constant second derivative μ (2) (t) = 2, and a constant within-suject covariance function derived from a random intercept ξ i.i.d. N(0, λ ), where λ = 0.0 and (t) =, 0 t. The measurement error in () was set ε ij i.i.d. N(0, σ 2 ), where σ 2 = 0.0. A random design was used, where the numers of oservations for each suject N i were chosen from {2, 3, 4, 5} with equal likelihood and the locations of the oservations were uniformly distriuted on [0, ], i.e., T ij i.i.d. U[0, ]. For comparison, we generated M = 200 samples of n = 50 i.i.d. random trajectories which have the same structure as in model () ut no within-suject i.i.d. correlation. Letting ξ i = 0 and ε ij N(0, λ + σ 2 ) leads to independent data with the same mean and variance functions. Therefore, the two sets of data have the same asymptotic distriution for the local polynomial mean estimators. We also generated M = 200 correlated and independent samples, respectively, consisting of n = 200 trajectories each for demonstrating the asymptotic ehavior with the increasing sample size n. Here we use the Epanechnikov kernel function, i.e., K (u) = 3/4( u 2 ) [,] (u), where A (u) = ifu A and 0 otherwise for any set A. Note that n() 2k+ d 2 in (B3), μ (2) (t) = 2, var(y T = t) = λ + σ 2 = 0.02, and the design density f(t) =, where k = 2 for local polynomial estimators and is the andwidth used for the mean estimation. From the aove construction, one can calculate the asymptotic variance and ias of the local polynomial mean estimators μ L (t) using Corollary 2 which is in fact applicale for oth correlated and independent data. Since the ias and variance terms are oth constant in our simulation framework, for convenience we compare the asymptotic integrated squared ias and variance with the empirical integrated squared ias and variance otained using Monte Carlo average from M = 200 simulated samples ased on 0 E[{ˆμ L(t) μ(t)} 2 ] dt = 0 {ˆμ L(t) E[ˆμ L (t)]} 2 dt + 0 {E[ˆμ L(t)] μ(t)} 2 dt. The asymptotic integrated squared ias and variance are given y AIBIAS = 2 σ2 K 4, AIVAR = 0.02 K 2, (20) n N and the asymptotic integrated mean squared error AIMSE = AIBIAS + AIVAR, where σ 2 K = u 2 K (u) du, K 2 = K 2(u) du and N = (/n) n i= N i, while the empirical integrated squared ias, variance and mean squared error are denoted y EIBIAS, EIVAR and EIMSE, The asymptotic and empirical quantities, such as the integrated squared ias, variance and mean squared error, are shown in Fig. for the correlated/independent data with sample size n = 50/n = 200, respectively. From Fig., it is ovious that the asymptotic approximation is improved y increasing the sample size. The asymptotic quantities AIBIAS, AIVAR and AIMSE agree with the

16 F. Yao / Journal of Multivariate Analysis 98 (2007) x n=50 Correlated 3 x n=50 Independent log () log () 3 x x n=200 Correlated n=200 Independent log () log () Fig.. Shown are the empirical quantities (solid, including EIBIAS, EIVAR, EIMSE) and asymptotic quantities (dashed, including AIBIAS, AIVAR, AIMSE) versus log() for correlated (left panels) and independent (right panels) data with different sample sizes n = 50 (top panels) and n = 200 (ottom panels), where is the andwidth used in the smoothing. In each panel, the integrated squared ias is the one with increasing pattern, the integrated variance is the one with decreasing pattern, and they cross each other, while the integrated mean squared error, which is larger than oth integrated squared ias and variance for any andwidth, usually decreases first and then increases after reaching a minimum. empirical quantities EIBIAS, EIVAR and EIMSE for oth correlated and independent data. For the simulated data with the same sample size n, such asymptotic approximations for correlated and independent data are well comparale in pattern and magnitude. This provides the evidence that the within-suject correlation indeed does not have ovious influence on the asymptotic ehavior of the local polynomial estimators compared to the standard rate otained from independent data, which is consistent with our theoretical derivations. 6. Discussion In this paper, the asymptotic distriutions of kernel-ased nonparametric regression estimators for functional or longitudinal data are studied. In particular, it derives general results for asymptotic distriutions of real-valued functions with arguments which are functionals formed y weighted averages otained from longitudinal or functional data. These asymptotic distriution results are comparale to those otained from identically distriuted and independent data. The conditions for the validity of local property of kernel-ased estimators are provided in (B3), (C3), (B3 ) and (C3 ) with the presence of within-suject correlation in such data, under random unalanced

17 56 F. Yao / Journal of Multivariate Analysis 98 (2007) design descried in (A.) and (A.2), fixed equally spaced design descried in (A ), and some case lying etween them. The proposed results could also e extended to more complicated cases, such as panel data where oservations for different sujects are otained at a series of common time points during a longitudinal follow-up. If considering random design, the density of the jth oservation time T j could e assumed to e f j (t), then the results are readily applied to this case with appropriate modifications with respect to the different marginal densities. The general asymptotic distriution results in univariate and ivariate smoothing settings are applied to the kernel-ased estimators of the mean and covariance functions, which yields asymptotic normal distriutions of these estimators. To the est of our knowledge, there are no asymptotic distriution results availale in literature for nonparametric estimators of covariance function otained from oserved noisy longitudinal or functional data. This provides theoretical asis and practical guidance for the nonparametric analysis of functional or longitudinal data with important potential applications that are ased on the asymptotic distriutions. For example, asymptotic confidence ands or regions for the regression curve or the covariance surface can e constructed ased on their asymptotic distriutions. Since, due to their heavy computational load, commonly used procedures (such as cross-validation) for andwidth selection in two-dimensional settings are not feasile, one important research prolem is to seek efficient approaches for choosing such smoothing parameters. Also functional principal component analysis, an increasingly popular tool for functional data analysis, is ased on eigen-decomposition of the estimated covariance function. Thus, the influence of the asymptotic properties of covariance estimators on the estimated eigenfunctions is another potential research of interest. References [] P.K. Bhattacharya, H.G. Müller, Asymptotics for nonparametric regression, Sankhyā 55 (993) [2] G. Boente, R. Fraiman, Kernel-ased functional principal components, Statist. Proa. Lett. 48 (999) [3] H. Cardot, F. Ferraty, P. Sarda, Functional linear model, Statist. Proa. Lett. 45 (999) 22. [4] P. Diggle, P. Hergerty, K.Y. Liang, S. Zeger, Analysis of Longitudinal Data, Oxford Unversity Press, Oxford, [5] J. Fan, I. Gijels, Local Polynomial Modelling and Its Applications, Chapman & Hall, London, 996. [6] P. Hall, N.I. Fisher, B. Hoffmann, On the nonparametric estimation of covariance functions, Ann. Statist. 22 (994) [7] P. Hall, P. Patil, Properties of nonparametic estimators of autocovariance for stationary random fields, Proa. Theory Related Fields 99 (994) [8] J.D. Hart, T.E. Wehrly, Kernel regression estimation using repeated measurements data, J. Amer. Statist. Assoc. 8 (986) [9] X. Lin, R.J. Carroll, Nonparametric function estimation for clustered data when the predictor is measured without/with error, J. Amer. Statist. Assoc. 95 (2000) [0] J.O. Ramsay, J.B. Ramsey, Functional data analysis of the dynamics of the monthly index of nondurale goods production, J. Econom. 07 (2002) [] T.A. Sererini, J.G. Staniswalis, Quasi-likelihood estimation in semiparametric models, J. Amer. Statist. Assoc. 89 (994) [2] J.G. Staniswalis, J.J. Lee, Nonparametric regression analysis of longitudinal data, J. Amer. Statist. Assoc. 93 (998) [3] A.P. Veryla, B.R. Cullis, M.G. Kenward, S.J. Welham, The analysis of designed experiments and longitudinal data using smoothing splines (with discussion), Appl. Statist. 48 (999) [4] C.J. Wild, T.W. Yee, Additive extensions to generalized estimating equation methods, J. Roy. Statist. Soc., Ser B 58 (996) [5] F. Yao, H.G. Müller, A.J. Clifford, S.R. Dueker, J. Lin Follett, B.A.Y. Buchholz, J.S. Vogel, Shrinkage estimation for functional principal component scores with application to the population kinetics of plasma folate, Biometrics 59 (2003) [6] F. Yao, H.G. Müller, J.L. Wang, Functional data analysis for sparse longitudinal data, J. Amer. Statist. Assoc. 00 (2005)