Least Squares Parameter Estimation for Sparse Functional Varying Coefficient Model

Transcription

1 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) Least Squares Parameter Estimatio for Sparse Fuctioal Varyig Coefficiet Model Behdad Mostafaiy Departmet of Statistics, Faculty of Mathematical Sciece, Shahid Beheshti Uiversity, Tehra, Ira; B_mostafaiy@sbu.ac.ir Mohammad Reza Faridrohai Departmet of Statistics, Faculty of Mathematical Sciece, Shahid Beheshti Uiversity, Tehra, Ira; m_faridrohai@sbu.ac.ir Received 15 Jue 015 Accepted 1 Jue 016 Abstract I the preset paper, we study fuctioal varyig coefficiet model i which both the respose ad the predictor are fuctios. We give estimates of the itercept ad the slope fuctios i the case that the observatios are sparse ad oise-cotamiated logitudial data by usig least squares represetatio of the model parameters. To estimate the parameter fuctios ivolved i the represetatio, we use a regularizatio method i some reproducig kerel Hilbert spaces. As we will see, our procedure is easy to implemet. Also, we obtai the covergece rates of the estimators i the L -sese. These covergece rates establish that the procedure performs well, especially, whe samplig frequecy or sample size icreases. Keywords: Fuctioal varyig coefficiet model; Logitudial data Aalysis; Rate of covergece; Regularizatio; Reproducig kerel Hilbert space; Sparsity. 000 Mathematics Subject Classificatio: 6G05, 34D05 1. Itroductio Nowadays, due to developmet of sciece ad techology, it is possible to collect data which are ifiite dimesioal. Fuctioal data aalysis deals with such observatios. See Ramsay ad Silverma (00; 005) for a overview of methods ad applicatios of fuctioal data aalysis. See also Ferraty ad Vieu (006) ad, Horváth ad Kokoszka (01). A extesio of parametric regressio models, commoly kow as varyig coefficiet models, were itroduced by Clevelad et al. (1991) ad, Hastie ad Tibshirai (1993). These models allow its regressio coefficiets to vary i respect to some predictors of iterest. A collectio of available methods is provided by Fa ad Zhag (008). The estimatio approaches give i Fa ad Zhag (008) are based o polyomial splie, smoothig splies ad local polyomial smoothig. Other refereces iclude Hoover et al. (1998), Wu et al. (1998), Kauerma ad Tutz (1999), Wu ad Chiag (000), Chiag et al. (001) ad Huag et al. (00; 004). Most of these approaches are ot applicable for sparse desigs. I the case of sparse desigs, Copyright 017, the Authors. Published by Atlatis Press. This is a ope access article uder the CC BY-NC licese ( 337

2 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) see for example Şetürk ad Müller (008), Noh ad Park (010), Şetürk ad Müller (010), Setürk ad Nguye (011), Chiou et al. (01), ad Şetürk et al. (013). I this paper, we cosider the followig fuctioal varyig coefficiet model Y(t) = β 0 (t) + β 1 (t)x(t) + Z(t), t T (1.1) where T R is a compact set, Y(t) ad X(t) are square itegrable respose ad predictor processes, respectively, β 0 (t) ad β 1 (t) are smoothed itercept ad slope fuctios, respectively, ad Z(t) is a mea zero radom process, idepedet of X(t). We wat to estimate fuctios β 0 (t) ad β 1 (t) i the situatio that the observatios are sparse ad irregular logitudial data ad combied with some measuremet errors. Followig Yao et al. (005), we model this situatio as follows. Let U ii ad V ii deote the observatios of the radom fuctios X i ad Y i respectively at the radom times T ii, cotamiated with measuremet errors ε ii ad ε i j respectively, which are assumed to be idepedet ad idetically distributed with meas zero ad variaces σ X ad σ Y respectively, ad idepedet of the radom fuctios. We may represet the observed data as U ii = X i (T ii ) + ε ii, j = 1,, ; i = 1,,, V ii = Y i (T ii ) + ε ii, j = 1,, ; i = 1,,. (1.) Cosider the populatio least squares represetatio for (β 0 (t), β 1 (t)) i model (1.1) as The solutios are (β 0 (t), β 1 (t)) = arg mi { E[Y(t) θ 0 (t) θ 1 (t)x(t)] }. (θ 0 (t),θ 1 (t)) β 0 (t) = μ Y (t) β 1 (t)μ X (t), β 1 (t) = C XX(t, t) C XX (t, t) where μ X (t) = E[X(t)] ad μ Y (t) = E[Y(t)] are mea fuctios of X ad Y respectively, ad C XX (s, t) = cov(x(s), X(t)) ad C XX (s, t) = cov(x(s), Y(t)) are respectively covariace fuctio of X ad crosscovariace fuctio of X ad Y. I the preset paper, we use some regularizatio methods to estimate the parameter fuctios ivolved i represetatio (1.3). I additio, by assumig that the sample paths of X ad Y are α times differetiable, we obtai covergece rates of the estimators. The obtaied results show that our proposed method performs well. The paper is orgaized as follows. I sectio, we review some basic facts of RKHS which are importat cocepts i the sequel. Sectio 3 cotais our estimatio methods ad, fially, covergece rates of our estimators are obtaied uder some certai regularity assumptios i sectio 4.. Reproducig Kerel Hilbert Spaces I this sectio, we will review, without proof, some basic facts about reproducig kerel Hilbert spaces (RKHSs). Verificatio of these results as well as more detailed discussios of RKHS theory ca be foud, for example, i Aroszaj (1950) ad Wahba (1990). Let H be a Hilbert space of fuctios o some set T ad deote by <, > H ad H, respectively, the ier product ad the or H. A bivariate fuctio o T T is said to be a reproducig kerel for H if for every t T ad everyf H, (i) K(, t) H, (ii) f(t) =< f, K(, t) > H. Whe (i) ad (ii) hold, H is said to be a RKHS with reproducig kerel K. Relatio (i) is called the reproducig property of K. If K is a reproducig kerel the it is symmetric, oegative defiite ad uique. (1.3) 338

3 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) Coversely, if K is a oegative defiite fuctio o T T, a uique RKHS of fuctios o T with K as its reproducig kerel ca be costructed. I the sequel, we will deote the Hilbert space with reproducig kerel K by H(K). Also the correspod ier product ad orm will be deoted by <, > H(K) ad H(K), respectively. Accordig to properties of ier product ad by usig properties (i) ad (ii), for ay N, N N, α 1,, α N, α 1,, α N R ad t 1,, t N, t mm 1,, t N T, we have N < α i K(, t i ), α j K(, t j) > H(K) = α i α j < K(, t i ), K(, t j) > H(K) N j=1 N N N N j=1 = α i α j K(t i, t j). (.1) j=1 Now, let H 1 ad H be two Hilbert spaces of real-valued fuctios with, respectively, ier products <, > H1 ad <, > H. The tesor product space of H 1 ad H, H 1 H, is defied i the followig fashio. For f 1 H 1 ad g 1 H defie the biliear form f 1 g 1 : H 1 H R by [f 1 g 1 ](f, g ) =< f 1, f > H1 < g 1, g > H, (f, g ) H 1 H. Let H 0 be the set of all fiite liear combiatios of such biliear forms. A ier product, <, > H0 o H 0 ca be defied as follows: < f 1 g 1, f g > H0 =< f 1, f > H1 < g 1, g > H, (.) where f 1, f H 1 ad g 1, g H. The the tesor product space H 1 H is defied as the completio of H 0 uder the ier product <, > H0 defied i () ad it is show that it is a Hilbert space. For a RKHS H(K), cosider the tesor product space H(K K): = H(K) H(K). It ca be show that H(K K) is a RKHS with reproducig kerel To defie our regularizatio method, we use RKHS. 3. Estimatio Procedures (K K)((s 1, t 1 ), (s, t )) = K(s 1, s )K(t 1, t ). I the preset sectio, we estimate the parameter fuctios ivolved i (1.3). Suppose the sample paths of radom fuctios X ad Y belog to the same, without loss of geerality, RKHS H(K) almost surely, such that E X H(K) <, E Y H(K) <. (3.1) A result i Cai ad Yua (010) shows that if coditio (3.1) holds the μ X, μ Y H(K) ad C XX H(K K). I additio, Mostafaiy et. al. (014) have show coditio (3.1) implies that C XX H(K K). These results let us to defie some regularizatio methods to estimate the parameter fuctios. First, we estimate mea fuctio of X. Notice that, mea fuctio of Y ca be estimated i the same way by usig V ii s istead of U ii s i the followig procedure. Cosider the followig regularizatio method: where μ X = arg mil 1 (f) + λ 1 f H(K) (3.) f H(K) 339

4 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) l 1 (f) = 1 1 ( U i f(t ii )) i j=1 ad λ 1 0 is a smoothig parameter that balaces the fidelity to the data measured by l 1 ad the smoothess of the estimate measured by the squared RKHS orm. Accordig to the so-called represeter lemma (see Wahba 1990), the solutio of the miimizatio problem (3.) ca be expressed as μ X,a(t) = a ii K(t, T ii ) (3.3) j=1 for some coefficiet vector a = [a 11,, a 1m1, a 1,, a m ] R N 1 with N 1 =. This result demostrates that although the miimizatio i (3.) is take over aifiite dimesioal space, the solutio ca be foud i a fiite dimesioal subspace, ad it suffices to evaluate coefficiet vector a i (3.3). To do this, let P be a N 1 N 1 matrix defied by where P 11 P 1 P 13 P 1 P P = 1 P P 3 P P 1 P P 3 P P i1 i = KT i1 j 1, T i j 1 j1 1, 1 j, 1 i 1, i. By usig (.1), we have Notice that μ X,a H(K) 1 =< a i1 j 1 Kt, T i1 j 1, a i j Kt, T i j > H(K) i 1 =1 j 1 =1 1 i =1 j =1 = a i1 j 1 a i j K(T i1 j 1, T i j ) i 1 =1 j 1 =1 i =1 j =1 = a PP l 1 (μ X,a) = U PP l, where l is the space of square summable sequeces ad U = [U 11,, U 1m1, U 1,, U m ]. Therefore we have l 1 (μ X,a)+ μ X,a H(K) = U PP l + a PP (3.4) The right had side of (4) is quadratic i a ad so it is ot hard to see that the miimizer of it is a = (P + Nλ 1 I) 1 U (3.5) where I is a N 1 N 1 idetity matrix. To estimate C XX ad C XX, let ad defie the followig regularizatio methods: C XX,i T ij1, T ij = U ij1 μ XT ij1 U ij μ XT ij, C XX,i (T ij1, T ij ) = U ij1 μ X(T ij1 )V ij μ Y(T ij ) 340

5 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) where C XX = C XX = l (C) = 1 1 ( 1) arg mi l (C) + λ C H(K) H(K), C H(K) H(K) arg mi l 3 (C) + λ 3 C H(K) H(K) C H(K) H(K) C XX,i T ij1, T ij CT ij1, T ij, 1 j 1 j l 3 (C) = 1 1 C m XX,i (T ij1, T ij ) C(T ij1, T ij ) i j 1 =1 j =1 ad λ ad λ 3 are smoothig parameters. By represeter lemma, we have C XX (s, t) = b ij1 j K(s, T ij1 )K(t, T ij ), j 1 =1 j =1 for some b = [b 111,, b 11m1, b 11,, b m m ] R N with N =, ad C XX (s, t) = c ij1 j K(s, T ij1 )K(t, T ij ), j 1 =1 j =1 for some c = [c 111,, c 11m1, c 11,, c m m ] R N. Coefficiet vectors b ad c ca be evaluated i the same maer as coefficiet vector a i the estimatio procedure of μ X. Now, from (1.3) the plug-i estimates of β 0 ad β 1 are give by β 1(t) = C XX (t, t) C XX (t, t), β 0(t) = μ Y(t) β 1(t)μ X(t). (3.6) Notice that the above estimators deped o smoothig parameters λ 1, λ ad λ 3. I practice these smoothig parameters ca be chose by some optimizatio criteria such as cross-validatio. 4. Rates of Covergece I this sectio, we obtai covergece rates of the estimators i terms of itegrated squared errors. To do this, we assume the followig coditios. These coditios also have bee adopted by Cai ad Yua (010). Let F(α; M, c) be the collectio of probability measures defied o (X, Y) such that a) The sample paths of α time differetiable X ad Y belog to H(K) almost surely ad there exits costat M > 0 such that E X H(K) < M ad E Y H(K) < M. b) K is a Mercer kerel with eigevalues, ρ k, satisfyig ρ k k α, where for two positive sequeces r ad s, r s meas that r k /s k is bouded away from 0 ad as k, that is, 0 < lif ( r k/s k ) lim sup( r k /s k ) <. k k c) For Z beig either X or Y, there exists a costat c > 0 such that ad E[Z 4 (t)] c (E[Z (t)]) for all t T, 341

6 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) for ay f L (T). 4 E Z (t) f(t) dd T c E Z (t) f(t) dd, T The coditio (a) imposes smoothess of the processes X ad Y. The boudedess requiremet E X H(K) < M ad E Y H(K) < M i (a) is a techical coditio. The coditio (b) guaraties the α smoothess of the kerel fuctio K. As show i Micchelli ad Wahba (1981), the Sobolev space W satisfies the coditio (b). Fially, the coditio (c) cocers the fourth momet of a liear fuctioal of Z. This coditio is satisfied with c = 3 for a Gaussia process because Z (t)f(t)dd is ormally distributed. T Let m be the harmoic mea of m 1,..., m, that is, For the smoothig parameters we assume λ 1 () α α+1, m: = α lll ( ) λ, λ 3 α+1 (4.1). Give a sequece c of positive costats, we shall use O p (c ) to deote radom variable R which satisfies lim lim sup sup P F { R > Dc } = 0. D F F(α;M,c) The followig result gives the rates of covergece for the regularized estimators μ X ad μ Y. Theorem 1. Assume that T ii s are idepedet ad idetically distributed with a desity bouded away from zero o T. Suppose the smoothig parameter λ 1 satisfies (4.1). The μ Z μ Z L = O p 1 + () α+1 α, fff Z = X oo Y. (4.) Proof. The proof is give i Cai ad Yua (011). The followig theorem gives covergece rates of C XX ad C XX. Theorem. Assume that E(ε 4 ) <, E(ε 4 ) < ad T ii s are idepedet ad idetically distributed with a desity bouded away from zero o T. Suppose the smoothig parameters λ 1, λ ad λ 3 satisfy (4.1). The C XX C XX L = O p 1 + lll ( ) α α+1 ad C XX C XX L = O p 1 + lll ( ) α α+1. Proof. The proof is give i Cai ad Yua (010), ad Mostafaiy et. al. (014). The covergece rates of β 0 ad β 1 are established i the followig theorem. 34

7 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) Theorem 3. Uder the coditios of Theorem, the varyig coefficiet estimators (3.6) satisfy β k β k L = O p 1 + lll ( ) α α+1, for k = 0,1. Proof. The proof follows from (3.6) ad Theorems 1,. 5. Coclusios ad Extesios We have show that the covergece rates for the proposed estimators of the slope ad the itercept fuctios i the L -sese are 1 α () + lll α+1. Whe s small, that is m 1 α log, the covergece rates are determied by factor α lll () α+1 which deped joitly o the values of both m ad. O the other had, whe mis large, that is m 1 α log, the rates tur to 1 regardless of m. I the preset paper, we have focused o stochastic processes defied o a compact subset T of the real lie R. We ote that this assumptio ca be exteded for more geeral compact domais. For example, for T = [0,1] r, it is ot hard to show that the covergece rates become 1 α () + lll α+r. Refereces [1] N. Aroszaj, Theory of reproducig kerel, Trasactio of America Mathematical Society 68 (1950) [] T. Cai ad M. Yua, Noparametric covariace estimatio for fuctioal ad logitudial data, Techical Report (Uiversity of Pesylvaia ad Georgia Istitute of Techology, 010). [3] T. Cai ad M. Yua, Optimal estimatio of the mea fuctio based o discretely sampled fuctioal data: Phase trasitio, The Aals of Statistics 39 (011) [4] C. Chiag, J. A. Rice ad C. O. Wu, Smoothig splie estimatio for varyig coefficiet models with repeatedly measured depedet variables, Joural of the America Statistical Associatio 96 (001) [5] J. M. Chiou, Y. Ma ad C. L. Tsai, Fuctioal radom effect time-varyig coefficiet model for logitudial data. Stat 1 (01) [6] W. S. Clevelad, E. Grosse, ad W. M. Shyu, Local regressio models. Statistical Models i S (Chambers, J. M. ad Hastie, T. J., eds, Wadsworth ad Books, Pacic Grove. 1991) pp [7] J. Fa ad W. Zhag, Statistical methods with varyig coefficiet models, Statistics ad Its Iterface 1 (008) [8] F. Ferraty ad P. Vieu, Noparametric Fuctioal Data Aalysis: Methods, Theory, Applicatios ad Implemetatios (Spriger, New York, 006). [9] T. J. Hastie ad R. J. Tibshirai, Varyig-coefficiet models, Joural of the Royal Statistical Society, B 55 (1993) [10] D. R. Hoover, J. A. Rice, C. O. Wu ad L.-P. Yag, Noparametric smoothig estimates of time-varyig coefficiet models with logitudial data, Biometrika, 85 (1998) [11] L. Horvath ad P. Kokoszka, Iferece for Fuctioal Data with Applicatios. Spriger, New York (01). [1] J. Z. Huag, C. O. Wu ad L. Zhou, Varyig-coefficiet models ad basis fuctio approximatios for the aalysis of repeated measuremets Biometrika 89 (00) [13] J. Z. Huag, C. O. Wu ad L. Zhou, Polyomial splie estimatio ad iferece for varyig coefficiet models with logitudial data, Statistica Siica 14 (004) [14] G. Kauerma ad G. Tutz, O model diagostics usig varyig coefficiet models, Biometrika 86 (1999) [15] C. Micchelli ad G. Wahba, Desig problems for optimal surface iterpolatio. I Approximatio Theory ad Applicatios (Z. Ziegler, ed.). (Academic Press, New York, 1981), pp [16] B. Mostafaiy, M. FridRohai, ad S. Cheouri, Sparse Fuctioal Liear Regressio ad Sigular Fuctios by Reproducig Kerel Hilbert Space Methods, Mauscript (014). [17] H. S. Noh ad B. U. Park, Sparse varyig coefficiet models for logitudial data, Statistica Siica 0 (010) [18] J. O. Ramsay ad B. W. Silverma, Applied Fuctioal Data Aalysis: Methods ad Case Studies (Spriger, New York, 00). 343

8 Joural of Statistical Theory ad Applicatios, Vol. 16, No. 3 (September 017) [19] J. O. Ramsay ad B. W. Silverma, Fuctioal Data Aalysis, d Editio. (Spriger, New York, 005). [0] D. Seturk ad H. G. Muller, Geeralized varyig coefficiet models for logitudial data, Biometrika 95 (008) [1] D. Seturk ad H. G. Muller, Fuctioal varyig coefficiet models for logitudial data, Joural of the America Statistical Associatio, 105 (010) [] D. Seturk ad D. V. Nguye, Vary coefficiet models for sparse oise- cotamiated logitudial data, Statistica Siica 1 (011) [3] D. Seturk, L. S. Dalrymple, S. M. Mohammed, G. A. Kayse ad D. V. Nguye, Modelig time-varyig effects with geeralized ad usychroized logitudial data, Statistics i Medicie 3 (013) [4] G. Wahba, Splie models for observatioal data. (SIAM, Philadelphia, 1990). [5] C. O. Wu, C. T. Chiag ad D. R. Hoover, Asymptotic cofidece regios for kerel smoothig of a varyig-coefficiet model with logitudial data, Joural of the America Statistical Associatio 93 (1998) [6] C. O. Wu ad C. T. Chiag, Kerel smoothig o varyig coefficiet models with logitudial depedet variable, Statistica Siica 10 (000) [7] F. Yao, H. G. Muller, ad J. Wag, Fuctioal data aalysis for sparse logitudial data, Joural of the America Statistical Associatio 100 (005)