Supplementary Document

Transcription

1 Saisica Sinica (2013): Preprin 1 Supplemenary Documen for Funcional Linear Model wih Zero-value Coefficien Funcion a Sub-regions Jianhui Zhou, Nae-Yuh Wang, and Naisyin Wang Universiy of Virginia, Johns Hopkins Universiy, and Universiy of Michigan Algorihm for he Refinemen Sage We presen a pracical algorihm here o implemen he null region refinmen and funcion esimaion sage in Secion 2.2 wih D = 1. Knos Placemen. Denoe he iniial esimae of T by ˆT (0) = J j=1 [a j, c j ], which is he union of he idenified subinervals in Secion 2.1. KP.1 Remove he iniial knos wihin [a j, c j ]. KP.2 On ˆT (0),c, evenly-spaced knos are placed, and he oal number of his se of knos is k 1,n + 1 wih k 1,n < k 0,n. Denoe his new se of knos by A. Working Null-region wih he One-sep Group SCAD Esimaor. An ieraion process is carried ou in his sep. WN.1 Le l = 0. WN.2 Take he working null region T l = J j=1 [a j + lδ n, c j lδ n ] when a 1 0 and c J T. When a 1 = 0 or c J = T, he inerval [0, c 1 lδ n ] or [a J + lδ n, T ] are couned ino he working null region. WN.3 The curren knos on [0, T ] conains he knos in A and he boundaries of working null regions T k for k = 0,..., l. Using his se of knos, compue he variables in he approximae model (2.4). WN.4 Ge he iniial value b 1 by leas squares, and divide b 1 ino b 1N,l and b 1S,l according o heir associaion o T l.

2 2 J. ZHOU, N.-Y. WANG AND N. WANG WN.5 Esimae b 1 by minimizing Q n (T l, λ, b) by LARS algorihm, where λ is seleced by he criion C(T l, λ) o be discussed below. WN.6 Le l = l + 1 and repea WN.2-WN.5 unil one inerval [a j, c j ] shrinks o he empy se. The crierion C(T l, λ) can be generalized cross validaion crierion (GCV), Akaike s informaion crierion (AIC), he Bayesian informaion crierion (BIC; Schwarz) and he residual informaion crierion (RIC). They are defined as GCV (T l, λ) = RSS/[n{1 d(λ)/n} 2 ], AIC(T l, λ) = nlog(rss/n) + 2d(λ), BIC(T l, λ) = nlog(rss/n) + log(n)d(λ), RIC(T l, λ) = {n d(λ)}log( σ 2 ) + d(λ){log(n) 1} + 4/{n d(λ) 2}, where RSS is he residual sum of squares, d(λ) is he number of non-zero esimaed coefficiens when he uning parameer is chosen o be λ, and σ 2 = RSS/{n d(λ)}. Final Deerminaion of he Refined Esimaion of T and β(). Idenify he l f ha reaches he smalles crierion value across l and λ. FD.1 Le l f = arg min l C(T l, arg min λ>0 C(T l, λ)). The refined esimae of he null region is ˆT = T lf. FD.2 Le ˆb 1 = arg min b Q n (ˆT, arg min λ>0 C(ˆT, λ), b). The refined esimae of β() is ˆβ() = B T 1 ()ˆb 1, where B T 1 () are he B-spline basis funcion generaed in Sep 2.3 using he knos in A and he boundaries of working null regions T k for k = 0,..., l f. Proofs We use a n > O p (b n ) and a n O p (b n ) o denoe ha, as n wih probabiliy ending o 1, b n /a n 0 and b n /a n is bounded from above, respecively. We need he following lemma.

3 Supplemenary Documen 3 Lemma 1 Le b 0 (n) = (b 0,1 (n), b 0,2 (n),..., b 0,k0,n +h(n)) T and assume ha β() has rh bounded derivaive on [0, T ] where r 3. There exiss a consan M 0 such ha for all b 0,j (n) which are associaed wih T, max b 0,j (n) M 0 k r 0,n. Lemma 1 is a direc resul of he local propery of he B-spline basis funcions. The proof of Lemma 1 is sraighforward, and is hus omied. Proof of he convergence rae of he iniial esimaor by leas squares: We firs prove he convergence rae of he iniial esimaor b 1 (n) of b 1 (n) by leas squares in he refinemen sage. Define ɛ 1 (n) = (ɛ 1,1,..., ɛ 1,n ) T and e(n) = (e 1,..., e n ) T. Le L n {b(n)} = n i=1 (Y i z 1,i b(n)) 2. Given b 1 (n) is he minimizer of L n {b(n)}, we have L n { b 1 (n)} L n {b 1 (n)} = [ b 1 (n) b 1 (n)] T Z T 1 (n)z 1 (n)[ b 1 (n) b 1 (n)] 2(Z T 1 (n)ɛ 1 (n))[ b 1 (n) b 1 (n)] 0. Given A 8, we have [ b 1 (n) b 1 (n)] T Z T 1 (n)z 1 (n)[ b 1 (n) b 1 (n)] c 1 (n/k 1,n) b 1 (n) b 1 (n) 2 l 2. Since he approximaion error e 1 () is bounded below Ck1,n r in absolue value for some consan C, A 2 ensures ha sup ɛ 1,i e i M Ck1,n 1. Thus, he erm Z T 1 (n)ɛ 1 (n) l2 = Z T 1 (n)e(n) + Z T 1 (n)(ɛ 1 (n) e(n)) l2 is dominaed by Z T 1 (n)e(n) l2. Given e(n) N(0, I n ), we have n 1/2 (Z T 1 (n)e(n)) N(0, n 1 Z T 1 (n)z 1 (n)), which indicaes (n 1 Z T 1 (n)z 1 (n)) 1/2 n 1/2 (Z T 1 (n)e(n)) N(0, I k1,n +h), where h+1 is he B-spline basis funcion order. Therefore we have (n 1 Z T 1 (n)z 1 (n)) 1/2 n 1/2 Z T 1 (n)e(n)) 2 l 2 χ 2 (k 1,n + h). Given A 8, we have Therefore, Z T 1 (n)ɛ 1 (n) l2 = O p (n 1/2 ). (1) c 1(n/k 1,n ) b 1 (n) b 1 (n) 2 l 2 [ b 1 (n) b 1 (n)] T Z T 1 (n)z 1 (n)[ b 1 (n) b 1 (n)] 2(Z T 1 (n)ɛ 1 (n)) T [ b 1 (n) b 1 (n)] 2 Z T 1 (n)ɛ 1 (n) l2 b 1 (n) b 1 (n) l2 = O p (n 1/2 ) b 1 (n) b 1 (n) l2,

4 4 J. ZHOU, N.-Y. WANG AND N. WANG which indicaes b 1 (n) b 1 (n) l2 = O p (n 1/2 k 1,n ). Proof of Theorem 1, Par (iii): Assuming A 6, wih probabiliy ending o 1, he coefficiens b 0,j (n) ha are associaed wih T are idenified correcly wih he hreshold value d n, and, hus, he subinervals I j ha are in T are idenified correcly ino ˆT (0). For a subinerval I j { [0, T ] : β() k0,n r+2 }, he associaed coefficiens are b 0,j (n),..., b 0,j+h (n). Taking 0 I j, we have β( 0 ) = h k=0 B 0,j+k( 0 )b 0,j+k (n) + e 0 ( 0 ), where e 0 () ck0,n r is he approximaion error. Given he B-spline basis funcions are all bounded beween 0 and 1, we have ha h h b 0,j+k (n) B 0,j+k ( 0 )b 0,j+k (n) = β( 0 ) e 0 ( 0 ) k0,n r+2 ck0,n r. k=0 k=0 Thus, we have ha, when k 0,n is large enough, h k=0 b 0,j+k(n) (1/2)k0,n r+2, and a leas one of he coefficiens b 0,j (n) associaed wih I j is larger han (1/2)(h + 1) 1 k0,n r+2 in absolue value. Given A 5, wih probabiliy ending o 1, a leas one of he esimaed coefficiens b 0,j (n) associaed wih I j is larger han (1/4)(h + 1) 1 k0,n r+2 in absolue value as k 0,n goes o infiniy. By A 6, he subinerval I j { [0, T ] : β() k0,n r+2 } is idenified correcly ino ˆT (0),c wih probabiliy ending o 1. In summary, we have ha he subinervals I j in T are idenified ino and he subinervals I j in { [0, T ] : β() k0,n r+2 } are idenified ino ˆT (0),c wih probabiliy ending o 1. As a resul, when he lengh of I j goes o 0 as k 0,n goes o, we have T ˆT (0) and ˆT (0) T c Ω(k 0,n ) wih probabiliy ending o 1, where Ω(k 0,n ) = { [0, T ] : 0 < β() < k0,n r+2 } as defined in Theorem 1. The sub-region Ω(k 0,n ) converges o he empy region as k 0,n. Par (iii) is proved. Proof of Theorem 2: Firs we prove ha ˆb 1 (n) b 1 (n) l2 O p (n 1/2 k 3/2 1,n ). This is a non-opimal bound for he convergence rae of ˆb 1 (n), bu i is sufficien o use o show he following Oracle propery. For he coefficien b 1,j (n) associaed wih T, given he consrucion of he k 1,n + 1 adapive knos, he resuls of Lemma 1 applies, i.e. b 1,j (n) Ck r 1,n for some consan C. Assume he coefficien b 1,j (n) is associaed wih he region Ω(k 0,n ). The consrucion of he k 1,n + 1 adapive knos indicaes ha he knos ˆT (0)

5 Supplemenary Documen 5 are evenly-spaced on Ω(k 0,n ). Since β() < k0,n r+2 when Ω(k 0,n ), as in Lemma 1, given A 5, i is rue ha b 1,j (n) < C k0,n r+2 for b 1,j (n) associaed wih Ω(k 0,n ), where C is a consan. Recall ha b 1N (n) and b 1S (n) are he division of b 1 (n) according o ˆT (0). Since b 1N (n) conains he coefficiens associaed wih ˆT (0), given he resuls in Theorem 1 (iii), hese coefficiens are eiher associaed wih T or wih Ω(k 0,n ). Also, here are only a finie number of coefficiens in b 1N (n) according o our mehod o place he k 1,n knos. Thus, given A 5, we have ha b 1N (n) l1 = O p (k r+2 0,n ). Le M be he maximum of β() on T c. Following he proofs of Par (iii) of Theorem 1, we have ha here is a leas one coefficien in b 1S (n) ha is greaer han M/[2(h + 1)] in absolue value, where h + 1 is he fixed spline order. Thus, b 1S (n) l1 O p (1). Recall ha b 1N (n) and b 1S (n) are he division of b 1 (n) according o ˆT (0). Given b 1N (n) b 1N (n) l1 C b 1N (n) b 1N (n) l2 = O p (n 1/2 k 1,n ), b 1N (n) l1 = O p (k0,n r+2 ) and A 5, we have ha b 1N (n) l1 = O p (k0,n r+2 ) and b 1S (n) l2 O p (1). Given A 7, wih probabiliy ending o 1, we have ha p λ n ( b 1N (n) l1 ) = λ n and p λ n ( b 1S (n) l1 ) = 0. Since ˆb 1 (n) minimizes Q n {b(n)}, wih probabiliy ending o 1, we have 0 Q n {ˆb 1 (n)} Q n {b 1 (n)} = [ˆb 1 (n) b 1 (n)] T Z T 1 (n)z 1 (n)[ˆb 1 (n) b 1 (n)] 2(Z T 1 (n)ɛ 1 (n)) T [ˆb 1 (n) b 1 (n)] + nλ n ( ˆb 1N (n) l1 b 1N (n) l1 ) [ˆb 1 (n) b 1 (n)] T Z T 1 (n)z 1 (n)[ˆb 1 (n) b 1 (n)] 2(Z T 1 (n)ɛ 1 (n)) T [ˆb 1 (n) b 1 (n)] + nλ n ( ˆb 1N (n) b 1N (n) l1 2 b 1N (n) l1 ), where ˆb 1N (n), ˆb 1S (n) and b 1N (n), b 1S (n) are he divisions of ˆb 1 (n) and b 1 (n), respecively, according o heir associaion wih ˆT (0). We firs show ha ˆb 1N (n) b 1N (n) l2 O p (n 1/2 k 3/2 1,n ). Suppose ha his is no rue and ha ˆb 1N (n) b 1N (n) l2 > O p (n 1/2 k 3/2 1,n ), which indicaes ˆb 1N (n) b 1N (n) l1 > O p (n 1/2 k 3/2 1,n ). Since b 1N (n) l1 = O p (k0,n r+2 ), given A 5, we have ˆb 1N (n) b 1N (n) l1 2 b 1N (n) l1 > 0 wih probabiliy ending o 1.

6 6 J. ZHOU, N.-Y. WANG AND N. WANG Given Q n {ˆb 1 (n)} Q n {b 1 (n)} 0 and A 8, we have, wih probabiliy ending o, c 1(n/k 1,n ) ˆb 1 (n) b 1 (n) 2 l 2 [ˆb 1 (n) b 1 (n)] T Z T 1 (n)z 1 (n)[ˆb 1 (n) b 1 (n)] 2(Z T 1 (n)ɛ 1 (n)) T [ˆb 1 (n) b 1 (n)] 2 Z T 1 (n)ɛ 1 (n) l2 ˆb 1 (n) b 1 (n) l2. Given (1), we have ˆb 1N (n) b 1N (n) l2 ˆb 1 (n) b 1 (n) l2 = O p (n 1/2 k 1,n ), which is conradicive o he assumpion ˆb 1N (n) b 1N (n) l2 > O p (n 1/2 k 3/2 1,n ). Therefore we have ˆb 1N (n) b 1N (n) l2 O p (n 1/2 k 3/2 1,n ). (2) Nex, we show ha ˆb 1S (n) b 1S (n) l2 = O p (n 1/2 k 3/2 1,n ). We firs define Q n,s {(b S (n)} = Q n {b(n) b N (n) = ˆb 1N (n)}. Since ˆb 1 (n) minimizes Q n {b(n)}, we have ha ˆb 1S (n) is he minimizer of Q n,s {b S (n)}. Therefore, when n is large, 0 Q n,s {ˆb 1S (n)} Q n,s {b 1S (n)} = [ˆb 1S (n) b 1S (n)] T Z T 1S(n)Z 1S (n)[ˆb 1S (n) b 1S (n)] 2[Z T 1S(n)ɛ 1 (n) Z T 1S(n)Z 1N (n)(ˆb 1N (n) b 1N (n))] T [ˆb 1S (n) b 1S (n)]. Given A 8, we have c 1(n/k 1,n ) ˆb 1S (n) b 1S (n) 2 l 2 [ˆb 1S (n) b 1S (n)] T Z T 1S(n)Z 1S (n)[ˆb 1S (n) b 1S (n)] 2[Z T 1S(n)ɛ 1 (n) Z T 1S(n)Z 1N (n)(ˆb 1N (n) b 1N (n))] T [ˆb 1S (n) b 1S (n)] 2 Z T 1S(n)ɛ 1 (n) Z T 1S(n)Z 1N (n)(ˆb 1N (n) b 1N (n)) l2 ˆb 1S (n) b 1S (n) l2 2{ Z T 1S(n)ɛ 1 (n) l2 + Z T 1S(n)Z 1N (n)(ˆb 1N (n) b 1N (n)) l2 } ˆb 1S (n) b 1S (n) l2. Following he seps o show (1), we obain ha Z T 1S(n)ɛ 1 (n) l2 = O p (n 1/2 ). Since ˆb 1N (n) b 1N (n) l2 = O p (n 1/2 k 3/2 1,n ), given A 8, we have Z T 1S(n)Z 1N (n)(ˆb 1N (n) b 1N (n)) 2 l 2 = [ˆb 1N (n) b 1N (n)] T Z T 1N(n)Z 1S (n)z T 1S(n)Z 1N (n)[ˆb 1N (n) b 1N (n)] c 3 (n/k 1,n ) ˆb 1N (n) b 1N (n) 2 l 2 = O p (k 2 1,n).

7 Supplemenary Documen 7 Thus, we have Z T 1S(n)Z 1N (n)(ˆb 1N (n) b 1N (n)) l2 = O p (k 1,n ), and Given (2) and (3), we have ˆb 1S (n) b 1S (n) l2 O p (n 1/2 k 3/2 1,n ). (3) ˆb 1 (n) b 1 (n) l2 O p (n 1/2 k 3/2 1,n ). Finally, we prove he oracle propery of he proposed esimaor. We firs show ha ˆb 1,j (n) = 0, wih probabiliy ending o 1, for any ˆb 1,j (n) associaed wih ˆT (0). We ake he parial derivaive of Q n {b(n)} a b(n) = ˆb 1 (n) wih respec o b 1,j (n) in b 1N (n). As shown above, we have p λ n ( b 1N (n) l1 ) = λ n and p λ n ( b 1S (n) l1 ) = 0 wih probabiliy ending o 1. The parial derivaive is hen = = Q n {b(n)} b j (n) b(n)=ˆb 1 (n) n 2[Y i z 1,iˆb 1 (n)]( z 1,i,j ) + nλ n sign[ˆb 1,j (n)] i=1 n 2{Y i z 1,i b 1 (n) + z 1,i [b 1 (n) ˆb 1 (n)]}( z 1,i,j ) + nλ n sign[ˆb 1,j (n)] i=1 = 2Z T 1,j(n)ɛ 1 (n) + 2[ˆb 1 (n) b 1 (n)] T Z T 1 (n)z 1,j (n) + nλ n sign[ˆb 1,j (n)] = I II + III, where Z 1,j (n) is he jh column of he marix Z 1 (n). Given A 2 and he uniformly bounded B-spline approximaion error, we have sup ɛ 1,i e i M Ck 1 1,n for some consan C. Thus, he erm ZT 1,j(n)ɛ 1 (n) is dominaed by Z 1,j (n)e n. Since e(n) N(0, I n ), we have (k 1,n /n) 1/2 Z T 1,j(n)e(n) N[0, (k 1,n /n)z T 1,j(n)Z 1,j (n)]. Given A 8, we know ha (k 1,n /n)z T 1,j(n)Z 1,j (n) is beween he consans c 1 and c 2. Therefore, (k 1,n /n) 1/2 I = N[0, (k 1,n /n)z T 1,j(n)Z 1,j (n))] + o p (1).

8 8 J. ZHOU, N.-Y. WANG AND N. WANG By A 8, we have Z T 1 (n)z 1,j (n) l2 = O p (nk1,n 1 ). Thus, we have (k 1,n /n) 1/2 II 2(k 1,n /n) 1/2 ˆb 1 (n) b 1 (n) l2 Z T 1 (n)z 1,j (n) l2 We also have = 2(k 1,n /n) 1/2 O p (n 1/2 k 3/2 1,n )O p(nk 1 1,n ) = O p (k 1,n ). (k 1,n /n) 1/2 III = n 1/2 λ n k 1/2 1,n. Since Q n {b(n)} minimizes a ˆb 1 (n), we have ha I + II = III. Given A 5 and A 7, we have I/III = o p (1) and II/III = o p (1). Therefore, P r(ˆb 1,j (n) 0) P r(i + II = III) 0, indicaing ha, wih probabiliy ending o 1, ˆb 1,j (n) = 0 for any ˆb 1,j (n) associaed wih ˆT (0). Since T ˆT (0), wih probabiliy ending o 1, as shown in Theorem 1, we have ha ˆβ() = 0 for T wih probabiliy ending o 1. Par (i) is proved. Nex, we show he asympoic disribuion of ˆβ() for T c. We firs define P n (b ) = n (Y i z 1S,i b ) 2, i=1 where z 1S,i are he elemens of z 1,i ha correspond o he coefficiens in b S (n). Wih probabiliy ending o 1, ˆb 1N (n) = 0 and p λ n ( b 1S (n) l1 ) = 0 as shown above. Since ˆb 1 (n) minimizes Q n {b(n)}, we know ha ˆb 1S (n) is he minimizer of P n (b ) and P n {ˆb 1S (n)} = 0, wih probabiliy ending o 1. Using he Taylor expansion of P n {ˆb 1S (n)} a b 1S (n), we have P n {ˆb 1S (n)} = P n {b 1S (n)} + 2 P n (b )[ˆb 1S (n) b 1S (n)], where b is a poin beween ˆb 1S (n) and b 1S (n). Thus, we have ˆb 1S (n) b 1S (n) = ( 2 P n (b )) 1 P n {b 1S (n)} = (Z T 1S(n)Z 1S (n)) 1 Z T 1S(n)[ɛ 1 (n) + Z 1N (n)b 1N (n)],

9 Supplemenary Documen 9 where Z 1N (n) and Z 1S (n) are sub-marices of Z 1 (n) corresponding o he coefficiens in b 1N (n) and b 1S (n), respecively. Recall ha B 1 (n, ) are he B-spline basis funcions evaluaed a. Le B 1N (n, ) and B 1S (n, ) be he pariioning of B 1 (n, ) according o b 1N (n) and b 1S (n). By Theorem 1, we have ˆT (0) T c Ω(k 0,n ), where Ω(k 0,n ) = { [0, T ] : 0 < β() < k0,n r+2 }. For T c, when n is large enough, we have β() > k0,n r+2. Thus, we have ha ˆT (0),c when n is large enough. As a resuls, when n is large enough, we have (n/k 1,n ) 1/2 ( ˆβ() β()) = (n/k 1,n ) 1/2 B T 1S(n, )[ˆb 1S (n) b 1S (n)] + (n/k 1,n ) 1/2 [B T 1 (n, )b 1 (n) β()] = B T 1S(n, )[(k 1,n /n)z T 1S(n)Z 1S (n)] 1 {(n/k 1,n ) 1/2 Z T 1S(n)[ɛ 1 (n) + Z T 1N(n)b 1N (n)]} + (n/k 1,n ) 1/2 [B T 1 (n, )b 1 (n) β()] = B T 1S(n, )[(k 1,n /n)z T 1S(n)Z 1S (n)] 1 [(n/k 1,n ) 1/2 Z T 1S(n)e(n)] + B T 1S(n, )[(k 1,n /n)z T 1S(n)Z 1S (n)] 1 [(n/k 1,n ) 1/2 Z T 1S(n)(ɛ 1 (n) e(n))] + B T 1S(n, )[(k 1,n /n)z T 1S(n)Z 1S (n)] 1 [(n/k 1,n ) 1/2 Z T 1S(n)Z 1N (n)b 1N (n)]] + (n/k 1,n ) 1/2 [B T 1 (n, )b 1 (n) β()] = U n () + (n/k 1,n ) 1/2 B n() + (n/k 1,n ) 1/2 B n() + (n/k 1,n ) 1/2 W n () By Huang (1998), U n () is he variance componen, B n () = B n() + B n() is he esimaion bias, and W n () is he approximaion error. Given ha e(n) N(0, I n ), we have ha, for T c, U n () D N[0, σ 2 ()] where σ 2 () = lim n B T 1S(n, )[(k 1,n /n)z T 1S(n)Z 1S (n)] 1 B 1S (n, ). Given A 8, we have ha λ max ((k 1,n /n)z 1S (n)z T 1S(n)) c 2. As shown above, we have sup ɛ 1,i e i M Ck1,n r for some consan C. Thus, we have ha (n/k 1,n ) 1 (ɛ 1 (n) e(n)) T Z 1S (n)z T 1S(n)(ɛ 1 (n) e(n)) = (ɛ 1 (n) e(n)) T [(k 1,n /n)z 1S (n)z T 1S(n)](ɛ 1 (n) e(n)) c 2(ɛ 1 (n) e(n)) T (ɛ 1 (n) e(n)) c 2(M C) 2 nk 2r 1,n.

10 10 J. ZHOU, N.-Y. WANG AND N. WANG Thus, we have (n/k 1,n ) 1/2 Z T 1S(n)(ɛ 1 (n) e(n)) l2 C n 1/2 k1,n r for some consan C. Since B 1S (n, ) are bounded and a mos h of hem are nonzero, given A 8, we have Given A 8, we have (n/k 1,n ) 1/2 B n() = O p (n 1/2 k r 1,n ). (n/k 1,n ) 1 b T 1N(n)Z T 1N(n)Z 1S (n)z T 1S(n)Z 1N (n)b 1N (n) c 2 2 b 1N (n) 2 l 2 Given A 5, each coefficien in b 1N (n) is bounded by C k r+2 0,n for some consan C when n is large enough, as shown in he proof above, and here are a finie number of coefficiens in b 1N (n). Thus, we obain ha b 1N (n) 2 l 2 = O p (k 2r+4 0,n ) and (n/k 1,n ) 1/2 Z T 1S(n)Z 1N (n)b 1N (n) l2 = O p (k r+2 0,n ). Given A 7, we have ha k r+2 0,n = o p (1). Therefore, (n/k 1,n ) 1/2 B n() = o p (1). Therefore we have (n/k 1,n ) 1/2 B n () = O p (n 1/2 k r 1,n ). The erm W n () is he B-spline approximaion error a β(). Given A 1 and he B-spline approximaion propery, we have Therefore we have, for T c, (n/k 1,n ) 1/2 W n () = O p (n 1/2 k r 1/2 1,n ). (n/k 1,n ) 1/2 [ ˆβ() β() B n () W n ()] D N[0, σ 2 ()]. Par (ii) is proved. Assuming he addiional sronger condiion n 1 k1,n 2r in A 5, i follows ha (n/k 1,n ) 1/2 B n () = o p (1) and (n/k 1,n ) 1/2 W n () = o p (1). Therefore we have, for T c, (n/k 1,n ) 1/2 [ ˆβ() β()] D N[0, σ 2 ()]. Par (iii) is proved. The proof of Theorem 2 is compleed..

11 Supplemenary Documen 11 Performance of GCV, AIC, BIC and RIC in Sudies 1 and 2: Table 1: Inegraed absolue biases of he leas squares, he Danzig selecor, he adapive LASSO (adplasso), and he one-sep group SCAD (gscad) esimaes for Sudy 1. Each enry is he Mone Carlo average of A j, j = 0 or 1; he corresponding sandard deviaion is repored in parenheses. β 1 () β 2 () Esimaor A 0 A 1 A 0 A 1 Oracle Esimaor (0.041) (0.046) Leas Squares (1.432) (2.549) (1.256) (2.716) Danizig Selecor (0.013) (0.094) (0.010) (0.132) adplasso GCV (0.031) (0.059) (0.028) (0.070) adplasso AIC (0.030) (0.059) (0.028) (0.069) adplasso BIC (0.031) (0.059) (0.029) (0.074) adplasso RIC (0.031) (0.059) (0.028) (0.074) gscad GCV (0.026) (0.038) (0.023) (0.046) gscad AIC (0.033) (0.038) (0.030) (0.048) gscad BIC (0.013) (0.037) (0.009) (0.049) gscad RIC (0.011) (0.037) (0.007) (0.049)

12 12 J. ZHOU, N.-Y. WANG AND N. WANG Table 2: Null region esimaes for Sudy 1. Each enry is he Mone Carlo average of esimaed boundary of he null region; he corresponding sandard deviaion is repored in parenheses. β 1 () β 2 () Esimaor lower upper lower upper Danzig Selecor (0.064) (0.175) (0.038) (0.202) gscad GCV (0.082) (0.268) (0.051) (0.292) gscad AIC (0.091) (0.479) (0.063) (0.528) gscad BIC (0.082) (0.171) (0.051) (0.181) gscad RIC (0.082) (0.168) (0.051) (0.179) Table 3: Mone Carlo bias, sandard deviaion (SD), mean squared error (MSE), and empirical coverage probabiliy (CP) of 95% poinwise confidence inervals of group SCAD (gscad) esimaes for Sudy 1. Each enry is he average over he seleced poins in he non-null region of β 1 () or β 2 (); he corresponding sandard deviaion is repored in parenheses. β 1 () Esimaor Ave. MC Bias Ave. MC SD Ave. MC MSE CP gscad GCV (0.013) (0.213) (0.328) (0.059) gscad AIC (0.013) (0.213) (0.331) (0.047) gscad BIC (0.019) (0.218) (0.339) (0.094) gscad RIC (0.022) (0.218) (0.338) (0.101) β 2 () Esimaor Ave. MC Bias Ave. MC SD Ave. MC MSE CP gscad GCV (0.033) (0.244) (0.386) (0.067) gscad AIC (0.031) (0.247) (0.394) (0.053) gscad BIC (0.043) (0.242) (0.378) (0.098) gscad RIC (0.044) (0.242) (0.379) (0.105)

13 Supplemenary Documen 13 Table 4: Inegraed absolue biases of he leas squares, he Danzig selecor, he adapive LASSO (adplasso), and he one-sep group SCAD (gscad) esimaes for Sudy 2. Each enry is he Mone Carlo average of A j, j = 0 or 1; he corresponding sandard deviaion is repored in parenheses. Esimaor A 0 A 1 Oracle Esimaor (0.054) Leas Squares (0.060) (0.054) Danzig Selecor (0.007) (0.069) adplasso GCV (0.062) (0.063) adplasso AIC (0.063) (0.063) adplasso BIC (0.041) (0.079) adplasso RIC (0.034) (0.084) gscad GCV (0.071) (0.054) gscad AIC (0.076) (0.054) gscad BIC (0.020) (0.056) gscad RIC (0.019) (0.056) Table 5: Null region esimaes for Sudy 2. Each enry is he Mone Carlo average of esimaed boundary of he null region; he corresponding sandard deviaion is repored in parenheses. [0.000, 0.200] [0.486, 0.771] Esimaor lower upper lower upper Danzig Selecor (0.009) (0.016) (0.014) (0.008) gscad GCV (0.009) (0.020) (0.019) (0.015) gscad AIC (0.009) (0.021) (0.019) (0.016) gscad BIC (0.009) (0.016) (0.014) (0.008) gscad RIC (0.009) (0.016) (0.014) (0.008)

14 14 J. ZHOU, N.-Y. WANG AND N. WANG Table 6: Mone Carlo bias, sandard deviaion (SD), mean squared error (MSE), and empirical coverage probabiliy (CP) of 95% poinwise confidence inervals of group SCAD (gscad) esimaes for Sudy 2. Each enry is he average over he seleced poins in he non-null region of β 1 () or β 2 (); he corresponding sandard deviaion is repored in parenheses. β 1 () Esimaor Ave. MC Bias Ave. MC SD Ave. MC MSE CP gscad GCV (0.058) (0.174) (0.266) (0.016) gscad AIC (0.055) (0.173) (0.265) (0.016) gscad BIC (0.072) (0.183) (0.272) (0.020) gscad RIC (0.072) (0.183) (0.272) (0.020)

15 Supplemenary Documen 15 Empirical CP of 95% poinwise CI GCV AIC Coverage probabiliy BIC RIC Figure 1: Empirical coverage probabiliies (CP) of 95% poinwise confidence inervals for coefficien esimae over non-null region of β 1 () for Sudy 1, by GCV, AIC, BIC and RIC, respecively. The poins are aken a = 6.1, 6.2,, 10.0.

16 16 J. ZHOU, N.-Y. WANG AND N. WANG Empirical CP of 95% poinwise CI GCV AIC Coverage probabiliy BIC RIC Figure 2: Empirical coverage probabiliies (CP) of 95% poinwise confidence inervals for coefficien esimae over non-null region of β 2 () for Sudy 1, by GCV, AIC, BIC and RIC, respecively. The poins are aken a = 7.1, 7.2,, 10.0.

17 Supplemenary Documen 17 Empirical CP of 95% poinwise CI GCV AIC Coverage probabiliy BIC RIC Figure 3: Empirical coverage probabiliies (CP) of 95% poinwise confidence inervals for coefficien esimae over non-null region of β() for Sudy 2, by GCV, AIC, BIC and RIC, respecively. The poins are aken a = 0.21, 0.22,, 0.48, 0.78, 0.79,, 0.99,