8.1 Introduction. 8. Nonparametric Inference Using Orthogonal Functions

Transcription

1 8. Noparametric Iferece Usig Orthogoal Fuctios 1. Itroductio. Noparametric Regressio 3. Irregular Desigs 4. Desity Estimatio 5. Compariso of Methods 8.1 Itroductio Use a orthogoal basis to covert oparametric regressio/desity estimatio ito may Normal meas problem Costruct estimates ad cofidece ball/sets usig the miimax theory. For regressio problems, orthogoal methods produce a liear smoother Noparametric Regressio Model: Y i = rx i + σɛ i ɛ i N0, 1 are iid. r L 0, 1. Covert Noparametric Iferece ito May Normal Meas estimatio Oe ca approximate r by the projectio of r: r x = θ j φ j x. Estimatig r is equivalet to estimatig θ = θ 1,..., θ Assume a regular desig: x i = i for i = 1,...,. A aive estimator of θ j =< f, φ j >: φ 1, φ,... : orthoormal basis Oe ca expad r as rx = θ j φ j x Z j N θ j, σ Z j = 1. Y i φ j x i, j = 1,...,. θ j = 1 0 φ jxrxdx. 3 4

2 Defie the itegrated squared bias of size B θ = r r = 1 0 rx r x dx = Lemma 1. Let Θm, c be a Sobolev ellipsoid. The I particular, if m > 1/, the sup B θ = O m θ Θ sup B θ = o 1. θ Θ j=+1 θ j. Modulatio Estimators James-Stei estimator: miimize the SURE over the class of liear estimator bz = bz 1,..., bz b [0, 1]. Wat to miimize the SURE over a larger class of estimators. Modulators: a vector such that 0 b j 1, j = 1,...,. Modulatio estimator: a compoetwise liear estimator. θ = bz = b 1 Z 1, b Z,..., b Z b = b 1,..., b is a modulator. Proof. For θ Θm, c, j=+1 θ j 1 m j=+1 j m θ j c m. 5 6 Examples of Modulators Set of costat modulators M CONS : b M CONS b = b,..., b. Set of ested subset selectio modulator M NSS : b M NSS b = 1,..., 1, 0,..., 0. Set of mootoe modulator M MON : b M MON 1 b 1... b 0. The fuctio estimator r is r t = θ j φ j x = l i t = 1 b j Z j φ j t = b j φ j tφ j x i Y i l i t How to choose b? Use the SURE formula to estimator the risk fuctio ad fid b that miimizes SURE over some class of liear estimators 7 8

3 Theorem 1. The risk of a modulator b is Rb = 1 b j θj + σ The modified SURE estimator of Rb is Rb = 1 b j Zj σ σ is a cosistet estimator of σ. + b j, + σ b j, Lemma. Let σ = J i= J +1 J ad J as. The σ is a cosistet estimate of σ Proof. For large j, θ j 0 ad hece, Zj = θj + σ σ ɛ j ɛ j Z i Therefore, E σ j = J J +1 E Z i J Note: as a default value, J = /4. J +1 σ E ɛ i = σ 9 10 Theorem. Let M be oe of M CONS, M NSS, M MON. Let Rb deote the true risk of the estimator bz = b 1 Z 1,..., b Z. Let ad let The, as, b = argmi Rb b M b = argmi b M Rb. R b Rb 0. For M = M MON, the estimator θ = b 1 Z 1,..., b Z achieves the Pisker boud. How to choose a optimal modulator? For b M NSS, Fidig b = argmi b MNSS Rb is equivalet to fidig Ĵ which miimizes RJ = J σ j + Z j σ + Set r = j=j+1 bj Z j φ j x 11 1

4 For b M MON, Rb ca be writte as Rb = g i = 1 Z i Fidig b = argmi b MMON b = argmi b M MON b i g i + σ Z i gi, σ Rb is equivalet to fidig b i g i Zi It is a weighted least squares problem subject to a mootoicity costrait. Use Pooled-Adjacet-Violators PAV algorithm Theorem 3. Let θ be the MON or NSS estimator ad let σ be a cosistet estimator of σ. Let B = θ = θ 1,..., θ θ j θ j s τ = σ4 s = R b τ + z α b j σ The, for ay c > 0 ad m > 1/, Z j σ 1 b j. lim sup Prθ B 1 α = 0 θ Θ Proof. Defie the pivot process B b: We will use the followig strategy. B b Lb Rb 1. By the fuctioal cetral limt theorem, oe ca show that B b coverges waekly to a mea 0 Gaussia process with covariace kerel Ks, t. 3. Show that τ is a cosistet estimate of K b, b 4. From step 1-3, lim if Pr θ Θm,c lim L b R b τ z α = if Prθ B 1 α. θ Θm,c. By stochastic equicotiuity of the pivot process, oe ca show that B b is stochastically very close to B b. Hece B b has a Guassia limit

5 REACT Cofidece Sets Set Cofidece bad for f: r x = bj Z j φ j x Ix = r x c α σ lx, r x + c α σ lx c α is from the tube formula ad lx 1 b jφ jx Cofidece sets for T f,fuctioals of f: Ix = if T f, sup T f θ B θ B 8.3 Irregular Desigs For a irregular desig, oe ca use a orthoormal basis {φ 1,..., φ } w.r.t {x 1,..., x }. For j = 1,...,, defie Z j = 1 Y i φ j x i. Margially Z j has a asymptotic Normal distributio. Z j N θ j, σ Choose basis for L P P = 1 δ i ad δ i is a poit mass at x i. For j = 1,...,, φ j = φ j xdp x = 1 φ x i = 1 For 1 j < k, < φ j, φ k >= φ j xφ k xdp x = 1 φ j x i φ k x i = 0 Gram-Schmidt orthogoalizatio Costruct a orthoormal basis by Gram-Schmidt orthogoalizatio Let g 1,..., g be a orthoormal basis for R. Let For r defie φ 1 x = ψ 1x ψ r ψ 1x = g 1 x φ r x = ψ rx ψ r ψ r 1 rx = g r x a r,j φ j x a r,j =< g r, φ j >. φ 1,..., φ r form a orthoormal basis w.r.t P. 19 0

6 8.4 Desity Estimatio X 1,..., X are iid from desity f Assume f L 0, 1. Oe ca expad f with a orthoormal basis set {φ 1, φ,...}: fx = θ j φ j x θ j = fxφ j xdx. Oe ca approximate f by the projectio of f: m f m = θ j φ j x. m ad m / 0 as. Defie m f m x = θ j φ j x. Defie Z j as a aive estimator of θ j = Eφ j X Z j = 1 φ j X i, for j = 1,...,. 1 Mea of Z j : E Z j = φ j xfxdx = E φ j X = θ j Variace of Z j : Var Z j = 1 Var φ jx = 1 σ j = Var φ jx. φ jxfxdx θ j σ j Modulatio estimator Modulatio estimator θ: θ = bz = b 1 Z 1,..., b m Z m The risk fuctio of θ is m m Rb = b jσ,j + 1 b jθj. Margially, Z j Nθ j, σ,j, σ,j = sigma j /. But Z j ad Z k are ot idepedet! 3 4

7 The SURE formula yields m m Rb = b j σ,j + 1 b jzj σ,j +, σ,j = σ j = 1 The optimal desity estimator is Z j φ j X i. m f m x = bj Z j φ j x, The desity estimator ca be egative remove the egative part reormalize it to itegrate to 1 Ulike regressio problems, desity estimatio problems are coverted ito may Normal meas with covariace Curretly m = o 1/3 but might be improved up to O 1/ Jag et al. 004 b = b 1,..., b m = argmi b M Rb ad M is a class of modulators Compariso of Methods Local polyomial smoothers have the advatage to correct boudary bias. Orthogoal methods ca easily covert oparametric iferece ito may Normal meas problem. Orthogoal methods ca be cosidered as kerel smoothig with a particular kerel ad vice versa. Härdle et al Use all methods for the problem ad see if they agree. Otherwise, the why? REFERENCES 1. Bera, R REACT scatterplot smoothers: Superefficiecy through basis ecoomy. Joural of the America Statistical Associatio Bera, R. ad Dümbge, L Modulatio of estimators ad cofidece sets. Aals of Statistics Efromovich, S Noparametric Curve Estimatio: Methods, Theory ad Applicatios. Spriger. New York. 4. Härdle, W. ad Kerkyacharia, G. Picard, D. ad Tsybakov, A Wavelets, Approximatio, ad Statistical Applicatio, Spriger. New York. 5. Jag, W. Geovese, C. ad Wasserma, L Noparametric cofidece sets for desities. Upublished mauscript. 7 8