Local Polynomial Regression
|
|
- Edwin Dawson
- 5 years ago
- Views:
Transcription
1 VI Local Polynomial Regression (1) Global polynomial regression We observe random pairs (X 1, Y 1 ),, (X n, Y n ) where (X 1, Y 1 ),, (X n, Y n ) iid (X, Y ). We want to estimate m(x) = E(Y X = x) based on (X 1, Y 1 ),, (X n, Y n ). (i) averaging as a constant regression model : m(u) u α (constant) LSE : α argmin (Y i α) 2 = Y α (ii) linear regression 1 model : m(u) = α 0 + α 1 u LSE : ( α 0, α 1 ) argmin (Y i α 0 α 1 X i ) 2 α 0,α 1 ( n = Y α 1 X, (X ) i X)Y i n (X i X) 2 2 model : m(u) = β 0 + β 1 (u x) where x is the point at which we want to estimate the value of m α 1 = β 1, α 0 = β 0 β 1 x LSE : ( β 0, β 1 ) argmin (Y i β 0 β 1 (X i x)) 2 ) β 0,β ( 1 n = Y β 1 (X x), (X ) i x (X x))y i n (X i x (X x)) 2 = m(x) = α 0 + α 1 x = β 0 m (x) = α 1 = β 1 (iii) polynomial regression ( n Y β 1 (X x), (X ) i X)Y i n (X i X) 2 1 model : m(u) = α 0 + α 1 u + α 2 u α p u p LSE : ( α 0,, α p ) argmin α 0,,α p 55 (Y i α 0 α 1 X i α p X p i )2
2 2 model : m(u) = β 0 + β 1 (u x) + β 2 (u x) β p (u x) p LSE : ( β 0,, β p ) argmin (Y i β 0 β 1 (X i x) β p (X i x) p ) 2 β 0,,β p p m (r) (x) = j(j 1) (j r + 1)α j x j r ; r = 1,..., p j=r = r! β r m ( x) = α 0 + α 1 x + + α p x p = β 0 p m (r) (x) = j(j 1) (j r + 1) α j x j r ; r = 1,..., p j=r = r! β r (2) Local polynomial regression : Basic idea Nothing is assumed for the structure of m (i) local constant regression 1 Idea : m(u) β 0 (constant) when u x Use only (X i, Y i ) s.t. X i x and approximate m(x i ) for such X i s by a (unknown) constant β 0 2 Suppose we use only (X i, Y i ) s.t. X i x h = β 0 argmin (Y i β 0 ) 2 I ( h,h) (X i x) β 0 = argmin (Y i β 0 ) 2 1 β 0 h 1 ( ) 2 I Xi x ( 1,1) h m(x; h) β 0 (See Figure 1, p.4 of the lecture note) 3 generalization of the weight function m(x; h) β 0 argmin (Y i β 0 ) 2 K h (X i x) = β 0 n K h(x i x)y i n K h(x i x) : This is the famous Nadaraya-Watson estimator 56
3 (ii) local polynomial regression 1 Idea : m(u) β 0 + β 1 (u x) + + β p (u x) p when u x (better approximate than local constant modelling!) β r = m(r) (x) r! 2 Definition : Note ( β 0,, β p ) = argmin (Y i β 0 β 1 (X i x) β p (X i x) p ) 2 K h (X i x) β 0,,β p m (r) (x; h) r! β r ; r = 0, 1,..., p (See Figure 2, p.6 of the lecture note) β r depends on x, the point of interest. (3) Theory for local constant fitting (Nadaraya-Watson estimator) f : density of X i s supported on [0, 1] v(u) var(y X = u) (i) Asymtotic MSE in Int(supp(f)) f has a continuous derivative at x, and f(x) > 0 v is continuous at x m has two continuous derivatives at x K is a symmetric probability function supported on a compact set, say [ 1, 1] ; K is bounded = bias( m(x; h) X 1,, X n ) = 1 { } m (x)f(x) + 2m (x)f (x) µ 2 (K)h 2 +o p (h 2 )+O p (n 1/2 h 1/2 ) 2 f(x) var( m(x; h) X 1,, X n ) = v(x) ( ) K 2 n 1 h 1 + o p (n 1 h 1 ) f(x) 57
4 Proof 1 (bias part) E { m(x; h) X 1,, X n } m(x) 1 n n = K h(x i x) {m(x i ) m(x)} 1 n n K h(x i x) Recall the formula given in p.8 and note that Z n = EZn + O p ( var(z n )) ( ) K h {m( ) m(x)}f( )(x) + O p n 1 h 1 (K 2 ) h {m( ) m(x)}2 f( )(x) = K h f(x) + O p(n 1/2 h 1/2 ) = 1 2 {m (x)f(x) + 2m (x)f (x)}µ 2 (K)h 2 + O p (n 1/2 h 1/2 ) + o p (h 2 ) f(x) + o(h) + O p (n 1/2 h 1/2 ) 2 (variance part) var { m(x; h) X 1,, X n } = [ ] 2 n 1 K h (x X i ) n 1 n j=1 K v(x i ) h(x X j ) = n 1 h 1 n 1 n (K2 ) h (x X i )v(x i ) {n 1 n K h(x X i )} 2 = n 1 h 1 [(K 2 ) h (vf)(x) + O p (n 1/2 h 1/2 )] {f(x) + o(h) + O p (n 1/2 h 1/2 )} 2 (ii) Asymptotic MSE at boundaries x x n = αh (0 α < 1) f is right continuous at 0, and f(0+) > 0 v is right continuous at 0 m is differentiable on (0, ɛ) for some ɛ > 0 and m is right continuous at 0 K is a symmetric probability function supported on [ 1, 1] and is bounded 58
5 = bias( m(x; h) X 1,, X n ) Proof Remark = µ 1(K; α) µ 0 (K; α) m (0+)h + o(h) + O p (n 1/2 h 1/2 ) var( m(x; h) X 1,, X n ) = µ 0(K 2 ; α) {µ 0 (K; α)} 2 v(0+) f(0+) n 1 h 1 + o p (n 1 h 1 ) Apply the formula in (8)-(i)- 1, Page (bias part) K h {m( ) m(x)}f( )(x) = µ 1(K; α)m (x)f(x)h + o(h) K h f(x) = µ 0(K; α)f(x) + o(1) (K 2 ) h {m( ) m(x)}2 f( )(x) = O(h 2 ) (K 2 ) h f(x) = µ 0(K 2 ; α)f(x) + o(1) bias( m(x; h) X 1,, X n ) 2 (variance part) = µ 1(K; α)m (x)f(x)h + o(h) + O p (n 1/2 h 1/2 ) µ 0 (K; α)f(x) + o(1) + O p (n 1/2 h 1/2 ) (K 2 ) h (vf)(x) = µ 0(K 2 ; α)v(x)f(x) + o(1) var( m(x; h) X 1,, X n ) = n 1 h 1 {µ 0 (K 2 ; α)v(x)f(x) + O p (n 1/2 h 1/2 ) + o(1)} {µ 0 (K; α)f(x) + o(1) + O p (n 1/2 h 1/2 )} 2 In fact, it is unnecessary fot K to be a symmetric probability density. (4) Theory for local linear fitting f : density of X i s supported on [0, 1] v(u) var(y X = u) (i) Asymtotic MSE in Int(supp(f)) 59
6 Note 1 Note 2 f is continuous at x, and f(x) > 0 v is continuous at x m has two continuous derivatives at x K is supported on [ 1, 1] and is bounded = bias( m(x; h) X 1,, X n ) = 1 µ 2 (K) 2 µ 1 (K)µ 3 (K) 2 µ 0 (K)µ 2 (K) µ 1 (K) 2 m (x)h 2 + o(h 2 ) + O p (n 1/2 h 3/2 ) var( m(x; h) X 1,, X n ) {µ2 (K) zµ 1 (K)} 2 K 2 (z)dz = v(x) {µ 0 (K)µ 2 (K) µ 1 (K) 2 } 2 f(x) n 1 h 1 + o p (n 1 h 1 ) We do not assume K is a symmetric probability density here. If we do, then the conditional bias and variance reduce to bias( m(x; h) X 1,, X n ) = 1 2 µ 2(K)m (x)h 2 + o(h 2 ) + O p (n 1/2 h 3/2 ) var( m(x; h) X 1,, X n ) = µ 0 (K 2 ) v(x) f(x) n 1 h 1 + o p (n 1 h 1 ) Why the conditional bias is still O p (h 2 ) even when we put unbalanced weights around the point x? 60
7 Local Constant Local Linear h x local constant vs local linear when one puts weights on the right hand side only of the point x We can expect O p (h 2 ) conditional bias at boundaries too, which shall be shown shortly. proof of the bias and the variance formula ( β 0, β 1 ) = argmin (Y i β 0 β 1 (X i x)) 2 K h (X i x) β 0,β 1 Write â 0 = β 0 m(x), â 1 = h( β 1 m (x)) = m(u) = m(x) + m (x)(u x) : linear approximation near x (Y i β 0 β 1 (X i x)) 2 K h (X i x) = = { ( )} 2 Y i m(x) m Xi x (x)(x i x) â 0 â 1 K h (X i x) h { ( )} 2 1 Y i m(x i ) (â 0, â 1 ) X i x K h (X i x) h (â 0, â 1 ) minimizes, w.r.t. a T = (a 0, a 1 ), 61
8 (Yi a T X i ) 2 K h (X i x) = (Y X a) T W(Y X a) where Y = (Y 1,, Y Y i = Y i m(x i ), X i = n ) T, X = (X 1,, X n) T ( 1 X i x h = (â 0, â 1 ) T = (X T WX ) 1 X T WY 1 (bias part) ), W = Diag(K h (X i x)) E(â X 1,, X n ) = (X T WX ) 1 X T WE(Y X 1,, X n ) 1 n (X T WX ) r,s = 1 ( ) r+s Xi x K h (X i x) ; r, s = 0, 1 n h = 1 (P r+s K) h (X i x) n = (P r+s K) h f(x) + O p(n 1/2 h 1/2 ) = µ r+s (K)f(x) + O p (n 1/2 h 1/2 ) + o(1) A r,s means the (r, s) component of the matrix A with indices being counted from zero. P l (u) = u l ; P l K(u) = u l K(u); (P l K) h (u) = 1 ( u ) l ( u K h h h) 1 n {X T WE(Y X 1,, X n )} r ( a r means the rth component of the vector a with indices being counted from zero = 1 ( ) r Xi x K h (X i x){m(x i ) m(x i )}; r = 0, 1 n h = 1 (P r K) h (X i x){m(x i ) m(x i )} n = (P r K) h {m( ) m( )}f( )(x) + O p(n 1/2 h 3/2 ) = 1 2 µ 2(P r K)m (x)f(x)h 2 + o(h 2 ) + O p (n 1/2 h 3/2 ) ) 62
9 = 1 2 µ r+2(k)m (x)f(x)h 2 + o(h 2 ) + O p (n 1/2 h 3/2 ) ( ) ( ) µ0 (K) µ 1 (K) µ2 (K) Write N =, γ = µ 1 (K) µ 2 (K) µ 3 (K) Then, E(â 0 X 1,, X n ) = 1 2 (N 1 γ) 0 m (x)h 2 + o(h 2 ) + O p (n 1/2 h 3/2 ); ( )( ) (N 1 γ) 0 = 1 µ2 (K) µ 1 (K) µ2 (K) µ 0 (K)µ 2 (K) µ 1 (K) 2 µ 1 (K) µ 0 (K) µ 3 (K) 2 (variance part) var(â X 1,, X n ) = µ 2(K) 2 µ 1 (K)µ 3 (K) µ 0 (K)µ 2 (K) µ 1 (K) 2 = (X T WX ) 1 X T W var(y X 1,, X n ) WX (X T WX ) 1 = (X T WX ) 1 X T ΣX (X T WX ) 1 where Σ = Diag(v(X i )(K h (X i x)) 2 ) 0 h ( X T ΣX ) n r,s = 1 ( ) r+s Xi x (K 2 ) h (X i x)v(x i ); r, s = 0, 1 n h = µ r+s (K 2 )v(x)f(x) + o p (1) ( ) µ0 (K 2 ) µ 1 (K 2 ) Write S =. Then, µ 1 (K 2 ) µ 2 (K 2 ) var(â 0 X 1,, X n ) = (N 1 SN 1 v(x) ) 0,0 f(x) n 1 h 1 + o p (n 1 h 1 ) 1 1 (N 1 SN 1 ) 0,0 = (N 1 ) 0,r S r,s (N 1 ) s,0 = r=0 s=0 {µ2 (K) zµ 1 (K)} 2 K(z) 2 dz {µ 0 (K)µ 2 (K) µ 1 (K) 2 } 2 63
10 Note 3 m I (x; h) 1 n {f(x i )} 1 L h (X i x)y i prototype of internal regression estimator If we take L(z) = K (z) µ 2 (K) zµ 1 (K) K(z), then µ 0 (K)µ 2 (K) µ 1 (K) 2 m I (x; h) has the same asymptotic property (first order) as m(x; h). We call K equivalent kernel (ii) Asymptotic MSE at boundaries x x n αh(0 α < 1) f is right continuous at 0, and f(0+) > 0 v is right continuous at 0 m is twice differentiable on (0, ɛ) for some ɛ > 0, and m is right continuous at 0 K is supported on [ 1, 1] and is bounded = bias( m(x; h) X 1,, X n ) = 1 µ 2 (K; α) 2 µ 1 (K; α)µ 3 (K; α) 2 µ 0 (K; α)µ 2 (K; α) µ 1 (K; α) 2 m (0+)h 2 +o(h 2 )+O p (n 1/2 h 3/2 ) var( m(x; h) X 1,, X n ) α 1 = {µ 2(K; α) zµ 1 (K; α)} 2 K(z) 2 dz v(0+) {µ 0 (K; α)µ 2 (K; α) µ 1 (K; α) 2 } 2 f(0+) n 1 h 1 +o p (n 1 h 1 ) (5) Theory for local polynomial fitting p : the order of the local polynomial fitting f : density of X i s supported on [0, 1] v(u) var(y X = u) (i) Asymptotic MSE in Int(supp(f)) : even p case 64
11 f has a continuous derivatives at x, and f(x) > 0 v is continuous at x m has (p + 2) continuous derivatives at x K is supported on [ 1, 1], and is bounded = bias( m(x; h) X 1,, X n ) = (N 1 m (p+1) (x) γ) 0 (p + 1)! hp+1 [ + (N 1 m (p+2) (x) δ) 0 (p + 2)! + o(h p+2 ) + O p (n 1/2 h (2p+1)/2 ) var( m(x; h) X 1,, X n ) + (N 1 δ N 1 JN 1 γ) 0 m (p+1) (x) (p + 1)! = (N 1 SN 1 ) 0,0 v(x) f(x) n 1 h 1 + o p (n 1 h 1 ) where N = (µ r+s (K)) : (p + 1) (p + 1) matrix Note 1 γ = (µ p+1+r (K)) : (p + 1) 1 vector δ = (µ p+2+r (K)) : (p + 1) 1 vector J = (µ r+s+1 (K)) : (p + 1) (p + 1) matrix S = (µ r+s (K 2 )) : (p + 1) (p + 1) matrix ] f (x) h p+2 f(x) Suppose all the odd moments of K vanish, i.e. µ j (K) = 0 for all odd j s. Then N r,s = 0 for r + s odd. This implies (N 1 ) r,s = 0 for r + s odd, too. (See Appendix (4) in the lecture note Nonparametric Regression Function Estimation ) p (N 1 γ) 0 = (N 1 ) 0,s µ p+1+s s=0 = (N 1 ) 0,s µ p+1+s s:even = 0 since p s is odd = bias( m(x; h) X 1,, X n ). 65
12 [ = (N 1 m (p+2) (x) δ) 0 (p + 2)! + o(h p+2 ) + O p (n 1/2 h (2p+1)/2 ) (ii) Asymptotic MSE at boundaries : even p case x x n = αh, 0 α < 1 f is right continuous at 0, and f(0+) > 0 + (N 1 δ N 1 JN 1 γ) 0 m (p+1) (x) (p + 1)! v is right continuous at 0 m is (p + 1) times differentiable on (0, ɛ) for some ɛ > 0, and m (p+1) is right continuous at 0 K is supported on [ 1, 1] and is bounded = bias( m(x; h) X 1,, X n ) = (N 1 γ) 0 m (p+1) (0+) (p + 1)! var( m(x; h) X 1,, X n ) h p+1 + o(h p+1 ) + O p (n 1/2 h (2p+1)/2 ) = (N 1 SN 1 ) 0,0 v(0+) f(0+) n 1 h 1 + o p (n 1 h 1 ) ] f (x) h p+2 f(x) where all the entries of N, γ and S are replaced by their corresponding incomplete moments of K and K 2 66
13 (iii) Asymptotic MSE in Int(supp(f)) : odd p f is continuous at x, and f(x) > 0 v is continuous at x m has (p + 1) continuous derivatives at x K is supported on [ 1, 1], and is bounded = bias( m(x; h) X 1,, X n ) = (N 1 γ) 0 m (p+1) (x) (p + 1)! var( m(x; h) X 1,, X n ) h p+1 + o(h p+1 ) + O p (n 1/2 h (2p+1)/2 ) = (N 1 SN 1 ) 0,0 v(x) f(x) n 1 h 1 + o p (n 1 h 1 ) (iv) Asymptotic MSE at boundaries : odd p The conditions and the formula for bias and variance are the same as those in (ii). Proof (i) (iv) It suffices to prove (i). Define ( β 0, β 1,, β p ) = argmin {Y i β 0 β 1 (X i x) β p (X i x) p } 2 K h (X i x). β 0,,β p ( ) Write â 0 = β 0 m(x), â 1 = h( β 1 m (x)),, â p = h p β p m(p) (x). p! m(u) = m(x) + m (x)(u x) + + m(p) (x) (u x) p. p! = {Y i β 0 β 1 (X i x) β p (X i x) p } 2 K h (X i x) = { ( ) ( ) p } 2 Xi x Xi x Y i m(x i ) â 0 â 1 â p h h K h (X i x) = (â 0,, â p ) minimizes, w.r.t. a T = (a 0,, a p ), 67
14 (Yi a T X i ) 2 K h (X i x) = (Y X a) T W(Y X a) = â = (â 0,, â p ) T = (X T WX ) 1 X T WY where Y = (Y 1,, Y n ) T, Y i = Y i m(x i ) X = (X 1,, X n) T, X i = ( 1, ( X i x h W = Diag (K h (X i x)) 1 (bias part) ),, ( Xi x h ) p ) T E(â X 1,, X n ) = (X T WX ) 1 X T WE(Y X 1,, X n ) 1 n (X T WX ) = N f(x) + J f (x)h + o(h) + O p (n 1/2 h 1/2 ) 1 n {X T WE(Y X 1,, X n )} { } = γ m(p+1) (x) m (p+2) (x) (p + 1)! f(x)hp+1 +δ (p + 2)! f(x) + m(p+1) (x) (p + 1)! f (x) + o(h p+2 ) + O p (n 1/2 h (2p+1)/2 ) [N f(x) + J f (x)h + o(h) + O p (n 1/2 h 1/2 )] 1 = N 1 2 (variance part) 1 f(x) N 1 JN 1 f (x) f(x) h + o(h) + O p(n 1/2 h 1/2 ) 2 var(â X 1,, X n ) = (X T WX ) 1 X T ΣX (X T WX ) 1 where Σ = diag(v(x i )(K h (X i x)) 2 ) h n (X T ΣX ) = S v(x)f(x) + o p (1) 1 n (X T WX ) = N f(x) + o p (1) + O p (n 1/2 h 1/2 ) var(â X 1,, X n ) = n 1 h 1 ( 1 n X T WX ) 1 ( h n X T ΣX ) ( 1 n X T WX ) 1 = n 1 h 1 N 1 SN 1 v(x) f(x) + o p(n 1 h 1 ) 68 h p+2
15 (v) Estimation of m (r) ( m (r) (x; h) = r! β r ), r = 0,..., p under the conditions in (i), bias( m (r) (x; h) X 1,, X n ) = r! h r E(â r X 1,, X n ) = r! (N 1 m (p+1) (x) γ) r (p + 1)! hp r+1 [ +r! (N 1 m (p+2) (x) δ) r (p + 2)! + o(h p r+2 ) + O p (n 1/2 h p r+1/2 ) var( m (r) (x; h) X 1,, X n ) = (r!) 2 h 2r var(â r X 1,, X n ) + (N 1 δ N 1 JN 1 γ) r m (p+1) (x) (p + 1)! = (r!) 2 (N 1 SN 1 ) r,r v(x) f(x) n 1 h 2r 1 + o p (n 1 h 2r 1 ) If (p r) is even and all the odd moments of K vanish, then bias( m (r) (x; h) X 1,, X n ) [ = r! (N 1 m (p+2) (x) δ) r (p + 2)! + (N 1 δ N 1 JN 1 γ) r m (p+1) (x) (p + 1)! + o(h p r+2 ) + O p (n 1/2 h p r+1/2 ) p ( ) (N 1 γ) r = (N 1 ) r,s µ p+1+s s=0 = s:r+s even = = 0 s:p+s even (N 1 ) r,s µ p+1+s f ] (x) h p r+2 f(x) (N 1 ) r,s µ p+1+s (p r : even) ] f (x) h p r+2 f(x) 69
16 under the condition in (ii) (p r : even), bias( m (r) (x; h) X 1,, X n ) = r!(n 1 γ) r m (p+1) (0+) (p + 1)! var( m (r) (x; h) X 1,, X n ) h p r+1 + o(h p r+1 ) + O p (n 1/2 h p r+ 1 2 ) = (r!) 2 (N 1 SN 1 ) r,r v(0+) f(0+) n 1 h 2r 1 + o p (n 1 h 2r 1 ) under the condition in (iii) (p r : odd), bias( m (r) (x; h) X 1,, X n ) = r!(n 1 m (p+1) (x) γ) r (p + 1)! hp r+1 + o(h p r+1 ) + O p (n 1/2 h p r+ 1 var( m (r) (x; h) X 1,, X n ) = (r!) 2 (N 1 SN 1 ) r,r v(x) f(x) n 1 h 2r 1 + o p (n 1 h 2r 1 ) under the condition in (iv) (p r : odd), the formula for bias and variance are the same as those in the second case. (vi) Optimal bandwidth for estimating m (r), r = 0,..., p [ ] C 1 = (r!) 2 (N 1 m p+1 2 (x) γ) r (p + 1)! [ C 2 = (r!) 2 (N 1 m p+2 (x) δ) r C 3 = (r!) 2 (N 1 SN 1 ) r,r v(x) f(x) Case 1 (p + 2)! + (N 1 δ N 1 JN 1 m (p+1) (x) γ) r (p + 1)! Interior point, even (p r), symmetric kernel { } 1/(2p+5) (2r + 1)C3 h opt = n 1/(2p+5) 2(p r + 2)C 2 and minimum (conditional) MSE = n 2(p r+2)/(2p+5) 2 ) ] f 2 (x) f(x) 70
17 Case 2 Interior point, odd (p r) or boundary point (x = αh, 0 α < 1), odd (p r) or boundary point, even (p r) { } 1/(2p+3) (2r + 1)C3 h opt = n 1/(2p+3) 2(p r + 1)C 1 and minimum (conditional) MSE = n 2(p r+1)/(2p+3) (vii) Which order to fit when estimating m (r)? polynomial order r r + 1 r + 2 r + 3 interior bias O(h 2 ) O(h 2 ) O(h 4 ) O(h 4 ) boundary bias O(h) O(h 2 ) O(h 3 ) O(h 4 ) variance c 0 n 1 h 2r 1 c 1 n 1 h 2r 1 c 2 n 1 h 2r 1 c 3 n 1 h 2r 1 For the cases of polynomial order r + j with j even, the orders of the interior biases are based on the use of symmetric kernels. c 0 = c 1 < c 2 = c 3 < for interior points (see the lemma below) c 0 < c 1 < c 2 < c 3 < for boundary points When p r is even, the leading bias term involves a complicated constant factor depending on m and f, and the bias order at boundary is inferior to that at interior. When p r is odd, the leading bias term involves a relatively simple constant factor depending only on m, and the boundary bias is of the same order as interior. = p with (p r) odd is recommended. Lemma A, B : k k matrices. Ã, B : (k + 1) (k + 1) matrices s.t. Ã r,s = A r,s and B r,s = B r,s for 0 r, s k 1. Suppose Ãr,s = B r,s = 0 for (r, s) with r + s odd, and also for A and B. Suppose that the matrices obtained by deleting all the odd-numbered columns and rows from A, 71
18 Ã, and those obtained by deleting all the even-numbered columns and rows are invertible. Then (Ã 1 BÃ 1 ) r,r = (A 1 BA 1 ) r,r when k r is odd. Proof See p.10-p.11, Nonparametric Regression Function Estimatoin (6) Minimax efficiency of the local linear smoother x : a fixed interior point of supp(f) (A1) f is continuous at x, and f(x) > 0 (A2) v is continuous at x C 2 = {m : m(u) m(x) m (x)(u x) C 1 2 (u x)2 } (i) Best linear smoother L : class of all linear estimators of the form m(x) = w i (x; X 1,, X n )Y i R L (n, C 2 ) inf sup m L m C 2 E{( m(x) m(x)) 2 X 1,, X n } : linear minimax risk Then, under (A1) and (A2) R L (n, C 2 ) sup m C2 E{( m(x; h) m(x)) 2 X 1,, X n } p 1 where m(x; h) is the local linear smoother with Epanechnikov kernel K and the bandwidth h = [ ( ) ] 1/5 K 2 v(x) n 1/5 µ 2 (K) 2 C 2 f(x) 72
19 (ii) 89.4% minimax efficiency T : class of all estimators of m(x) R(n, C 2 ) = inf sup m T m C 2 E{( m(x) m(x)) 2 X 1,, X n } : minimax risk under (A1) and (A2) R(n, C 2 ) sup m C2 E{( m(x; h) m(x)) 2 X 1,, X n } (0.894)2 + o p (1) where m(x; h) is the local linear smoother with Epanechnikov kernel and the bandwidth h given at (i). Reference Fan(1993) Local linear regression smoothers and their minimax efficiency The Annals of Statistics, vol 21, pp Note Minimax efficiency in the case of general local polynomial estimators and derivative estimation has been dealt in Fan, Gasser, Gijbels, Brockmann and Engel (1997), Annals of Institute of Statistical Mathematics, vol 49, pp The fact that the local linear smoother is the best among all linear estimators at boundary points too has been proved by Cheng, Fan and Marron (1997), The Annals of Statistics, vol 25, pp (7) Bandwidth selection (i) Cross-validation Prediction error : CV (h) ĥ CV 1 n [Y i m(x i ; h)] 2 [Y i m i (X i ; h)] 2 argmin h>0 CV (h) 73
20 where m i ( ; h) is the leave-one-out estimator with the i-th pair (X i, Y i ) being removed in its construction. Reference Härdle, W., Hall, P. and Marron, J. S. (1988). How far are automatically chosen regression smoothing parameters from their optimum (with discussion), Journal of the American Statistical Association, vol. 83, pp (ii) Plug-in methods (p : odd) MISE ( m(x : h) X 1,..., X n ) [ ] = E { m(x; h) m(x)} 2 w(x)dx X 1,..., X n { } (N = h 2p γ) 0 {m } (p+1) 2 w(x)dx (p + 1)! + n 1 h 1 (N 1 SN 1 ) 0,0 {v(x)w(x)/f(x)} dx + o p (h 2p+2 + n 1 h 1 ) When w = f and v(x) σ 2, [ ] 1/(2p+3) h MISE (p + 1)!p!(N 1 SN 1 ) 0,0 σ 2 = 2 {(N 1 γ) 0 } 2 {m (p+1) (x)} 2 n 1/(2p+3) f(x)dx θ p+1 (g) 1 n { 2 m(p+1) (X i ; g)} σ 2 : a reasonable estimator of σ 2 g(h) = C(K)h α with C(K) and α determined in a way similar to Park and Marron (1990) or Sheather and Jones (1991). ĥ SE : the solution of the equation [ ] 1/(2p+3) (p + 1)!p!(N 1 SN 1 ) 0,0 σ 2 h = n 1/(2p+3) 2 {(N 1 γ) 0 } 2 θp+1 (g(h)) 74
21 Reference Ruppert, D., Sheather, S. J. and Wand, M. P. (1995). An effective bandwidth selector for local least squares regression, Journal of the American Statistical Association, vol. 90, pp See Chapter 4 of Fan and Gijbels (1996) for other methods and references therein. (8) Difficulties with design sparseness K is compactly supported, say on [ 1, 1]. = X T WX is singular if I [x h,x+h] (X i ) = 0. [ ] But, P I [x h,x+h] (X i ) = 0 > 0 for all n. = P [ X T WX is singular ] > 0 for all n = Unconditional bias and variance of the local polynomial estimator do not exist. (i) Local polynomial ridge regression 1 (X 1 x) (X 1 x) p X =.. Y = 1 (X n x) (X n x) p W = Diag (K h (X i x)) β ( β 0 (x),..., β p (x)) = argmin (Y Xβ) T W(Y Xβ) β = (X T WX) 1 X T WY β ridge (x) (H + X T WX) 1 X T WY where H is a nonnegative definite matrix such that H + X T WX is nonsingular. 75 Y 1. Y n
22 Reference Seifert, B. and Gasser, T. (1996). Finite-sample variance of local polynomials : analysis and solutions, Journal of the American Statistical Association, vol. 91, pp (ii) Addding pseudo data Idea X (1) X (n), Y [i] X (i). If X (i+1) X (i) is large for some i, then add k i equally spaced pseudo design points to the interval [ ] X (i), X (i+1). For each pseudo design point X (X (i), X (i+1) ), define Y by linear interpolation between the pairs (X (i), Y [i] ) and (X (i+1), Y [i+1] ). Y [i+1] Y [i] pseudo data X (i) X X (i+1) Reference Hall, P. and Turlach, B. A. (1997). Interpolation methods for adapting to sparse design in nonparametric regression, Journal of the American Statistical Association, vol. 92, pp
23 (9) Quantile regression (X 1, Y 1 ),..., (X n, Y n ) iid with a joint cdf F (x, y) and pdf f(x, y) (i) quantile function y = q p (x) F Y X (y x) = p (0 < p < 1) q p (x) = argmin β E[ρ p (Y β) X = x] where ρ p is the check function defined by ρ p (z) = 1 { z + (2p 1)z} 2 = pzi [0, ) (z) (1 p)zi (,0) (z) p = 1 2 p < 1 2 p > 1 2 ( ) E[ρ p (Y β) X = x] = p β yf Y X (y x)dy (1 p) β yf Y X (y x)dy pβ [ 1 F Y X (β x) ] + (1 p)βf Y X (β x) Solving β E[ρ p(y β) X = x] = 0 leads to the result. (ii) applications constructing prediction interval [ q α/2 (x), q 1 α/2 (x) ] detecting heteroscedasticity 77
24 (iii) local polynomial quantile regression q p (x) = β 0 (x) where ( β 0 (x),..., β p (x)) = argmin ρ p (Y i β 0 β 1 (X 1 x) β p (X i x) p ) K h (X i x) β Reference Yu, K. and Jones, M. C. (1998). Local linear quantile regression, Journal of the American Statistical Association, vol. 93, pp (10) Robust regression very useful when the error variance is large. m(x; h) = β 0 (x) where ( β 0 (x),..., β p (x)) = argmin l (Y i β 0 β 1 (X 1 x) β p (X i x) p ) K h (X i x) β Choice of l more resistent to outliers than the squared error loss (i) Huber s Ψ : l = Ψ c where Ψ c (z) = max{ c, min(c, z)} c c c c 78
25 (ii) Huber s bisquare : l = B c where { ( z ) } 2 2 ( z ) B c (z) = z 1 I [ 1,1] c c c c 5 c 5 c References 1 Härdle, W. and Gasser, T. (1984). Robust non-parametric function fitting, Journal of the Royal Statistical Society, Series B, vol. 46, pp Fan, J., Hu, T. -C. and Truong, Y. K. (1994). Robust nonparametric function estimation, Scandinavian Journal of Statistics, vol. 21, pp (iii) Robust locally weighted regression (LOWESS : LOcally WEighted Scatter plot Smoothing due to Cleveland (1979), JASA, vol. 74, pp ) Step 1 For each k, minimize {Y i β 0 β 1 (X i X k ) β p (X i X k ) p } 2 K hk (X i X k ) with respect to β 0,, β p where h k = nd -th smallest number among { X k X j ; j = 1,..., n}. ( nd means the nearest integer to nd, 0 < d 1. Cleveland used K(t) = (1 t 3 ) 3 I ( 1,1) (t), and suggest 0.2 d 0.8). Ŷ k = β 0 β 0 (X k ), k = 1,..., n r k = Y k Ŷk, k = 1,..., n 79
26 Step 2 M = median{ r 1,, r n } δ i = L(r i /6M) where L is a kernel (Cleveland used the biweight kernel L(t) = (1 t 2 ) 2 I ( 1,1) (t).) Step 3 For each k, minimize {Y i β 0 β 1 (X i X k ) β p (X i X k ) p } 2 δ i K hk (X i X k ) Ŷ k = β 0 β 0 (X k ) r k = Y k Ŷk Repeat Step 2 and 3 a total of N times. Get the final fitted values Ŷ k, k = 1,..., n. Get the estimated values at x different from the design points {X k : k = 1,..., n} by interpolation. (Cleveland suggested p = 1, N = 3 ; See Figure 2.4, p.27 of FG) Note When N, Ŷk converges to the robust regression estimator m(x k, h k ) with l (z) = B 6M (z). 80
Nonparametric Regression. Changliang Zou
Nonparametric Regression Institute of Statistics, Nankai University Email: nk.chlzou@gmail.com Smoothing parameter selection An overall measure of how well m h (x) performs in estimating m(x) over x (0,
More informationLocal linear multiple regression with variable. bandwidth in the presence of heteroscedasticity
Local linear multiple regression with variable bandwidth in the presence of heteroscedasticity Azhong Ye 1 Rob J Hyndman 2 Zinai Li 3 23 January 2006 Abstract: We present local linear estimator with variable
More informationNonparametric Regression Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction
Härdle, Müller, Sperlich, Werwarz, 1995, Nonparametric and Semiparametric Models, An Introduction Tine Buch-Kromann Univariate Kernel Regression The relationship between two variables, X and Y where m(
More informationLocal Polynomial Modelling and Its Applications
Local Polynomial Modelling and Its Applications J. Fan Department of Statistics University of North Carolina Chapel Hill, USA and I. Gijbels Institute of Statistics Catholic University oflouvain Louvain-la-Neuve,
More informationLocal linear multivariate. regression with variable. bandwidth in the presence of. heteroscedasticity
Model ISSN 1440-771X Department of Econometrics and Business Statistics http://www.buseco.monash.edu.au/depts/ebs/pubs/wpapers/ Local linear multivariate regression with variable bandwidth in the presence
More information4 Nonparametric Regression
4 Nonparametric Regression 4.1 Univariate Kernel Regression An important question in many fields of science is the relation between two variables, say X and Y. Regression analysis is concerned with the
More informationEcon 582 Nonparametric Regression
Econ 582 Nonparametric Regression Eric Zivot May 28, 2013 Nonparametric Regression Sofarwehaveonlyconsideredlinearregressionmodels = x 0 β + [ x ]=0 [ x = x] =x 0 β = [ x = x] [ x = x] x = β The assume
More informationIntroduction. Linear Regression. coefficient estimates for the wage equation: E(Y X) = X 1 β X d β d = X β
Introduction - Introduction -2 Introduction Linear Regression E(Y X) = X β +...+X d β d = X β Example: Wage equation Y = log wages, X = schooling (measured in years), labor market experience (measured
More informationECON 721: Lecture Notes on Nonparametric Density and Regression Estimation. Petra E. Todd
ECON 721: Lecture Notes on Nonparametric Density and Regression Estimation Petra E. Todd Fall, 2014 2 Contents 1 Review of Stochastic Order Symbols 1 2 Nonparametric Density Estimation 3 2.1 Histogram
More informationNonparametric Econometrics
Applied Microeconometrics with Stata Nonparametric Econometrics Spring Term 2011 1 / 37 Contents Introduction The histogram estimator The kernel density estimator Nonparametric regression estimators Semi-
More informationOn the Robust Modal Local Polynomial Regression
International Journal of Statistical Sciences ISSN 683 5603 Vol. 9(Special Issue), 2009, pp 27-23 c 2009 Dept. of Statistics, Univ. of Rajshahi, Bangladesh On the Robust Modal Local Polynomial Regression
More informationModelling Non-linear and Non-stationary Time Series
Modelling Non-linear and Non-stationary Time Series Chapter 2: Non-parametric methods Henrik Madsen Advanced Time Series Analysis September 206 Henrik Madsen (02427 Adv. TS Analysis) Lecture Notes September
More informationNonparametric Estimation of Regression Functions In the Presence of Irrelevant Regressors
Nonparametric Estimation of Regression Functions In the Presence of Irrelevant Regressors Peter Hall, Qi Li, Jeff Racine 1 Introduction Nonparametric techniques robust to functional form specification.
More informationMinimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model.
Minimax Rate of Convergence for an Estimator of the Functional Component in a Semiparametric Multivariate Partially Linear Model By Michael Levine Purdue University Technical Report #14-03 Department of
More informationDensity estimation Nonparametric conditional mean estimation Semiparametric conditional mean estimation. Nonparametrics. Gabriel Montes-Rojas
0 0 5 Motivation: Regression discontinuity (Angrist&Pischke) Outcome.5 1 1.5 A. Linear E[Y 0i X i] 0.2.4.6.8 1 X Outcome.5 1 1.5 B. Nonlinear E[Y 0i X i] i 0.2.4.6.8 1 X utcome.5 1 1.5 C. Nonlinearity
More informationMultivariate Locally Weighted Polynomial Fitting and Partial Derivative Estimation
journal of multivariate analysis 59, 8705 (996) article no. 0060 Multivariate Locally Weighted Polynomial Fitting and Partial Derivative Estimation Zhan-Qian Lu Geophysical Statistics Project, National
More informationIntroduction to Regression
Introduction to Regression p. 1/97 Introduction to Regression Chad Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/97 Acknowledgement Larry Wasserman, All of Nonparametric
More informationNonparametric Regression. Badr Missaoui
Badr Missaoui Outline Kernel and local polynomial regression. Penalized regression. We are given n pairs of observations (X 1, Y 1 ),...,(X n, Y n ) where Y i = r(x i ) + ε i, i = 1,..., n and r(x) = E(Y
More informationA Note on Data-Adaptive Bandwidth Selection for Sequential Kernel Smoothers
6th St.Petersburg Workshop on Simulation (2009) 1-3 A Note on Data-Adaptive Bandwidth Selection for Sequential Kernel Smoothers Ansgar Steland 1 Abstract Sequential kernel smoothers form a class of procedures
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer May 20, 2015 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures Cross Validation Local Polynomial Regression
More informationWEIGHTED QUANTILE REGRESSION THEORY AND ITS APPLICATION. Abstract
Journal of Data Science,17(1). P. 145-160,2019 DOI:10.6339/JDS.201901_17(1).0007 WEIGHTED QUANTILE REGRESSION THEORY AND ITS APPLICATION Wei Xiong *, Maozai Tian 2 1 School of Statistics, University of
More informationIntroduction to Regression
Introduction to Regression Chad M. Schafer cschafer@stat.cmu.edu Carnegie Mellon University Introduction to Regression p. 1/100 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression
More informationModel-free prediction intervals for regression and autoregression. Dimitris N. Politis University of California, San Diego
Model-free prediction intervals for regression and autoregression Dimitris N. Politis University of California, San Diego To explain or to predict? Models are indispensable for exploring/utilizing relationships
More informationDEPARTMENT MATHEMATIK ARBEITSBEREICH MATHEMATISCHE STATISTIK UND STOCHASTISCHE PROZESSE
Estimating the error distribution in nonparametric multiple regression with applications to model testing Natalie Neumeyer & Ingrid Van Keilegom Preprint No. 2008-01 July 2008 DEPARTMENT MATHEMATIK ARBEITSBEREICH
More informationLocal Modal Regression
Local Modal Regression Weixin Yao Department of Statistics, Kansas State University, Manhattan, Kansas 66506, U.S.A. wxyao@ksu.edu Bruce G. Lindsay and Runze Li Department of Statistics, The Pennsylvania
More informationRemedial Measures for Multiple Linear Regression Models
Remedial Measures for Multiple Linear Regression Models Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Remedial Measures for Multiple Linear Regression Models 1 / 25 Outline
More informationSome Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model
Some Theories about Backfitting Algorithm for Varying Coefficient Partially Linear Model 1. Introduction Varying-coefficient partially linear model (Zhang, Lee, and Song, 2002; Xia, Zhang, and Tong, 2004;
More informationNonparametric Inference in Cosmology and Astrophysics: Biases and Variants
Nonparametric Inference in Cosmology and Astrophysics: Biases and Variants Christopher R. Genovese Department of Statistics Carnegie Mellon University http://www.stat.cmu.edu/ ~ genovese/ Collaborators:
More informationO Combining cross-validation and plug-in methods - for kernel density bandwidth selection O
O Combining cross-validation and plug-in methods - for kernel density selection O Carlos Tenreiro CMUC and DMUC, University of Coimbra PhD Program UC UP February 18, 2011 1 Overview The nonparametric problem
More information3 Nonparametric Density Estimation
3 Nonparametric Density Estimation Example: Income distribution Source: U.K. Family Expenditure Survey (FES) 1968-1995 Approximately 7000 British Households per year For each household many different variables
More informationA Design Unbiased Variance Estimator of the Systematic Sample Means
American Journal of Theoretical and Applied Statistics 2015; 4(3): 201-210 Published online May 29, 2015 (http://www.sciencepublishinggroup.com/j/ajtas) doi: 10.1148/j.ajtas.20150403.27 ISSN: 232-8999
More informationLecture Notes 15 Prediction Chapters 13, 22, 20.4.
Lecture Notes 15 Prediction Chapters 13, 22, 20.4. 1 Introduction Prediction is covered in detail in 36-707, 36-701, 36-715, 10/36-702. Here, we will just give an introduction. We observe training data
More informationLocal regression I. Patrick Breheny. November 1. Kernel weighted averages Local linear regression
Local regression I Patrick Breheny November 1 Patrick Breheny STA 621: Nonparametric Statistics 1/27 Simple local models Kernel weighted averages The Nadaraya-Watson estimator Expected loss and prediction
More informationNonparametric Methods
Nonparametric Methods Michael R. Roberts Department of Finance The Wharton School University of Pennsylvania July 28, 2009 Michael R. Roberts Nonparametric Methods 1/42 Overview Great for data analysis
More informationIntroduction to Regression
Introduction to Regression David E Jones (slides mostly by Chad M Schafer) June 1, 2016 1 / 102 Outline General Concepts of Regression, Bias-Variance Tradeoff Linear Regression Nonparametric Procedures
More informationTime Series and Forecasting Lecture 4 NonLinear Time Series
Time Series and Forecasting Lecture 4 NonLinear Time Series Bruce E. Hansen Summer School in Economics and Econometrics University of Crete July 23-27, 2012 Bruce Hansen (University of Wisconsin) Foundations
More informationNonparametric Modal Regression
Nonparametric Modal Regression Summary In this article, we propose a new nonparametric modal regression model, which aims to estimate the mode of the conditional density of Y given predictors X. The nonparametric
More informationA Primer of Nonparametric Econometrics and Their Applications to Economics and Finance
A Primer of Nonparametric Econometrics and Their Applications to Economics and Finance Zongwu Cai University of North Carolina at Charlotte, USA and Xiamen University, China E-mail:zcai@uncc.edu WHY DO
More informationSimple and Efficient Improvements of Multivariate Local Linear Regression
Journal of Multivariate Analysis Simple and Efficient Improvements of Multivariate Local Linear Regression Ming-Yen Cheng 1 and Liang Peng Abstract This paper studies improvements of multivariate local
More informationSingle Index Quantile Regression for Heteroscedastic Data
Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University JSM, 2015 E. Christou, M. G. Akritas (PSU) SIQR JSM, 2015
More informationVariance Function Estimation in Multivariate Nonparametric Regression
Variance Function Estimation in Multivariate Nonparametric Regression T. Tony Cai 1, Michael Levine Lie Wang 1 Abstract Variance function estimation in multivariate nonparametric regression is considered
More informationOn variable bandwidth kernel density estimation
JSM 04 - Section on Nonparametric Statistics On variable bandwidth kernel density estimation Janet Nakarmi Hailin Sang Abstract In this paper we study the ideal variable bandwidth kernel estimator introduced
More informationSUPPLEMENTARY MATERIAL FOR PUBLICATION ONLINE 1
SUPPLEMENTARY MATERIAL FOR PUBLICATION ONLINE 1 B Technical details B.1 Variance of ˆf dec in the ersmooth case We derive the variance in the case where ϕ K (t) = ( 1 t 2) κ I( 1 t 1), i.e. we take K as
More informationPreface. 1 Nonparametric Density Estimation and Testing. 1.1 Introduction. 1.2 Univariate Density Estimation
Preface Nonparametric econometrics has become one of the most important sub-fields in modern econometrics. The primary goal of this lecture note is to introduce various nonparametric and semiparametric
More informationNonparametric Function Estimation with Infinite-Order Kernels
Nonparametric Function Estimation with Infinite-Order Kernels Arthur Berg Department of Statistics, University of Florida March 15, 2008 Kernel Density Estimation (IID Case) Let X 1,..., X n iid density
More informationRelative error prediction via kernel regression smoothers
Relative error prediction via kernel regression smoothers Heungsun Park a, Key-Il Shin a, M.C. Jones b, S.K. Vines b a Department of Statistics, Hankuk University of Foreign Studies, YongIn KyungKi, 449-791,
More informationNADARAYA WATSON ESTIMATE JAN 10, 2006: version 2. Y ik ( x i
NADARAYA WATSON ESTIMATE JAN 0, 2006: version 2 DATA: (x i, Y i, i =,..., n. ESTIMATE E(Y x = m(x by n i= ˆm (x = Y ik ( x i x n i= K ( x i x EXAMPLES OF K: K(u = I{ u c} (uniform or box kernel K(u = u
More informationBoundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta
Boundary Correction Methods in Kernel Density Estimation Tom Alberts C o u(r)a n (t) Institute joint work with R.J. Karunamuni University of Alberta November 29, 2007 Outline Overview of Kernel Density
More informationERROR VARIANCE ESTIMATION IN NONPARAMETRIC REGRESSION MODELS
ERROR VARIANCE ESTIMATION IN NONPARAMETRIC REGRESSION MODELS By YOUSEF FAYZ M ALHARBI A thesis submitted to The University of Birmingham for the Degree of DOCTOR OF PHILOSOPHY School of Mathematics The
More informationSmooth functions and local extreme values
Smooth functions and local extreme values A. Kovac 1 Department of Mathematics University of Bristol Abstract Given a sample of n observations y 1,..., y n at time points t 1,..., t n we consider the problem
More information2 Two-Point Boundary Value Problems
2 Two-Point Boundary Value Problems Another fundamental equation, in addition to the heat eq. and the wave eq., is Poisson s equation: n j=1 2 u x 2 j The unknown is the function u = u(x 1, x 2,..., x
More informationTransformation and Smoothing in Sample Survey Data
Scandinavian Journal of Statistics, Vol. 37: 496 513, 2010 doi: 10.1111/j.1467-9469.2010.00691.x Published by Blackwell Publishing Ltd. Transformation and Smoothing in Sample Survey Data YANYUAN MA Department
More informationDESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA
Statistica Sinica 18(2008), 515-534 DESIGN-ADAPTIVE MINIMAX LOCAL LINEAR REGRESSION FOR LONGITUDINAL/CLUSTERED DATA Kani Chen 1, Jianqing Fan 2 and Zhezhen Jin 3 1 Hong Kong University of Science and Technology,
More information41903: Introduction to Nonparametrics
41903: Notes 5 Introduction Nonparametrics fundamentally about fitting flexible models: want model that is flexible enough to accommodate important patterns but not so flexible it overspecializes to specific
More informationOptimal bandwidth selection for differences of nonparametric estimators with an application to the sharp regression discontinuity design
Optimal bandwidth selection for differences of nonparametric estimators with an application to the sharp regression discontinuity design Yoichi Arai Hidehiko Ichimura The Institute for Fiscal Studies Department
More informationIntegral approximation by kernel smoothing
Integral approximation by kernel smoothing François Portier Université catholique de Louvain - ISBA August, 29 2014 In collaboration with Bernard Delyon Topic of the talk: Given ϕ : R d R, estimation of
More informationSmooth simultaneous confidence bands for cumulative distribution functions
Journal of Nonparametric Statistics, 2013 Vol. 25, No. 2, 395 407, http://dx.doi.org/10.1080/10485252.2012.759219 Smooth simultaneous confidence bands for cumulative distribution functions Jiangyan Wang
More informationA Novel Nonparametric Density Estimator
A Novel Nonparametric Density Estimator Z. I. Botev The University of Queensland Australia Abstract We present a novel nonparametric density estimator and a new data-driven bandwidth selection method with
More informationSingle Index Quantile Regression for Heteroscedastic Data
Single Index Quantile Regression for Heteroscedastic Data E. Christou M. G. Akritas Department of Statistics The Pennsylvania State University SMAC, November 6, 2015 E. Christou, M. G. Akritas (PSU) SIQR
More informationNew Local Estimation Procedure for Nonparametric Regression Function of Longitudinal Data
ew Local Estimation Procedure for onparametric Regression Function of Longitudinal Data Weixin Yao and Runze Li The Pennsylvania State University Technical Report Series #0-03 College of Health and Human
More informationChapter 2: Fundamentals of Statistics Lecture 15: Models and statistics
Chapter 2: Fundamentals of Statistics Lecture 15: Models and statistics Data from one or a series of random experiments are collected. Planning experiments and collecting data (not discussed here). Analysis:
More informationFast learning rates for plug-in classifiers under the margin condition
Fast learning rates for plug-in classifiers under the margin condition Jean-Yves Audibert 1 Alexandre B. Tsybakov 2 1 Certis ParisTech - Ecole des Ponts, France 2 LPMA Université Pierre et Marie Curie,
More informationQuantitative Economics for the Evaluation of the European Policy. Dipartimento di Economia e Management
Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti 1 Davide Fiaschi 2 Angela Parenti 3 9 ottobre 2015 1 ireneb@ec.unipi.it. 2 davide.fiaschi@unipi.it.
More informationNon-parametric Inference and Resampling
Non-parametric Inference and Resampling Exercises by David Wozabal (Last update 3. Juni 2013) 1 Basic Facts about Rank and Order Statistics 1.1 10 students were asked about the amount of time they spend
More informationDay 3B Nonparametrics and Bootstrap
Day 3B Nonparametrics and Bootstrap c A. Colin Cameron Univ. of Calif.- Davis Frontiers in Econometrics Bavarian Graduate Program in Economics. Based on A. Colin Cameron and Pravin K. Trivedi (2009,2010),
More informationProceedings of the 2016 Winter Simulation Conference T. M. K. Roeder, P. I. Frazier, R. Szechtman, E. Zhou, T. Huschka, and S. E. Chick, eds.
Proceedings of the 2016 Winter Simulation Conference T. M. K. Roeder, P. I. Frazier, R. Szechtman, E. Zhou, T. Huschka, and S. E. Chick, eds. THE EFFECTS OF ESTIMATION OF HETEROSCEDASTICITY ON STOCHASTIC
More informationA Resampling Method on Pivotal Estimating Functions
A Resampling Method on Pivotal Estimating Functions Kun Nie Biostat 277,Winter 2004 March 17, 2004 Outline Introduction A General Resampling Method Examples - Quantile Regression -Rank Regression -Simulation
More informationSimple and Honest Confidence Intervals in Nonparametric Regression
Simple and Honest Confidence Intervals in Nonparametric Regression Timothy B. Armstrong Yale University Michal Kolesár Princeton University June, 206 Abstract We consider the problem of constructing honest
More informationAdaptive Kernel Estimation of The Hazard Rate Function
Adaptive Kernel Estimation of The Hazard Rate Function Raid Salha Department of Mathematics, Islamic University of Gaza, Palestine, e-mail: rbsalha@mail.iugaza.edu Abstract In this paper, we generalized
More informationBETA KERNEL SMOOTHERS FOR REGRESSION CURVES
Statistica Sinica 1(2), 73-91 BETA KERNEL SMOOTHERS FOR REGRESSION CURVES Song Xi Chen La Trobe University Abstract: This paper proposes beta kernel smoothers for estimating curves with compact support
More informationOptimal global rates of convergence for interpolation problems with random design
Optimal global rates of convergence for interpolation problems with random design Michael Kohler 1 and Adam Krzyżak 2, 1 Fachbereich Mathematik, Technische Universität Darmstadt, Schlossgartenstr. 7, 64289
More informationInference on distributions and quantiles using a finite-sample Dirichlet process
Dirichlet IDEAL Theory/methods Simulations Inference on distributions and quantiles using a finite-sample Dirichlet process David M. Kaplan University of Missouri Matt Goldman UC San Diego Midwest Econometrics
More informationAdditive Isotonic Regression
Additive Isotonic Regression Enno Mammen and Kyusang Yu 11. July 2006 INTRODUCTION: We have i.i.d. random vectors (Y 1, X 1 ),..., (Y n, X n ) with X i = (X1 i,..., X d i ) and we consider the additive
More informationPREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY
REVSTAT Statistical Journal Volume 7, Number 1, April 2009, 37 54 PREWHITENING-BASED ESTIMATION IN PARTIAL LINEAR REGRESSION MODELS: A COMPARATIVE STUDY Authors: Germán Aneiros-Pérez Departamento de Matemáticas,
More informationOPTIMAL POINTWISE ADAPTIVE METHODS IN NONPARAMETRIC ESTIMATION 1
The Annals of Statistics 1997, Vol. 25, No. 6, 2512 2546 OPTIMAL POINTWISE ADAPTIVE METHODS IN NONPARAMETRIC ESTIMATION 1 By O. V. Lepski and V. G. Spokoiny Humboldt University and Weierstrass Institute
More informationSTAT 512 sp 2018 Summary Sheet
STAT 5 sp 08 Summary Sheet Karl B. Gregory Spring 08. Transformations of a random variable Let X be a rv with support X and let g be a function mapping X to Y with inverse mapping g (A = {x X : g(x A}
More informationAutomatic Local Smoothing for Spectral Density. Abstract. This article uses local polynomial techniques to t Whittle's likelihood for spectral density
Automatic Local Smoothing for Spectral Density Estimation Jianqing Fan Department of Statistics University of North Carolina Chapel Hill, N.C. 27599-3260 Eva Kreutzberger Department of Mathematics University
More informationThe EM Algorithm for the Finite Mixture of Exponential Distribution Models
Int. J. Contemp. Math. Sciences, Vol. 9, 2014, no. 2, 57-64 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ijcms.2014.312133 The EM Algorithm for the Finite Mixture of Exponential Distribution
More informationOn Asymptotic Normality of the Local Polynomial Regression Estimator with Stochastic Bandwidths 1. Carlos Martins-Filho.
On Asymptotic Normality of the Local Polynomial Regression Estimator with Stochastic Bandwidths 1 Carlos Martins-Filho Department of Economics IFPRI University of Colorado 2033 K Street NW Boulder, CO
More informationChapter 4. Replication Variance Estimation. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 1 / 28
Chapter 4 Replication Variance Estimation J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 1 / 28 Jackknife Variance Estimation Create a new sample by deleting one observation n 1 n n ( x (k) x) 2 = x (k) = n
More informationFall 2017 STAT 532 Homework Peter Hoff. 1. Let P be a probability measure on a collection of sets A.
1. Let P be a probability measure on a collection of sets A. (a) For each n N, let H n be a set in A such that H n H n+1. Show that P (H n ) monotonically converges to P ( k=1 H k) as n. (b) For each n
More informationStatistical inference on Lévy processes
Alberto Coca Cabrero University of Cambridge - CCA Supervisors: Dr. Richard Nickl and Professor L.C.G.Rogers Funded by Fundación Mutua Madrileña and EPSRC MASDOC/CCA student workshop 2013 26th March Outline
More informationERROR-DEPENDENT SMOOTHING RULES IN LOCAL LINEAR REGRESSION
Statistica Sinica 12(2002), 429-447 ERROR-DEPENDENT SMOOTHING RULES IN LOCAL LINEAR REGRESSION Ming-Yen Cheng 1,2 and Peter Hall 1 1 Australian National University and 2 National Taiwan University Abstract:
More informationSMOOTHED BLOCK EMPIRICAL LIKELIHOOD FOR QUANTILES OF WEAKLY DEPENDENT PROCESSES
Statistica Sinica 19 (2009), 71-81 SMOOTHED BLOCK EMPIRICAL LIKELIHOOD FOR QUANTILES OF WEAKLY DEPENDENT PROCESSES Song Xi Chen 1,2 and Chiu Min Wong 3 1 Iowa State University, 2 Peking University and
More informationIntroduction to Nonparametric Regression
Introduction to Nonparametric Regression Nathaniel E. Helwig Assistant Professor of Psychology and Statistics University of Minnesota (Twin Cities) Updated 04-Jan-2017 Nathaniel E. Helwig (U of Minnesota)
More informationReminders. Thought questions should be submitted on eclass. Please list the section related to the thought question
Linear regression Reminders Thought questions should be submitted on eclass Please list the section related to the thought question If it is a more general, open-ended question not exactly related to a
More information12 - Nonparametric Density Estimation
ST 697 Fall 2017 1/49 12 - Nonparametric Density Estimation ST 697 Fall 2017 University of Alabama Density Review ST 697 Fall 2017 2/49 Continuous Random Variables ST 697 Fall 2017 3/49 1.0 0.8 F(x) 0.6
More informationarxiv: v5 [stat.me] 12 Jul 2016
Reliable Prediction Intervals for Local Linear Regression Mohammad Ghasemi Hamed a,, Masoud Ebadi Kivaj b a IFSTTAR-COSYS-LIVIC, 25 Allée des Marronniers 78000 Versailles, France b Independent Researcher,
More informationStatistica Sinica Preprint No: SS
Statistica Sinica Preprint No: SS-017-0013 Title A Bootstrap Method for Constructing Pointwise and Uniform Confidence Bands for Conditional Quantile Functions Manuscript ID SS-017-0013 URL http://wwwstatsinicaedutw/statistica/
More informationEstimation of a quadratic regression functional using the sinc kernel
Estimation of a quadratic regression functional using the sinc kernel Nicolai Bissantz Hajo Holzmann Institute for Mathematical Stochastics, Georg-August-University Göttingen, Maschmühlenweg 8 10, D-37073
More informationA New Method for Varying Adaptive Bandwidth Selection
IEEE TRASACTIOS O SIGAL PROCESSIG, VOL. 47, O. 9, SEPTEMBER 1999 2567 TABLE I SQUARE ROOT MEA SQUARED ERRORS (SRMSE) OF ESTIMATIO USIG THE LPA AD VARIOUS WAVELET METHODS A ew Method for Varying Adaptive
More informationError distribution function for parametrically truncated and censored data
Error distribution function for parametrically truncated and censored data Géraldine LAURENT Jointly with Cédric HEUCHENNE QuantOM, HEC-ULg Management School - University of Liège Friday, 14 September
More informationMeasurement Error in Nonparametric Item Response Curve Estimation
Research Report ETS RR 11-28 Measurement Error in Nonparametric Item Response Curve Estimation Hongwen Guo Sandip Sinharay June 2011 Measurement Error in Nonparametric Item Response Curve Estimation Hongwen
More informationFrontier estimation based on extreme risk measures
Frontier estimation based on extreme risk measures by Jonathan EL METHNI in collaboration with Ste phane GIRARD & Laurent GARDES CMStatistics 2016 University of Seville December 2016 1 Risk measures 2
More informationAn Asymptotic Study of Variable Bandwidth Selection for Local Polynomial Regression with Application to Density Estimation
An Asymptotic Study of Variable Bandwidth Selection for Local Polynomial Regression with Application to Density Estimation Jianqing Fant Department of Statistics University of North Carolina Chapel Hill,
More informationMultiscale Exploratory Analysis of Regression Quantiles using Quantile SiZer
Multiscale Exploratory Analysis of Regression Quantiles using Quantile SiZer Cheolwoo Park Thomas C. M. Lee Jan Hannig April 6, 2 Abstract The SiZer methodology proposed by Chaudhuri & Marron (999) is
More informationThe Closed Form Reproducing Polynomial Particle Shape Functions for Meshfree Particle Methods
The Closed Form Reproducing Polynomial Particle Shape Functions for Meshfree Particle Methods by Hae-Soo Oh Department of Mathematics, University of North Carolina at Charlotte, Charlotte, NC 28223 June
More informationChapter 4. Chapter 4 sections
Chapter 4 sections 4.1 Expectation 4.2 Properties of Expectations 4.3 Variance 4.4 Moments 4.5 The Mean and the Median 4.6 Covariance and Correlation 4.7 Conditional Expectation SKIP: 4.8 Utility Expectation
More informationOptimal Estimation of a Nonsmooth Functional
Optimal Estimation of a Nonsmooth Functional T. Tony Cai Department of Statistics The Wharton School University of Pennsylvania http://stat.wharton.upenn.edu/ tcai Joint work with Mark Low 1 Question Suppose
More informationIndependent and conditionally independent counterfactual distributions
Independent and conditionally independent counterfactual distributions Marcin Wolski European Investment Bank M.Wolski@eib.org Society for Nonlinear Dynamics and Econometrics Tokyo March 19, 2018 Views
More information