Quantile Processes for Semi and Nonparametric Regression

Transcription

1 Quantile Processes for Semi and Nonparametric Regression Shih-Kang Chao Department of Statistics Purdue University IMS-APRM 2016 A joint work with Stanislav Volgushev and Guang Cheng

2 Quantile Response Y, predictors X. Conditional quantile curve Q( ; τ) of Y R conditional on X is defined through P (Y Q(X; τ) X = x) = τ x. Examples: Y = 0.5*X + N(0,1) Y = X + (.5+X)*N(0,1) Y X=x ~ Beta(1.5 x,.5+x)

3 Quantile regression vs mean regression Mean regression: Y i = m(x i ) + ε i, E[ε X = x] = 0 m: Regression function, object of interest. ε i : errors. Quantile regression: P (Y Q(x; τ) X = x) = τ No strict distinction between signal and noise. Object of interest: properties of conditional distribution of Y X = x. Contains much richer information than just conditional mean.

4 Quantile Regression: Estimation Koenker and Bassett (1978): if Q(x; τ) = β(τ) x, estimate by β(τ) := arg min b ρ τ (Y i b X i ) (1.1) i where ρ τ (u) := τu + + (1 τ)u check function. convex optimization. Well-behaved Q(x 0 ; τ) := x 0 β(τ) for any x 0. Series Model Partial Linear Model

5 Quantile Regression Process The quantile regression process (QRP) at x 0 is a n ( Q(x 0 ; τ) Q(x 0 ; τ)) l (T ), (1.2) where a n appropriately chosen, T (0, 1) is compact For fixed x 0, and τ L, τ U close to 0 and 1, Q 1 (x 0 ; y) = F (y x 0 ) τ L + τu τ L 1{Q(x 0 ; τ) < y}dτ =: Φ y (Q(x 0 ; τ)) BMD Q^( Age x 0 = 13 ; τ) Q^( Age x 0 = 18 ; τ) Q^( Age x 0 = 23 ; τ) CDF F^(y Age x 0 = 13) F^(y Age x 0 = 18) F^(y Age x 0 = 23) τ BMD

6 Convergence of QRP If for a n, a n ( Q(x 0 ; ) Q(x 0 ; )) G( ) in l (T ), (1.3) where G( ): Gaussian process, l (T ): set of all uniformly bounded, real functions on T, then, a n { FY X ( x 0 ) F Y X ( x 0 ) } f Y X ( x 0 )G ( x 0 ; F Y X ( x 0 ) ) in l (Y). (1.4) Proof: Φ y (Q(x; τ)) is Hadamard differentiable (Chernozhukov et al., 2010) (tangentially to C(0, 1) at any strictly increasing, differentiable function); functional delta method

7 Study (1.3): Fixed dimension linear model Q(x; τ) = x β(τ): Koenker and Xiao (2002); Angrist et al. (2006) Kernel nonparametric estimation: Qu and Yoon (2015) A Unified framework (Belloni et al., 2011): Q(x; τ) Z(x) β(τ) Z(X i ) can be a higher dimensional ( ) transformation Q(x; τ) = Z(x) β(τ), where β(τ) is estimated by replacing X i by Z(X i ) in Quantile Regression Need to control the bias Q(x; τ) Z(x) β(τ)

8 Overview We present process convergence results for the models: a n ( Q(x 0 ; ) Q(x 0 ; )) G( ) in l (T ) General series estimator B-splines: Z(x) = B(x) An application: partial linear models Z = (V, Z(W ) ) R k+k Notation: Z i := Z(X i ) general basis function (e.g. trigonometric, power, etc.); B i := B(X i ) local basis (e.g. B-spline)

9 Technical Assumptions Assumption (A): data (X i, Y i ) i=1,...,n form triangular array and are row-wise i.i.d. with (A1) m = Z(x). Assume that Z i ξ m <, and there exits some fixed constant M so that 1/M λ min (E[ZZ T ]) λ max (E[ZZ T ]) M (A2) The conditional distribution F Y X (y x) is twice differentiable w.r.t. y. Denote the corresponding derivatives by f Y X (y x) and f Y X (y x). Assume that f := sup y,x uniformly in n. f Y X (y x) <, f := sup f Y X (y x) < y,x (A3) 0 < f min inf τ T inf x f Y X (Q(x; τ) x) uniformly in n.

10 A Bahadur Representation Under Assumption (A), mξm 2 log n = o(n). For any β n ( ) satisfying [ { g n (β n ) := sup E Zi FY X (Z i β n (τ) X) τ }] = o(ξ 1 τ T c n (β n ) := sup Q(x; τ) Z(x) β n (τ) = o(1) x,τ T we have β(τ) β n (τ) = 1 n J m(τ) 1 m ) n Z i (1{Y i Z i b} τ) +Remainder, i=1 } {{ } ( ) where J m (τ) := E[f Y X (Q(X; τ))z(x)z(x) ].

11 Assumption ( ): mc n log n = o(1), m 3 ξm(log 2 n) 3 = o(n), g n = o(n 1/2 ) and for any u = 1, [ u J m (τ) 1 ( E Z i 1{Yi Q(X i ; τ)} 1{Y i Z i β n (τ)} )] = o(n 1/2 ). sup τ T Under Assumption ( ), we can replace ( ) by U n (τ) := n 1 J 1 m (τ) n ( Z i 1{Yi Q(X i ; τ)} τ ). i=1

12 τ J 1 m (τ)z i ( 1{Yi Q(X i ; τ)} τ ) : A triangular array Not Lipschitz in τ Asymptotic Equicontinuity of Quantile Process: Under Assumption (A) and ξm(log 2 n) 2 = o(n), we have for any ε > 0 and vector u n R m with u n = 1, ( ) lim lim sup P n 1/2 u n U n (τ 1 ) u n U n (τ 2 ) > ε = 0. δ 0 n sup τ 1 τ 2 δ τ 1,τ 2 T Proof: Stochastic Equicontinuity + Chaining (Kley et al., 2015; van der Vaart and Wellner, 1996)

13 Weak Convergence: General Series Estimator Under Assumption (A) and ( ). For a sequence u n, if H(τ 1, τ 2 ; u n ) := lim n u n 2 u n J 1 m (τ 1 )E[ZZ ]J 1 m (τ 2 )u n (τ 1 τ 2 τ 1 τ 2 ) exists for any τ 1, τ 2 T, then n ( u n u n β( ) ) u n β n (τ) G( ) in l (T ), (2.1) where G( ) is a centered Gaussian process with the covariance function H. In particular, there exists a version of G with almost surely continuous sample paths. Proof: Asymptotic equicontinuity, and verifying the Lindeberg condition.

14 Splines B-Splines B = (b 1 (x), b 2 (x),..., b m (x)) are local basis functions Notation: For I {1,..., m} and a R m, a (I) = (a j )m j=1 Rm where a j = 0 for j I

15 Benefit from Using Splines J m (τ) is a block matrix, where J m (τ) = E[f Y X (Q(X; τ))b(x)b(x) ], entries in J 1 m (τ) decay geometrically from the main diagonal (Demko et al., 1984) (a) J m(τ), dim(x) = 1 (b) J 1 m (τ), dim(x) =

16 If a has at most a 0 nonzero consecutive entries, we can find set I(a) {1,..., m} with I(a) log n such that a J 1 m (τ) (a J 1 m (τ)) (I(a)) a a 0 n c where c > 0 is arbitrary If u n is nonzero at the position in an index set I, which consists of L < consecutive entries u n U n (τ) 1 n n u n J 1 i=1 m (τ)b (I(un)) i (1{Y i B i β n (τ) (I (u n)) } τ) where I (u n ) = {1 j m : i I(u n ) such that j i r} A reduction from dimension m to log n!

17 Define γ n (τ) := argmin b R m E [ (B b Q(X; τ)) 2 f Y X (Q(X; τ) X) ], Assumption ( ): c 2 n = o(n 1/2 ), ξ 4 m(log n) 6 = o(n), where c n (γ n ) := sup Q(x; τ) Z(x) β n (τ) x,τ T Compare to Assumption ( ): ξ m = O(m 1/2 ): ξ 4 m(log n) 6 = o(n) is much weaker than m 3 ξ 2 m(log n) 3 = o(n) in Assumption ( ) If Q(x; τ) is smooth in x for all τ, then c n = o(n 2/5 ) when x R

18 Weak Convergence: Spline Series Estimator Under Assumption (A) and ( ). For a sequence u n, if H(τ 1, τ 2 ; u n ) := lim n u n 2 u n J 1 m (τ 1 )E[BB ]J 1 m (τ 2 )u n (τ 1 τ 2 τ 1 τ 2 ) exists for any τ 1, τ 2 T, then n ( u n u n β( ) ) u n γ n ( ) G( ) in l (T ), (3.1) where G( ) is a centered Gaussian process with the covariance function H. In particular, there exists a version of G with almost surely continuous sample paths.

19 Quantile Regression General Series Model Local Basis Model References Tensor-Product Spline 1 (τ ) Conjecture(Demko et al., 1984, Sec.5): the elements of Jm from the tensor-product spline B are also decreasing geometrically away from the main diagonal 1 (d) Jm (τ ), dim(x) = 2 (c) Jm (τ ), dim(x) =

20 Application: Partial Linear Model Partial Linear Model: Q(X; τ) = V α(τ) + h(w ; τ), (3.2) where X = (V, W ) R k+k and k, k N are fixed. Expanding w h(w; τ) in terms of basis vectors w Z(w), we can approximate (3.2): Q(x; τ) Z(x) β n (τ) where Z(x) = (v, Z(w) ) and β n = (α(τ), β n(τ) ) Quantile Regression gives α(τ) and ĥ(w; τ) = Z(w) β n (τ)

21 Assumption (B): (B1) Define c n := sup τ,w Z(w) β n(τ) h(w; τ) and assume that ξ m c n = o(1); sup E[f Y X (Q(X; τ) X) h V W (W ; τ) A(τ) Z(W )) 2 ] = O(λ 2 n) τ T with ξ m λ 2 n = o(1) (B2) We have max j k V j < C almost surely for some constant C > 0. (B3) ( 2/3 mξ m n log n ) 3/4 + c 2 n ξ m = o(n 1/2 ). Moreover, assume that c nλ n = o(n 1/2 ) and mc n log n = o(1).

22 Joint Process Convergence Under Assumption (A) and (B), suppose at w 0, Γ 22 (τ 1, τ 2 ) = lim n Z(w 0 ) 2 Z(w0 ) M 2 (τ 1 ) 1 E[ Z(W ) Z(W ) ]M 2 (τ 2 ) 1 Z(w0 ) exists, then ( n { α( ) α( ) } ) n {ĥ(w0 ; ) h(w 0 ; ) } ( G 1 ( ),..., G k ( ), G h (w 0 ; ) ), Z(w 0 ) in (l (T )) k+1 where (G 1 ( ),..., G k ( ), G h (w 0 ; )) are centered Gaussian processes with joint covariance function ( ) Γ(τ 1, τ 2 ; Z(w Γ11 (τ 0 )) = (τ 1 τ 2 τ 1 τ 2 ) 1, τ 2 ) 0 k 0 k Γ 22 (τ 1, τ 2 ) Detail In particular, there exists a version of G h (w 0 ; ) with almost surely continuous sample paths.

23 Remarks Most asymptotic results for partial linear model separately study the parametric and nonparametric part Joint asymptotics phenomenon for mean partial linear model: Cheng and Shang (2015) We show such joint asymptotics holds in process sense for quantile model

24 Thank you for your attention arxiv

25 References I Angrist, J., Chernozhukov, V., and Fernández-Val, I. (2006). Quantile regression under misspecification, with an application to the U.S. wage structure. Econometrica, 74(2): Belloni, A., Chernozhukov, V., and Fernández-Val, I. (2011). Conditional quantile processes based on series or many regressors. arxiv preprint arxiv: Cheng, G. and Shang, Z. (2015). Joint asymptotics for semi-nonparametric regression models with partially linear structure. Annals of Statistics, 43(3): Chernozhukov, V., Fernández-Val, I., and Galichon, A. (2010). Quantile and probability curves without crossing. Econometrica, 78(3): Demko, S., Moss, W. F., and Smith, P. W. (1984). Decay rates for inverses of band matrices. Mathematics of Computation, 43(168):

26 References II Kley, T., Volgushev, S., Dette, H., and Hallin, M. (2015+). Quantile spectral processes: Asymptotic analysis and inference. Bernoulli. Koenker, R. and Xiao, Z. (2002). Inference on the quantile regression process. Econometrica, 70(4): Qu, Z. and Yoon, J. (2015). Nonparametric estimation and inference on conditional quantile processes. Journal of Econometrics, 185:1 19. van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer.

27 Detail for PLM M 2 (τ) := E [ Z(W ) Z(W ) f Y X (Q(X; τ) X) ] Γ 11 (τ 1, τ 2 ) = M 1,h (τ 1 ) 1 E [ (V h V W (W ; τ 1 ))(V h V W (W ; τ 2 )) ] M 1,h (τ 2 ) 1 where M 1,h (τ) = E [ (V h V W (W ; τ))(v h V W (W ; τ)) f Y X (Q(X; τ) X) ] Partial Linear Model