On Fractile Transformation of Covariates in Regression 1

Transcription

1 1 / 13 On Fractile Transformation of Covariates in Regression 1 Bodhisattva Sen Department of Statistics Columbia University, New York ERCIM December, Joint work with Probal Chaudhuri, Indian Statistical Institute, Calcutta

2 Example 1 Household Expenditure and Income Data Investigate the inequality in income and compare the economic condition of Poland (blue) and Bulgaria (red) X = total expenditure; Y = proportion of expenditure on food as a fraction of X per capita per household 0.65 Regression Curves 0.65 Standardized Regression Curves Prop. of Expenditure on Food Prop. of Expenditure on Food Total Expenditure (in USD) Usual regression functions Total Expenditure (in USD) Standardized reg. functions 2 / 13

3 3 / 13 Example 2 Data on the sales (in Indian rupees) and profit (as a fraction of sales) for companies over different years Compare the Y = profitability of the companies against X = sales for years 1997 (red) and 2003 (blue) 0.2 Regression Curves 0&14 Standardized Regression Curves 0& &1 Profit to Sales Pro=it to Sales 0&0) 0&0( 0&04 0&02 0 0! Sales x 10 6 Usual regression functions!0&02! Sales Standardized reg. functions

4 Problem: Comparison of two regression functions Two bivariate populations (X 1, Y 1 ) and (X 2, Y 2 ) We usually look at µ i (x) = E(Y i X i = x), i = 1, 2 Instead, compare the fractile regression functions m i (t) = E{Y i F i (X i ) = t}, t (0, 1) where F i is the c.d.f. of X i 0.7 Fractile Graphs 0.08 Fractile Graphs Prop. of Expenditure on Food Profit to Sales 0!0.02! !0.06! ! Fractiles of Total Expenditure (in USD)! Fractiles of Sales Fractile regression functions in Examples 1 and 2 4 / 13

5 Other applications of fractile regression 5 / 13 Hertz-Picciotto and Din-Dzietham (Epidemiology, 1998) compare the infant mortality of African and European Americans with gestational age Nordhaus (PNAS, 2006) compares the dependence of log of output density with key geographic variables Fractile regression enables us to simultaneously compare the effect of different covariates on one response variable

6 Why the fractile transformation X 1 F 1 (X 1 )? 6 / 13 Transformed covariates F 1 (X 1 ) and F 2 (X 2 ) both have a Unif (0, 1) distribution; thereby adjusting for covariate skewness/data sparsity Distribution-free nonparametric standardization Compare m 1 (t) and m 2 (t), the means of Y 1 and Y 2 at the t-th quantile of the covariates Makes the fractile regression functions invariant under all strictly increasing transformations of the covariate, e.g., if X 2 = φ(x 1 ), Y 1 = Y 2, then E{Y 1 F 1 (X 1 )} = E{Y 2 F 2 (X 2 )} Mahalanobis (Econometrica, 1960), Sen and Chaudhuri (JASA, 2010),...

7 Extension to multi-dimension Questions: How do we standardize the distribution of the covariates that will enable a more meaningful comparison of the regression functions? Suppose (X 1, Y ) and (X 2, Y ) in R d+1 for d 1, X 2 = g(x 1 ) and g : R d R d is an (unknown) invertible function. How to standardize the covariates and conclude that the two regression functions are essentially the same? 7 / 13

8 Notation 8 / 13 (X, Y ) is a random vector having a continuous distribution on R d+1, d 1, where X = (X 1, X 2,..., X d ) R d Standardization of the covariate: T : P R d E R d such that x T(P, x) T(X, x) is an invertible map from X P, the support of P, onto E, for every X P P, a class of distributions on R d. T ls (P, x) = Γ(P) 1/2 {x µ(p)}, Γ(P) = diag(σ 2 1,..., σ2 d ) The standardized regression function is then defined as m X (t) = E{Y T(P, X) = t} for t E. G: group of one-one transformations acting on the space of all predictors X P. We say that T is invariant under G if T(g(X), g(x)) = T(X, x), for all x R d and g G.

9 Fractile Standardization 9 / 13 For X P, define R P : R d (0, 1) d, as R P (x) = ( F 1 (x 1 ), F 2 1 (x 2 x 1 ),..., F d 1,...,d 1 (x d x 1,..., x d 1 ) ), where F 1 (x 1 ) = P(X 1 x 1 ), F 2 1 (x 2 ) = P(X 2 x 2 X 1 = x 1 ),... Fractile regression: m X (t) = E{Y R P (X) = t}, t (0, 1) d Distributional standardization: R P (X) Uniform(0, 1) d Multivariate analogue of X 1 F 1 (X 1 )

10 Invariance 10 / 13 Consider the group F, x (g 1 (x 1 ),..., g d (x d )), where g i : R i R, is a func. in x i for every fixed (x 1,..., x i 1 ), and (g 1,..., g i ) : R i R i is invertible for every i Invariance: for g F, R X (x) = R g(x) (g(x)) for all x R d {all coordinate-wise increasing transformations} F If we want the standardized regression function to be invariant under the group action F, then the standardization T(X, ) has to be a function of R P Furthermore, if we assume that T(X, X) Unif (0, 1) d and T(X, ) F then T(X, x) = R P (x) for all x, for all X P P

11 Curse of dimensionality! 11 / 13 Computation of R P X 1, X 2,..., X n i.i.d. P R P requires estimation of conditional distribution functions may use a kernel estimate of the multivariate density of X 1, and then use it to get the various conditional densities f n;1,2,...,d (x) = 1 n(h 1,n h 2,n... h d,n ) n ( ) x Xi K i=1 f n;j 1,...,j 1 (x j x 1,..., x j 1 ) = f n;1,...,j(x 1,..., x j ) f n;1,...,j 1 (x 1,..., x j 1 ) Standardized covariates: R n (X 1 ), R n (X n ),..., R n (X n ) h n Under appropriate conditions, sup x R n (x) R P (x) P 0.

12 Computation of fractile regression Smooth estimate of fractile regression: n m n (t) = Y i W n,i (t), t (0, 1) d i=1 Nadaraya-Watson type weight: W n,i (t) = ( ) K t R n(x i ) ( hn ) n t R j=1 K n(x j ) hn Y on X 1 and X 2 Y on X 2 and X 1 12 / 13

13 13 / 13 Summary Usual comparison of regression functions not always possible and meaningful R P acheives distributional standardization R P has nice invariance properties, but computationally challanging for d large Alternatives: marginal standardization, centered rank function (multivariate distribution transform), etc. Thank You! Questions?