The dr Package. R topics documented: September 1, Version Date 2005/08/31. Title Methods for dimension reduction for regression

Transcription

1 The dr Package September 1, 2005 Version Date 2005/08/31 Title Methods for dimension reduction for regression Author Sanford Weisberg Maintainer Sanford Weisberg Functions, methods, and datasets for fitting dimension reduction regression, including phd and inverse regression methods SIR and SAVE. These methods are described, for example, in R. D. Cook (1998), Regression Graphics, Wiley, New York. Also included is code for computing permutation tests of dimension. Adding additional methods of estimating dimension is straightforward. One function depends on the lqs package. Minor revision to be compatible with R 2.0.0, and bug-fixes. For documentation, see License GPL version 2 or newer URL R topics documented: ais cosangle fullquad.fit dr.m dr dr.coplot dr.directions dr.fit dr.indep.test.phdres dr.permutation.test dr.persp dr.slices dr.test dr.weights dr.x dr.y givens.rotation markby mussels

2 2 cosangle rotplot wood.pvalue Index 22 ais Australian athletes data Format Data on 102 male and 100 female athletes collected at the Australian Institute of Sport, courtesy of Richard Telford and Ross Cunningham. This data frame contains the following columns: Sex (0 = male or 1 = female) Ht height (cm) Wt weight (kg) LBM lean body mass RCC red cell count WCC white cell count Hc Hematocrit Hg Hemoglobin Source R. D. Cook and S. Weisberg (1999). Applied Statistics Including Computing and Graphics. New York: Wiley. cosangle Compute the cosine of the angle between a vector and a subspace cosangle1 returns the cosine of the angle between a vector vecs and the subspace spanned by the columns of the matrix mat. For cosangle, vecs can be a matrix, in which case cosangle1 is called for each column in vecs. cosangle(mat, vecs) cosangle1(mat, vec)

3 fullquad.fit 3 mat vecs vec A matrix the provides the basis for a subspace A vector with the same number of rows as mat, or a matrix with the same number of rows as mat A vector with the same number of rows as mat cosangle1 computes the cosine of the angle between vec and the orthogonal projection of vec onto the column space of mat. cosangle repeats this computation for each column of vecs. cosangle1 returns a single value and cosangle returns a vector. Sanford Weisberg <sandy@stat.umn.edu> fullquad.fit Internal function This function recovers a linear predictor from a dr object, and then uses lm to fit a full quadratic regression fullquad.fit(object) object An object of type dr Returns an object of type lm. Sanford Weisberg

4 4 dr.m dr.m function to do... Compute the kernel matrix for a dr method dr.m(object,...) object An object of type dr.... Additional method-specific arguments; see description below This is a method-specific function to compute the kernel matrix M of a dimension reduction method. For sir, save, msir and save, the additional arguements are nslices, the number of slices to be used, and slice.info, which if present is the a list produced by dr.slices. For ols and the phd methods, no additional arguments are required. New dimension reduction methods can be added to dr by writing a new dr.m method. Returns a list with components M slice.info The kernel matrix Information about the slices, if used by the dr method.... Sanford Weisberg, sandy@stat.umn.edu objects to as dr, Examples ##---- Should be DIRECTLY executable!! ---- ##-- ==> Define data, use random, ##-- or do help(data=index) for the standard data sets. ## The function is currently defined as function(object,...){usemethod("dr.m")}

5 dr 5 dr Dimension reduction regression The function dr implements dimension reduction methods, including SIR, SAVE and phd. dr calls dr.compute, so only the former will be needed by most users. dr (formula, data, subset, na.action = na.fail, weights,...) dr.compute (x, y, weights, method = "sir",...) formula data subset weights na.action x y method a symbolic description of the model to be fit. The details of the model are the same as for lm. Although factors may not be appropriate for dr methods, they are permitted. Full rank models are recommended, although rank deficient models are permitted. an optional data frame containing the variables in the model. By default the variables are taken from the environment from which dr is called. an optional vector specifying a subset of observations to be used in the fitting process. an optional vector of weights to be used where appropriate. In the context of dimension reduction methods, weights are used to obtain elliptical symmetry, not constant variance; see dr.weights. a function which indicates what should happen when the data contain NA s. The default is na.fail, which will stop calculations. The option na.omit is also permitted, but it may not work correctly when weights are used. The design matrix The response vector or matrix This character string specifies the method of fitting. sir" specifies sliced inverse regression and save" specifies sliced average variance estimation. phdy" uses principal hessian directions using the response as suggested by Li, and phdres" uses the LS residuals as suggested by Cook. Other methods may be added.... For dr, arguments passed to dr.compute. For dr.compute, arguments required for particular dimension reduction method. nslices is the number of slices used by sir and save. numdir is the maximum number of directions to compute, with default equal to 4. other methods may have other defaults. The general regression problem studies F (y x), the conditional distribution of a response y given a set of predictors x. This function provides methods for estimating the dimension and central subspace of a general regression problem. That is, we want to find a p d matrix B such that F (y x) = F (y B x)

6 6 dr Both the dimension d and the subspace R(B) are unknown. These methods make few assumptions. All the methods available in this function estimate the unknowns by study of the inverse problem, F (x y). In each, a kernel matrix M is estimated such that the column space of M should be close to the central subspace. Eigenanalysis of M is then used to estimate the central subspace. Objects created using this function have appropriate print, summary and plot methods. Weights can be used, essentially to specify the relative frequency of each case in the data. Empirical weights that make the contours of the weighted sample closer to elliptical can be computed using dr.weights. This will usually result in zero weight for some cases. The function will set zero estimated weights to missing. Several functions are provided that require a dr object as input. dr.permutation.tests uses a permutation test to obtain significance levels for tests of dimension. dr.coplot allows visualizing the results using a coplot of either two selected directions conditioning on a third and using color to mark the response, or the resonse versus one direction, conditioning on a second direction. plot.dr provides the default plot method for dr objects, based on a scatterplot matrix. dr returns an object that inherits from dr (the name of the type is the value of the method argument), with attributes: x y The design matrix The response vector weights The weights used, normalized to add to n. qr QR factorization of x. cases call M evalues evectors numdir slice.info method Number of cases used. The initial call to dr. A matrix that depends on the method of computing. The column space of M should be close to the central subspace. The eigenvalues of M (or squared singular values if M is not symmetric). The eigenvectors of M (or of M M if M is not square and symmetric) ordered according to the eigenvalues. The maximum number of directions to be found. The output value of numdir may be smaller than the input value. output from sir.slice, used by sir and save. the dimension reduction method used. dr.weights returns a vector of weights estimated weights, scaled to add to the number of cases. Sanford Weisberg, sandy@stat.umn.edu For weights, see R. D. Cook and C. Nachtsheim (1994), Reweighting to achieve elliptically contoured predictors in regression. Journal of the American Statistical Association, 89, References The details of these methods are given by R. D. Cook (1998). Regression Graphics. New York: Wiley. Equivalent methods are also available in Arc, R. D. Cook and S. Weisberg (1999). Applied Regression Including Computing and Graphics, New York: Wiley,

7 dr.coplot 7 dr.permutation.test,dr.x,dr.y, dr.direction,dr.coplot,dr.weights Examples library(dr) data(ais) attach(ais) # the Australian athletes data #fit dimension reduction using sir m1 <- dr(lbm~wt+ht+rcc+wcc, method="sir", nslices = 8) summary(m1) # repeat, using save: m2 <- update(m1,method="save") summary(m2) # repeat, using phd: m3 <- update(m2, method="phdres") summary(m3) # repeat, using weights: w1 <- dr.weights(lbm~wt+ht+rcc+wcc, covmethod="mve") m4 <- dr(lbm~wt+ht+rcc+wcc, method="sir", nslices = 8, weights=w1) dr.coplot Visualize dimension reduction regression using a coplot This function produces a coplot of the response versus specified directions in dimension reduction regression. dr.coplot(object, which=1:object$numdir, mark.by.y=false,...) object which mark.by.y a dr object specify the indices of the directions to be plotted if FALSE, plot the response versus the first specified direction conditioning on the second specified direction. If TRUE, plot first direction versus second direction conditioning on the third direction, using the response as a marking variable to color the points.... other arguments passed to coplot The function dr.permutation.test.statistic is not to be called by the user, but it computes the test statistic for each different type of dr object. A coplot.

8 8 dr.directions Sanford Weisberg dr dr.directions Directions selected by dimension reduction regressiosn Dimension reduction regression returns a set of p orthogonal direction vectors each of length p, the first d of which are estimates a basis of a d dimensional central subspace. The function returns the estimated directions in the original n dimensional space for plotting. dr.direction(object, which, norm, x) dr.directions(object, which, norm, x) dr.direction.default(object, which=1:object$numdir,norm=false,x=dr.x(object)) object which norm x a dimension reduction regression object select the directions wanted, default is all directions if TRUE, direction vectors are normalized to length 1, otherwise their length is arbitrary select the X matrix, the default is dr.x(object)... additional arguments are passed to dr.direction.default Dimension reduction regression is used to estimate a basis of the central subspace of a regression. If there are p predictors, the dimension reduction regression object includes a p by p matrix of C of eigenvectors. This method returns (X-m1 )C where m is the vector of column means of X. If X is equal to the original matrix of predictors given by dr.x(object), then this gives the directions in the coordinates of the orginal n dimensional space. These directions are used in graphical methods and elsewhere. Returns a matrix. The same function has two names. Sanford Weisberg <sandy@stat.umn.edu> References See R. D. Cook (1998). Regression Graphics. New York: Wiley.

9 dr.fit 9 dr Examples library(dr) data(ais) attach(ais) # the Australian athletes data #fit dimension reduction using sir m1 <- dr(lbm~wt+ht+rcc+wcc, method="sir", nslices = 8) summary(m1) dr.directions(m1) dr.fit Fit dimension reduction regression Internal generic function that estimates the central subspace. dr.fit(object, numdir=4,...) object numdir tol dimension reduction regression object maximum number of dimensions to consider tolerance passed to singular value decomposition... other arguments passed to dr.fit.m These functions will not typically be called directly by the user. At present, the same dr.fit method works for all dimension reduction methods implemented in this package, but one could potentially write a special dr.fit method if needed. The general outline of this method is as follows. (1) A matrix M is computed by a call to dr.fit.m(object,...), such that the columns of M are estimated to fall in the subspace of interest (either the central subspace or the central mean subspace). (2) If M is square, its eigenvalues and eigenvectors are computed; if M is not square, the eigenvalues of M M are computed. (3) M was computed with scaled and centered predictors. The eigenvectors are backtransformed to the original scale. evectors evalues numdir ordered eigenvectors that describe the estimates of the dimension reduction subspace ordered eigenvalues number of eigenvalues raw.evectors eigenvectors of the rotated data M The kernel matrix.

10 10 dr.indep.test.phdres Sanford Weisberg dr dr.indep.test.phdres Test of independence for phd based on residuals Compute the p-value for the hypothesis that the central subspace for OLS residuals has dimension 0 against the alternative that it is greater than zero. This is an internal method not usually needed by the user. dr.indep.test.phdres(object, stat) object stat phdres object value of the test statistic Sanford Weisberg <sandy@stat.umn.edu> References R. D. Cook (1998) Regression Graphics, New York: Wiley. dr

11 dr.permutation.test 11 dr.permutation.test Dimension Reduction Regression Functions These functions require a dimension reduction regression object as input to produce output of various types. dr.permutation.test(object, npermute=50,numdir=object$numdir,permute.weights=tru object a dimension reduction regression object created by dr npermute number of permutations to compute, default is 50 numdir maximum permitted value of the dimension, with the default from the object permute.weights If TRUE, permute weights as well as data in the permutation test. If FALSE, do not permute the weights. Approximates significance levels for tests of dimension using a permutation test. Returns an object of type dr.permutation.test that can be printed or summarized to give the summary of the test. Sanford Weisberg, sandy@stat.umn.edu References See and then select the article on dimension reduction regression or inverse regression. dr

12 12 dr.persp Examples data(ais) attach(ais) # the Australian athletes data #fit dimension reduction regression using sir m1 <- dr(lbm~wt+ht+rcc+wcc, method="sir", nslices = 8) summary(m1) dr.permutation.test(m1,npermute=100) plot(m1) dr.coplot(m1) dr.persp Produce a perspective plot of dimension reduction regression objects. This function uses the sm library to draw a perspective 3D plot of two of the directions obtained by dimension reduction regression versus the response. dr.persp(object, which=1:2, h=c(0.1, 0.1),...) object a dimension reduction regression object which a vector of two directions to use h bandwidths for smoothing passed to sm... graphical parameters passed to sm Returns a perspective plot Sanford Weisberg <sandy@stat.umn.edu> dr, sm

13 dr.slices 13 dr.slices Divide a vector into slices of approximately equal size Divides a vector into slices of approximately equal size. dr.slices(y, nslices) y nslices a vector of length n or an n by p matrix the number of slices, no larger than n, or a vector of p numbers giving the number of slices in each direction. If y has p columns and nslices is a number, then the number of slices in each direction is the smallest integer greater than the p-th root of nslices. If y is an n-vector, order y. The goal for the number of observations per slice is m, the smallest integer in nslices/n. Allocate the first m observations to slice 1. If there are duplicates in y, keep adding observations to the first slice until the next value of y is not equal to the largest value in the first slice. Allocate the next m values to the next slice, and again check for ties. Continue until all values are allocated to a slice. This does not guarantee that nslices will be obtained, nor does it guarantee an equal number of observations per slice. This method of choosing slices is invariant under rescaling, but not under multiplication by -1, so the slices of y will not be the same as the slices of -y. If y is a matrix of p columns, slice the first column as described above. Then, within each of the slices determined for the first column, slice based on the second column, so that each of the "cells" has approximately the same number of observations. Continue through all the columns. This method is not invariant under reordering of the columns, or under multiplication by -1. slice.sizes slice.indicator nslices gives the actual number of slices produced Sanford Weisberg, <sandy@stat.umn.edu> References This function produces the same slice indicators as does the equivalent function in Arc, Cook and Weisberg (1999), Applied Regression Including Computing and Graphics, New York: Wiley.

14 14 dr.test dr dr.test Dimension reduction regression tests This method computes tests of dimension in dimension reduction regression. A separate method is needed for each dimension reduction regression method. Called by the dr function, and not usually accessed directly by the user. dr.test(object, nd) dr.test.default(object, nd) object nd dimension reduction regresion object Maximum number of dimensions to test Returns test statistics and significance levels in a data frame. Sanford Weisberg, <sandy@stat.umn.edu> References R. C. Cook (1998). Regression Graphics. New York: Wiley. dr

15 dr.weights 15 dr.weights Estimate weighting for elliptical symmetry This function estimate weights to apply to the rows of a data matrix to make the resulting weighted matrix as close to multivariate normality as possible. This method is usually not called directly by the user. dr.weights(formula, data = list(), subset, na.action = na.fail, covmethod = "mve", sigma=1, nsamples=null,...) robust.center.scale(x, method,... ) formula data subset na.action covmethod nsamples sigma x method A one-sided or two-sided formula. The right hand side is used to define the design matrix. An optional data frame. A list of cases to be used in computing the weights. The default is na.fail, to prohibit computations. If set to na.omit, the function will return a list of weights of the wrong length for use with dr. covmethod is passed as the argument method to the function cov.rob in the required package lqs. The choices are "classical", "mve" and "mcd". are different with S-Plus. If classical is selected, the usual estimate of the covariance matrix is used, but the center is the medians, not the means. The weights are determined by random sampling from a data-determined normal distribution. This controls the number of samples. The default is 10 times the number of cases. Scale factor, set to one by default An n p data matrix with no missing values see covmethod above... Additional args are passed to cov.rob The basic outline is: (1) Estimate a mean m and covariance matrix S using a possibly robust method; (2) For each iteration, obtain a random vector from N(m,sigma*S). Add 1 to a counter for observation i if the i-th row of the data matrix is closest to the random vector; (3) return as weights the sample faction allocated to each observation. If you set the keyword weights.only to T on the call to dr, then only the list of weights will be returned. Returns a list of n weights, some of which may be zero.

16 16 dr.x Sanford Weisberg, References R. D. Cook and C. Nachtsheim (1994), Reweighting to achieve elliptically contoured predictors in regression. Journal of the American Statistical Association, 89, dr, lqs Examples data(ais) w1 <- dr.weights(~ Ht +Wt +RCC, data = ais) m1 <- dr(lbm~ht+wt+rcc,data=ais,weights=w1) dr.x Accessor functions for dr. objects A concise (1-5 lines) description of what the function does. dr.x(object) dr.wts(object) dr.qr(object) dr.q(object) dr.r(object) dr.z(object) dr.y.name(object) object An object that inherits from dr. Returns a component of a dr object. For example, dr.qr returns object$qr, dr.wts returns object$weights, dr.q returns the Q component of object$qr, dr.z returns the scaled and centered design matrix. Sanford Weisberg, sandy@stat.umn.edu objects to as dr.

17 dr.y 17 Examples ##---- Should be DIRECTLY executable!! ---- ##-- ==> Define data, use random, ##-- or do help(data=index) for the standard data sets. ## The function is currently defined as function(object) {object$x} dr.y function to do... Generic function to return the response for a dr fit. dr.y(object) object A dr object This returns the response variable for a dr fit. For most, dr methods, this is just the response as specified in the call to dr. For phd and phdres, this returns the residuals from the OLS regression of the original response on the design matrix. For phdy, it returns the response minus its mean. A vector. Sanford Weisberg, sandy@stat.umn.edu objects to as dr. Examples ##---- Should be DIRECTLY executable!! ---- ##-- ==> Define data, use random, ##-- or do help(data=index) for the standard data sets. ## The function is currently defined as function(object) {UseMethod("dr.y")}

18 18 markby givens.rotation Create givens rotation matrix For a given angle theta, returns a p by p Givens rotation matrix. givens.rotation(theta, p=2, which=c(1, 2)) theta p which an angle in radians the dimension of the matrix to be produced two numbers between 1 and p giving the columns/rows for the nonzero elements of the result. Returns a p by p matrix z of all zeroes, except z[which,which] has elements cos(theta), -sin(theta), sin(theta) and cos(theta), in column order. sandy@stat.umn.edu References Gene H. Golub and Charles F. Van Loan (1989). Matrix Computations, Second Edition. Baltimore: Johns Hopkins Press, p Examples givens.rotation(1,4,c(1,3)) markby Produce a vector of colors/symbols to mark points This function creates marking of points by color or symbol for use in graphs. markby(z, use="color", values=null, color.fn=rainbow, na.action="na.use")

19 mussels 19 z use values color.fn na.action a variable with a few distict values that will define the groups for marking. if equal to "color", will returna list of colors. If anything else, it will return a list of symbols for marking. a list with as many values as unique values of z that determine the colors or symbols. If this is not set, then the function rainbow is used for colors and 1:length(unique(z)) is used for symbols. By default, missing values (NAs) are treated as another category in the color marking. Setting this argument to anything else will return NA for the missing category names. This will cause an error in R graphics routines, but see below for usage of this options. Returns length(z) values that specify the color or symbol for each point. Sanford Weisberg, <sandy@stat.umn.edu> References This function is to help users familiar with Arc, as discussed in R. D. Cook and S. Weisberg (1999). Applied Regression Including Computing and Graphics, New York: Wiley. Examples x <- rnorm(100) y <- rnorm(100) z <- cut(rnorm(100),3) # Scatterplot, mark using color with groups determined by Status plot(x,y,col=markby(z)) # Scatterplot, mark using symbols with groups determined by Status plot(x,y,pch=markby(z,use="symbols")) # handing of missing values: z[1:10] <- NA # set to missing marks <- markby(z,na.action="omit") sel <-!is.na(marks) plot(x[sel],y[sel],col=marks[sel]) mussels Mussels muscles data Data were furnished by Mike Camden, Wellington Polytechnic, Wellington, New Zealand. Horse mussels, (Atrinia), were sampled from the Marlborough Sounds. The response is the mussels Muscle Mass.

20 20 rotplot Format This data frame contains the following columns: H Shell height in mm L Shell length in mm M Muscle mass in g S Shell mass in g W Shell width in mm Source R. D. Cook and S. Weisberg (1999). Applied Statistics Including Computing and Graphics. New York: Wiley. rotplot draw many 2D projections of a 3D plot This function draws several 2D views of a 3D plot, sort of like a spinning plot. rotplot(x, y, number=9, theta=seq(0, pi/2, length = number),...) x a matrix with 2 columns giving the horizontal axes of the full 3D plot. y the vertical axis of the 3D plot. number Number of 2D views of the 3D plot. theta a list of rotation angles... additional arguments passed to coplot For each value of theta, draw the plot of cos(theta)*x[,1]+sin(theta)*x[,2] versus y. returns a graph object. Sanford Weisberg, sandy@stat.umn.edu

21 wood.pvalue 21 Examples data(ais) attach(ais) m1 <- dr(lbm ~ Ht + Wt + WCC) rotplot(dr.direction(m1,which=1:2),dr.y(m1),col=markby(sex)) wood.pvalue Chi-square approximation to a sum of Chi-squares Returns an approximation to P(coef X > f) for X=(X1,...,Xk), a vector of iid one df chi-squared rvs. coef is a list of positive coefficients. tol is used to check for near-zero conditions. wood.pvalue(coef, f, tol=0, print=false) coef f tol print Vector of positive numbers, usually from the eigenvalues of a random matrix observed value of statistic A small number for tolerance If TRUE, print the output; if FALSE return the pvalue Following Wood (1989), use either a Saterthwaite-Welsh, inverse gamma or three-parameter Pearson Type IV approximation, depending on the values of the coefficients. Returns the tail area probability. Sanford Weisberg <sandy@stat.umn.edu>. Arc version by Dennis Cook. References See Wood (1989), Communications in Statistics, Simulation Translated from Arc function wood-pvalue. dr

22 Index Topic datasets ais, 1 mussels, 19 Topic internal dr.fit, 9 dr.m, 3 dr.x, 16 dr.y, 17 fullquad.fit, 3 Topic regression cosangle, 2 dr, 4 dr.coplot, 7 dr.directions, 8 dr.fit, 9 dr.indep.test.phdres, 10 dr.permutation.test, 11 dr.persp, 12 dr.slices, 13 dr.test, 14 dr.weights, 15 givens.rotation, 18 markby, 18 rotplot, 20 wood.pvalue, 21 AIS (ais), 1 ais, 1 cosangle, 2 cosangle1 (cosangle), 2 cov.ew.matrix (dr.indep.test.phdres), 10 dr, 4, 4, 7, 8, 10 12, 14, 16, 17, 21 dr.coplot, 6, 7 dr.direction, 6 dr.direction (dr.directions), 8 dr.directions, 8 dr.fit, 9 dr.indep.test.phdres, 10 dr.m, 3 dr.permutation.test, 6, 11 dr.permutation.test.statistic.default (dr.permutation.test), 11 dr.permutation.test.statistic.phd (dr.permutation.test), 11 dr.permutation.test.statistic.phdres (dr.permutation.test), 11 dr.permutation.test.statistic.phdy (dr.permutation.test), 11 dr.persp, 12 dr.q (dr.x), 16 dr.qr (dr.x), 16 dr.r (dr.x), 16 dr.slice.1d (dr.slices), 13 dr.slice1 (dr.slices), 13 dr.slice2 (dr.slices), 13 dr.slices, 3, 13 dr.test, 14 dr.test2.phdres (dr.test), 14 dr.weights, 5, 6, 15 dr.wts (dr.x), 16 dr.x, 6, 16 dr.y, 6, 17 dr.y.name (dr.x), 16 dr.z (dr.x), 16 fullquad.fit, 3 givens.rotation, 18 lqs, 16 markby, 18 mussels, 19 plot.dr (dr), 4 print.dr (dr), 4 print.dr.permutation.test (dr.permutation.test), 11 print.summary.dr (dr), 4 robust.center.scale (dr.weights), 15 rotplot, 20 sm, 12 22

23 INDEX 23 summary.dr (dr), 4 summary.dr.permutation.test (dr.permutation.test), 11 wood.pvalue, 21