22s:152 Applied Linear Regression. Chapter 2: Regression Analysis. a class of statistical methods for

Transcription

1 22s:152 Applied Linear Regression Chapter 2: Regression Analysis Regression analysis a class of statistical methods for studying relationships between variables that can be measured e.g. predicting blood pressure from age using known values of certain variables to predict the values of other variables for the same subjects e.g. given a person s age, cholesterol, and weight, predict blood pressure 1

2 Well-known Example: Space Shuttle Challenger On January 27, 1986, the night before a planned launch, a 3-hour discussion took place. The discussion was about the forecasted low temperature for the next day of 31 F, and the effect of low temperature on O-ring performance. (O-rings seal joints). In their discussion they utilized the following plot showing the relationship between the number of O-rings having some thermal distress and the temperature to decide whether the shuttle should take-off as planned. Number of incidents temperature 2

3 The final decision was to launch the shuttle as planned. - 7 astronauts were killed - combustion gas leak through an O-ring was the cause of the accident Post-tragedy, a commission noted that a mistake in the analysis of the data was that the flights with zero incidents were left off because it was felt that these flights did not contribute any information about the temperature effect. Number of incidents temperature 3

4 What may have helped in the decision making process? - use off all the data (rather than using data conditional on the occurrence of an incident) - quantification of the relationship between temperature and O-ring failure (perhaps as a conditional probability) - prediction of the probability of O-ring failure at 31 F (logistic regression, Dalal et al. used this approach in the their 1989 article) Dalal, S.R, Fowlkes, E.B. and Hoadley, B. (1989). Risk analysis of the Space Shuttle: Pre-Chellenger Predicton of Failure. Journal of the American Statistical Association, v.84,

5 Investing it: duffers need not apply New York Times, May 31, 1998 An example of inappropriate removal of outliers - An investment compensation expert carried out a study purporting to show that the major companies, whose C.E.O s had low golf scores, had high performing stocks. - The expert obtained data for golf scores from the journal Golf Digest and used his own data on the stock market performance of the companies of 51 chief executives. - He created a Stock Rating which gave each company a stock rating based on how investors who held their stock did with 100 being highest and 0 lowest. 5

6 All data points Points considered outliers All data points 'Outliers' marked stock rating corr = 0.04 stock rating X X X X X X X handicap handicap Data in final analysis 'Outliers' removed stock rating corr = handicap King, B. (1998) Critique of Investing it: duffers need not apply. Chance News

7 Ch.2 Regression analysis... (as stated in book p. 16) examines the relationship between a quantitative dependent variable Y and one or more quantitative independent variables, X 1,..., X k. (He reserves the term regression for quantitative variables) Regression analysis traces the conditional distribution of Y - or some aspect of the distribution, such as its mean - as a function of the X s Examples: - General relationship between X and Y (where ɛ represents a random error). Y = f(x) + ɛ May be a linear or non-linear relationship. 7

8 Linear Models (linear in the parameters) - Simple linear relationship: Model the conditional mean response of a continuous variable using a linear relationship to a single continuous variable assuming normal errors Y = β 0 +β 1 X +ɛ with ɛ N(0, σ 2 ) Given X, Y has a normal distribution with a mean(center) of [β 0 + β 1 X] and a variance of σ 2. Also written as: Y X N(β 0 + β 1 X, σ 2 ) Sketch of plot showing normal conditional distributions: 8

9 - Quadratic relationship: Model the conditional mean response of a continuous variable as a quadratic relationship to a single continuous variable (this is still a linear model as it s linear in the parameters) Y = β 0 + β 1 X + β 2 X 2 + ɛ with ɛ N(0, σ 2 ) - Multiple linear relationships: Model the conditional mean response of a continuous variable as a linear relationship with each of two continuous variables (no interaction) Y = β 0 + β 1 X 1 + β 2 X 2 + ɛ with ɛ N(0, σ 2 ) Mean response surface shown on next page... 9

10 Z Mean response surface (errors not shown): y x1 This surface is a plane in space. 10

11 Non-Linear Models (not linear in the parameters) - Specific relationship: β β Y = β0 + β1x1 2 + β3x2 4 + with N (0, σ 2) - Specific relationship: Y = f (X1, X2) + with N (0, σ 2) Mean response surface (errors not shown): 11

12 Non-normality The conditional distribution of Y given X does not have to be normal. BUT the validity of many of our common hypothesis tests depends on normality. Y = β 0 +β 1 X +ɛ with ɛ a right-skewed distribution sketch - Might attain normality of errors through transformations if so, common statistical tests valid - Could use the original skewed data and maximum likelihood methods for estimation (with a specified non-normal distribution) 12

13 Nonparametric Regression LOWESS (locally weighted scatterplot smoother) Average Income, USD Prestige - The lowess smoother estimates the function... Y i = f(x i ) + ɛ i - The predicted Y i for a given x i is determined by considering only local points in a window around x i - Often a simple linear regression is fit to the local points, and the prediction falls on this line - Researcher chooses width of window 13

14 Other analyses The type of data will affect how the data is modeled and the choice of analysis Binary response (0/1) with covariate predictors: Logistic regression Relationship between categorical/ordinal variables: Contingency tables, chi-squared test (we won t cover this in this class) Relationship between a quantitative dependent variable (Y) and qualitative predictor: t-test or ANOVA 14

15 Predicting a continuous response from both quantitative and qualitative variables: Dummy-variable regression or ANCOVA Response is a count (Poisson distribution) and the Poisson distribution mean is dependent on the covariates: Poisson regression 15