Simple Linear Regression for the Advertising Data

Transcription

1 Revenue Pages of Advertising Simple Linear Regression for the Advertising Data

2 What do we do with the data? y i = Revenue of i th Issue x i = Pages of Advertisement in i th Issue i =1,...,n n = 37 Sample Size Primary Research Questions: 1. How does pages of advertising relate to revenue?

3 Exploratory Results r =0.82 Revenue Pages of Advertising Cov(X, Y ) = Form Linear? 2. Direction Positive or Negative 3. Strength 4. Outliers

4 SLR Model Fit ŷ = (Pages) ˆ =7.432 Revenue Pages of Advertising Is this model any good? Or, does it explain the structure in the data well?

5 SLR Model Fit Measure of Goodness of Fit to Observed data: Total Sums of Squares (SST) = nx (y i ȳ) 2 i=1 Sum of Square Errors (SSE) = nx (y i i=1 ˆ0+ ˆ1x i z} { ŷ i ) 2 Sum of Squares from Regression (SSR) = SST {z} Total Error = SSE {z} Error After Regression nx ( i=1 ˆ0+ ˆ1x i z} { ŷ i ȳ) 2 + SSR {z} Error Taken Away by Regression

6 SLR Model Fit Measure of Goodness of Fit to Observed data: R 2 2 [0, 1] = SSR SST =1 SSE SST = Interpretation: R 2 is the percent of variation in revenue that is explained away by pages of advertisement. Intuition: Percent better off in predicting revenue when you include information from pages of advertisement. Issue: R 2 only says how good your model is at explaining the data, it says nothing about how good your model is at prediction.

7 Cross Validation Measures of Goodness of Fit to Unobserved Data: Predictive Accuracy via cross validation i. Randomly remove p% of your data (called a test set) ii. Fit model to remaining (1-p)% of your data (called a training set) iii. Use fit to predict the held-out test data.

8 Cross Validation Test Data Set Compare predictions to observed revenue Repeat multiple times! Use Pages (the x) from the test data to get a prediction of revenue for each obs in the test data Training Data Set Fit a model (a line) to training dataset only

9 Cross Validation Cross Validation Metrics Predictive Bias = 1 n test X (ŷ i y i ) n test Bias: Systematic errors in estimation. For example, if bias<0 then our predictions are too low, if bias>0 then predictions are too high and if bias=0 predictions are just right. RPMSE = v u t 1 n test n test Root predicted mean square error: How far off my predictions are on average. i=1 X (ŷ i y i ) 2 i=1

10 SLR Assumptions ŷ = (Pages) ˆ =7.432 Revenue Are our assumptions OK? 1. Linear maybe 2. Independent maybe 3. Normal maybe 4. Equal Variance no Pages of Advertising

11 Assessing Model Assumptions Residuals vs. Fitted Values Scatterplot for Assessing Linear, Independent and Equal Variance Assumptions: ˆ i = y i ˆ0 ˆ1x i ) Should be independent (no pattern) of ŷ i with constant variance (if pattern then likely dependent). Residuals Fitted Values

12 Assessing Model Assumptions Residuals vs. Fitted Values Scatterplot for Assessing Linear, Independent and Equal Variance Assumptions: ˆ i = y i ˆ0 ˆ1x i ) Should be independent (no pattern) of ŷ i with constant variance (if pattern then likely dependent).

13 Assessing Model Assumptions Residuals vs. Fitted Values Scatterplot for Assessing Linear, Independent and Equal Variance Assumptions: What is close enough to homoskedastic (equal variance)? Breusch-Pagan Test (mathematical details are beyond the prereqs for this course): H 0 : Data are homoskedastic H A : Data are heteroskedastic p value : 0.01 Warning: Breusch-Pagan Test is highly sensitive so always check with fitted value vs. residual plot.

14 Assessing Model Assumptions Histogram of standardized residuals to assess normality (and, to some extent, outliers): ˆ i /SE(ˆ i ) N (0, 1) (in theory) Histogram of Standardized Residuals Frequency Do we have outliers? Standardized Residuals

15 Assessing Model Assumptions Normal Quantile-Quantile (QQ) Plot to Assess Normality (and, to some extent, outliers) ˆ i /SE(ˆ i ) N (0, 1) (in theory) i. Sort ˆ i /SE(ˆ i ) so that ˆ (1) /SE(ˆ (1) ) < < ˆ (n) /SE(ˆ (n) ) ii. Find z (1),...,z (n) so that Prob(z <z i ) i/n from normal iii. if the normal assumption holds ˆ (i) /SE(ˆ (i) ) z (i) Normal Q Q Plot Sample Quantiles Here s that outlier again! Theoretical Quantiles

16 Assessing Model Assumptions When assessing model assumptions, what is close enough to normal? Kolmogorov-Smirnov (KS) Test (mathematical details are beyond the pre-reqs for this class): H 0 : Data come from a normal H A : Data don t come from a normal p value :

17 Outliers in SLR Two questions: 1. Should we worry about outliers? ˆ1 = Corr(Y,X) s y s x Correlation is sensitive to outliers so our regression line will be sensitive as well 2. How do we identify outliers? i. Graphical Histogram/QQ plot of standardized residuals ii. Cooks Distance D i = P n j=1 (ŷ j ŷ j( i) ) 2 2ˆ2 ŷ j : Prediction of j th point using all data. ŷ j( i) : Prediction of j th point using all data EXCEPT the i th point.

18 Outliers in SLR Cooks Distance Use cutoff (rule of thumb) of 4/n as influential or outlier D 1 =0.414 D 2 =0.395 D 8 =0.409 Revenue Pages of Advertising

19 Dealing with Violations of the SLR Assumptions Based on the above diagnostics we know: 1. Linear assumption is a bit sketchy but not too bad 2. Homoskedasticity is certainly an issue 3. Normality OK with the exception of a few outliers. So, what do we do? 1. Change your assumptions (hard but preferred) 2. Transformations Idea t Y (y i )= t X (x i )+ i t Y (y i ) = Transformation of y t X (y i ) = Transformation of x Example p log(y i )= xi + i t Y (y i ) = log(y i ) t X (x i )= p x i

20 Transformations Name Transformation Fixes Issues log ln(y i ) Nonlinearity, Heteroskedasticity Only if positive Square root p yi Nonlinearity, Heteroskedasticity Only if positive y i /x i Heteroskedasticity Reverses relationship Power y i, 2 ( 1, 1) Heteroskedasticity Hard to interpret Box-Cox ( yi 1 if 6= 0 ln(y i ) if =0 Non-normality Impossible to interpret With Box-Cox transformations, λcan be treated as a parameter and estimated using least squares OR maximum likelihood.

21 Transformations Revenue ln(revenue) ln(pages) Pages ln(revenue) sqrt(revenue) ln(pages) sqrt(pages)

22 Transformations Issues with transformations: 1. With the exception of Box-Cox, you re guessing so your choice of transformation is subjective. 2. Changes the interpretation of the parameters. Need to back-transform in order to produce anything interpretable 3. Changes the standard errors of the parameters. 4. Not always easy to keep track of all your assumptions.

23 Advertisement Example ln(y i )= ln(x i )+ i ) [ ln(y) = ln(x) ) ŷ =exp{ ln(x)} Cooks distance values are better now. Histogram of Standardized Residuals Normal Q Q Plot Revenue Frequency Sample Quantiles Pages of Advertising Standardized Residuals Theoretical Quantiles

24 End of Advertisement Analysis (see webpage for R and SAS code)