7.1. Correlation analysis. Regression.

Transcription

1 7.1 Correlation analysis. Regression.

2 7.12

3 7.13

4 7.13

5 7.33

6 7.15 Values from the same group tend to be similar. There is no tendency for values from the same group to be similar.

7 7.14

8 Modelling of data: Linear regression 7.16

9 7.17 Overview

10 7.18 Overview of the model fitting process

11 7.20 Linear regression

12 7.21 Estimation using ordinary least squares (OLS)

13 7.22 Normal equations

14 7.23 OLS solutions and predictions

15 7.24 A statistical model

16 7.25 Model assumptions

17 7.26 Maximum likehood estimation

18 7.28 Correlation coefficient

19 7.29

20 7.30 Hypothesis testing

21 7.31

22 7.32 Hypothesis testing for intercept and slope

23 7.34

24 7.35 Check data before doing a regression!!

25 7.36

26 7.37 Model diagnostic: variance and linearity

27 7.38 variance stabilizing methods

28 7.39 Normal residuals

29 7.40 Non-normal errors

30 Correlated residuals (autocorrelation)

31 7.42 Other variable transformation s

32 7.43 Box-Cox family of transformations

33 7.44 Parameter tuning

34 7.45

35 7.46 Model building

36 7.47 Separate linear regressions. Examples: consider the following scenario

37 Br(x) 7.48 Basis functions

38 7.53 Goodness of fit criteria

39 7.54 Recap

40 Linear regression in R: case<-read.csv("case1.txt", header=t, sep="\t") plot(case[,5],case[,6]) t0=lm(case[,6]~case[,5]) k = summary(t0)[[4]][2,1] b = summary(t0)[[4]][1,1] x=seq(5,70,by=1) points(x,k*x+b,type="l",col="red") OR abline(t0) 7.55

41 Before accepting the result of a linear regression it is important to evaluate it suitability at explaining the data. layout(matrix(1:4,2,2)); plot(t0) 7.56

42 One more example: x=seq(1,10,by=1) y=x+rnorm(10,0,1) y[5]=50 t0=lm(y~x) plot(x,y);abline(t0) 7.57

43 layout(matrix(1:4,2,2)) plot(t0) 7.58

44 Leverage and Cook s distance: Cook s distance measures the effect of deleting a given observation. Points with large Cook s distance are considered to merit closer analysis. It is sum over a squared difference between the prediction from the full regression model and the prediction in which this point was deleted. P- the number of fitted parameters. MSE the mean square error of the regression model. 7.59

45 Robust regression. As the residual goes down, the weight goes up. 7.60

46 Nonlinear regression in R: x=seq(0,10,by=0.1) y=3*sin(x)+1+rnorm(length(x),mean=0,sd=0.3) plot(x,y) t1=nls(y~b*sin(x)+a,start=list(a=0.1,b=0.1)) 7.61

47 Nonlinear regression in R: x=seq(0,10,by=0.1) y=3*sin(x)+1+rnorm(length(x),mean=0,sd=0.3) plot(x,y) t1=nls(y~b*sin(x)+a,start=list(a=0.1,b=0.1)) 7.62 points(x,summary(t1)[[10]][2,1]*sin(x)+summary(t1)[[10]][1,1],type="l",col="red")

48 7.55 Defining Models in R 7.63 It is necessary to understand the syntax for defining models in R. Let s assume that the dependent variable being modeled is Y and that A, B and C are independent variables that might affect Y. The table below provides some useful examples. Note that the mathematical symbols used to define models do not have their normal meanings!

49 Risk, Odds, Odds ration and Logistic regression

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65 b0+b1x b0+b1x 7.65

66 mylogit<-gml(y~x,family=binomial(link="logit"));b0=mylogit$coefficients[1]; b1=mylogit$coefficients[2]; summary(t0) 7.66 Logistic regression in R

67 7.67

68 7.68

69 7.69

70 7.70

71 Extracting parameters of logistic regression mylogit<- glm(as.formula(data[,1]~data[,2]+data[,3]), family=binomial(link="logit"), na.action=na.pass) koef1=exp(mylogit$coefficients[2]) ##### Odds ratio koef2=exp(confint(mylogit))[2,1] ##### Confifence interval of odds ratio left koef3=exp(confint(mylogit))[2,2] ##### Confifence interval of odds ratio right koef4=summary(mylogit)[["coefficients"]][,"pr(> z )"][2] ##### P-value of odds ratio 71

72 Stepwise regression Any stepwise procedure in logistic regression is based on a statistical algorithm that checks for the "importance" of variables, and either includes or excludes them on the basis of a fixed decision rule. The "importance" of a variable is defined in terms of a measure of the statistical significance of the coefficient for the variable. The statistic used depends on the assumptions of the model. In stepwise linear regression an F-test is used since the errors are assumed to be normally distributed. In logistic regression the errors are assumed to follow a binomial distribution, and significance is assessed via likelihood ratio chisquare test. Thus at any step in the procedure the most important variable, in statistical terms, is the one that produces the greatest change in the log-likelihood relative to a model not containing the variable.

73 Stepwise regression in R Any stepwise regression procedure is an algorithm for forward selection followed by backward elimination. stepaic(object, direction = c("both", "backward", "forward") X=runif(250,-2.5,2.5) Y=runif(250, -2.5,2.5) Z=runif(250,-2.5,2.55) K=round(1/(1+exp(-X))+runif(50,-0.01,0.01)) data=cbind(k,x,y,z) library(mass) mylogit<- glm(as.formula(data[,1]~data[,2]+data[,3]+data[,4]+data[,2]*data[, 3]+data[,2]*data[,4]+data[,3]*data[,4]), family=binomial(link="logit"), na.action=na.pass) step <- stepaic(mylogit, direction="both")