MSc / PhD Course Advanced Biostatistics. dr. P. Nazarov

Transcription

1 MSc / PhD Course Advanced Biostatistics dr. P. Nazarov petr.nazarov@crp-sante.lu L4. Linear models edu.sablab.net/abs013 1

2 Outline ANOVA (L3.4) 1-factor ANOVA Multifactor ANOVA Experimental design Linear regression (L3.5) L4. Linear models edu.sablab.net/abs013

3 L4.1 ANOVA Why ANOVA? Means for more than populations We have measurements for 5 conditions. Are the means for these conditions equal? If we would use pairwise comparisons, what will be the probability of getting error? 5 5! Number of comparisons: C 10!3! Probability of an error: 1 (0.95) 10 = 0.4 Validation of the effects We assume that we have several factors affecting our data. Which factors are most significant? Which can be neglected? ANOVA example from Partek L4. Linear models edu.sablab.net/abs013 3

4 L4.1 ANOVA Example As part of a long-term study of individuals 65 years of age or older, sociologists and physicians at the Wentworth Medical Center in upstate New York investigated the relationship between geographic location and depression. A sample of 60 individuals, all in reasonably good health, was selected; 0 individuals were residents of Florida, 0 were residents of New York, and 0 were residents of North Carolina. Each of the individuals sampled was given a standardized test to measure depression. The data collected follow; higher test scores indicate higher levels of depression. Q: Is the depression level same in all 3 locations? depression.txt 1. Good health respondents Florida New York N. Carolina L4. Linear models H0: 1= = 3 Ha: not all 3 means are equal edu.sablab.net/abs013 4

5 FL FL FL FL FL FL FL NY NY NY NY NY NY NY NC NC NC NC NC NC Depression level L4.1 ANOVA Meaning H 0 : 1 = = 3 H a : not all 3 means are equal m m 3 6 m Measures L4. Linear models edu.sablab.net/abs013 5

6 Assumptions for Analysis of Variance L4.1 ANOVA Assumption for ANOVA 1. For each population, the response variable is normally distributed. The variance of the respond variable, denoted as is the same for all of the populations. 3. The observations must be independent. L4. Linear models edu.sablab.net/abs013 6

7 L4.1 ANOVA Parameter Florida New York N. Carolina m= overall mean= var= Let s estimate the variance of sampling distribution. If H 0 is true, then all m i belong to the same distribution m k i1 mi m ( ) k ( ) 31 ( ) 1.96 n this is called between-treatment estimate, works only at H m 0 At the same time, we can estimate the variance just by averaging out variances for each populations: k i1 k i Some Calculations 7 L4. Linear models edu.sablab.net/abs013 7 this is called within-treatment estimate Does between-treatment estimate and within-treatment estimate give variances of the same population?

8 H 0 : 1 = = = k H a : not all k means are equal L4.1 ANOVA Theory Means for treatments m j n j Variances treatments n j xij xij m j i i1 1 j i s j m n j n j 1 nt Total mean k n j 1 1 x ij due to treatment Sum squares SSTR k j1 n j m j m n n n T 1 n k Mean squares, beetween due to error Sum squares SSTR MSTR k 1 SSE k 1 j1 n j s j Test of variance equality F MSE SSTR p-value for the treatment effect p value SSE Mean squares, within MSE nr k 8 L4. Linear models edu.sablab.net/abs013 8

9 Total sum squares SST n k j xij m j1 i1 SST L4.1 ANOVA The Main Equation SSTR SSE Total variability of the data include variability due to treatment and variability due to error SS due to treatment SSTR SSE k j1 n j m SS due to error k 1 j1 j n j s j m d. f.( SST) d. f.( SSTR) d. f.( SSE) n T 1 k 1 n k T Partitioning The process of allocating the total sum of squares and degrees of freedom to the various components. 9 L4. Linear models edu.sablab.net/abs013 9

10 FL FL FL FL FL FL FL NY NY NY NY NY NY NY NC NC NC NC NC NC Depression level L4.1 ANOVA Example m m 3 6 m Measures SST SSTR SSE 10 L4. Linear models edu.sablab.net/abs013 10

11 L4.1 ANOVA Example ANOVA table A table used to summarize the analysis of variance computations and results. It contains columns showing the source of variation, the sum of squares, the degrees of freedom, the mean square, and the F value(s). In Excel use: Data Data Analysis ANOVA Single Factor Let s perform for dataset 1: good health depression.txt In R use: aov(...) Df Sum Sq Mean Sq F value Pr(>F) Location ** Residuals Signif. codes: 0 *** ** 0.01 * L4. Linear models edu.sablab.net/abs013 11

12 L4.1 ANOVA Some Definitions Factor Another word for the independent variable of interest. Treatments Different levels of a factor. Factorial experiment An experimental design that allows statistical conclusions about two or more factors. good health bad health depression.txt Factor 1: Health Factor : Location Florida New York North Carolina Depression = + Health + Location + HealthLocation + Interaction The effect produced when the levels of one factor interact with the levels of another factor in influencing the response variable. 1 L4. Linear models edu.sablab.net/abs013 1

13 Replications The number of times each experimental condition is repeated in an experiment. L4.1 ANOVA ANOVA Table 13 L4. Linear models edu.sablab.net/abs013 13

14 F-statistitcs L4.1 ANOVA Example depression.txt Factor 1: Health res = aov(depression ~ Location + Health + Location*Health, Data) summary(res) source(" drawanova(res) ANOVA Factor : Location ANOVA model: Factor Location Health Location:Health error Df Depression = m + Location + Health + Location * Health + e Sum Sq Mean Sq F value p-value e e-7.373e-01 * *** Location Health Location*Health error 14 L4. Linear models edu.sablab.net/abs013 14

15 L4.1 ANOVA Experimental Design Aware of Batch Effect! When designing your experiment always remember about various factors which can effect your data: batch effect, personal effect, lab effect... Day 1 Day? T = +30C T = +10C 15 L4. Linear models edu.sablab.net/abs013 15

16 L4.1 ANOVA Experimental Design Completely randomized design An experimental design in which the treatments are randomly assigned to the experimental units. We can nicely randomize: Day effect Batch effect 16 L4. Linear models edu.sablab.net/abs013 16

17 L4.1 ANOVA Experimental Design Blocking The process of using the same or similar experimental units for all treatments. The purpose of blocking is to remove a source of variation from the error term and hence provide a more powerful test for a difference in population or treatment means. Day 1 Day 17 L4. Linear models edu.sablab.net/abs013 17

18 L4.1 ANOVA A good suggestion Block what you can block, randomize what you cannot, and try to avoid unnecessary factors 18 L4. Linear models edu.sablab.net/abs013 18

19 L4.1 ANOVA ################################################################################ # L4.1 ANOVA ################################################################################ ## clear memory rm(list = ls()) ## load data Data = read.table(" str(data) DataGH = Data[Data$Health == "good",] ## build 1-factor ANOVA model res1 = aov(depression ~ Location, DataGH) summary(res1) ## build the ANOVA model res = aov(depression ~ Location + Health + Location*Health, Data) ## show the ANOVA table summary(res) ## Load function source(" x11() drawanova(res) L4. Linear models edu.sablab.net/abs013 19

20 Ending weight Blood ph L3.5. Linear Regression Dependent and Independent Variables mice.xls Ending weight vs. Starting weight Blood ph vs. Starting weight Starting weight Starting weight L4. Linear models edu.sablab.net/abs013 0

21 L3.5. Linear Regression Dependent and Independent Variables L4. Linear models edu.sablab.net/abs013 1

22 Number of cells L3.5. Linear Regression Example Temperature Cell Number y Temperature x Cells are grown under different temperature conditions from 0 to 40. A researched would like to find a dependency between T and cell number. cells.txt Dependent variable The variable that is being predicted or explained. It is denoted by y. Independent variable The variable that is doing the predicting or explaining. It is denoted by x. L4. Linear models edu.sablab.net/abs013

23 Number of cells L3.5. Linear Regression Regression Model and Regression Line Simple linear regression Regression analysis involving one independent variable and one dependent variable in which the relationship between the variables is approximated by a straight line. Building a regression means finding and tuning the model to explain the behaviour of the data Temperature L4. Linear models edu.sablab.net/abs013 3

24 Number of cells L3.5. Linear Regression Regression Model and Regression Line Regression model The equation describing how y is related to x and an error term; in simple linear regression, the regression model is y = x + Regression equation The equation that describes how the mean or expected value of the dependent variable is related to the independent variable; in simple linear regression, E(y) = x 450 Model for a simple linear regression: y( x) x Temperature L4. Linear models edu.sablab.net/abs013 4

25 y( x) x 1 0 L3.5. Linear Regression Regression Model and Regression Line L4. Linear models edu.sablab.net/abs013 5

26 Number of cells Number of cells L3.5. Linear Regression Estimation Estimated regression equation The estimate of the regression equation developed from sample data by using the least squares method. For simple linear regression, the estimated regression equation is y = b 0 + b 1 x cells.xls 1. Make a scatter plot for the data. E y( x) x yˆ ( x) 1 0 b x b y( x) b1 x b Temperature. Right click to Add Trendline. Show equation y = x Temperature L4. Linear models edu.sablab.net/abs013 6

27 L3.5. Linear Regression Overview L4. Linear models edu.sablab.net/abs013 7

28 L3.5. Linear Regression Slope and Intercept Least squares method A procedure used to develop the estimated regression equation. ˆi The objective is to minimize y y i Slope: b 1 x m y m i x m 1 x x i y Intercept: b0 my b1m x L4. Linear models edu.sablab.net/abs013 8

29 Number of cells L3.5. Linear Regression Coefficient of Determination Sum squares due to error SSE y i y ˆ i y = x Sum squares total SST y i my Sum squares due to regression SSR yˆ i my Temperature The Main Equation SST SSR SSE L4. Linear models edu.sablab.net/abs013 9

30 FL FL FL FL FL FL FL NY NY NY NY NY NY NY NC NC NC NC NC NC Depression level Number of cells L3.5. Linear Regression ANOVA and Regression m 8 m 3 6 m Measures Temperature SST SSTR SSE SST SSR SSE L4. Linear models edu.sablab.net/abs013 30

31 Number of cells L3.5. Linear Regression Coefficient of Determination SSE SST y i y ˆ y i my i SST SSR SSE y = x R = SSR yˆ i my Coefficient of determination A measure of the goodness of fit of the estimated regression equation. It can be interpreted as the proportion of the variability in the dependent variable y that is explained by the estimated regression equation R Temperature SSR SST Correlation coefficient A measure of the strength of the linear relationship between two variables (previously discussed in Lecture 1). r sign b R 1 L4. Linear models edu.sablab.net/abs013 31

32 y( x) x 1 0 L3.5. Linear Regression Assumptions Assumptions for Simple Linear Regression 1. The error term is a random variable with 0 mean, i.e. E[]=0. The variance of, denoted by, is the same for all values of x 3. The values of are independent 3. The term is a normally distributed variable L4. Linear models edu.sablab.net/abs013 3

33 i-th residual L3.5. Linear Regression Estimation of The difference between the observed value of the dependent variable and the value predicted using the estimated regression equation; for the i-th observation the i-th residual is: y yˆ Mean square error The unbiased estimate of the variance of the error term. It is denoted by MSE or s. Standard error of the estimate: the square root of the mean square error, denoted by s. It is the estimate of, the standard deviation of the error term. i i s MSE SSE n s MSE SSE n L4. Linear models edu.sablab.net/abs013 33

34 L3.5. Linear Regression Sampling Distribution for b1 If assumptions for are fulfilled, then the sampling distribution for b 1 is as follows: y( x) x 1 0 yˆ ( x) b x b 1 0 Expected value E[ b1] 1 Variance b 1 x m i x = Standard Error Distribution: normal Interval Estimation for 1 1 b 1 t n / x m i x b 1 1 t n SE / L4. Linear models edu.sablab.net/abs013 34

35 H 0 : 1 = 0 H a : 1 0 insignificant L3.5. Linear Regression Test for Significance 1. Build a F-test statistics. MSR F MSE 1. Build a t-test statistics. t b 1 b 1 b 1 s x m i x. Calculate a p-value. Calculate p-value for t L4. Linear models edu.sablab.net/abs013 35

36 L3.5. Linear Regression Example cells.xls 1. Calculate manually b 1 and b 0 Intercept b0= Slope b1= In Excel use the function: = INTERCEPT(y,x) = SLOPE(y,x). Let s do it automatically SUMMARY OUTPUT Regression Statistics Multiple R R Square Adjusted R Square Standard Error Observations 1 Data Data Analysis Regression ANOVA df SS MS F Significance F Regression E-11 Residual Total Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Lower 95.0% Upper 95.0% Intercept E X Variable E L4. Linear models edu.sablab.net/abs013 36

37 L3.5. Linear Regression Confidence and Prediction Confidence interval The interval estimate of the mean value of y for a given value of x. Prediction interval The interval estimate of an individual value of y for a given value of x. L4. Linear models edu.sablab.net/abs013 37

38 y L3.5. Linear Regression Example cells.txt x = data$temperature y = data$cell.number res = lm(y~x) res summary(res) # draw the data x11() x plot(x,y) # draw the regression and its confidence (95%) lines(x, predict(res,int = "confidence")[,1],col=4,lwd=) lines(x, predict(res,int = "confidence")[,],col=4) lines(x, predict(res,int = "confidence")[,3],col=4) # draw the prediction for the values (95%) lines(x, predict(res,int = "pred")[,],col=) lines(x, predict(res,int = "pred")[,3],col=) L4. Linear models edu.sablab.net/abs013 38

39 L3.5. Linear Regression Residuals L4. Linear models edu.sablab.net/abs013 39

40 L3.5. Linear Regression Example rana.txt A biology student wishes to determine the relationship between temperature and heart rate in leopard frog, Rana pipiens. He manipulates the temperature in increment ranging from to 18C and records the heart rate at each interval. His data are presented in table rana.txt 1) Build the model and provide the p-value for linear dependency ) Provide interval estimation for the slope of the dependency 3) Estmate 95% prediction interval for heart rate at L4. Linear models edu.sablab.net/abs013 40

41 L3.5. Linear Regression Multiple Regression L4. Linear models edu.sablab.net/abs013 41

42 L3.5. Linear Regression Multiple Regression L4. Linear models edu.sablab.net/abs013 4

43 edu.sablab.net/abs013 L4. Linear models 43 L3.5. Linear Regression Non-Linear Regression p p p p p x x x x x x x x x y P y E... exp 1... exp,...,, 1 ) (

44 Questions? Thank you for your attention to be continued L4. Linear models edu.sablab.net/abs013 44