Regression Analysis. Simple Regression Multivariate Regression Stepwise Regression Replication and Prediction Error EE290H F05
|
|
- Terence Bridges
- 6 years ago
- Views:
Transcription
1 Regression Analysis Simple Regression Multivariate Regression Stepwise Regression Replication and Prediction Error 1
2 Regression Analysis In general, we "fit" a model by minimizing a metric that represents the error. n Σ min (y i - y i ) 2 i=1 The sum of squares gives closed form solutions and minimum variance for linear models. 2
3 The Simplest Regression Model Line through the origin: y y=bx x y u =βx u +ε u u=1,2,...,n ε u ~N(0, σ R 2 ) n min S = min Σ (y u - βx u ) 2 : estimate of 2 σ R u=1 y=bx η u =βx u b: estimate of β y: estimate of η u, the true value of the model. 3
4 Using the Normal Equation to fit line through the origin model Our model only has one degree of freedom This is why our choices are confined on this line min Σ (y-y) 2 y2 y y=bx (1 d.f.) y1 4
5 Using the Normal Equation (cont) (fitting line through the origin model) Choose b so that the residual vector is perpendicular to the model vector... Σ (y-y) x =0 Σ (y - bx) x = 0 b= Σ xy Σx 2 (est. of β) s 2 = S R n-1 (est. of σ R 2 ) V(b) = s2 Σx 2 67% conf: b ± s 2 Σx 2 Significance test: t = b-β* s 2 Σx 2 ~ t n-1 5
6 Etch time vs. removed material: y = bx 500 R e m o v ed ( n m ) Etch Time (sec) x 10^3 Variable Name Coefficient Std. Err. Estimate t Statistic Prob > t Etch Time (sec)
7 Model Validation through ANOVA The idea is to decompose the sum of squares into orthogonal components. Assuming that there is no need for a model at all* (always a good null Hypothesis!): H 0 : β * =0 Σ y u 2 = Σ y u 2 + Σ (y u - y u ) 2 n p n-p total model residual * This is equivalent to saying that y~n(μ,σ 2 ), where μ and σ are constants, independent of x. 7
8 Model Validation through ANOVA (cont) Assuming a specific model: H 0 : β * = b Σ(y u - β * x u ) 2 = Σ (y u - β * x u ) 2 + Σ (y u - y u ) 2 n p n-p total model residual The ANOVA table will answer the question: Is Is there a relationship between x and y? y? 8
9 ANOVA table and Residual Plot Source Sum of Squares Deg. of Freedom Mean Squares F-Ratio Prob>F Model 1.83e e e e-6 Error 6.47e e+2 Total 1.89e+5 8 R es i d u a l s Etch Time (sec) x 10^
10 A More Complex Regression Equation - a straight line with two parameters actual estimated η = α + β (x - x ) y = a + b (x - x ) y i ~ N (η i, σ 2 ) Minimize R = Σ(y i -y i ) 2 to estimate α and β a=y b= Σ(x i -x)y i Σ(x i -x) 2 =Σ(x i -x)(y i-y) Σ(x i -x) 2 Are a and b good estimators of α and β? E[a] = α E[b] = Σ(x i -x)e[y i] Σ(x i -x) 2 = β 10
11 Variance Estimation: Note that all variability comes from y i! V[a] = V V[b] = V Σ y i k = 1 k 2 Σ V[ y i] = σ 2 k Σ (x i -x)y i Σ (x i -x) 2 = σ 2 Σ (x i -x) 2 min var. thanks to to least squares! 11
12 LTO thickness vs deposition time: y = a + bx L T O t h i c k A x 1 0^ Dep time x 10^3 Variable Name Coefficient Std. Err. Estimate t Statistic Prob > t Constant Dep time 6.04e e e e e e
13 Source Anova table and Residual Plot Sum of Squares Deg. of Freedom Mean Squares F-Ratio Prob>F Model 4.77e e e Error 5.09e e+3 Total 4.82e R es i d u a l s Dep time x 10^3 13
14 ANOVA Representation (x i,y i ) (y i -y i ) y (y i -η i ) b(x i -x) (y i -η i ) (a-α) y i = a+b(x i -x) η i = α+β(x i -x) β(x i -x) x x i x Note differences between "true" and "estimated" model. 14
15 ANOVA Representation (cont) (y i -η i ) = (a- α ) + (b- β )(x i -x) + ( y i - y i ) Σ(y i -η i ) 2 = k(a-α ) 2 + (b-β) 2 Σ(x i -x)+ (k) (1) (1) ~σ 2 χ 2 (k) ~σ 2 χ 2 (1) ~ σ 2 χ 2 (1) Σ(y i -y i ) 2 (k-2) ~σ 2 χ 2 (k-2) In In this way, the significance of of the model can be be analyzed in in detail. 15
16 Confidence Limits of an Estimate y0= y+b(x0 -x ) V(y0) = V(y)+(x0 -x ) 2 V(b) V(y0) = 1 n + (x 0 -x ) 2 Σ (x -x ) 2 s2 prediction interval: y0 +/- tα 2 V(y0) 16
17 Confidence Interval of Prediction (all points) p L T O T h i c k n e s s Dep time Leverage 17
18 Confidence Interval of Prediction (half the points) L T O 3000 T h i c k n e s s Dep time Leverage 18
19 Confidence Interval of Prediction (1/4 of points) L T O T h i c k n e s s Dep time Leverage 19
20 Prediction Error vs Experimental Error Experimental Error y Prediction error Estimated Model True model x Experimental Error Error Does Does not not depend on on location or or sample sample size. size. Prediction Error Error depends on on location gets gets smaller smaller as as sample sample size size increases. 20
21 Multivariate Regression η = β 1 x 1 +β 2 x 2 β 2 y y x 2 R The Residual is is to to y,, x 1,, x 2.. β 1 x 1 Coefficient Estimation: Σ(y-y)x 1 =0 Σ(y-y)x 2 =0 Σyx 1 -b 1 Σx 1 2 -b 2 Σx 1 x 2 = 0 Σyx 2 -b 2 Σx 2 2 -b 1 Σx 1 x 2 = 0 21
22 Variance Estimation: s 2 = S R n-p V(b 1 ) = 1 1-ρ 2 s 2 Σx 1 2 V(b 2 ) = 1 1-ρ 2 s 2 Σx 2 2 ρ = - Σx 1 x 2 Σx 12 Σx
23 Thickness vs time, temp: y = a + b1 x1 + b2 x2 Variable Name Coefficient Std. Err. Estimate t Statistic Prob > t Constant temp time min -7.04e e e e e e e e e
24 Anova table and Correlation of Estimates Source Sum of Squares Deg. of Freedom Mean Squares F-Ratio Prob>F Model 2.58e e e Error 7.71e e+1 Total 2.66e+4 20 Data File: tox nm temp time min regression Tox Temp Time
25 Multiple Regression in General x 1 x 2 x n b = y + e minimize Xb - y 2 = e 2 = ( y - Xb ) T ( y - Xb ) or, min -e T Xb + e T y which is equiv. to: ( y - Xb ) T Xb = 0 X T Xb = X T y b = ( X T X ) -1 X T y V(b) = ( X T X ) -1 σ 2 25
26 Joint Confidence Region for x 1 x 2 S = S R 1 + p n-p F α(p, n-p) Σ 2 β 1 -b 1 Σ x β 1 -b 1 β 2 -b 2 Σ 2 x 1 x 2 + β 2 -b 2 Σ x 2 2= S-S R 26
27 What if a linear model is not enough? 300 d e p r a t e inlet temp Variable Name Coefficient Std. Err. Estimate t Statistic Prob > t Constant inlet temp -1.85e e e e e e
28 ANOVA table and Residual Plot Source Sum of Squares Deg. of Freedom Mean Squares F-Ratio Prob>F Model 3.65e e e Error 4.06e e+1 Total 3.69e R es i d u a l s inlet temp 28
29 Multiple Regression with Replication S E = 1 2 Σ (y i1 -y i2 ) 2 S LF =S R -S E (a-α) 2 k Σ i η i k k i Σ v n i (y iv -ηi) 2 k Σ i η i + (b-β) 2 Σ η i (x i -x) 2 + η i (y i. -y i ) 2 + (y iv -y i. ) 2 i Σ k Σ i = k n i Σ Σ 1 1 k-2 η i -k i v k Σ i k Σ i Σ v n i (y iv -y) 2 k Σ Σ v n i = (y iv -y i. ) 2 i k + Σ η i i k (y i. -y i ) 2 + Σ η i i (y-y i ) 2 29
30 Pure Error vs. Lack of Fit Example Lack Of Fit Source Lack Of Fit Pure Error Total Error DF Sum of Squares Mean Square F Ratio Prob > F Parameter Estimates Term Intercept inlet temp Estimate Std Error t Ratio Prob> t Model Test Source inlet temp DF 1 Sum of Squares F Ratio Prob > F
31 Dep. rate vs temperature: y = a + bx + cx d e p r a t e 200 Variable Name inlet temp Std. Err. t Coefficient Estimate Statistic Prob > t Constant inlet temp inlet temp ^2 8.34e e e e e e e e e
32 Pure Error vs. Lack of Fit Example (cont) Lack Of Fit Source Lack Of Fit Pure Error Total Error DF Sum of Squares Mean Square F Ratio 8.37 Prob > F Parameter Estimates Term Intercept inlet temp^1 inlet temp^2 Estimate Std Error t Ratio Prob> t Model Test Source Poly(inlet temp,2) DF 2 Sum of Squares F Ratio Prob > F
33 Source ANOVA table and Residual Plot Sum of Squares Deg. of Freedom Mean Squares F-Ratio Prob>F Model 3.67e e e Error 1.55e e+0 Total 3.69e+4 22 R es i d u a l s inlet temp 33
34 Use regression line to predict LTO thickness... y = x R 2 = y = x R 2 = LTO Thick A 90%LimitLow 90%LimitHigh Dep Time Sec LTO Thick A
35 Response Surface Methodology Objectives: get a feel of I/O relationships find setting(s) that satisfy multiple constraints find settings that lead to optimum performance Observations: Function is nearly linear away from the peak Function is nearly quadratic at the peak 35
36 Building the planar model A Factorial experiment with center points is enough to build and confirm a planar model. b1, b2, b12 = /-0.75 b11+b22=1/4σp+1/3σc= /
37 Quadratic Model and Confirmation Run Close to the peak, a quadratic model can be built and confirmed by an expanded two-phase experiment. 37
38 Response Surface Methodology RSM consists of creating models that lead to visual images of a response. The models are usually linear or quadratic in nature. Either expanded factorial experiments, or regression analysis can be used. All empirical models have a random prediction error. In RSM, the average variance of the model is: V(y) = 1 n n Σ i=1 V(y i ) = pσ2 n where p is the number of model parameters and n is the number of experiments. 38
39 Response Surface Exploration 39
40 "Popular" RSM Use singe-stage Box-B or Box-W designs Use computer (simulated) experiments Rely on "goodness of fit" measures Automate model structure generation Problems? 40
Regression Analysis. Regression Analysis
Regresson Analyss Smple Regresson Multvarate Regresson Stepwse Regresson Replcaton and Predcton Error 1 Regresson Analyss In general, we "ft" a model by mnmzng a metrc that represents the error. n mn (y
More informationUNIVERSITY OF CALIFORNIA College of Engineering Department of Electrical Engineering and Computer Sciences. PROBLEM SET No. 5 Official Solutions
1 UNIVERSITY OF CALIFORNIA College of Engineering Department of Electrical Engineering and Computer Sciences C. SPANOS Special Issues in Semiconductor Manufacturing EECS 290H Fall 0 PROBLEM SET No. Official
More informationEE290H F05. Spanos. Lecture 5: Comparison of Treatments and ANOVA
1 Design of Experiments in Semiconductor Manufacturing Comparison of Treatments which recipe works the best? Simple Factorial Experiments to explore impact of few variables Fractional Factorial Experiments
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationSMA 6304 / MIT / MIT Manufacturing Systems. Lecture 10: Data and Regression Analysis. Lecturer: Prof. Duane S. Boning
SMA 6304 / MIT 2.853 / MIT 2.854 Manufacturing Systems Lecture 10: Data and Regression Analysis Lecturer: Prof. Duane S. Boning 1 Agenda 1. Comparison of Treatments (One Variable) Analysis of Variance
More informationResponse Surface Methodology
Response Surface Methodology Bruce A Craig Department of Statistics Purdue University STAT 514 Topic 27 1 Response Surface Methodology Interested in response y in relation to numeric factors x Relationship
More informationregression analysis is a type of inferential statistics which tells us whether relationships between two or more variables exist
regression analysis is a type of inferential statistics which tells us whether relationships between two or more variables exist sales $ (y - dependent variable) advertising $ (x - independent variable)
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationA discussion on multiple regression models
A discussion on multiple regression models In our previous discussion of simple linear regression, we focused on a model in which one independent or explanatory variable X was used to predict the value
More information14.0 RESPONSE SURFACE METHODOLOGY (RSM)
4. RESPONSE SURFACE METHODOLOGY (RSM) (Updated Spring ) So far, we ve focused on experiments that: Identify a few important variables from a large set of candidate variables, i.e., a screening experiment.
More informationEXST Regression Techniques Page 1. We can also test the hypothesis H :" œ 0 versus H :"
EXST704 - Regression Techniques Page 1 Using F tests instead of t-tests We can also test the hypothesis H :" œ 0 versus H :" Á 0 with an F test.! " " " F œ MSRegression MSError This test is mathematically
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More information4/22/2010. Test 3 Review ANOVA
Test 3 Review ANOVA 1 School recruiter wants to examine if there are difference between students at different class ranks in their reported intensity of school spirit. What is the factor? How many levels
More informationStatistics and Quantitative Analysis U4320. Segment 10 Prof. Sharyn O Halloran
Statistics and Quantitative Analysis U4320 Segment 10 Prof. Sharyn O Halloran Key Points 1. Review Univariate Regression Model 2. Introduce Multivariate Regression Model Assumptions Estimation Hypothesis
More informationVariance. Standard deviation VAR = = value. Unbiased SD = SD = 10/23/2011. Functional Connectivity Correlation and Regression.
10/3/011 Functional Connectivity Correlation and Regression Variance VAR = Standard deviation Standard deviation SD = Unbiased SD = 1 10/3/011 Standard error Confidence interval SE = CI = = t value for
More informationTable 1: Fish Biomass data set on 26 streams
Math 221: Multiple Regression S. K. Hyde Chapter 27 (Moore, 5th Ed.) The following data set contains observations on the fish biomass of 26 streams. The potential regressors from which we wish to explain
More information(ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box.
FINAL EXAM ** Two different ways to submit your answer sheet (i) Use MS-Word and place it in a drop-box. (ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box. Deadline: December
More informationHypothesis Testing hypothesis testing approach
Hypothesis Testing In this case, we d be trying to form an inference about that neighborhood: Do people there shop more often those people who are members of the larger population To ascertain this, we
More informationStatistical Modelling in Stata 5: Linear Models
Statistical Modelling in Stata 5: Linear Models Mark Lunt Arthritis Research UK Epidemiology Unit University of Manchester 07/11/2017 Structure This Week What is a linear model? How good is my model? Does
More informationLinear Regression. Simple linear regression model determines the relationship between one dependent variable (y) and one independent variable (x).
Linear Regression Simple linear regression model determines the relationship between one dependent variable (y) and one independent variable (x). A dependent variable is a random variable whose variation
More informationChapter 10. Simple Linear Regression and Correlation
Chapter 10. Simple Linear Regression and Correlation In the two sample problems discussed in Ch. 9, we were interested in comparing values of parameters for two distributions. Regression analysis is the
More informationSection Least Squares Regression
Section 2.3 - Least Squares Regression Statistics 104 Autumn 2004 Copyright c 2004 by Mark E. Irwin Regression Correlation gives us a strength of a linear relationship is, but it doesn t tell us what it
More informationCorrelation and Simple Linear Regression
Correlation and Simple Linear Regression Sasivimol Rattanasiri, Ph.D Section for Clinical Epidemiology and Biostatistics Ramathibodi Hospital, Mahidol University E-mail: sasivimol.rat@mahidol.ac.th 1 Outline
More information" M A #M B. Standard deviation of the population (Greek lowercase letter sigma) σ 2
Notation and Equations for Final Exam Symbol Definition X The variable we measure in a scientific study n The size of the sample N The size of the population M The mean of the sample µ The mean of the
More informationStatistical Techniques II EXST7015 Simple Linear Regression
Statistical Techniques II EXST7015 Simple Linear Regression 03a_SLR 1 Y - the dependent variable 35 30 25 The objective Given points plotted on two coordinates, Y and X, find the best line to fit the data.
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationEcon 300/QAC 201: Quantitative Methods in Economics/Applied Data Analysis. 12th Class 6/23/10
Econ 300/QAC 201: Quantitative Methods in Economics/Applied Data Analysis 12th Class 6/23/10 In God we trust, all others must use data. --Edward Deming hand out review sheet, answer, point to old test,
More informationsociology sociology Scatterplots Quantitative Research Methods: Introduction to correlation and regression Age vs Income
Scatterplots Quantitative Research Methods: Introduction to correlation and regression Scatterplots can be considered as interval/ratio analogue of cross-tabs: arbitrarily many values mapped out in -dimensions
More informationProblems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
More information9. Linear Regression and Correlation
9. Linear Regression and Correlation Data: y a quantitative response variable x a quantitative explanatory variable (Chap. 8: Recall that both variables were categorical) For example, y = annual income,
More informationSimple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
More informationSTAT420 Midterm Exam. University of Illinois Urbana-Champaign October 19 (Friday), :00 4:15p. SOLUTIONS (Yellow)
STAT40 Midterm Exam University of Illinois Urbana-Champaign October 19 (Friday), 018 3:00 4:15p SOLUTIONS (Yellow) Question 1 (15 points) (10 points) 3 (50 points) extra ( points) Total (77 points) Points
More informationAddition of Center Points to a 2 k Designs Section 6-6 page 271
to a 2 k Designs Section 6-6 page 271 Based on the idea of replicating some of the runs in a factorial design 2 level designs assume linearity. If interaction terms are added to model some curvature results
More information2.830J / 6.780J / ESD.63J Control of Manufacturing Processes (SMA 6303) Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 2.830J / 6.780J / ESD.63J Control of Processes (SMA 6303) Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/term
More informationFactorial designs. Experiments
Chapter 5: Factorial designs Petter Mostad mostad@chalmers.se Experiments Actively making changes and observing the result, to find causal relationships. Many types of experimental plans Measuring response
More information7. Response Surface Methodology (Ch.10. Regression Modeling Ch. 11. Response Surface Methodology)
7. Response Surface Methodology (Ch.10. Regression Modeling Ch. 11. Response Surface Methodology) Hae-Jin Choi School of Mechanical Engineering, Chung-Ang University 1 Introduction Response surface methodology,
More informationMultiple Regression Analysis. Basic Estimation Techniques. Multiple Regression Analysis. Multiple Regression Analysis
Multiple Regression Analysis Basic Estimation Techniques Herbert Stocker herbert.stocker@uibk.ac.at University of Innsbruck & IIS, University of Ramkhamhaeng Regression Analysis: Statistical procedure
More informationChapte The McGraw-Hill Companies, Inc. All rights reserved.
12er12 Chapte Bivariate i Regression (Part 1) Bivariate Regression Visual Displays Begin the analysis of bivariate data (i.e., two variables) with a scatter plot. A scatter plot - displays each observed
More informationDesign of Engineering Experiments Chapter 5 Introduction to Factorials
Design of Engineering Experiments Chapter 5 Introduction to Factorials Text reference, Chapter 5 page 170 General principles of factorial experiments The two-factor factorial with fixed effects The ANOVA
More informationSimple Linear Regression Analysis
LINEAR REGRESSION ANALYSIS MODULE II Lecture - 6 Simple Linear Regression Analysis Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur Prediction of values of study
More informationWhat If There Are More Than. Two Factor Levels?
What If There Are More Than Chapter 3 Two Factor Levels? Comparing more that two factor levels the analysis of variance ANOVA decomposition of total variability Statistical testing & analysis Checking
More informationy ˆ i = ˆ " T u i ( i th fitted value or i th fit)
1 2 INFERENCE FOR MULTIPLE LINEAR REGRESSION Recall Terminology: p predictors x 1, x 2,, x p Some might be indicator variables for categorical variables) k-1 non-constant terms u 1, u 2,, u k-1 Each u
More informationAlternatives to Difference Scores: Polynomial Regression and Response Surface Methodology. Jeffrey R. Edwards University of North Carolina
Alternatives to Difference Scores: Polynomial Regression and Response Surface Methodology Jeffrey R. Edwards University of North Carolina 1 Outline I. Types of Difference Scores II. Questions Difference
More informationSimple Linear Regression
Simple Linear Regression MATH 282A Introduction to Computational Statistics University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/ eariasca/math282a.html MATH 282A University
More informationDifference in two or more average scores in different groups
ANOVAs Analysis of Variance (ANOVA) Difference in two or more average scores in different groups Each participant tested once Same outcome tested in each group Simplest is one-way ANOVA (one variable as
More information: The model hypothesizes a relationship between the variables. The simplest probabilistic model: or.
Chapter Simple Linear Regression : comparing means across groups : presenting relationships among numeric variables. Probabilistic Model : The model hypothesizes an relationship between the variables.
More informationSix Sigma Black Belt Study Guides
Six Sigma Black Belt Study Guides 1 www.pmtutor.org Powered by POeT Solvers Limited. Analyze Correlation and Regression Analysis 2 www.pmtutor.org Powered by POeT Solvers Limited. Variables and relationships
More informationDESAIN EKSPERIMEN Analysis of Variances (ANOVA) Semester Genap 2017/2018 Jurusan Teknik Industri Universitas Brawijaya
DESAIN EKSPERIMEN Analysis of Variances (ANOVA) Semester Jurusan Teknik Industri Universitas Brawijaya Outline Introduction The Analysis of Variance Models for the Data Post-ANOVA Comparison of Means Sample
More informationMa 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA
Ma 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA March 6, 2017 KC Border Linear Regression II March 6, 2017 1 / 44 1 OLS estimator 2 Restricted regression 3 Errors in variables 4
More informationMultiple Regression. More Hypothesis Testing. More Hypothesis Testing The big question: What we really want to know: What we actually know: We know:
Multiple Regression Ψ320 Ainsworth More Hypothesis Testing What we really want to know: Is the relationship in the population we have selected between X & Y strong enough that we can use the relationship
More informationAcknowledgements. Outline. Marie Diener-West. ICTR Leadership / Team INTRODUCTION TO CLINICAL RESEARCH. Introduction to Linear Regression
INTRODUCTION TO CLINICAL RESEARCH Introduction to Linear Regression Karen Bandeen-Roche, Ph.D. July 17, 2012 Acknowledgements Marie Diener-West Rick Thompson ICTR Leadership / Team JHU Intro to Clinical
More informationRegression Analysis. Table Relationship between muscle contractile force (mj) and stimulus intensity (mv).
Regression Analysis Two variables may be related in such a way that the magnitude of one, the dependent variable, is assumed to be a function of the magnitude of the second, the independent variable; however,
More informationInverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1
Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is
More informationLecture Slides. Elementary Statistics Tenth Edition. by Mario F. Triola. and the Triola Statistics Series. Slide 1
Lecture Slides Elementary Statistics Tenth Edition and the Triola Statistics Series by Mario F. Triola Slide 1 Chapter 10 Correlation and Regression 10-1 Overview 10-2 Correlation 10-3 Regression 10-4
More informationGregory Carey, 1998 Regression & Path Analysis - 1 MULTIPLE REGRESSION AND PATH ANALYSIS
Gregory Carey, 1998 Regression & Path Analysis - 1 MULTIPLE REGRESSION AND PATH ANALYSIS Introduction Path analysis and multiple regression go hand in hand (almost). Also, it is easier to learn about multivariate
More informationCHAPTER 6 MACHINABILITY MODELS WITH THREE INDEPENDENT VARIABLES
CHAPTER 6 MACHINABILITY MODELS WITH THREE INDEPENDENT VARIABLES 6.1 Introduction It has been found from the literature review that not much research has taken place in the area of machining of carbon silicon
More informationRegression. Estimation of the linear function (straight line) describing the linear component of the joint relationship between two variables X and Y.
Regression Bivariate i linear regression: Estimation of the linear function (straight line) describing the linear component of the joint relationship between two variables and. Generally describe as a
More informationCorrelation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
More informationConfidence Intervals, Testing and ANOVA Summary
Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0
More informationStatistics - Lecture Three. Linear Models. Charlotte Wickham 1.
Statistics - Lecture Three Charlotte Wickham wickham@stat.berkeley.edu http://www.stat.berkeley.edu/~wickham/ Linear Models 1. The Theory 2. Practical Use 3. How to do it in R 4. An example 5. Extensions
More informationLecture 3: Inference in SLR
Lecture 3: Inference in SLR STAT 51 Spring 011 Background Reading KNNL:.1.6 3-1 Topic Overview This topic will cover: Review of hypothesis testing Inference about 1 Inference about 0 Confidence Intervals
More informationMEMORIAL UNIVERSITY OF NEWFOUNDLAND DEPARTMENT OF MATHEMATICS AND STATISTICS FINAL EXAM - STATISTICS FALL 1999
MEMORIAL UNIVERSITY OF NEWFOUNDLAND DEPARTMENT OF MATHEMATICS AND STATISTICS FINAL EXAM - STATISTICS 350 - FALL 1999 Instructor: A. Oyet Date: December 16, 1999 Name(Surname First): Student Number INSTRUCTIONS
More informationMeasuring the fit of the model - SSR
Measuring the fit of the model - SSR Once we ve determined our estimated regression line, we d like to know how well the model fits. How far/close are the observations to the fitted line? One way to do
More information13 Simple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 3 Simple Linear Regression 3. An industrial example A study was undertaken to determine the effect of stirring rate on the amount of impurity
More informationOPTIMIZATION OF FIRST ORDER MODELS
Chapter 2 OPTIMIZATION OF FIRST ORDER MODELS One should not multiply explanations and causes unless it is strictly necessary William of Bakersville in Umberto Eco s In the Name of the Rose 1 In Response
More informationLecture 6 Multiple Linear Regression, cont.
Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression
More informationOverview Scatter Plot Example
Overview Topic 22 - Linear Regression and Correlation STAT 5 Professor Bruce Craig Consider one population but two variables For each sampling unit observe X and Y Assume linear relationship between variables
More informationAnalysis of variance
Analysis of variance 1 Method If the null hypothesis is true, then the populations are the same: they are normal, and they have the same mean and the same variance. We will estimate the numerical value
More informationVisual interpretation with normal approximation
Visual interpretation with normal approximation H 0 is true: H 1 is true: p =0.06 25 33 Reject H 0 α =0.05 (Type I error rate) Fail to reject H 0 β =0.6468 (Type II error rate) 30 Accept H 1 Visual interpretation
More informationGeneral Linear Model (Chapter 4)
General Linear Model (Chapter 4) Outcome variable is considered continuous Simple linear regression Scatterplots OLS is BLUE under basic assumptions MSE estimates residual variance testing regression coefficients
More informationSTK4900/ Lecture 3. Program
STK4900/9900 - Lecture 3 Program 1. Multiple regression: Data structure and basic questions 2. The multiple linear regression model 3. Categorical predictors 4. Planned experiments and observational studies
More informationChapter 1 Linear Regression with One Predictor
STAT 525 FALL 2018 Chapter 1 Linear Regression with One Predictor Professor Min Zhang Goals of Regression Analysis Serve three purposes Describes an association between X and Y In some applications, the
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More informationLIST OF FORMULAS FOR STK1100 AND STK1110
LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function
More informationRecent Advances in the Field of Trade Theory and Policy Analysis Using Micro-Level Data
Recent Advances in the Field of Trade Theory and Policy Analysis Using Micro-Level Data July 2012 Bangkok, Thailand Cosimo Beverelli (World Trade Organization) 1 Content a) Classical regression model b)
More informationLecture 14. Analysis of Variance * Correlation and Regression. The McGraw-Hill Companies, Inc., 2000
Lecture 14 Analysis of Variance * Correlation and Regression Outline Analysis of Variance (ANOVA) 11-1 Introduction 11-2 Scatter Plots 11-3 Correlation 11-4 Regression Outline 11-5 Coefficient of Determination
More informationLecture 14. Outline. Outline. Analysis of Variance * Correlation and Regression Analysis of Variance (ANOVA)
Outline Lecture 14 Analysis of Variance * Correlation and Regression Analysis of Variance (ANOVA) 11-1 Introduction 11- Scatter Plots 11-3 Correlation 11-4 Regression Outline 11-5 Coefficient of Determination
More informationStats Review Chapter 14. Mary Stangler Center for Academic Success Revised 8/16
Stats Review Chapter 14 Revised 8/16 Note: This review is meant to highlight basic concepts from the course. It does not cover all concepts presented by your instructor. Refer back to your notes, unit
More informationMATH c UNIVERSITY OF LEEDS Examination for the Module MATH1725 (May-June 2009) INTRODUCTION TO STATISTICS. Time allowed: 2 hours
01 This question paper consists of 11 printed pages, each of which is identified by the reference. Only approved basic scientific calculators may be used. Statistical tables are provided at the end of
More informationLecture 1 Linear Regression with One Predictor Variable.p2
Lecture Linear Regression with One Predictor Variablep - Basics - Meaning of regression parameters p - β - the slope of the regression line -it indicates the change in mean of the probability distn of
More informationPredicted Y Scores. The symbol stands for a predicted Y score
REGRESSION 1 Linear Regression Linear regression is a statistical procedure that uses relationships to predict unknown Y scores based on the X scores from a correlated variable. 2 Predicted Y Scores Y
More informationCHAPTER EIGHT Linear Regression
7 CHAPTER EIGHT Linear Regression 8. Scatter Diagram Example 8. A chemical engineer is investigating the effect of process operating temperature ( x ) on product yield ( y ). The study results in the following
More informationSTATISTICS 110/201 PRACTICE FINAL EXAM
STATISTICS 110/201 PRACTICE FINAL EXAM Questions 1 to 5: There is a downloadable Stata package that produces sequential sums of squares for regression. In other words, the SS is built up as each variable
More informationAnswers to Problem Set #4
Answers to Problem Set #4 Problems. Suppose that, from a sample of 63 observations, the least squares estimates and the corresponding estimated variance covariance matrix are given by: bβ bβ 2 bβ 3 = 2
More informationInference. ME104: Linear Regression Analysis Kenneth Benoit. August 15, August 15, 2012 Lecture 3 Multiple linear regression 1 1 / 58
Inference ME104: Linear Regression Analysis Kenneth Benoit August 15, 2012 August 15, 2012 Lecture 3 Multiple linear regression 1 1 / 58 Stata output resvisited. reg votes1st spend_total incumb minister
More informationAdvanced Engineering Statistics - Section 5 - Jay Liu Dept. Chemical Engineering PKNU
Advanced Engineering Statistics - Section 5 - Jay Liu Dept. Chemical Engineering PKNU Least squares regression What we will cover Box, G.E.P., Use and abuse of regression, Technometrics, 8 (4), 625-629,
More informationEconometrics. 8) Instrumental variables
30C00200 Econometrics 8) Instrumental variables Timo Kuosmanen Professor, Ph.D. http://nomepre.net/index.php/timokuosmanen Today s topics Thery of IV regression Overidentification Two-stage least squates
More informationCorrelation. Bivariate normal densities with ρ 0. Two-dimensional / bivariate normal density with correlation 0
Correlation Bivariate normal densities with ρ 0 Example: Obesity index and blood pressure of n people randomly chosen from a population Two-dimensional / bivariate normal density with correlation 0 Correlation?
More informationSimple Linear Regression
Simple Linear Regression September 24, 2008 Reading HH 8, GIll 4 Simple Linear Regression p.1/20 Problem Data: Observe pairs (Y i,x i ),i = 1,...n Response or dependent variable Y Predictor or independent
More informationStatistics Handbook. All statistical tables were computed by the author.
Statistics Handbook Contents Page Wilcoxon rank-sum test (Mann-Whitney equivalent) Wilcoxon matched-pairs test 3 Normal Distribution 4 Z-test Related samples t-test 5 Unrelated samples t-test 6 Variance
More informationChapter 14 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics. Chapter 14 Multiple Regression
Chapter 14 Student Lecture Notes 14-1 Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Multiple Regression QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing
More informationEstimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
More informationStat/F&W Ecol/Hort 572 Review Points Ané, Spring 2010
1 Linear models Y = Xβ + ɛ with ɛ N (0, σ 2 e) or Y N (Xβ, σ 2 e) where the model matrix X contains the information on predictors and β includes all coefficients (intercept, slope(s) etc.). 1. Number of
More informationProbability and Statistics Notes
Probability and Statistics Notes Chapter Seven Jesse Crawford Department of Mathematics Tarleton State University Spring 2011 (Tarleton State University) Chapter Seven Notes Spring 2011 1 / 42 Outline
More informationIntroduction to Regression
Introduction to Regression Using Mult Lin Regression Derived variables Many alternative models Which model to choose? Model Criticism Modelling Objective Model Details Data and Residuals Assumptions 1
More informationThe linear model. Our models so far are linear. Change in Y due to change in X? See plots for: o age vs. ahe o carats vs.
8 Nonlinear effects Lots of effects in economics are nonlinear Examples Deal with these in two (sort of three) ways: o Polynomials o Logarithms o Interaction terms (sort of) 1 The linear model Our models
More informationSTT 843 Key to Homework 1 Spring 2018
STT 843 Key to Homework Spring 208 Due date: Feb 4, 208 42 (a Because σ = 2, σ 22 = and ρ 2 = 05, we have σ 2 = ρ 2 σ σ22 = 2/2 Then, the mean and covariance of the bivariate normal is µ = ( 0 2 and Σ
More informationLinear regression. We have that the estimated mean in linear regression is. ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. The standard error of ˆµ Y X=x is.
Linear regression We have that the estimated mean in linear regression is The standard error of ˆµ Y X=x is where x = 1 n s.e.(ˆµ Y X=x ) = σ ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. 1 n + (x x)2 i (x i x) 2 i x i. The
More informationInference in Regression Analysis
Inference in Regression Analysis Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 4, Slide 1 Today: Normal Error Regression Model Y i = β 0 + β 1 X i + ǫ i Y i value
More informationCorrelation and the Analysis of Variance Approach to Simple Linear Regression
Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation
More information