Multiple Regression. Dan Frey Associate Professor of Mechanical Engineering and Engineering Systems
|
|
- Christina Morrison
- 5 years ago
- Views:
Transcription
1 ultiple Regression Dan Frey Associate Professor of echanical Engineering and Engineering Systems
2 Plan for Today ultiple Regression Estimation of the parameters Hypothesis testing Regression diagnostics Testing lack of fit Case study Net steps
3 The odel Equation ε X y + For a single variable For multiple variables ε α + + Y nk n n k k L O L L X k 0 ε n ε ε ε y n y y y α is renamed 0 These s allow 0 to enter the equation without being mult by s + k p
4 The odel Equation y X + ε Each row of X is paired with an observation y y y y n X Each column of X is paired with a coefficient L n n L O L k k nk ε E( ε i ) ε ε ε n 0 Var( ε ) σ i There are n observations of the response 0 k There are k coefficients Each observation is affected by an independent homoscedastic normal variates
5 Accounting for Indices ε X y + y n y y y nk n n k k L O L L X k 0 ε n ε ε ε + k p n np p n
6 Concept Question Which of these is a valid X matri? X 5.0m 0.3sec 7.m 0.sec 3.m 0.7 sec 5.4m 0.4sec A X 5.0m 7.V 3.sec 5.4A B 0.3m 0.V 0.7 sec 0.4A X 5.0m 7.m C 0.sec 0.3sec ) A only ) B only 3) C only 4) A and B 5) B and C 6) A and C 7) A, B, & C 8) None 9) I don t know
7 Adding h.o.t. to the odel Equation y n y y y n n n n n O X Each row of X is paired with an observation There are n observations of the response You can add interactions You can add curvature 0
8 Estimation of the Parameters Assume the model equation y X + ε We wish to minimize the sum squared error L ε T ε ( ) T y X ( y X) To minimize, we take the derivative and set it equal to zero L ˆ X T y + X T Xˆ The solution is ( T ) T X X X y And we define the fitted model ˆ yˆ Xˆ
9 Done in athcad: athcad Demo ontgomery Eample 0- ontgomery, D. C., 00, Design and Analysis of Eperiments, John Wiley & Sons.
10 Breakdown of Sum Squares Grand Total Sum of Squares GTSS n i y i SS due to mean ny SS T n i ( y i y) n SS R (yˆ y) i i SS E n i e i SSPE SSLOF
11 Breakdown of DOF n number of y values due to the mean n- total sum of squares k for the regression n-k- for error
12 Estimation of the Error Variance σ Remember the the model equation y X + ε ε ~ N(0, σ ) If assumptions of the model equation hold, then E ( ) SS ( n k ) σ E So an unbiased estimate of σ is σ ˆ SS E ( n k )
13 a.k.a. coefficient of multiple determination R and Adjusted R What fraction of the total sum of squares (SS T ) is accounted for jointly by all the parameters in the fitted model? R SS R SS T SS SS E T R can only rise as parameters are added R adj R adj SS SS E T ( n ( n p) ) n ( n p can rise or drop as parameters are added R )
14 Back to athcad Demo ontgomery Eample 0- ontgomery, D. C., 00, Design and Analysis of Eperiments, John Wiley & Sons.
15 Why Hypothesis Testing is Important in ultiple Regression Say there are 0 regressor variables Then there are coefficients in a linear model To make a fully nd order model requires 0 curvature terms in each variable 0 choose 45 interactions You d need 68 samples just to get the matri X T X to be invertible You need a way to discard insignificant terms
16 Test for Significance of Regression The hypotheses are H 0 : K k 0 H : j 0 for at least one j The test statistic is F 0 SS E SSR k ( n k ) F > F k n k Reject H 0 if 0 α,,
17 The hypotheses are Test for Significance Individual Coefficients H 0 : j 0 H : j 0 j C ( The test statistic is t0 X T X) ˆ σ C jj Standard error ˆ t > t k Reject H 0 if 0 α /, n σˆ C jj
18 Test for Significance of Groups of Coefficients Partition the coefficients into two groups to be removed to remain ε X y + Reduced model nk n n k k L O L L X Basically, you form X by removing the columns associated with the coefficients you are testing for significance X : : H H
19 Test for Significance Groups of Coefficients Reduced model y X + ε The regression sum of squares for the reduced model is SS R ( T y H y ny ) Define the sum squares of the removed set given the other coefficients are in the model The partial F test F 0 SS SS R E ( ) ( n SS R r p) ( ) SSR ( ) SSR ( ) Reject H 0 if F 0 > Fα, r, n p
20 Ecel Demo -- ontgomery E0- ontgomery, D. C., 00, Design and Analysis of Eperiments, John Wiley & Sons.
21 Factorial Eperiments Cuboidal Representation Ehaustive search of the space of discrete -level factors is the full factorial 3 eperimental design
22 Adding Center Points Center points allow an eperimenter to check for curvature and, if replicated, allow for an estimate of pure eperimental error
23 Plan for Today ud cards ultiple Regression Estimation of the parameters Hypothesis testing Regression diagnostics Testing lack of fit Case study Net steps
24 The Hat atri Since ˆ ( T ) T X X X y and yˆ Xˆ therefore yˆ ( T ) T X X y X X So we define Which maps from observations y to predictions ŷ H ( T ) T X X X X y ˆ Hy
25 Influence Diagnostics The relative disposition of points in space determines their effect on the coefficients The hat matri H gives us an ability to check for leverage points h ij is the amount of leverage eerted by point y j on Usually the diagonal elements ~p/n and it is good to check whether the diagonal elements within X of that ŷ i
26 athcad Demo on Distribution of Samples and Its Effect on Regression
27 Standardized Residuals The residuals are defined as e y yˆ So an unbiased estimate of σ is ˆ σ SS E ( n p) The standardized residuals are defined as d e σˆ If these elements were z-scores then with probability 99.7% 3 < d < 3 i
28 Studentized Residuals The residuals are defined as therefore e y yˆ e y Hy ( I H) y So the covariance matri of the residuals is The studentized residuals are defined as Cov( e) σ Cov( I r i e ˆ ( i σ h ii ) H) If these elements were z-scores then with probability 99.7%???? 3 < r < 3 i
29 Testing for Lack of Fit (Assuming a Central Composite Design) Compute the standard deviation of the center points and assume that represents the S PE S PE i center points n C ( y y) S LOF SS p LOF SS ( n ) PE S PE SS + SS PE LOF SS E F 0 S S LOF PE
30 Concept Test You perform a linear regression of 00 data points (n00). There are two independent variables and. The regression R is 0.7. Both and pass a t test for significance. You decide to add the interaction to the model. Select all the things that cannot happen: ) Absolute value of decreases ) changes sign 3) R decreases 4) fails the t test for significance
31 Plan for Today ud cards ultiple Regression Estimation of the parameters Hypothesis testing Regression diagnostics Testing lack of fit Case study Net steps
32 Scenario The FAA and EPA are interested in reducing CO emissions Some parameters of airline operations are thought to effect CO (e.g., Speed, Altitude, Temperature, Weight) Imagine flights have been made with special equipment that allowed CO emission to be measured (data provided) You will report to the FAA and EPA on your analysis of the data and make some recommendations
33 Phase One Open a atlab window Load the data (load FAAcase3.mat) Eplore the data
34 Phase Two Do the regression Eamine the betas and their intervals Plot the residuals y[co./ground_speed]; ones(:3538); X[ones' TAS alt temp weight]; [b,bint,r,rint,stats] regress(y,x,0.05); yhatx*b; plot(yhat,r,'+')
35 dimssize(x); i:dims()-; climb(); climb(dims())0; des()0; des(dims()); climb(i)(alt(i)>(alt(i-)+00)) (alt(i+)>(alt(i)+00)); des(i)(alt(i)<(alt(i-)-00)) (alt(i+)<(alt(i)-00)); for idims():-: if climb(i) des(i) y(i,:)[]; X(i,:)[]; yhat(i,:)[]; r(i,:)[]; end end hold off plot(yhat,r,'or') This code will remove the points at which the aircraft is climbing or descending
36 Try The Regression Again on Cruise Only Portions What were the effects on the residuals? What were the effects on the betas? hold off [b,bint,r,rint,stats] regress(y,x,0.05); yhatx*b; plot(yhat,r,'+')
37 See What Happens if We Remove Variables Remove weight & temp Do the regression (CO vs TAS & alt) Eamine the betas and their intervals [b,bint,r,rint,stats] regress(y,x(:,:3),0.05);
38 Phase Three Try different data (flight34.mat) Do the regression Eamine the betas and their intervals Plot the residuals y[fuel_burn]; ones(:34); X[ones' TAS alt temp]; [b,bint,r,rint,stats] regress(y,x,0.05); yhatx*b; plot(yhat,r,'+')
39 Adding Interactions X(:,5)X(:,).*X(:,3); This line will add a interaction What s the effect on the regression?
40 Case Wrap-Up What were the recommendations? What other analysis might be done? What were the key lessons?
41 Net Steps Wenesday 5 April Design of Eperiments Please read "Statistics as a Catalyst to Learning" Friday 7 April Recitation to support the term project onday 30 April Design of Eperiments Wednesday ay Design of Computer Eperiments Friday 4 ay?? Eam review?? onday 7 ay Frey at NSF Wednesday 9 ay Eam #
Multiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationCh 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationCorrelation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
More information14 Multiple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 14 Multiple Linear Regression 14.1 The multiple linear regression model In simple linear regression, the response variable y is expressed in
More informationLecture 9 SLR in Matrix Form
Lecture 9 SLR in Matrix Form STAT 51 Spring 011 Background Reading KNNL: Chapter 5 9-1 Topic Overview Matrix Equations for SLR Don t focus so much on the matrix arithmetic as on the form of the equations.
More informationLecture 10 Multiple Linear Regression
Lecture 10 Multiple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: 6.1-6.5 10-1 Topic Overview Multiple Linear Regression Model 10-2 Data for Multiple Regression Y i is the response variable
More informationMultiple Linear Regression
Multiple Linear Regression University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html 1 / 42 Passenger car mileage Consider the carmpg dataset taken from
More information2.830J / 6.780J / ESD.63J Control of Manufacturing Processes (SMA 6303) Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 2.830J / 6.780J / ESD.63J Control of Processes (SMA 6303) Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationDr. Maddah ENMG 617 EM Statistics 11/28/12. Multiple Regression (3) (Chapter 15, Hines)
Dr. Maddah ENMG 617 EM Statistics 11/28/12 Multiple Regression (3) (Chapter 15, Hines) Problems in multiple regression: Multicollinearity This arises when the independent variables x 1, x 2,, x k, are
More informationDESAIN EKSPERIMEN Analysis of Variances (ANOVA) Semester Genap 2017/2018 Jurusan Teknik Industri Universitas Brawijaya
DESAIN EKSPERIMEN Analysis of Variances (ANOVA) Semester Jurusan Teknik Industri Universitas Brawijaya Outline Introduction The Analysis of Variance Models for the Data Post-ANOVA Comparison of Means Sample
More informationBasic Business Statistics 6 th Edition
Basic Business Statistics 6 th Edition Chapter 12 Simple Linear Regression Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of a dependent variable based
More informationSTAT5044: Regression and Anova
STAT5044: Regression and Anova Inyoung Kim 1 / 49 Outline 1 How to check assumptions 2 / 49 Assumption Linearity: scatter plot, residual plot Randomness: Run test, Durbin-Watson test when the data can
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More information2.830 Homework #6. April 2, 2009
2.830 Homework #6 Dayán Páez April 2, 2009 1 ANOVA The data for four different lithography processes, along with mean and standard deviations are shown in Table 1. Assume a null hypothesis of equality.
More informationStatistics for Managers using Microsoft Excel 6 th Edition
Statistics for Managers using Microsoft Excel 6 th Edition Chapter 13 Simple Linear Regression 13-1 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value of
More informationSteps in Regression Analysis
MGMG 522 : Session #2 Learning to Use Regression Analysis & The Classical Model (Ch. 3 & 4) 2-1 Steps in Regression Analysis 1. Review the literature and develop the theoretical model 2. Specify the model:
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More informationUnit 10: Simple Linear Regression and Correlation
Unit 10: Simple Linear Regression and Correlation Statistics 571: Statistical Methods Ramón V. León 6/28/2004 Unit 10 - Stat 571 - Ramón V. León 1 Introductory Remarks Regression analysis is a method for
More informationEcon 3790: Statistics Business and Economics. Instructor: Yogesh Uppal
Econ 3790: Statistics Business and Economics Instructor: Yogesh Uppal Email: yuppal@ysu.edu Chapter 14 Covariance and Simple Correlation Coefficient Simple Linear Regression Covariance Covariance between
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationMultiple Regression Analysis
Multiple Regression Analysis y = β 0 + β 1 x 1 + β 2 x 2 +... β k x k + u 2. Inference 0 Assumptions of the Classical Linear Model (CLM)! So far, we know: 1. The mean and variance of the OLS estimators
More informationThe Simple Regression Model. Simple Regression Model 1
The Simple Regression Model Simple Regression Model 1 Simple regression model: Objectives Given the model: - where y is earnings and x years of education - Or y is sales and x is spending in advertising
More informationSection 4.6 Simple Linear Regression
Section 4.6 Simple Linear Regression Objectives ˆ Basic philosophy of SLR and the regression assumptions ˆ Point & interval estimation of the model parameters, and how to make predictions ˆ Point and interval
More informationMATH 644: Regression Analysis Methods
MATH 644: Regression Analysis Methods FINAL EXAM Fall, 2012 INSTRUCTIONS TO STUDENTS: 1. This test contains SIX questions. It comprises ELEVEN printed pages. 2. Answer ALL questions for a total of 100
More informationFormal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
More informationLECTURE 6. Introduction to Econometrics. Hypothesis testing & Goodness of fit
LECTURE 6 Introduction to Econometrics Hypothesis testing & Goodness of fit October 25, 2016 1 / 23 ON TODAY S LECTURE We will explain how multiple hypotheses are tested in a regression model We will define
More informationAnalysis of Variance (and discussion of Bayesian and frequentist statistics)
Analysis of Variance (and discussion of Bayesian and frequentist statistics) Dan Frey Assistant Professor of Mechanical Engineering and Engineering Systems Efron, 2004 Plan for Today Bayesians, Frequentists,
More informationFactorial designs. Experiments
Chapter 5: Factorial designs Petter Mostad mostad@chalmers.se Experiments Actively making changes and observing the result, to find causal relationships. Many types of experimental plans Measuring response
More information18.S096 Problem Set 3 Fall 2013 Regression Analysis Due Date: 10/8/2013
18.S096 Problem Set 3 Fall 013 Regression Analysis Due Date: 10/8/013 he Projection( Hat ) Matrix and Case Influence/Leverage Recall the setup for a linear regression model y = Xβ + ɛ where y and ɛ are
More informationChapter 4: Randomized Blocks and Latin Squares
Chapter 4: Randomized Blocks and Latin Squares 1 Design of Engineering Experiments The Blocking Principle Blocking and nuisance factors The randomized complete block design or the RCBD Extension of the
More informationChapter 5 Matrix Approach to Simple Linear Regression
STAT 525 SPRING 2018 Chapter 5 Matrix Approach to Simple Linear Regression Professor Min Zhang Matrix Collection of elements arranged in rows and columns Elements will be numbers or symbols For example:
More informationWeighted Least Squares
Weighted Least Squares The standard linear model assumes that Var(ε i ) = σ 2 for i = 1,..., n. As we have seen, however, there are instances where Var(Y X = x i ) = Var(ε i ) = σ2 w i. Here w 1,..., w
More informationMultivariate Regression (Chapter 10)
Multivariate Regression (Chapter 10) This week we ll cover multivariate regression and maybe a bit of canonical correlation. Today we ll mostly review univariate multivariate regression. With multivariate
More informationCorrelation 1. December 4, HMS, 2017, v1.1
Correlation 1 December 4, 2017 1 HMS, 2017, v1.1 Chapter References Diez: Chapter 7 Navidi, Chapter 7 I don t expect you to learn the proofs what will follow. Chapter References 2 Correlation The sample
More informationHomoskedasticity. Var (u X) = σ 2. (23)
Homoskedasticity How big is the difference between the OLS estimator and the true parameter? To answer this question, we make an additional assumption called homoskedasticity: Var (u X) = σ 2. (23) This
More informationRegression Analysis for Data Containing Outliers and High Leverage Points
Alabama Journal of Mathematics 39 (2015) ISSN 2373-0404 Regression Analysis for Data Containing Outliers and High Leverage Points Asim Kumer Dey Department of Mathematics Lamar University Md. Amir Hossain
More informationEstadística II Chapter 5. Regression analysis (second part)
Estadística II Chapter 5. Regression analysis (second part) Chapter 5. Regression analysis (second part) Contents Diagnostic: Residual analysis The ANOVA (ANalysis Of VAriance) decomposition Nonlinear
More informationReview of Statistics
Review of Statistics Topics Descriptive Statistics Mean, Variance Probability Union event, joint event Random Variables Discrete and Continuous Distributions, Moments Two Random Variables Covariance and
More informationChapter 13 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics
Chapter 13 Student Lecture Notes 13-1 Department of Quantitative Methods & Information Sstems Business Statistics Chapter 14 Introduction to Linear Regression and Correlation Analsis QMIS 0 Dr. Mohammad
More informationRegression used to predict or estimate the value of one variable corresponding to a given value of another variable.
CHAPTER 9 Simple Linear Regression and Correlation Regression used to predict or estimate the value of one variable corresponding to a given value of another variable. X = independent variable. Y = dependent
More informationChapter 10. Supplemental Text Material
Chater 1. Sulemental Tet Material S1-1. The Covariance Matri of the Regression Coefficients In Section 1-3 of the tetbook, we show that the least squares estimator of β in the linear regression model y=
More information2.1 Linear regression with matrices
21 Linear regression with matrices The values of the independent variables are united into the matrix X (design matrix), the values of the outcome and the coefficient are represented by the vectors Y and
More informationREVIEW 8/2/2017 陈芳华东师大英语系
REVIEW Hypothesis testing starts with a null hypothesis and a null distribution. We compare what we have to the null distribution, if the result is too extreme to belong to the null distribution (p
More information2.830J / 6.780J / ESD.63J Control of Manufacturing Processes (SMA 6303) Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 2.830J / 6.780J / ESD.63J Control of Processes (SMA 6303) Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationRegression Review. Statistics 149. Spring Copyright c 2006 by Mark E. Irwin
Regression Review Statistics 149 Spring 2006 Copyright c 2006 by Mark E. Irwin Matrix Approach to Regression Linear Model: Y i = β 0 + β 1 X i1 +... + β p X ip + ɛ i ; ɛ i iid N(0, σ 2 ), i = 1,..., n
More informationEstimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
More informationEstadística II Chapter 4: Simple linear regression
Estadística II Chapter 4: Simple linear regression Chapter 4. Simple linear regression Contents Objectives of the analysis. Model specification. Least Square Estimators (LSE): construction and properties
More informationDesign of Engineering Experiments Chapter 5 Introduction to Factorials
Design of Engineering Experiments Chapter 5 Introduction to Factorials Text reference, Chapter 5 page 170 General principles of factorial experiments The two-factor factorial with fixed effects The ANOVA
More informationDesign & Analysis of Experiments 7E 2009 Montgomery
1 What If There Are More Than Two Factor Levels? The t-test does not directly apply ppy There are lots of practical situations where there are either more than two levels of interest, or there are several
More informationOPTIMIZATION OF FIRST ORDER MODELS
Chapter 2 OPTIMIZATION OF FIRST ORDER MODELS One should not multiply explanations and causes unless it is strictly necessary William of Bakersville in Umberto Eco s In the Name of the Rose 1 In Response
More informationConfidence Intervals, Testing and ANOVA Summary
Confidence Intervals, Testing and ANOVA Summary 1 One Sample Tests 1.1 One Sample z test: Mean (σ known) Let X 1,, X n a r.s. from N(µ, σ) or n > 30. Let The test statistic is H 0 : µ = µ 0. z = x µ 0
More informationLectures on Simple Linear Regression Stat 431, Summer 2012
Lectures on Simple Linear Regression Stat 43, Summer 0 Hyunseung Kang July 6-8, 0 Last Updated: July 8, 0 :59PM Introduction Previously, we have been investigating various properties of the population
More informationThe t-test: A z-score for a sample mean tells us where in the distribution the particular mean lies
The t-test: So Far: Sampling distribution benefit is that even if the original population is not normal, a sampling distribution based on this population will be normal (for sample size > 30). Benefit
More information1 Least Squares Estimation - multiple regression.
Introduction to multiple regression. Fall 2010 1 Least Squares Estimation - multiple regression. Let y = {y 1,, y n } be a n 1 vector of dependent variable observations. Let β = {β 0, β 1 } be the 2 1
More informationEconometrics I KS. Module 2: Multivariate Linear Regression. Alexander Ahammer. This version: April 16, 2018
Econometrics I KS Module 2: Multivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: April 16, 2018 Alexander Ahammer (JKU) Module 2: Multivariate
More informationAnswers to Problem Set #4
Answers to Problem Set #4 Problems. Suppose that, from a sample of 63 observations, the least squares estimates and the corresponding estimated variance covariance matrix are given by: bβ bβ 2 bβ 3 = 2
More informationAny of 27 linear and nonlinear models may be fit. The output parallels that of the Simple Regression procedure.
STATGRAPHICS Rev. 9/13/213 Calibration Models Summary... 1 Data Input... 3 Analysis Summary... 5 Analysis Options... 7 Plot of Fitted Model... 9 Predicted Values... 1 Confidence Intervals... 11 Observed
More informationLecture 14 Simple Linear Regression
Lecture 4 Simple Linear Regression Ordinary Least Squares (OLS) Consider the following simple linear regression model where, for each unit i, Y i is the dependent variable (response). X i is the independent
More informationREGRESSION ANALYSIS AND INDICATOR VARIABLES
REGRESSION ANALYSIS AND INDICATOR VARIABLES Thesis Submitted in partial fulfillment of the requirements for the award of degree of Masters of Science in Mathematics and Computing Submitted by Sweety Arora
More informationRegression Analysis and Forecasting Prof. Shalabh Department of Mathematics and Statistics Indian Institute of Technology-Kanpur
Regression Analysis and Forecasting Prof. Shalabh Department of Mathematics and Statistics Indian Institute of Technology-Kanpur Lecture 10 Software Implementation in Simple Linear Regression Model using
More informationPrepared by: Prof. Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti
Prepared by: Prof Dr Bahaman Abu Samah Department of Professional Development and Continuing Education Faculty of Educational Studies Universiti Putra Malaysia Serdang M L Regression is an extension to
More informationDesign and Analysis of Experiments Prof. Jhareswar Maiti Department of Industrial and Systems Engineering Indian Institute of Technology, Kharagpur
Design and Analysis of Experiments Prof. Jhareswar Maiti Department of Industrial and Systems Engineering Indian Institute of Technology, Kharagpur Lecture 26 Randomized Complete Block Design (RCBD): Estimation
More informationMSc / PhD Course Advanced Biostatistics. dr. P. Nazarov
MSc / PhD Course Advanced Biostatistics dr. P. Nazarov petr.nazarov@crp-sante.lu 04-1-013 L4. Linear models edu.sablab.net/abs013 1 Outline ANOVA (L3.4) 1-factor ANOVA Multifactor ANOVA Experimental design
More informationInverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1
Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is
More information6. Multiple Linear Regression
6. Multiple Linear Regression SLR: 1 predictor X, MLR: more than 1 predictor Example data set: Y i = #points scored by UF football team in game i X i1 = #games won by opponent in their last 10 games X
More informationProperties of the least squares estimates
Properties of the least squares estimates 2019-01-18 Warmup Let a and b be scalar constants, and X be a scalar random variable. Fill in the blanks E ax + b) = Var ax + b) = Goal Recall that the least squares
More informationContents. 1 Review of Residuals. 2 Detecting Outliers. 3 Influential Observations. 4 Multicollinearity and its Effects
Contents 1 Review of Residuals 2 Detecting Outliers 3 Influential Observations 4 Multicollinearity and its Effects W. Zhou (Colorado State University) STAT 540 July 6th, 2015 1 / 32 Model Diagnostics:
More informationK. Model Diagnostics. residuals ˆɛ ij = Y ij ˆµ i N = Y ij Ȳ i semi-studentized residuals ω ij = ˆɛ ij. studentized deleted residuals ɛ ij =
K. Model Diagnostics We ve already seen how to check model assumptions prior to fitting a one-way ANOVA. Diagnostics carried out after model fitting by using residuals are more informative for assessing
More informationAnalysis of Variance and Design of Experiments-II
Analysis of Variance and Design of Experiments-II MODULE VIII LECTURE - 36 RESPONSE SURFACE DESIGNS Dr. Shalabh Department of Mathematics & Statistics Indian Institute of Technology Kanpur 2 Design for
More informationAdvanced Experimental Design
Advanced Experimental Design Topic 8 Chapter : Repeated Measures Analysis of Variance Overview Basic idea, different forms of repeated measures Partialling out between subjects effects Simple repeated
More informationBusiness Statistics. Lecture 9: Simple Regression
Business Statistics Lecture 9: Simple Regression 1 On to Model Building! Up to now, class was about descriptive and inferential statistics Numerical and graphical summaries of data Confidence intervals
More informationAnalysis of Variance. ภาว น ศ ร ประภาน ก ล คณะเศรษฐศาสตร มหาว ทยาล ยธรรมศาสตร
Analysis of Variance ภาว น ศ ร ประภาน ก ล คณะเศรษฐศาสตร มหาว ทยาล ยธรรมศาสตร pawin@econ.tu.ac.th Outline Introduction One Factor Analysis of Variance Two Factor Analysis of Variance ANCOVA MANOVA Introduction
More informationRandomized Blocks, Latin Squares, and Related Designs. Dr. Mohammad Abuhaiba 1
Randomized Blocks, Latin Squares, and Related Designs Dr. Mohammad Abuhaiba 1 HomeWork Assignment Due Sunday 2/5/2010 Solve the following problems at the end of chapter 4: 4-1 4-7 4-12 4-14 4-16 Dr. Mohammad
More informationMidterm 2 - Solutions
Ecn 102 - Analysis of Economic Data University of California - Davis February 23, 2010 Instructor: John Parman Midterm 2 - Solutions You have until 10:20am to complete this exam. Please remember to put
More information5.1 Model Specification and Data 5.2 Estimating the Parameters of the Multiple Regression Model 5.3 Sampling Properties of the Least Squares
5.1 Model Specification and Data 5. Estimating the Parameters of the Multiple Regression Model 5.3 Sampling Properties of the Least Squares Estimator 5.4 Interval Estimation 5.5 Hypothesis Testing for
More informationWe like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
Statistical Methods in Business Lecture 5. Linear Regression We like to capture and represent the relationship between a set of possible causes and their response, by using a statistical predictive model.
More informationDesign & Analysis of Experiments 7E 2009 Montgomery
Chapter 5 1 Introduction to Factorial Design Study the effects of 2 or more factors All possible combinations of factor levels are investigated For example, if there are a levels of factor A and b levels
More informationAGEC 621 Lecture 16 David Bessler
AGEC 621 Lecture 16 David Bessler This is a RATS output for the dummy variable problem given in GHJ page 422; the beer expenditure lecture (last time). I do not expect you to know RATS but this will give
More informationLinear Regression with 1 Regressor. Introduction to Econometrics Spring 2012 Ken Simons
Linear Regression with 1 Regressor Introduction to Econometrics Spring 2012 Ken Simons Linear Regression with 1 Regressor 1. The regression equation 2. Estimating the equation 3. Assumptions required for
More informationBusiness Statistics. Tommaso Proietti. Linear Regression. DEF - Università di Roma 'Tor Vergata'
Business Statistics Tommaso Proietti DEF - Università di Roma 'Tor Vergata' Linear Regression Specication Let Y be a univariate quantitative response variable. We model Y as follows: Y = f(x) + ε where
More informationLinear regression Class 25, Jeremy Orloff and Jonathan Bloom
1 Learning Goals Linear regression Class 25, 18.05 Jerem Orloff and Jonathan Bloom 1. Be able to use the method of least squares to fit a line to bivariate data. 2. Be able to give a formula for the total
More informationApplied Statistics and Econometrics
Applied Statistics and Econometrics Lecture 6 Saul Lach September 2017 Saul Lach () Applied Statistics and Econometrics September 2017 1 / 53 Outline of Lecture 6 1 Omitted variable bias (SW 6.1) 2 Multiple
More informationAddition of Center Points to a 2 k Designs Section 6-6 page 271
to a 2 k Designs Section 6-6 page 271 Based on the idea of replicating some of the runs in a factorial design 2 level designs assume linearity. If interaction terms are added to model some curvature results
More informationPredictive Analytics : QM901.1x Prof U Dinesh Kumar, IIMB. All Rights Reserved, Indian Institute of Management Bangalore
What is Multiple Linear Regression Several independent variables may influence the change in response variable we are trying to study. When several independent variables are included in the equation, the
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More information[y i α βx i ] 2 (2) Q = i=1
Least squares fits This section has no probability in it. There are no random variables. We are given n points (x i, y i ) and want to find the equation of the line that best fits them. We take the equation
More informationChapter 14 Student Lecture Notes Department of Quantitative Methods & Information Systems. Business Statistics. Chapter 14 Multiple Regression
Chapter 14 Student Lecture Notes 14-1 Department of Quantitative Methods & Information Systems Business Statistics Chapter 14 Multiple Regression QMIS 0 Dr. Mohammad Zainal Chapter Goals After completing
More informationcoefficients n 2 are the residuals obtained when we estimate the regression on y equals the (simple regression) estimated effect of the part of x 1
Review - Interpreting the Regression If we estimate: It can be shown that: where ˆ1 r i coefficients β ˆ+ βˆ x+ βˆ ˆ= 0 1 1 2x2 y ˆβ n n 2 1 = rˆ i1yi rˆ i1 i= 1 i= 1 xˆ are the residuals obtained when
More information(ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box.
FINAL EXAM ** Two different ways to submit your answer sheet (i) Use MS-Word and place it in a drop-box. (ii) Scan your answer sheets INTO ONE FILE only, and submit it in the drop-box. Deadline: December
More informationSTAT Final Practice Problems
STAT 48 -- Final Practice Problems.Out of 5 women who had uterine cancer, 0 claimed to have used estrogens. Out of 30 women without uterine cancer 5 claimed to have used estrogens. Exposure Outcome (Cancer)
More informationWeighted Least Squares
Weighted Least Squares The standard linear model assumes that Var(ε i ) = σ 2 for i = 1,..., n. As we have seen, however, there are instances where Var(Y X = x i ) = Var(ε i ) = σ2 w i. Here w 1,..., w
More informationECO220Y Simple Regression: Testing the Slope
ECO220Y Simple Regression: Testing the Slope Readings: Chapter 18 (Sections 18.3-18.5) Winter 2012 Lecture 19 (Winter 2012) Simple Regression Lecture 19 1 / 32 Simple Regression Model y i = β 0 + β 1 x
More information2 Prediction and Analysis of Variance
2 Prediction and Analysis of Variance Reading: Chapters and 2 of Kennedy A Guide to Econometrics Achen, Christopher H. Interpreting and Using Regression (London: Sage, 982). Chapter 4 of Andy Field, Discovering
More informationOutline. Remedial Measures) Extra Sums of Squares Standardized Version of the Multiple Regression Model
Outline 1 Multiple Linear Regression (Estimation, Inference, Diagnostics and Remedial Measures) 2 Special Topics for Multiple Regression Extra Sums of Squares Standardized Version of the Multiple Regression
More informationAnalysis of Covariance. The following example illustrates a case where the covariate is affected by the treatments.
Analysis of Covariance In some experiments, the experimental units (subjects) are nonhomogeneous or there is variation in the experimental conditions that are not due to the treatments. For example, a
More informationTHE MULTIVARIATE LINEAR REGRESSION MODEL
THE MULTIVARIATE LINEAR REGRESSION MODEL Why multiple regression analysis? Model with more than 1 independent variable: y 0 1x1 2x2 u It allows : -Controlling for other factors, and get a ceteris paribus
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 51 Outline 1 Matrix Expression 2 Linear and quadratic forms 3 Properties of quadratic form 4 Properties of estimates 5 Distributional properties 3 / 51 Matrix
More informationConfidence Interval for the mean response
Week 3: Prediction and Confidence Intervals at specified x. Testing lack of fit with replicates at some x's. Inference for the correlation. Introduction to regression with several explanatory variables.
More informationStatistical Modelling in Stata 5: Linear Models
Statistical Modelling in Stata 5: Linear Models Mark Lunt Arthritis Research UK Epidemiology Unit University of Manchester 07/11/2017 Structure This Week What is a linear model? How good is my model? Does
More information