STAT 540: Data Analysis and Regression
|
|
- Jodie Bruce
- 5 years ago
- Views:
Transcription
1 STAT 540: Data Analysis and Regression Wen Zhou Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State University) STAT 540 July 6th, 205 / 62
2 Contents Multiple Linear Regression Model 2 Inference on Multiple Regression 3 Inference about Regression Parameters 4 Estimation and Prediction 5 Geometric View of Regression and Linear Models 6 Estimating estimable function of coefficient W. Zhou (Colorado State University) STAT 540 July 6th, / 62
3 Multiple Linear Regression I Multiple linear regression model Multiple linear regression model in matrix terms 2 Estimation of regression coefficients Inference ANOVA results 2 Inference about regression parameters 3 Estimation of mean response and prediction of new observation Inference about regression parameters Estimation and prediction Geometric interpretation of linear model and regression Estimating estimable function of regression or linear coefficient β W. Zhou (Colorado State University) STAT 540 July 6th, / 62
4 Multiple Linear Regression Model 2 Inference on Multiple Regression 3 Inference about Regression Parameters 4 Estimation and Prediction 5 Geometric View of Regression and Linear Models 6 Estimating estimable function of coefficient W. Zhou (Colorado State University) STAT 540 July 6th, / 62
5 Multiple Linear Regression Example: # of predictor variables = 2. Y i = β 0 + β X i + β 2 X i2 + ɛ i, ɛ i iid N(0, σ 2 ), for i =,..., n. Response surface: E(Y i ) = Example: Y = Pine bark beetle density X = Temperature X2 = Tree species W. Zhou (Colorado State University) STAT 540 July 6th, / 62
6 Interpretation of Coefficients β 0 : Intercept. When the model scope includes X = X 2 = 0. β0 is interpreted as the mean response E(Y ) at X = X 2 = 0. β j : Slope in the direction of X j (effect). E(Y )/ Xj = EY X=(X,X 2 )(Y ) E Y X=(X,X 2 )(Y ) = Interpreted as the change in the mean response E(Y ) per unit increase in X j, when X j are held constant. What if X j is qualitative? W. Zhou (Colorado State University) STAT 540 July 6th, / 62
7 Multiple Linear Regression A general linear regression model is, for i =,, n Response surface: Y i = β 0 + p X ij β j + ɛ i, ɛ i iid N(0, σ 2 ). j= E(Y i ) = β 0 + p X ij β j j= Regression coefficients: β 0, β,..., β p, β p. Predictor variables: X,..., X p are known constants/values. The model is linear in the parameters, not necessarily in the shape of the response surface. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
8 Response Surface Examples Polynomial regression E(Y ) = β 0 + β X + β 2 X 2 + β 3 X 3. Transformed variables E(log(Y )) = β 0 + β X + β 2 X2. Interaction effects E(Y ) = β 0 + β X + β 2 X2 + β 3 X X 2. The change in the mean response corresponding to a unit change in X depends on X 2 and vice versa. Testing whether β3 = 0 or not is very challenging in high-dimensional (n = o(p)). W. Zhou (Colorado State University) STAT 540 July 6th, / 62
9 Qualitative Predictor Variables Example: Let Y = length of hospital stay, X = age, and X 2 = gender: 0 for male and for female. An additive model is Thus the response surface for males is and for females is β 2 is This kind of model sometimes is called ANVOCA model. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
10 Qualitative Predictor Variables Interaction: the relationship between X and Y for a fixed value of X 2 = x 2 depends on x 2. An interaction model is Thus the response surface for males is and for females is W. Zhou (Colorado State University) STAT 540 July 6th, / 62
11 Notation n observations, response variable, p β s with predictors (i.e. β 0 is the pth). Response variable: Y n = (Y, Y 2,..., Y n ) T. The predictors are arranged in the design matrix X X 2 X,p X 2 X 22 X 2,p X n p =... X n X n2 X n,p Random error: ɛ n = (ɛ, ɛ 2,..., ɛ n ) T. Regression coefficients: β p = (β 0, β,..., β p ) T. W. Zhou (Colorado State University) STAT 540 July 6th, 205 / 62
12 Multiple Linear Regression Model in Matrix Terms The multiple linear regression model can be written as where as we have seen before Thus, E(ɛ) = 0 n, Var{ɛ} = σ 2 I n n. and Y W. Zhou (Colorado State University) STAT 540 July 6th, / 62
13 Least Squares Estimation Consider the criterion: n p Q = (Y i β 0 β j X ij ) 2 = i= j= The least squares estimate of β is assuming that X T X is invertible. This is also the MLE. What condition on X do we need to have X T X invertible? What if X T X is not invertible? W. Zhou (Colorado State University) STAT 540 July 6th, / 62
14 Fitted Values and Residuals Fitted values: Ŷ = where the hat matrix is Residuals: e = W. Zhou (Colorado State University) STAT 540 July 6th, / 62
15 Multiple Linear Regression Model 2 Inference on Multiple Regression 3 Inference about Regression Parameters 4 Estimation and Prediction 5 Geometric View of Regression and Linear Models 6 Estimating estimable function of coefficient W. Zhou (Colorado State University) STAT 540 July 6th, / 62
16 Sums of Squares We have sums of squares in matrix forms that SSR = SSE = SST O = n (Ŷi Ȳ )2 = i= n (Y i Ŷi) 2 = i= n (Y i Ȳ )2 = i= Partitioning of total sum of squares and particularly the df are SST O }{{} df=n = SSR }{{} df=p + SSE }{{}. df=n p W. Zhou (Colorado State University) STAT 540 July 6th, / 62
17 Mean Squares Define mean squares MSR = SSR p, SSE MSE = n p. It can be shown that E(MSE) = Also can be shown that = σ 2 if β j = 0 for j E(M SR). > σ 2 otherwise W. Zhou (Colorado State University) STAT 540 July 6th, / 62
18 ANOVA Table The ANOVA table is Source SS df MS F Regression SSR MSR F = MSR/MSE Error SSE M SE Total SST O If then in which case MSR/MSE. E(MSE) = E(MSR) = σ 2 W. Zhou (Colorado State University) STAT 540 July 6th, / 62
19 Overall F Test for Regression Relation Test H 0 : v.s. H a :. It can be shown that under H0, F = MSR MSE Thus we can perform an F -test at level α by the decision rule: Conditional on H 0 being rejected, we may want to find (or a.s.) Identification/Selection. S = {j β j 0} W. Zhou (Colorado State University) STAT 540 July 6th, / 62
20 Coefficient of Multiple Determination, R 2 The coefficient of multiple determination is denoted by R 2 and is defined as R 2 = SSR SST O = SSE SST O Interpretation: The proportion of variation in the Y i s explained by the regression relation. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
21 More on R 2 As more predictors are added to the model (p ), R 2 must increase. Why? Recall SST O = SSR + SSE SST O is fixed for Y while SSE is a minimum of the unconstraint convex optimization problem β = arg min SSE(β 0,..., β p ). Suppose we consider an extra predictor and thus consider SSE(β0,..., β p). The β that minimizes this SSE cannot be inferior to the previous minimizer because β p = 0 is a special case within the new minimization problem that incorporates the previous one. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
22 Adjusted R 2 R 2 depends on p (even for p n), how to remove that dependence? The adjusted coefficient of multiple determination is denoted by Ra 2 and is defined as Ra 2 = SSE/n p ( ) n SSE SST O/n = n p SST O. The adjusted coefficient of multiple determination R 2 a may decrease when more predictors are in the model. Many other statistics such as AIC, BIC, Mallow s C p, etc. will be discussed and they are superior over R 2 a. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
23 Multiple Linear Regression Model 2 Inference on Multiple Regression 3 Inference about Regression Parameters 4 Estimation and Prediction 5 Geometric View of Regression and Linear Models 6 Estimating estimable function of coefficient W. Zhou (Colorado State University) STAT 540 July 6th, / 62
24 Estimation of Regression Coefficients Mean satisfies E( ˆβ) = β. That is, the LS estimate ˆβ is an unbiased estimate of β. Variance-covariance matrix: Σ β := Var{ ˆβ} = σ 2 ( X T X ). (Σβ ) kk = (Σβ ) kl = W. Zhou (Colorado State University) STAT 540 July 6th, / 62
25 Inference about Regression Coefficients The estimated variance-covariance matrix. ˆΣ β := s 2 { ˆβ} = MSE (X T X ) s 2 { ˆβ 0 } s{ ˆβ 0, ˆβ } s{ ˆβ 0, ˆβ p } s{ := ˆβ, ˆβ 0 } s 2 { ˆβ } s{ ˆβ, ˆβ p }... s{ ˆβ p, ˆβ 0 } s{ ˆβ p, ˆβ } s 2 { ˆβ p } Under the multiple linear regression model, we have for k = 0,,..., p. ˆβ k β k s{ ˆβ k } W. Zhou (Colorado State University) STAT 540 July 6th, / 62
26 Inference about Regression Coefficients Thus the ( α) confidence interval for β k is ˆβ k ± t α/2;n p s{ ˆβ k }. Test H 0 : β k = β k0 versus H a : β k β k0. Under H 0, we have t = ˆβ k β k0 s{ ˆβ k } t n p Thus we can perform a t-test at level α by the decision rule: W. Zhou (Colorado State University) STAT 540 July 6th, / 62
27 Multiple Linear Regression Model 2 Inference on Multiple Regression 3 Inference about Regression Parameters 4 Estimation and Prediction 5 Geometric View of Regression and Linear Models 6 Estimating estimable function of coefficient W. Zhou (Colorado State University) STAT 540 July 6th, / 62
28 Estimation of Mean Response Hidden Extrapolation Define X h = (, X h,..., X h,p ) T. Caution about hidden extrapolations. The region (with respect to X0) defined by d(x 0) = X T 0 (X T X) X 0 h max where h max = max i h ii, is an ellipsoid enclosing all data points inside the regressor variable hull (RVH). Predictions for any X0 outside the RVH (i.e., d(x 0) > h max) is hidden extrapolation, at least to some degree. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
29 Estimation of Mean Response The estimated mean response corresponding to X h = Mean E(Ŷh) = Variance Var{Ŷh} = Estimated variance is s 2 {Ŷh} = W. Zhou (Colorado State University) STAT 540 July 6th, / 62
30 Confidence Intervals for Mean Response The ( α) confidence interval for E(Y h ) is Ŷ h ± t α/2;n p s{ŷh} The Working-Hotelling ( α) confidence band for the regression surface is Ŷ h ± W s{ŷh} where W 2 = pf ( α; p, n p). The Bonferroni ( α) joint confidence intervals for g mean responses are Ŷ h ± Bs{Ŷh} where B = t α/(2g);n p. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
31 Prediction of New Observation The predicted new observation corresponding to X h is Ŷh = X T h ˆβ, and Mean E( Ŷ h ) = X T h β = E(Y h(new) ). Prediction error variance σ 2 pred = Var(Ŷh Y h(new) ) = Estimated prediction error variance is s 2 {pred} = W. Zhou (Colorado State University) STAT 540 July 6th, / 62
32 Prediction Intervals for New Observation The ( α) prediction interval for Y h(new) is Ŷ h ± t α/2;n p s{pred} The Scheffé ( α) joint confidence intervals for g new observations are Ŷ h ± Ss{pred} where S 2 = gf ( α; g, n p). The Bonferroni ( α) joint confidence intervals for g new observations are Ŷ h ± Bs{pred} where B = t α/(2g);n p. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
33 Multiple Linear Regression Model 2 Inference on Multiple Regression 3 Inference about Regression Parameters 4 Estimation and Prediction 5 Geometric View of Regression and Linear Models 6 Estimating estimable function of coefficient W. Zhou (Colorado State University) STAT 540 July 6th, / 62
34 Geometric Viewpoint: The Column Space of the Design Matrix Xβ is a linear combination of the columns of X β Xβ = [x,..., x p ]. = β x β p x p β p The set of all possible linear combinations of the columns of X is called the column space of X and is denoted by C(X) = {Xa : a R p } The Gauss-Markov linear model says y is a random vector whose mean is in the column space of X and whose variance is σ 2 I for some positive real number σ 2, i.e. E(y) C(X) and Var(y) = σ 2 I, σ 2 R + W. Zhou (Colorado State University) STAT 540 July 6th, / 62
35 An Example Column Space X = [ ] C(X) = {Xa : a R p } {[ ] } = [a ] : a R [ ] } = {a : a R {[ ] } = a a : a R W. Zhou (Colorado State University) STAT 540 July 6th, / 62
36 Another Example Column Space X = [ ] C(X) = 0 a : a R 2 0 a = a 0 + a 0 2 : a, a 2 R 0 a 0 a = a 2 : a, a 2 R 0 a 2 a a = a 2 : a, a 2 R a 2 W. Zhou (Colorado State University) STAT 540 July 6th, / 62
37 Another Example Column Space X = , X 2 = x C(X ) x = X a for some a R 2 [ ] 0 x = X 2 for some a R 2 a x = X 2 b for some b R 3 x C(X 2 ) Thus C(X ) C(X 2 ) W. Zhou (Colorado State University) STAT 540 July 6th, / 62
38 Another Example Column Space (continued) x C(X 2 ) x = X 2 a for some a R 3 0 x = a + a a 0 3 for some a R3 0 a + a 2 a + a 2 x = a + a 3 for some a, a 2, a 3 R a + a 3 [ a + a 2 x = X a + a 3 ] for some a, a 2, a 3 R W. Zhou (Colorado State University) STAT 540 July 6th, / 62
39 Another Example Column Space (continued) x = X [ a + a 2 a + a 3 x = X b for some b R 2 x C(X ) ] for some a, a 2, a 3 R Thus, C(X 2 ) C(X ), as we have shown C(X ) C(X 2 ). It follows that C(X ) = C(X 2 ). W. Zhou (Colorado State University) STAT 540 July 6th, / 62
40 Estimation of E(y) A fundamental goal of linear model analysis is to estimate E(y) We could, of course, use y to estimate E(y) y is obviously an unbiased estimator of E(y), but it is often not a very sensible estimator. For example, suppose [ ] [ ] [ y = µ + y 2 ɛ ɛ 2 Should we estimate E(y) = [µ, µ] by y = [6., 2.3]? ], and we observe y = [6., 2.3] W. Zhou (Colorado State University) STAT 540 July 6th, / 62
41 Estimation of E(y) The Gauss-Markov linear models says that E(y) C(X), so we should use that information when estimating E(y) Consider estimating E(y) by the point in C(X) that is closest to y (as measured by the usual Euclidean distance). This unique point is called the orthogonal projection of y onto C(X) and denoted by ŷ (although it could be argued that Ê(y) might be better notation). By definition, y ŷ = min z C(X) y z where a = n i= a2 i W. Zhou (Colorado State University) STAT 540 July 6th, / 62
42 Geometric Viewpoint on Multiple Regression (and LM) Geometrically, how to minimize the distance between Y and C(X)? That point is The vector between Y and X β is, and the distance is For R 2 : if we add another predictor, C(X) gains more dimension, so e can only decrease. C(X) Note: if dim(s) = n then W. Zhou (Colorado State University) STAT 540 July 6th, / 62
43 W. Zhou (Colorado State University) STAT 540 July 6th, / 62
44 Orthogonal Projection Matrices It can be shown that, as we did for least square estimators y R n, ŷ = P X y is the optimal one, i.e. ŷ = P X y is the best estimator of E(y) in the class of linear unbiased estimators for the unique matrix P X = H, the hat matrix, and is called orthogonal projection matrix HH = H, idempotent H = H, symmetric HX = X and X H = X (Why? Intuitively...) If (X X) is not invertible, we use its generalized inverse (X X) where AA A = A. The H is invariant to the choice of (X X), which is itself not unique ŷ and y ŷ are orthogonal (Why?) W. Zhou (Colorado State University) STAT 540 July 6th, / 62
45 An Example Orthogonal Projection Suppose [ y y 2 ] = [ ] µ + [ X(X X) X = = ɛ ɛ 2 [ [ ], and we observe y = [6., 2.3]. Then ] ([ ] [ ] = [ = 2 [ = ] [ [2] [ ] [ ][ ] 2 ] /2 /2 /2 /2 ] ]) [ ] W. Zhou (Colorado State University) STAT 540 July 6th, / 62
46 An Example Orthogonal Projection Thus, [ the orthogonal ] projection of y = [6., 2.3] onto the column space of X = is [ ] [ ] [ ] /2 / P X y = Hy = = /2 / W. Zhou (Colorado State University) STAT 540 July 6th, / 62
47 Geometric illustration Suppose X = [ 2 ] and y = [ 2 3/4 ] W. Zhou (Colorado State University) STAT 540 July 6th, / 62
48 Geometric illustration Suppose X = [ 2 ] and y = [ 2 3/4 ] W. Zhou (Colorado State University) STAT 540 July 6th, / 62
49 Geometric illustration Suppose X = [ 2 ] and y = [ 2 3/4 ] W. Zhou (Colorado State University) STAT 540 July 6th, / 62
50 Geometric illustration Suppose X = [ 2 ] and y = [ 2 3/4 ] W. Zhou (Colorado State University) STAT 540 July 6th, / 62
51 Geometric illustration The angle between ŷ and residual y ŷ is 90. So, orthogonal projection. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
52 Multiple Linear Regression Model 2 Inference on Multiple Regression 3 Inference about Regression Parameters 4 Estimation and Prediction 5 Geometric View of Regression and Linear Models 6 Estimating estimable function of coefficient W. Zhou (Colorado State University) STAT 540 July 6th, / 62
53 What if X is not full column rank? X T X is not invertible, then (X T X) has to be defined based on the generalized inverse matrix. If X is not of full column rank, then there are infinitely many vectors in the set {b : Xb = Xβ} for any fixed value of β. Thus, no matter what the value of E(y), there will be infinitely many vectors b such that Xb = E(y) when X is not of full column rank. Our response vector y can help us learn about E(y) = Xβ, but when X is NOT of full column rank, there is NO hope of learning about β alone unless additional information about β is available. How, we could estimate estimable function of β W. Zhou (Colorado State University) STAT 540 July 6th, / 62
54 Treatment Effects Model Researchers randomly assigned a total of six experimental units to two treatments and measured a response of interest. y ij = µ + τ i + ɛ ij, i =, 2; j =, 2, 3 y y 2 y 3 y 2 y 22 = µ + τ µ + τ µ + τ µ + τ 2 µ + τ 2 + ɛ ɛ 2 ɛ 3 ɛ 2 ɛ 22 Question: what is X, β? y 23 µ + τ 2 ɛ 23 W. Zhou (Colorado State University) STAT 540 July 6th, / 62
55 Treatment Effects Model (continued) In this case, it makes no sense to estimate β = [µ, τ, τ 2 ] because there are multiple (infinitely many, in fact) choices of β that define the same mean for y. For example µ τ τ 2 = all yield same Xβ = E(y). 5, 0 4 6, When multiple values for β define the same E(y), we say that β is non-estimable. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
56 Estimable Functions of β A linear function of β, Cβ, is said to be estimable if there is a linear function of y, say Ay, that is an unbiased estimator for Cβ. Otherwise, nonexistence of such linear function implies that Cβ is non-estimable. Note that Ay is an unbiased estimator of Cβ if and only if E(Ay) = Cβ, for β R p AXβ = Cβ AX = C This says that we can estimate Cβ as long as Cβ = AXβ = AE(y) for some A, i.e. as long as Cβ is a linear function of E(y) The bottom line is that we can always estimate E(y) and all linear functions of E(y); all other linear functions of β are non-estimable W. Zhou (Colorado State University) STAT 540 July 6th, / 62
57 Treatment Effects Model (continued) Xβ = µ τ τ 2 = µ + τ µ + τ µ + τ µ + τ 2 µ + τ 2 µ + τ 2 so that [, 0, 0, 0, 0, 0]Xβ = [,, 0]β = µ + τ [0, 0, 0,, 0, 0]Xβ = [, 0, ]β = µ + τ 2 [, 0, 0,, 0, 0]Xβ = [0,, ]β = τ τ 2 are estimable functions of β W. Zhou (Colorado State University) STAT 540 July 6th, / 62
58 Estimating Estimable Functions of β If Cβ is estimable, then there exists a matrix A such that C = AX and Cβ = AXβ = AE(y) for any β R p It makes sens to estimate Cβ by AÊ(y) = Aŷ = AP Xy = AX(X X) X y = AX(X X) X X ˆβ = AP X X ˆβ = AX ˆβ = Cβ C ˆβ is called an Ordinary Least Squares (OLS) estimator of Cβ Note that although the hat is on β, it is Cβ that we are estimating Invariance of C ˆβ to the choice of ˆβ: Although there are infinitely many solutions to the normal equations when X is not of full column rank, C ˆβ is the same for all normal equation solutions ˆβ whenever Cβ is estimable (STAT 640) W. Zhou (Colorado State University) STAT 540 July 6th, / 62
59 Treatment Effects Model (continued) Suppose our aim is to estimate τ τ 2 As noted before Xβ = µ τ τ 2 = µ + τ µ + τ µ + τ µ + τ 2 µ + τ 2 µ + τ 2, so that [, 0, 0,, 0, 0]Xβ = [0,, ]β = τ τ 2 Thus, we can compute the OLS estimator of τ τ 2 as [, 0, 0,, 0, 0]ŷ = [0,, ] ˆβ where ˆβ is any solution to the normal equations. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
60 Treatment Effects Model (continued) The normal equation in this case is b b 2 b 3 = y y 2 y 3 y 2 y 22 y 23 so that b b 2 b 3 = y.. y. y 2. W. Zhou (Colorado State University) STAT 540 July 6th, / 62
61 Treatment Effects Model (continued) ȳ.. 0 ˆβ = ȳ. ȳ.. and ˆβ 2 = ȳ. are both solutions to the normal equation ȳ 2. ȳ.. ȳ 2. (Check this). Thus, the OLS estimator of Cβ = [0,, ]β = τ τ 2 is ȳ.. C ˆβ = [0,, ] ȳ. ȳ.. = ȳ. ȳ 2. = [0,, ] ȳ 2. ȳ.. 0 ȳ. ȳ 2. = C ˆβ 2 HW: Can you find two different generalized inverse of (X X), A and A 2 that (X X)A i (X X) = (X X) so that A i = (X X) for each i, and they will give you ˆβ and ˆβ 2, respectively? W. Zhou (Colorado State University) STAT 540 July 6th, / 62
62 The Gauss-Markov Theorem Under the Gauss-Markov Linear Model, the OLS estimator c ˆβ of an estimable linear function c β is the unique Best Linear Unbiased Estimator (BLUE) in the sense that Var(c ˆβ) is strictly less than the variance of any other linear unbiased estimator of c β for all β R p and all σ 2 R +. The Gauss-Markov Theorem says that if we want to estimate an estimable linear function c β using a linear estimator that is unbiased, we should always use the OLS estimator. In our simple example of the treatment effects model, we could have used y y 2 to estimate τ τ 2. It is easy to see that y y 2 is a linear estimator that is unbiased for τ τ 2, but its variance is clearly larger than the variance of the OLS estimator ȳ. ȳ 2. (as guaranteed by the Gauss-Markov Theorem). W. Zhou (Colorado State University) STAT 540 July 6th, / 62
Estimating Estimable Functions of β. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 17
Estimating Estimable Functions of β Copyright c 202 Dan Nettleton (Iowa State University) Statistics 5 / 7 The Response Depends on β Only through Xβ In the Gauss-Markov or Normal Theory Gauss-Markov Linear
More informationMatrix Approach to Simple Linear Regression: An Overview
Matrix Approach to Simple Linear Regression: An Overview Aspects of matrices that you should know: Definition of a matrix Addition/subtraction/multiplication of matrices Symmetric/diagonal/identity matrix
More information2. A Review of Some Key Linear Models Results. Copyright c 2018 Dan Nettleton (Iowa State University) 2. Statistics / 28
2. A Review of Some Key Linear Models Results Copyright c 2018 Dan Nettleton (Iowa State University) 2. Statistics 510 1 / 28 A General Linear Model (GLM) Suppose y = Xβ + ɛ, where y R n is the response
More informationLecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is Q = (Y i β 0 β 1 X i1 β 2 X i2 β p 1 X i.p 1 ) 2, which in matrix notation is Q = (Y Xβ) (Y
More informationCh 2: Simple Linear Regression
Ch 2: Simple Linear Regression 1. Simple Linear Regression Model A simple regression model with a single regressor x is y = β 0 + β 1 x + ɛ, where we assume that the error ɛ is independent random component
More informationXβ is a linear combination of the columns of X: Copyright c 2010 Dan Nettleton (Iowa State University) Statistics / 25 X =
The Gauss-Markov Linear Model y Xβ + ɛ y is an n random vector of responses X is an n p matrix of constants with columns corresponding to explanatory variables X is sometimes referred to as the design
More informationSimultaneous Inference: An Overview
Simultaneous Inference: An Overview Topics to be covered: Joint estimation of β 0 and β 1. Simultaneous estimation of mean responses. Simultaneous prediction intervals. W. Zhou (Colorado State University)
More informationLinear models and their mathematical foundations: Simple linear regression
Linear models and their mathematical foundations: Simple linear regression Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/21 Introduction
More informationEstimation of the Response Mean. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 27
Estimation of the Response Mean Copyright c 202 Dan Nettleton (Iowa State University) Statistics 5 / 27 The Gauss-Markov Linear Model y = Xβ + ɛ y is an n random vector of responses. X is an n p matrix
More informationOutline. Remedial Measures) Extra Sums of Squares Standardized Version of the Multiple Regression Model
Outline 1 Multiple Linear Regression (Estimation, Inference, Diagnostics and Remedial Measures) 2 Special Topics for Multiple Regression Extra Sums of Squares Standardized Version of the Multiple Regression
More informationNeed for Several Predictor Variables
Multiple regression One of the most widely used tools in statistical analysis Matrix expressions for multiple regression are the same as for simple linear regression Need for Several Predictor Variables
More informationFinal Review. Yang Feng. Yang Feng (Columbia University) Final Review 1 / 58
Final Review Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Final Review 1 / 58 Outline 1 Multiple Linear Regression (Estimation, Inference) 2 Special Topics for Multiple
More informationCh 3: Multiple Linear Regression
Ch 3: Multiple Linear Regression 1. Multiple Linear Regression Model Multiple regression model has more than one regressor. For example, we have one response variable and two regressor variables: 1. delivery
More informationFormal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
More informationLinear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
More informationLecture 10 Multiple Linear Regression
Lecture 10 Multiple Linear Regression STAT 512 Spring 2011 Background Reading KNNL: 6.1-6.5 10-1 Topic Overview Multiple Linear Regression Model 10-2 Data for Multiple Regression Y i is the response variable
More information6. Multiple Linear Regression
6. Multiple Linear Regression SLR: 1 predictor X, MLR: more than 1 predictor Example data set: Y i = #points scored by UF football team in game i X i1 = #games won by opponent in their last 10 games X
More informationTopic 7 - Matrix Approach to Simple Linear Regression. Outline. Matrix. Matrix. Review of Matrices. Regression model in matrix form
Topic 7 - Matrix Approach to Simple Linear Regression Review of Matrices Outline Regression model in matrix form - Fall 03 Calculations using matrices Topic 7 Matrix Collection of elements arranged in
More informationSSR = The sum of squared errors measures how much Y varies around the regression line n. It happily turns out that SSR + SSE = SSTO.
Analysis of variance approach to regression If x is useless, i.e. β 1 = 0, then E(Y i ) = β 0. In this case β 0 is estimated by Ȳ. The ith deviation about this grand mean can be written: deviation about
More informationMa 3/103: Lecture 24 Linear Regression I: Estimation
Ma 3/103: Lecture 24 Linear Regression I: Estimation March 3, 2017 KC Border Linear Regression I March 3, 2017 1 / 32 Regression analysis Regression analysis Estimate and test E(Y X) = f (X). f is the
More informationContents. 1 Review of Residuals. 2 Detecting Outliers. 3 Influential Observations. 4 Multicollinearity and its Effects
Contents 1 Review of Residuals 2 Detecting Outliers 3 Influential Observations 4 Multicollinearity and its Effects W. Zhou (Colorado State University) STAT 540 July 6th, 2015 1 / 32 Model Diagnostics:
More informationLecture 6 Multiple Linear Regression, cont.
Lecture 6 Multiple Linear Regression, cont. BIOST 515 January 22, 2004 BIOST 515, Lecture 6 Testing general linear hypotheses Suppose we are interested in testing linear combinations of the regression
More informationApplied Regression Analysis
Applied Regression Analysis Chapter 3 Multiple Linear Regression Hongcheng Li April, 6, 2013 Recall simple linear regression 1 Recall simple linear regression 2 Parameter Estimation 3 Interpretations of
More informationChapter 6 Multiple Regression
STAT 525 FALL 2018 Chapter 6 Multiple Regression Professor Min Zhang The Data and Model Still have single response variable Y Now have multiple explanatory variables Examples: Blood Pressure vs Age, Weight,
More informationMultiple Regression. Dr. Frank Wood. Frank Wood, Linear Regression Models Lecture 12, Slide 1
Multiple Regression Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 12, Slide 1 Review: Matrix Regression Estimation We can solve this equation (if the inverse of X
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is
More informationSummer School in Statistics for Astronomers V June 1 - June 6, Regression. Mosuk Chow Statistics Department Penn State University.
Summer School in Statistics for Astronomers V June 1 - June 6, 2009 Regression Mosuk Chow Statistics Department Penn State University. Adapted from notes prepared by RL Karandikar Mean and variance Recall
More informationChapter 5 Matrix Approach to Simple Linear Regression
STAT 525 SPRING 2018 Chapter 5 Matrix Approach to Simple Linear Regression Professor Min Zhang Matrix Collection of elements arranged in rows and columns Elements will be numbers or symbols For example:
More informationData Mining Stat 588
Data Mining Stat 588 Lecture 02: Linear Methods for Regression Department of Statistics & Biostatistics Rutgers University September 13 2011 Regression Problem Quantitative generic output variable Y. Generic
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 47 Outline 1 Regression 2 Simple Linear regression 3 Basic concepts in regression 4 How to estimate unknown parameters 5 Properties of Least Squares Estimators:
More informationMAT2377. Rafa l Kulik. Version 2015/November/26. Rafa l Kulik
MAT2377 Rafa l Kulik Version 2015/November/26 Rafa l Kulik Bivariate data and scatterplot Data: Hydrocarbon level (x) and Oxygen level (y): x: 0.99, 1.02, 1.15, 1.29, 1.46, 1.36, 0.87, 1.23, 1.55, 1.40,
More informationSimple and Multiple Linear Regression
Sta. 113 Chapter 12 and 13 of Devore March 12, 2010 Table of contents 1 Simple Linear Regression 2 Model Simple Linear Regression A simple linear regression model is given by Y = β 0 + β 1 x + ɛ where
More informationMultivariate Regression
Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationSimple Linear Regression
Simple Linear Regression ST 430/514 Recall: A regression model describes how a dependent variable (or response) Y is affected, on average, by one or more independent variables (or factors, or covariates)
More informationLinear Algebra Review
Linear Algebra Review Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Linear Algebra Review 1 / 45 Definition of Matrix Rectangular array of elements arranged in rows and
More informationRegression Models for Quantitative and Qualitative Predictors: An Overview
Regression Models for Quantitative and Qualitative Predictors: An Overview Polynomial regression models Interaction regression models Qualitative predictors Indicator variables Modeling interactions between
More informationInference for Regression
Inference for Regression Section 9.4 Cathy Poliak, Ph.D. cathy@math.uh.edu Office in Fleming 11c Department of Mathematics University of Houston Lecture 13b - 3339 Cathy Poliak, Ph.D. cathy@math.uh.edu
More information[4+3+3] Q 1. (a) Describe the normal regression model through origin. Show that the least square estimator of the regression parameter is given by
Concordia University Department of Mathematics and Statistics Course Number Section Statistics 360/1 40 Examination Date Time Pages Final June 2004 3 hours 7 Instructors Course Examiner Marks Y.P. Chaubey
More informationChapter 12 - Lecture 2 Inferences about regression coefficient
Chapter 12 - Lecture 2 Inferences about regression coefficient April 19th, 2010 Facts about slope Test Statistic Confidence interval Hypothesis testing Test using ANOVA Table Facts about slope In previous
More informationChapter 2 Multiple Regression I (Part 1)
Chapter 2 Multiple Regression I (Part 1) 1 Regression several predictor variables The response Y depends on several predictor variables X 1,, X p response {}}{ Y predictor variables {}}{ X 1, X 2,, X p
More information3 Multiple Linear Regression
3 Multiple Linear Regression 3.1 The Model Essentially, all models are wrong, but some are useful. Quote by George E.P. Box. Models are supposed to be exact descriptions of the population, but that is
More informationMath 423/533: The Main Theoretical Topics
Math 423/533: The Main Theoretical Topics Notation sample size n, data index i number of predictors, p (p = 2 for simple linear regression) y i : response for individual i x i = (x i1,..., x ip ) (1 p)
More informationLECTURE 2 LINEAR REGRESSION MODEL AND OLS
SEPTEMBER 29, 2014 LECTURE 2 LINEAR REGRESSION MODEL AND OLS Definitions A common question in econometrics is to study the effect of one group of variables X i, usually called the regressors, on another
More informationSTAT420 Midterm Exam. University of Illinois Urbana-Champaign October 19 (Friday), :00 4:15p. SOLUTIONS (Yellow)
STAT40 Midterm Exam University of Illinois Urbana-Champaign October 19 (Friday), 018 3:00 4:15p SOLUTIONS (Yellow) Question 1 (15 points) (10 points) 3 (50 points) extra ( points) Total (77 points) Points
More informationChapter 2 Inferences in Simple Linear Regression
STAT 525 SPRING 2018 Chapter 2 Inferences in Simple Linear Regression Professor Min Zhang Testing for Linear Relationship Term β 1 X i defines linear relationship Will then test H 0 : β 1 = 0 Test requires
More informationLinear regression. We have that the estimated mean in linear regression is. ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. The standard error of ˆµ Y X=x is.
Linear regression We have that the estimated mean in linear regression is The standard error of ˆµ Y X=x is where x = 1 n s.e.(ˆµ Y X=x ) = σ ˆµ Y X=x = ˆβ 0 + ˆβ 1 x. 1 n + (x x)2 i (x i x) 2 i x i. The
More informationMultivariate Linear Regression Models
Multivariate Linear Regression Models Regression analysis is used to predict the value of one or more responses from a set of predictors. It can also be used to estimate the linear association between
More informationMathematics for Economics MA course
Mathematics for Economics MA course Simple Linear Regression Dr. Seetha Bandara Simple Regression Simple linear regression is a statistical method that allows us to summarize and study relationships between
More informationassumes a linear relationship between mean of Y and the X s with additive normal errors the errors are assumed to be a sample from N(0, σ 2 )
Multiple Linear Regression is used to relate a continuous response (or dependent) variable Y to several explanatory (or independent) (or predictor) variables X 1, X 2,, X k assumes a linear relationship
More informationNonparametric Regression and Bonferroni joint confidence intervals. Yang Feng
Nonparametric Regression and Bonferroni joint confidence intervals Yang Feng Simultaneous Inferences In chapter 2, we know how to construct confidence interval for β 0 and β 1. If we want a confidence
More informationTHE ANOVA APPROACH TO THE ANALYSIS OF LINEAR MIXED EFFECTS MODELS
THE ANOVA APPROACH TO THE ANALYSIS OF LINEAR MIXED EFFECTS MODELS We begin with a relatively simple special case. Suppose y ijk = µ + τ i + u ij + e ijk, (i = 1,..., t; j = 1,..., n; k = 1,..., m) β =
More informationPART I. (a) Describe all the assumptions for a normal error regression model with one predictor variable,
Concordia University Department of Mathematics and Statistics Course Number Section Statistics 360/2 01 Examination Date Time Pages Final December 2002 3 hours 6 Instructors Course Examiner Marks Y.P.
More informationSTAT 705 Chapter 16: One-way ANOVA
STAT 705 Chapter 16: One-way ANOVA Timothy Hanson Department of Statistics, University of South Carolina Stat 705: Data Analysis II 1 / 21 What is ANOVA? Analysis of variance (ANOVA) models are regression
More informationF-tests and Nested Models
F-tests and Nested Models Nested Models: A core concept in statistics is comparing nested s. Consider the Y = β 0 + β 1 x 1 + β 2 x 2 + ǫ. (1) The following reduced s are special cases (nested within)
More informationBias Variance Trade-off
Bias Variance Trade-off The mean squared error of an estimator MSE(ˆθ) = E([ˆθ θ] 2 ) Can be re-expressed MSE(ˆθ) = Var(ˆθ) + (B(ˆθ) 2 ) MSE = VAR + BIAS 2 Proof MSE(ˆθ) = E((ˆθ θ) 2 ) = E(([ˆθ E(ˆθ)]
More informationLinear Regression. September 27, Chapter 3. Chapter 3 September 27, / 77
Linear Regression Chapter 3 September 27, 2016 Chapter 3 September 27, 2016 1 / 77 1 3.1. Simple linear regression 2 3.2 Multiple linear regression 3 3.3. The least squares estimation 4 3.4. The statistical
More information17: INFERENCE FOR MULTIPLE REGRESSION. Inference for Individual Regression Coefficients
17: INFERENCE FOR MULTIPLE REGRESSION Inference for Individual Regression Coefficients The results of this section require the assumption that the errors u are normally distributed. Let c i ij denote the
More informationInference in Normal Regression Model. Dr. Frank Wood
Inference in Normal Regression Model Dr. Frank Wood Remember We know that the point estimator of b 1 is b 1 = (Xi X )(Y i Ȳ ) (Xi X ) 2 Last class we derived the sampling distribution of b 1, it being
More informationPeter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8
Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall
More information14 Multiple Linear Regression
B.Sc./Cert./M.Sc. Qualif. - Statistics: Theory and Practice 14 Multiple Linear Regression 14.1 The multiple linear regression model In simple linear regression, the response variable y is expressed in
More informationEstimating σ 2. We can do simple prediction of Y and estimation of the mean of Y at any value of X.
Estimating σ 2 We can do simple prediction of Y and estimation of the mean of Y at any value of X. To perform inferences about our regression line, we must estimate σ 2, the variance of the error term.
More information4 Multiple Linear Regression
4 Multiple Linear Regression 4. The Model Definition 4.. random variable Y fits a Multiple Linear Regression Model, iff there exist β, β,..., β k R so that for all (x, x 2,..., x k ) R k where ε N (, σ
More informationLecture 1 Linear Regression with One Predictor Variable.p2
Lecture Linear Regression with One Predictor Variablep - Basics - Meaning of regression parameters p - β - the slope of the regression line -it indicates the change in mean of the probability distn of
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini June 9, 2018 1 / 56 Table of Contents Multiple linear regression Linear model setup Estimation of β Geometric interpretation Estimation of σ 2 Hat matrix Gram matrix
More informationRegression Analysis. Regression: Methodology for studying the relationship among two or more variables
Regression Analysis Regression: Methodology for studying the relationship among two or more variables Two major aims: Determine an appropriate model for the relationship between the variables Predict the
More informationCorrelation and the Analysis of Variance Approach to Simple Linear Regression
Correlation and the Analysis of Variance Approach to Simple Linear Regression Biometry 755 Spring 2009 Correlation and the Analysis of Variance Approach to Simple Linear Regression p. 1/35 Correlation
More informationSimple Linear Regression
Simple Linear Regression In simple linear regression we are concerned about the relationship between two variables, X and Y. There are two components to such a relationship. 1. The strength of the relationship.
More informationLINEAR REGRESSION MODELS W4315
LINEAR REGRESSION MODELS W431 HOMEWORK ANSWERS March 9, 2010 Due: 03/04/10 Instructor: Frank Wood 1. (20 points) In order to get a maximum likelihood estimate of the parameters of a Box-Cox transformed
More informationSTAT763: Applied Regression Analysis. Multiple linear regression. 4.4 Hypothesis testing
STAT763: Applied Regression Analysis Multiple linear regression 4.4 Hypothesis testing Chunsheng Ma E-mail: cma@math.wichita.edu 4.4.1 Significance of regression Null hypothesis (Test whether all β j =
More informationLecture 12 Inference in MLR
Lecture 12 Inference in MLR STAT 512 Spring 2011 Background Reading KNNL: 6.6-6.7 12-1 Topic Overview Review MLR Model Inference about Regression Parameters Estimation of Mean Response Prediction 12-2
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 51 Outline 1 Matrix Expression 2 Linear and quadratic forms 3 Properties of quadratic form 4 Properties of estimates 5 Distributional properties 3 / 51 Matrix
More informationInferences for Regression
Inferences for Regression An Example: Body Fat and Waist Size Looking at the relationship between % body fat and waist size (in inches). Here is a scatterplot of our data set: Remembering Regression In
More informationMa 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA
Ma 3/103: Lecture 25 Linear Regression II: Hypothesis Testing and ANOVA March 6, 2017 KC Border Linear Regression II March 6, 2017 1 / 44 1 OLS estimator 2 Restricted regression 3 Errors in variables 4
More informationSTAT5044: Regression and Anova
STAT5044: Regression and Anova Inyoung Kim 1 / 25 Outline 1 Multiple Linear Regression 2 / 25 Basic Idea An extra sum of squares: the marginal reduction in the error sum of squares when one or several
More informationHomoskedasticity. Var (u X) = σ 2. (23)
Homoskedasticity How big is the difference between the OLS estimator and the true parameter? To answer this question, we make an additional assumption called homoskedasticity: Var (u X) = σ 2. (23) This
More informationRegression Review. Statistics 149. Spring Copyright c 2006 by Mark E. Irwin
Regression Review Statistics 149 Spring 2006 Copyright c 2006 by Mark E. Irwin Matrix Approach to Regression Linear Model: Y i = β 0 + β 1 X i1 +... + β p X ip + ɛ i ; ɛ i iid N(0, σ 2 ), i = 1,..., n
More informationBIOS 2083 Linear Models c Abdus S. Wahed
Chapter 5 206 Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter
More informationSTA 302 H1F / 1001 HF Fall 2007 Test 1 October 24, 2007
STA 302 H1F / 1001 HF Fall 2007 Test 1 October 24, 2007 LAST NAME: SOLUTIONS FIRST NAME: STUDENT NUMBER: ENROLLED IN: (circle one) STA 302 STA 1001 INSTRUCTIONS: Time: 90 minutes Aids allowed: calculator.
More informationRegression and Statistical Inference
Regression and Statistical Inference Walid Mnif wmnif@uwo.ca Department of Applied Mathematics The University of Western Ontario, London, Canada 1 Elements of Probability 2 Elements of Probability CDF&PDF
More informationLinear Models in Machine Learning
CS540 Intro to AI Linear Models in Machine Learning Lecturer: Xiaojin Zhu jerryzhu@cs.wisc.edu We briefly go over two linear models frequently used in machine learning: linear regression for, well, regression,
More informationApplied Regression. Applied Regression. Chapter 2 Simple Linear Regression. Hongcheng Li. April, 6, 2013
Applied Regression Chapter 2 Simple Linear Regression Hongcheng Li April, 6, 2013 Outline 1 Introduction of simple linear regression 2 Scatter plot 3 Simple linear regression model 4 Test of Hypothesis
More informationChapter 2. Continued. Proofs For ANOVA Proof of ANOVA Identity. the product term in the above equation can be simplified as n
Chapter 2. Continued Proofs For ANOVA Proof of ANOVA Identity We are going to prove that Writing SST SSR + SSE. Y i Ȳ (Y i Ŷ i ) + (Ŷ i Ȳ ) Squaring both sides summing over all i 1,...n, we get (Y i Ȳ
More informationStatement: With my signature I confirm that the solutions are the product of my own work. Name: Signature:.
MATHEMATICAL STATISTICS Homework assignment Instructions Please turn in the homework with this cover page. You do not need to edit the solutions. Just make sure the handwriting is legible. You may discuss
More informationGeneral Linear Model: Statistical Inference
Chapter 6 General Linear Model: Statistical Inference 6.1 Introduction So far we have discussed formulation of linear models (Chapter 1), estimability of parameters in a linear model (Chapter 4), least
More informationMultiple Linear Regression
Multiple Linear Regression University of California, San Diego Instructor: Ery Arias-Castro http://math.ucsd.edu/~eariasca/teaching.html 1 / 42 Passenger car mileage Consider the carmpg dataset taken from
More informationSTA 2101/442 Assignment Four 1
STA 2101/442 Assignment Four 1 One version of the general linear model with fixed effects is y = Xβ + ɛ, where X is an n p matrix of known constants with n > p and the columns of X linearly independent.
More informationLeast Squares Estimation-Finite-Sample Properties
Least Squares Estimation-Finite-Sample Properties Ping Yu School of Economics and Finance The University of Hong Kong Ping Yu (HKU) Finite-Sample 1 / 29 Terminology and Assumptions 1 Terminology and Assumptions
More informationLectures on Simple Linear Regression Stat 431, Summer 2012
Lectures on Simple Linear Regression Stat 43, Summer 0 Hyunseung Kang July 6-8, 0 Last Updated: July 8, 0 :59PM Introduction Previously, we have been investigating various properties of the population
More informationProblems. Suppose both models are fitted to the same data. Show that SS Res, A SS Res, B
Simple Linear Regression 35 Problems 1 Consider a set of data (x i, y i ), i =1, 2,,n, and the following two regression models: y i = β 0 + β 1 x i + ε, (i =1, 2,,n), Model A y i = γ 0 + γ 1 x i + γ 2
More informationy ˆ i = ˆ " T u i ( i th fitted value or i th fit)
1 2 INFERENCE FOR MULTIPLE LINEAR REGRESSION Recall Terminology: p predictors x 1, x 2,, x p Some might be indicator variables for categorical variables) k-1 non-constant terms u 1, u 2,, u k-1 Each u
More informationChapter 4: Regression Models
Sales volume of company 1 Textbook: pp. 129-164 Chapter 4: Regression Models Money spent on advertising 2 Learning Objectives After completing this chapter, students will be able to: Identify variables,
More informationChapter 14. Linear least squares
Serik Sagitov, Chalmers and GU, March 5, 2018 Chapter 14 Linear least squares 1 Simple linear regression model A linear model for the random response Y = Y (x) to an independent variable X = x For a given
More informationMIT Spring 2015
Regression Analysis MIT 18.472 Dr. Kempthorne Spring 2015 1 Outline Regression Analysis 1 Regression Analysis 2 Multiple Linear Regression: Setup Data Set n cases i = 1, 2,..., n 1 Response (dependent)
More informationCorrelation Analysis
Simple Regression Correlation Analysis Correlation analysis is used to measure strength of the association (linear relationship) between two variables Correlation is only concerned with strength of the
More informationBusiness Statistics. Tommaso Proietti. Linear Regression. DEF - Università di Roma 'Tor Vergata'
Business Statistics Tommaso Proietti DEF - Università di Roma 'Tor Vergata' Linear Regression Specication Let Y be a univariate quantitative response variable. We model Y as follows: Y = f(x) + ε where
More informationMaximum Likelihood Estimation
Maximum Likelihood Estimation Merlise Clyde STA721 Linear Models Duke University August 31, 2017 Outline Topics Likelihood Function Projections Maximum Likelihood Estimates Readings: Christensen Chapter
More informationPart IB Statistics. Theorems with proof. Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua. Lent 2015
Part IB Statistics Theorems with proof Based on lectures by D. Spiegelhalter Notes taken by Dexter Chua Lent 2015 These notes are not endorsed by the lecturers, and I have modified them (often significantly)
More informationMeasuring the fit of the model - SSR
Measuring the fit of the model - SSR Once we ve determined our estimated regression line, we d like to know how well the model fits. How far/close are the observations to the fitted line? One way to do
More informationLecture 11: Regression Methods I (Linear Regression)
Lecture 11: Regression Methods I (Linear Regression) Fall, 2017 1 / 40 Outline Linear Model Introduction 1 Regression: Supervised Learning with Continuous Responses 2 Linear Models and Multiple Linear
More information