Economics 620, Lecture 4: The K-Variable Linear Model I. y 1 = + x 1 + " 1 y 2 = + x 2 + " 2 :::::::: :::::::: y N = + x N + " N
|
|
|
- Ashlynn Shields
- 5 years ago
- Views:
Transcription
1 1 Economics 620, Lecture 4: The K-Variable Linear Model I Consider the system y 1 = + x 1 + " 1 y 2 = + x 2 + " 2 :::::::: :::::::: y N = + x N + " N or in matrix form y = X + " where y is N 1, X is N 2, is 21, and " is N 1.
2 2 K-Variable Linear Model X = x 1 1 x 2 :. 1 x N 3 7 5, = " # : Good statistical practice requires inclusion of the column of ones. Consider the general model y = X + " Convention: " is N 1. X = y is N 1, X is N K, is K 1, and 1 x 21 ::::x K1 1 x 22 ::::x K2 : : :::: : 1 x 2N ::::x Kn 3 7 5, = : : K :
3 3 A typical row looks like: More on the Linear Model y i = x 2i + 3 x 3i + ::: + K x Ki + " i The Least Squares Method: First Assumption: Ey = X S(b) = (y Xb) 0 (y Xb) = y 0 y 2b 0 X 0 y + b 0 X 0 Xb Normal Equations X 0 X ^ X 0 y = 0 These equations always have a solution. geometry to come) (Clear from If X 0 X is invertible ^ = (X 0 X) 1 X 0 y.
4 4 More on the Linear Model Proposition: ^ is a minimizer. Proof : Let b be any other K-vector. (y Xb) 0 (y Xb) = (y X ^ + X(^ b)) 0 (y X ^ + X(^ b)) = (y X ^) 0 (y X ^) + (^ b) 0 X 0 X(^ b) (y X ^)(y X ^). (Why?) De nition: e = y X ^ is the vector of residuals. Note: Ee = 0 and X 0 e = 0. Proposition: The LS estimator is unbiased. Proof : E^ = E[(X 0 X) 1 X 0 y] = E[(X 0 X) 1 X 0 (X + ")] =
5 5 Geometry of Least Squares Consider y = X + " with y = " y1 y 2 #, X = " x1 x 2 # : De nition: The space spanned by matrix X is the vector space which consists of all linear combinations of the column vectors of X. De nition: X(X 0 X) 1 X 0 y is the orthogonal projection of y to the space spanned by X. Proposition: e is perpendicular to X, i.e., X 0 e = 0. Proof : e = y X ^ = y X(X 0 X) 1 X 0 y e = (I X(X 0 X) 1 X 0 )y ) X 0 e = (X 0 X 0 )y = 0.
6 6 Geometry of Least Squares (cont d) Thus the equation y = X ^ + e gives y as the sum of a vector in R[X] and a vector in N[X 0 ]. Common (friendly) projection matrices: 1. The matrix which projects to the space orthogonal to the space spanned by X (i.e. to N[X 0 ] is M = I X(X 0 X) 1 X 0 : Note: e = My. If X is full column rank, M has rank (N K). 2. The matrix which projects to the space spanned by X is I M = X(X 0 X) 1 X 0 : Note: ^y = y e = y My = (I M)y. If X is full column rank, (I M) has rank K.
7 7 Example in R 2 y i = x i + " i i = 1; 2 y e xb X x What is the case of singular X 0 X?
8 8 Properties of projection matrices 1. Projection matrices are idempotent. I.G. (I M)(I M) = (I M): Proof : (I M)(I M) = (X(X 0 X) 1 X 0 )(X(X 0 X) 1 X 0 ) = X(X 0 X) 1 X 0 = (I M) 2. Idempotent matrices have eigenvalues equal to zero or one. Proof : Consider the characteristic equation Mz = z ) M 2 z = Mz = 2 z. Since M is idempotent, M 2 z = Mz. Thus, 2 z = z, which implies that is either 0 or 1:
9 9 Properties of projection matrices 3. The number of nonzero eigenvalues of a matrix is equal to its rank. ) For idempotent matrices, trace = rank. More assumptions to the K-variable linear model: Second assumption: and " are N-vectors. V (y) = V (") = 2 I N where y With this assumption, we can obtain the sampling variance of ^: Proposition: V (^) = 2 (X 0 X) 1 Proof : ^ = (X 0 X) 1 X 0 y
10 10 = (X 0 X) 1 X 0 X + (X 0 X) 1 X 0 " hence ^ = + (X 0 X) 1 X 0 "
11 11 cont d V (^) = E(^ E^)(^ E^) 0 = E(X 0 X) 1 X 0 "" 0 X(X 0 X) 1 V (^) = (X 0 X) 1 X 0 (E"" 0 )X(X 0 X) 1 = 2 (X 0 X) 1 Gauss-Markov Theorem: The LS estimator is BLUE. Proof : Consider estimating c 0 for some c. A possible estimator is c 0^ with variance 2 c 0 (X 0 X) 1 c. An alternative linear unbiased estimator: b = a 0 y. Eb = a 0 Ey = a 0 X. Since both c 0^ and b are unbiased, a 0 X = c 0.
12 12 Gauss-Markov Theorem(cont d) Thus, b = a 0 y = a 0 (X + ") = a 0 X + a 0 " = c 0 + a 0 ". Hence, V (b) = 2 a 0 a: Now, V (c 0^) = 2 a 0 X(X 0 X) 1 X 0 a since c 0 = a 0 X. So V (b) V (c 0^) = 2 a 0 Ma, p.s.d. Hence, V (b) V (c 0^)
13 13 Estimation of 2 Proposition: s 2 = e 0 e=(n for 2. K) is an unbiased estimator Proof : e = y X ^ = My = M" ) e 0 e = " 0 M" Ee 0 e = E" 0 M" = E tr " 0 M" (Why?) = tr E" 0 M" = tr EM"" 0 (important trick) = tr M E"" 0 = 2 tr M = 2 (N K) ) s 2 = e 0 e=(n K) is unbiased for 2 :
14 14 Fit: Does the Regression Model Explain the Data? We will need the useful idempotent matrix A = I 1(1 0 1) = I 11 0 =N which sweeps out means. Here 1 is an N-vector of ones. Note that AM = M when X contains a constant term. De nition: case is The correlation coe cient in the K-variable R 2 = (Sum of squares due to X)/(Total sum of squares) = 1 (e 0 e=y 0 Ay).
15 15 More Fit Using A, y 0 Ay =: NP i=1 (y i y) 2 y 0 Ay = (Ay) 0 (Ay) = (A^y + Ae) 0 (A^y + Ae) = ^y 0 A^y + e 0 Ae since ^y 0 e = 0 Thus, y 0 Ay = ^y 0 A^y + e 0 e since Ae = e: Scaling yields: 1 = ^y0 A^y y 0 Ay + e0 e y 0 Ay What are the two terms of this splitup?
16 16 More Fit R 2 gives the fraction of variation explained by X: R 2 = 1 (e 0 e=y 0 Ay): Note: The adjusted squared correlation coe cient is given by R 2 = 1 e 0 e=(n K) y 0 Ay=(N 1) (Why might this be preferable?)
17 17 Reporting Always report characteristics of the sample, i.e. means, standard deviations, anything unusual or surprising, how the data set is collected and how the sample is selected. Report ^ and standard errors (not t-statistics). The usual format is ^ (s:e: of ^) Specify S 2 or 2 ML : Report N and R 2 : Plots are important. For example, predicted vs. actual values or predicted and actual values over time in time series studies should be presented.
18 18 Comments on Linearity Consider the following argument: Economic functions don t change suddenly. Therefore they are continuous. Thus they are di erentiable and hence nearly linear by Taylor s Theorem. This argument is false (but irrelevant). f(x) f(x) Continuous, not diff, but well-approximated by a line. x Continuous, diff, and not wellapproximated x
19 19 NOTE ON THE GAUSS-MARKOV THEOREM Consider estimation of a mean based on an observation X. Assume: X F and Z x df = ; Z x 2 df = : () Suppose that the estimator for is h(x). Unbiasedness implies that Z h(x) df = : Theorem: The only function h unbiased for all F and satisfying () is h(x) = x.
20 20 Proof : Let h(x) = x + (x). Then Z (x) df = 0 for all F. Suppose, on the contrary, that (x) > 0 on some set A. Let 2 A and F = (the distribution assigning point mass to x = ). Then () is satis ed and Z h(x)d = + () 6= ; which is a contradiction. Argument is the same if (x) < 0. (x) = 0: This shows that The logic is that if the estimator is nonlinear, we can choose a distribution so that it is biased.
Economics 620, Lecture 4: The K-Varable Linear Model I
Economics 620, Lecture 4: The K-Varable Linear Model I Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 4: The K-Varable Linear Model I 1 / 20 Consider the system
Economics 620, Lecture 5: exp
1 Economics 620, Lecture 5: The K-Variable Linear Model II Third assumption (Normality): y; q(x; 2 I N ) 1 ) p(y) = (2 2 ) exp (N=2) 1 2 2(y X)0 (y X) where N is the sample size. The log likelihood function
Economics 620, Lecture 7: Still More, But Last, on the K-Varable Linear Model
Economics 620, Lecture 7: Still More, But Last, on the K-Varable Linear Model Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 7: the K-Varable Linear Model IV
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation
Economics 620, Lecture 9: Asymptotics III: Maximum Likelihood Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 9: Asymptotics III(MLE) 1 / 20 Jensen
Econ 620. Matrix Differentiation. Let a and x are (k 1) vectors and A is an (k k) matrix. ) x. (a x) = a. x = a (x Ax) =(A + A (x Ax) x x =(A + A )
Econ 60 Matrix Differentiation Let a and x are k vectors and A is an k k matrix. a x a x = a = a x Ax =A + A x Ax x =A + A x Ax = xx A We don t want to prove the claim rigorously. But a x = k a i x i i=
Economics 620, Lecture 2: Regression Mechanics (Simple Regression)
1 Economics 620, Lecture 2: Regression Mechanics (Simple Regression) Observed variables: y i ; x i i = 1; :::; n Hypothesized (model): Ey i = + x i or y i = + x i + (y i Ey i ) ; renaming we get: y i =
LECTURE 2 LINEAR REGRESSION MODEL AND OLS
SEPTEMBER 29, 2014 LECTURE 2 LINEAR REGRESSION MODEL AND OLS Definitions A common question in econometrics is to study the effect of one group of variables X i, usually called the regressors, on another
Linear Regression. In this problem sheet, we consider the problem of linear regression with p predictors and one intercept,
Linear Regression In this problem sheet, we consider the problem of linear regression with p predictors and one intercept, y = Xβ + ɛ, where y t = (y 1,..., y n ) is the column vector of target values,
Multiple Regression Model: I
Multiple Regression Model: I Suppose the data are generated according to y i 1 x i1 2 x i2 K x ik u i i 1...n Define y 1 x 11 x 1K 1 u 1 y y n X x n1 x nk K u u n So y n, X nxk, K, u n Rks: In many applications,
Ma 3/103: Lecture 24 Linear Regression I: Estimation
Ma 3/103: Lecture 24 Linear Regression I: Estimation March 3, 2017 KC Border Linear Regression I March 3, 2017 1 / 32 Regression analysis Regression analysis Estimate and test E(Y X) = f (X). f is the
Assignment 1 Math 5341 Linear Algebra Review. Give complete answers to each of the following questions. Show all of your work.
Assignment 1 Math 5341 Linear Algebra Review Give complete answers to each of the following questions Show all of your work Note: You might struggle with some of these questions, either because it has
Economics 620, Lecture 18: Nonlinear Models
Economics 620, Lecture 18: Nonlinear Models Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 18: Nonlinear Models 1 / 18 The basic point is that smooth nonlinear
YORK UNIVERSITY. Faculty of Science Department of Mathematics and Statistics MATH M Test #1. July 11, 2013 Solutions
YORK UNIVERSITY Faculty of Science Department of Mathematics and Statistics MATH 222 3. M Test # July, 23 Solutions. For each statement indicate whether it is always TRUE or sometimes FALSE. Note: For
REVIEW (MULTIVARIATE LINEAR REGRESSION) Explain/Obtain the LS estimator () of the vector of coe cients (b)
REVIEW (MULTIVARIATE LINEAR REGRESSION) Explain/Obtain the LS estimator () of the vector of coe cients (b) Explain/Obtain the variance-covariance matrix of Both in the bivariate case (two regressors) and
LECTURE 11: GENERALIZED LEAST SQUARES (GLS) In this lecture, we will consider the model y = Xβ + ε retaining the assumption Ey = Xβ.
LECTURE 11: GEERALIZED LEAST SQUARES (GLS) In this lecture, we will consider the model y = Xβ + ε retaining the assumption Ey = Xβ. However, we no longer have the assumption V(y) = V(ε) = σ 2 I. Instead
STAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini June 9, 2018 1 / 56 Table of Contents Multiple linear regression Linear model setup Estimation of β Geometric interpretation Estimation of σ 2 Hat matrix Gram matrix
Solutions for Econometrics I Homework No.1
Solutions for Econometrics I Homework No.1 due 2006-02-20 Feldkircher, Forstner, Ghoddusi, Grafenhofer, Pichler, Reiss, Yan, Zeugner Exercise 1.1 Structural form of the problem: 1. q d t = α 0 + α 1 p
Outline. Remedial Measures) Extra Sums of Squares Standardized Version of the Multiple Regression Model
Outline 1 Multiple Linear Regression (Estimation, Inference, Diagnostics and Remedial Measures) 2 Special Topics for Multiple Regression Extra Sums of Squares Standardized Version of the Multiple Regression
The General Linear Model. Monday, Lecture 2 Jeanette Mumford University of Wisconsin - Madison
The General Linear Model Monday, Lecture 2 Jeanette Mumford University of Wisconsin - Madison How we re approaching the GLM Regression for behavioral data Without using matrices Understand least squares
1 A complete Fourier series solution
Math 128 Notes 13 In this last set of notes I will try to tie up some loose ends. 1 A complete Fourier series solution First here is an example of the full solution of a pde by Fourier series. Consider
Variable Selection and Model Building
LINEAR REGRESSION ANALYSIS MODULE XIII Lecture - 37 Variable Selection and Model Building Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur The complete regression
Peter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8
Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall
Linear Regression. y» F; Ey = + x Vary = ¾ 2. ) y = + x + u. Eu = 0 Varu = ¾ 2 Exu = 0:
Linear Regression 1 Single Explanatory Variable Assume (y is not necessarily normal) where Examples: y» F; Ey = + x Vary = ¾ 2 ) y = + x + u Eu = 0 Varu = ¾ 2 Exu = 0: 1. School performance as a function
Introduction to Estimation Methods for Time Series models. Lecture 1
Introduction to Estimation Methods for Time Series models Lecture 1 Fulvio Corsi SNS Pisa Fulvio Corsi Introduction to Estimation () Methods for Time Series models Lecture 1 SNS Pisa 1 / 19 Estimation
plim W 0 " 1 N = 0 Prof. N. M. Kiefer, Econ 620, Cornell University, Lecture 16.
1 Lecture 16: Estimation of Simultaneous Equations Models Consider y 1 = Y 2 + X 1 + " 1 which is an equation from a system. We can rewrite this at y 1 = Z +" 1 where Z = [Y 2 X 1 ] and = [ 0 0 ] 0. Note
Review of Classical Least Squares. James L. Powell Department of Economics University of California, Berkeley
Review of Classical Least Squares James L. Powell Department of Economics University of California, Berkeley The Classical Linear Model The object of least squares regression methods is to model and estimate
STA 302f16 Assignment Five 1
STA 30f16 Assignment Five 1 Except for Problem??, these problems are preparation for the quiz in tutorial on Thursday October 0th, and are not to be handed in As usual, at times you may be asked to prove
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is Q = (Y i β 0 β 1 X i1 β 2 X i2 β p 1 X i.p 1 ) 2, which in matrix notation is Q = (Y Xβ) (Y
Lecture 6 & 7. Shuanglin Shao. September 16th and 18th, 2013
Lecture 6 & 7 Shuanglin Shao September 16th and 18th, 2013 1 Elementary matrices 2 Equivalence Theorem 3 A method of inverting matrices Def An n n matrice is called an elementary matrix if it can be obtained
Inverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1
Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is
can only hit 3 points in the codomain. Hence, f is not surjective. For another example, if n = 4
.. Conditions for Injectivity and Surjectivity In this section, we discuss what we can say about linear maps T : R n R m given only m and n. We motivate this problem by looking at maps f : {,..., n} {,...,
Need for Several Predictor Variables
Multiple regression One of the most widely used tools in statistical analysis Matrix expressions for multiple regression are the same as for simple linear regression Need for Several Predictor Variables
Linear models. Linear models are computationally convenient and remain widely used in. applied econometric research
Linear models Linear models are computationally convenient and remain widely used in applied econometric research Our main focus in these lectures will be on single equation linear models of the form y
1 Correlation between an independent variable and the error
Chapter 7 outline, Econometrics Instrumental variables and model estimation 1 Correlation between an independent variable and the error Recall that one of the assumptions that we make when proving the
3 Multiple Linear Regression
3 Multiple Linear Regression 3.1 The Model Essentially, all models are wrong, but some are useful. Quote by George E.P. Box. Models are supposed to be exact descriptions of the population, but that is
Estimation of the Response Mean. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 27
Estimation of the Response Mean Copyright c 202 Dan Nettleton (Iowa State University) Statistics 5 / 27 The Gauss-Markov Linear Model y = Xβ + ɛ y is an n random vector of responses. X is an n p matrix
Introduc)on to linear algebra
Introduc)on to linear algebra Vector A vector, v, of dimension n is an n 1 rectangular array of elements v 1 v v = 2 " v n % vectors will be column vectors. They may also be row vectors, when transposed
Economics 620, Lecture 14: Lags and Dynamics
Economics 620, Lecture 14: Lags and Dynamics Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 14: Lags and Dynamics 1 / 15 Modern time-series analysis does not
Advanced Econometrics I
Lecture Notes Autumn 2010 Dr. Getinet Haile, University of Mannheim 1. Introduction Introduction & CLRM, Autumn Term 2010 1 What is econometrics? Econometrics = economic statistics economic theory mathematics
Finansiell Statistik, GN, 15 hp, VT2008 Lecture 15: Multiple Linear Regression & Correlation
Finansiell Statistik, GN, 5 hp, VT28 Lecture 5: Multiple Linear Regression & Correlation Gebrenegus Ghilagaber, PhD, ssociate Professor May 5, 28 Introduction In the simple linear regression Y i = + X
the error term could vary over the observations, in ways that are related
Heteroskedasticity We now consider the implications of relaxing the assumption that the conditional variance Var(u i x i ) = σ 2 is common to all observations i = 1,..., n In many applications, we may
Regression Models - Introduction
Regression Models - Introduction In regression models there are two types of variables that are studied: A dependent variable, Y, also called response variable. It is modeled as random. An independent
STAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini April 27, 2018 1 / 1 Table of Contents 2 / 1 Linear Algebra Review Read 3.1 and 3.2 from text. 1. Fundamental subspace (rank-nullity, etc.) Im(X ) = ker(x T ) R
Appendix A: Matrices
Appendix A: Matrices A matrix is a rectangular array of numbers Such arrays have rows and columns The numbers of rows and columns are referred to as the dimensions of a matrix A matrix with, say, 5 rows
Introduction to the Simultaneous Equations Model
1 Lecture 15: Introduction to the Simultaneous Equations Model The statistics framework for the simultaneous equations model (SEM) is the multivariate regression. For example, let E(yjX; ) = X and V (yjx;
18.06SC Final Exam Solutions
18.06SC Final Exam Solutions 1 (4+7=11 pts.) Suppose A is 3 by 4, and Ax = 0 has exactly 2 special solutions: 1 2 x 1 = 1 and x 2 = 1 1 0 0 1 (a) Remembering that A is 3 by 4, find its row reduced echelon
4 Multiple Linear Regression
4 Multiple Linear Regression 4. The Model Definition 4.. random variable Y fits a Multiple Linear Regression Model, iff there exist β, β,..., β k R so that for all (x, x 2,..., x k ) R k where ε N (, σ
Xβ is a linear combination of the columns of X: Copyright c 2010 Dan Nettleton (Iowa State University) Statistics / 25 X =
The Gauss-Markov Linear Model y Xβ + ɛ y is an n random vector of responses X is an n p matrix of constants with columns corresponding to explanatory variables X is sometimes referred to as the design
Applied Econometrics (QEM)
Applied Econometrics (QEM) The Simple Linear Regression Model based on Prinicples of Econometrics Jakub Mućk Department of Quantitative Economics Jakub Mućk Applied Econometrics (QEM) Meeting #2 The Simple
Linear Algebra March 16, 2019
Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented
YORK UNIVERSITY. Faculty of Science Department of Mathematics and Statistics MATH M Test #2 Solutions
YORK UNIVERSITY Faculty of Science Department of Mathematics and Statistics MATH 3. M Test # Solutions. (8 pts) For each statement indicate whether it is always TRUE or sometimes FALSE. Note: For this
LECTURE 13: TIME SERIES I
1 LECTURE 13: TIME SERIES I AUTOCORRELATION: Consider y = X + u where y is T 1, X is T K, is K 1 and u is T 1. We are using T and not N for sample size to emphasize that this is a time series. The natural
Lecture 4: Multivariate Regression, Part 2
Lecture 4: Multivariate Regression, Part 2 Gauss-Markov Assumptions 1) Linear in Parameters: Y X X X i 0 1 1 2 2 k k 2) Random Sampling: we have a random sample from the population that follows the above
FIRST MIDTERM EXAM ECON 7801 SPRING 2001
FIRST MIDTERM EXAM ECON 780 SPRING 200 ECONOMICS DEPARTMENT, UNIVERSITY OF UTAH Problem 2 points Let y be a n-vector (It may be a vector of observations of a random variable y, but it does not matter how
E2 212: Matrix Theory (Fall 2010) Solutions to Test - 1
E2 212: Matrix Theory (Fall 2010) s to Test - 1 1. Let X = [x 1, x 2,..., x n ] R m n be a tall matrix. Let S R(X), and let P be an orthogonal projector onto S. (a) If X is full rank, show that P can be
STAT 540: Data Analysis and Regression
STAT 540: Data Analysis and Regression Wen Zhou http://www.stat.colostate.edu/~riczw/ Email: riczw@stat.colostate.edu Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State
Uses of Asymptotic Distributions: In order to get distribution theory, we need to norm the random variable; we usually look at n 1=2 ( X n ).
1 Economics 620, Lecture 8a: Asymptotics II Uses of Asymptotic Distributions: Suppose X n! 0 in probability. (What can be said about the distribution of X n?) In order to get distribution theory, we need
Econometrics I. Professor William Greene Stern School of Business Department of Economics 3-1/29. Part 3: Least Squares Algebra
Econometrics I Professor William Greene Stern School of Business Department of Economics 3-1/29 Econometrics I Part 3 Least Squares Algebra 3-2/29 Vocabulary Some terms to be used in the discussion. Population
Topic 7 - Matrix Approach to Simple Linear Regression. Outline. Matrix. Matrix. Review of Matrices. Regression model in matrix form
Topic 7 - Matrix Approach to Simple Linear Regression Review of Matrices Outline Regression model in matrix form - Fall 03 Calculations using matrices Topic 7 Matrix Collection of elements arranged in
Multivariate Regression
Multivariate Regression The so-called supervised learning problem is the following: we want to approximate the random variable Y with an appropriate function of the random variables X 1,..., X p with the
Topic 3: Inference and Prediction
Topic 3: Inference and Prediction We ll be concerned here with testing more general hypotheses than those seen to date. Also concerned with constructing interval predictions from our regression model.
Economics 620, Lecture 15: Introduction to the Simultaneous Equations Model
Economics 620, Lecture 15: Introduction to the Simultaneous Equations Model Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 15: SEM I 1 / 16 The statistics framework
Chapter 6. Orthogonality
6.4 The Projection Matrix 1 Chapter 6. Orthogonality 6.4 The Projection Matrix Note. In Section 6.1 (Projections), we projected a vector b R n onto a subspace W of R n. We did so by finding a basis for
Inference in Regression Analysis
Inference in Regression Analysis Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 4, Slide 1 Today: Normal Error Regression Model Y i = β 0 + β 1 X i + ǫ i Y i value
Chapter 5 Matrix Approach to Simple Linear Regression
STAT 525 SPRING 2018 Chapter 5 Matrix Approach to Simple Linear Regression Professor Min Zhang Matrix Collection of elements arranged in rows and columns Elements will be numbers or symbols For example:
Economics 620, Lecture 13: Time Series I
Economics 620, Lecture 13: Time Series I Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 13: Time Series I 1 / 19 AUTOCORRELATION Consider y = X + u where y is
ECONOMET RICS P RELIM EXAM August 19, 2014 Department of Economics, Michigan State University
ECONOMET RICS P RELIM EXAM August 19, 2014 Department of Economics, Michigan State University Instructions: Answer all ve (5) questions. Be sure to show your work or provide su cient justi cation for your
Simple Linear Regression for the MPG Data
Simple Linear Regression for the MPG Data 2000 2500 3000 3500 15 20 25 30 35 40 45 Wgt MPG What do we do with the data? y i = MPG of i th car x i = Weight of i th car i =1,...,n n = Sample Size Exploratory
We begin by thinking about population relationships.
Conditional Expectation Function (CEF) We begin by thinking about population relationships. CEF Decomposition Theorem: Given some outcome Y i and some covariates X i there is always a decomposition where
Chapter 3. Matrices. 3.1 Matrices
40 Chapter 3 Matrices 3.1 Matrices Definition 3.1 Matrix) A matrix A is a rectangular array of m n real numbers {a ij } written as a 11 a 12 a 1n a 21 a 22 a 2n A =.... a m1 a m2 a mn The array has m rows
Econ 2120: Section 2
Econ 2120: Section 2 Part I - Linear Predictor Loose Ends Ashesh Rambachan Fall 2018 Outline Big Picture Matrix Version of the Linear Predictor and Least Squares Fit Linear Predictor Least Squares Omitted
EE5120 Linear Algebra: Tutorial 3, July-Dec
EE5120 Linear Algebra: Tutorial 3, July-Dec 2017-18 1. Let S 1 and S 2 be two subsets of a vector space V such that S 1 S 2. Say True/False for each of the following. If True, prove it. If False, justify
Topic 3: Inference and Prediction
Topic 3: Inference and Prediction We ll be concerned here with testing more general hypotheses than those seen to date. Also concerned with constructing interval predictions from our regression model.
Brief Suggested Solutions
DEPARTMENT OF ECONOMICS UNIVERSITY OF VICTORIA ECONOMICS 366: ECONOMETRICS II SPRING TERM 5: ASSIGNMENT TWO Brief Suggested Solutions Question One: Consider the classical T-observation, K-regressor linear
Heteroskedasticity. We now consider the implications of relaxing the assumption that the conditional
Heteroskedasticity We now consider the implications of relaxing the assumption that the conditional variance V (u i x i ) = σ 2 is common to all observations i = 1,..., In many applications, we may suspect
10-725/36-725: Convex Optimization Prerequisite Topics
10-725/36-725: Convex Optimization Prerequisite Topics February 3, 2015 This is meant to be a brief, informal refresher of some topics that will form building blocks in this course. The content of the
LECTURE 18: NONLINEAR MODELS
LECTURE 18: NONLINEAR MODELS The basic point is that smooth nonlinear models look like linear models locally. Models linear in parameters are no problem even if they are nonlinear in variables. For example:
Basic Distributional Assumptions of the Linear Model: 1. The errors are unbiased: E[ε] = The errors are uncorrelated with common variance:
![Basic Distributional Assumptions of the Linear Model: 1. The errors are unbiased: E[ε] = The errors are uncorrelated with common variance:](/thumbs/95/122989574.jpg "Basic Distributional Assumptions of the Linear Model: 1. The errors are unbiased: E[ε] = The errors are uncorrelated with common variance:")
8. PROPERTIES OF LEAST SQUARES ESTIMATES 1 Basic Distributional Assumptions of the Linear Model: 1. The errors are unbiased: E[ε] = 0. 2. The errors are uncorrelated with common variance: These assumptions
MS&E 226: Small Data. Lecture 6: Bias and variance (v2) Ramesh Johari
MS&E 226: Small Data Lecture 6: Bias and variance (v2) Ramesh Johari ramesh.johari@stanford.edu 1 / 47 Our plan today We saw in last lecture that model scoring methods seem to be trading o two di erent
Homoskedasticity. Var (u X) = σ 2. (23)
Homoskedasticity How big is the difference between the OLS estimator and the true parameter? To answer this question, we make an additional assumption called homoskedasticity: Var (u X) = σ 2. (23) This
EC3062 ECONOMETRICS. THE MULTIPLE REGRESSION MODEL Consider T realisations of the regression equation. (1) y = β 0 + β 1 x β k x k + ε,
THE MULTIPLE REGRESSION MODEL Consider T realisations of the regression equation (1) y = β 0 + β 1 x 1 + + β k x k + ε, which can be written in the following form: (2) y 1 y 2.. y T = 1 x 11... x 1k 1
Multiple Regression Analysis
Multiple Regression Analysis y = 0 + 1 x 1 + x +... k x k + u 6. Heteroskedasticity What is Heteroskedasticity?! Recall the assumption of homoskedasticity implied that conditional on the explanatory variables,
Notes on Time Series Modeling
Notes on Time Series Modeling Garey Ramey University of California, San Diego January 17 1 Stationary processes De nition A stochastic process is any set of random variables y t indexed by t T : fy t g
EE263: Introduction to Linear Dynamical Systems Review Session 2
EE263: Introduction to Linear Dynamical Systems Review Session 2 Basic concepts from linear algebra nullspace range rank and conservation of dimension EE263 RS2 1 Prerequisites We assume that you are familiar
Regression. Oscar García
Regression Oscar García Regression methods are fundamental in Forest Mensuration For a more concise and general presentation, we shall first review some matrix concepts 1 Matrices An order n m matrix is
Properties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
Lecture 05 Geometry of Least Squares
Lecture 05 Geometry of Least Squares 16 September 2015 Taylor B. Arnold Yale Statistics STAT 312/612 Goals for today 1. Geometry of least squares 2. Projection matrix P and annihilator matrix M 3. Multivariate
Quantitative Analysis of Financial Markets. Summary of Part II. Key Concepts & Formulas. Christopher Ting. November 11, 2017
Summary of Part II Key Concepts & Formulas Christopher Ting November 11, 2017 christopherting@smu.edu.sg http://www.mysmu.edu/faculty/christophert/ Christopher Ting 1 of 16 Why Regression Analysis? Understand
Economics 620, Lecture 19: Introduction to Nonparametric and Semiparametric Estimation
Economics 620, Lecture 19: Introduction to Nonparametric and Semiparametric Estimation Nicholas M. Kiefer Cornell University Professor N. M. Kiefer (Cornell University) Lecture 19: Nonparametric Analysis
M340L Final Exam Solutions May 13, 1995
M340L Final Exam Solutions May 13, 1995 Name: Problem 1: Find all solutions (if any) to the system of equations. Express your answer in vector parametric form. The matrix in other words, x 1 + 2x 3 + 3x
Lecture 12: Diagonalization
Lecture : Diagonalization A square matrix D is called diagonal if all but diagonal entries are zero: a a D a n 5 n n. () Diagonal matrices are the simplest matrices that are basically equivalent to vectors
Conceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
5.1 Consistency of least squares estimates. We begin with a few consistency results that stand on their own and do not depend on normality.
88 Chapter 5 Distribution Theory In this chapter, we summarize the distributions related to the normal distribution that occur in linear models. Before turning to this general problem that assumes normal
MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian.
MATH 304 Linear Algebra Lecture 10: Linear independence. Wronskian. Spanning set Let S be a subset of a vector space V. Definition. The span of the set S is the smallest subspace W V that contains S. If
The General Linear Model. How we re approaching the GLM. What you ll get out of this 8/11/16
8// The General Linear Model Monday, Lecture Jeanette Mumford University of Wisconsin - Madison How we re approaching the GLM Regression for behavioral data Without using matrices Understand least squares
Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam
Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system
Lecture 13: Simple Linear Regression in Matrix Format. 1 Expectations and Variances with Vectors and Matrices
Lecture 3: Simple Linear Regression in Matrix Format To move beyond simple regression we need to use matrix algebra We ll start by re-expressing simple linear regression in matrix form Linear algebra is
Math 321: Linear Algebra
Math 32: Linear Algebra T. Kapitula Department of Mathematics and Statistics University of New Mexico September 8, 24 Textbook: Linear Algebra,by J. Hefferon E-mail: kapitula@math.unm.edu Prof. Kapitula,
Chapter 11 Specification Error Analysis
Chapter Specification Error Analsis The specification of a linear regression model consists of a formulation of the regression relationships and of statements or assumptions concerning the explanator variables