Lecture 6: Geometry of OLS Estimation of Linear Regession


 Muriel Robinson
 1 years ago
 Views:
Transcription
1 Lecture 6: Geometry of OLS Estimation of Linear Regession Xuexin Wang WISE Oct / 22
2 Matrix Algebra An n m matrix A is a rectangular array that consists of nm elements arranged in n rows and m columns A typical element of A might be denoted by either A ij or a ij, where i = 1,, n and j = 1,, m If a matrix has only one column or only one row, it is called a vector There are two types of vectors, column vectors and row vectors If a matrix has the same number of columns and rows, it is said to be square A square matrix A is symmetric ifa ij = A ji for all i and j A square matrix is said to be diagonal if A ij = 0 for all i j The transpose of A is obtained by interchanging its row and column subscripts, denote A T or A 2 / 22
3 Arithmetic Operations on Matrices Addition and subtraction of matrices works exactly the way it does for scalars Matrix multiplication: A B = C n mm l AB BA except in special cases Identity matrix: I, IB = BI n l Assuming that the dimensions of the matrices are conformable for the various operations Distributive properties Associative properties A(B + C) = AB + AC (B + C)A = BA + CA (A + B) + C = A + (B + C) (AB)C = A(BC) 3 / 22
4 Transpose and Inverse (AB) T = B T A T If A is invertible, then it has an inverse matrix A 1 with the property that AA 1 = A 1 A = I If A is symmetric, then so is A 1 If A is triangular, then so is A 1 If an n n square matrix A is invertible, then its rank is n Such a matrix is said to have full rank If a square matrix does not have full rank, and therefore is not invertible, it is said to be singular for matrices that are not necessarily square, the rank is the largest number m for which an m m nonsingular matrix can be constructed by omitting some rows and some columns from the original matrix 4 / 22
5 Regression Models and Matrix Notation Model with one regressor y 1 = β 0 + β 0 x 11 + u 1 y 2 = β 0 + β 0 x 12 + u 2 y n = β 0 + β 0 x 1n + u n where y = Matrix notation: y 1 y 2 y n 1 x 11 1 x 12 y = Xβ + u,, X =, u = 1 x 1n u 1 u 2 u n, β = ( β0 β 1 ) 5 / 22
6 Multiple Regression Model Matrix notation: where y 1 y 2 y =, X = y n y = Xβ + u, x 11 x k1 x 12 x k2 x 1n x kn Incorporate the case of intercept, β = β 1 β k 6 / 22
7 Partitioned Matrices [ X = x 1 n k n 1 X = n k X = n k X 1 X 2 X n [ k1 x 2 n 1 x k n 1 1 k 1 k 1 k ] X 11 k 2 X 12 X 21 X 22 ], n 1 n 2 If two matrices A and B of the same dimensions are partitioned in exactly the same way, they can be added or subtracted block by block A + B = [ A 1 +B 1 A 2 +B 2 ] 7 / 22
8 OLS Estimation of One Linear Regressor Model First derivative: 1 n 1 n n (y i β 0 β 1 x 1i ) = 0 i=1 n x 1i (y i β 0 β 1 x 1i ) = 0 i=1 [ n n i=1 x 1i Matrix form n i=1 x 1i n i=1 x2 1i ] [ β0 β 1 ] = X T Xβ = X T y [ n i=1 y ] i n i=1 x 1iy i ˆβ = ( X T X ) 1 X T y 8 / 22
9 OLS Estimation of Multiple Linear Regressor Model SSR(β) = (y Xβ) T (y Xβ) First Derivative X T (y Xβ) = 0 Remark: Method of Moments E (x i u) = 0, i = 1,, k X T Xβ = X T y 9 / 22
10 The Geometry of Linear Regression: Introduction n observations of a linear regression model with k regressors y = Xβ + u, where y and u are n vectors, X is an n k matrix ˆβ = ( X T X ) 1 X T y Numerical properties of these OLS estimates they have nothing to do with how the data were actually generated Euclidean geometry 10 / 22
11 The Geometry of Vector Spaces an n vector was defined as a column vector with n elements, that is, an n 1 matrix Euclidean space in n dimensions, which we will denote as E n Scalar or inner product: For any two vectors x, y E n, their scalar product is x, y = x T y Comutative: x, y = y, x norm of a vector x is x = ( x T x ) 1/2 11 / 22
12 Vector Geometry in Two Dimensions Cartesian coordinates: x = (x 1, x 2 ), y = (y 1, y 2 ) Adding: x + y O A C B x x y x + y y x 2 y 2 y 1 x 1 addition of vectors 12 / 22
13 The Geometry of Scalar Products Multiplying: αx = α x x, y = x y cos θ cos θ = 0, θ = π 2, x, y are said to be orthogonal CauchySchwartz inequality: x, y x y 13 / 22
14 Subspaces of Euclidean Space Defining a subspace of E n is in terms of a set of basis vectors A subspace that is of particular interest to us is the one for which the columns of X provide the basis vectors We may denote the k columns of X as x 1, x 2,, x k Then the subspace associated with these k basis vectors will be denoted by (X) or (x 1, x 2,, x k ) { } k (x 1, x 2,, x k ) z E n z = b i x i, b i R The subspace defined above is called the subspace spanned by the x 1, x 2,, x k or the column space of X The orthogonal complement of (X) is denoted as (X) i=1 (X) { w E n w T z =0, z (X) } If the dimension of (X) is k, then the dimension of (X) is n k 14 / 22
15 The Geometry of OLS Estimation ] X = [x 1 x 2 x k ] Xβ = [x 1 x 2 x k β 1 β 2 β k = k β i x i i=1 OLS estimator ˆβ ( X T y X ˆˆβ ) = 0 15 / 22
16 The Geometry of OLS Estimation Pythagoras Theorem y 2 = X ˆβ 2 + û 2 y T y = ˆβ T X T X ˆβ + û T û T SS = ESS + SSR O y θ û X ˆβ Residuals and fitted values 16 / 22
17 The Geometry of OLS Estimation x 2 O θ y X ˆβ B û A S(x 1, x 2) x 1 a) y projected on two regressors x 2 A ˆβ 2x 2 X ˆβ O ˆβ 1x 1 x 1 O θ y X ˆβ B û A b) The span S(x 1, x 2) of the regressors c) The vertical plane through y Linear regression in three dimensions 17 / 22
18 Orthogonal Projections A projection is a mapping that takes each point of E n into a point in a subspace of E n, while leaving all points in that subspace unchanged An orthogonal projection maps any point into the point of the subspace that is closest to it OLS is an example of orthogonal projection Projection matrix P X = X ( X T X ) 1 X T M X = I X ( X T X ) 1 X T = I P X X ˆβ = P X y M X y = û 18 / 22
19 Orthogonal Projections P X P X = P X, M X M X = M X The pair of projections P X and M X are said to be complementary projections, since the sum of P X y and M X y restores the original vector y y 2 = P X y 2 + M X y 2 P X y y P Z would be the matrix that projects on to (Z), P X,W would be the matrix that projects of (X, W) 19 / 22
20 Linear Transformations of Regressors Nonsingular linear transformation: k k A XA = X [ a 1 a 2 ] [ a k = Xa1 Xa 2 ] Xa k (X) = (XA) Xβ = XAA 1 β Fitted values and residuals are invariant to any nonsingular linear transformation of the columns of X, even though ˆβ will change Special Case: Units of measurement of the regressors 20 / 22
21 The FrischWaughLovell Theorem Two Groups of Regressors: y = X 1 β 1 + X 2 β 2 + u, where X 1 is n k 1 matrix, X 2 is n k 2 matrix, X = [ X 1 X 2 ] with k = k1 + k 2 X T 2 X 1 = O : OLS estimator of β 2 in y = X 2 β 2 + u 2 is the same as the OLS estimator β 2 in y = X 1 β 1 + X 2 β 2 + u(second condition in the omitted variable bias) P 1 = P X1 = X 1 ( X T 1 X 1 ) 1 X T 1 P 1 P X = P X P 1 = P 1 M 1 = I P 1 M 1 M X = M X 21 / 22
22 The FrischWaughLovell Theorem Consider two regression model y = X 1 β 1 + X 2 β 2 + u M 1 y = M 1 X 2 β 2 + residual Theorem (The FrischWaughLovell Theorem) The OLS estimates of β 2 from the two regressions above are numerically identical The residuals from regressions above are numerically identical 22 / 22
The Geometry of Linear Regression
Chapter 2 The Geometry of Linear Regression 21 Introduction In Chapter 1, we introduced regression models, both linear and nonlinear, and discussed how to estimate linear regression models by using the
More informationMath Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
More information. a m1 a mn. a 1 a 2 a = a n
Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by
More informationLinear Algebra V = T = ( 4 3 ).
Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a dimensional column vector and V is a 5dimensional
More informationPhys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
More informationLecture 3: Matrix and Matrix Operations
Lecture 3: Matrix and Matrix Operations Representation, row vector, column vector, element of a matrix. Examples of matrix representations Tables and spreadsheets ScalarMatrix operation: Scaling a matrix
More informationCS100: DISCRETE STRUCTURES. Lecture 3 Matrices Ch 3 Pages:
CS100: DISCRETE STRUCTURES Lecture 3 Matrices Ch 3 Pages: 246262 Matrices 2 Introduction DEFINITION 1: A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called an m x n
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A pdimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationQuantum Computing Lecture 2. Review of Linear Algebra
Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces
More informationMath Bootcamp An pdimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An pdimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationSTAT 350: Geometry of Least Squares
The Geometry of Least Squares Mathematical Basics Inner / dot product: a and b column vectors a b = a T b = a i b i a b a T b = 0 Matrix Product: A is r s B is s t (AB) rt = s A rs B st Partitioned Matrices
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationMatrix & Linear Algebra
Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84 Vectors Vectors Vector: A
More informationIn the bivariate regression model, the original parameterization is. Y i = β 1 + β 2 X2 + β 2 X2. + β 2 (X 2i X 2 ) + ε i (2)
RNy, econ460 autumn 04 Lecture note Orthogonalization and reparameterization 5..3 and 7.. in HN Orthogonalization of variables, for example X i and X means that variables that are correlated are made
More informationLinear Models Review
Linear Models Review Vectors in IR n will be written as ordered ntuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign
More informationMatrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices
Matrices A. Fabretti Mathematics 2 A.Y. 2015/2016 Table of contents Matrix Algebra Determinant Inverse Matrix Introduction A matrix is a rectangular array of numbers. The size of a matrix is indicated
More informationMatrix Algebra. Matrix Algebra. Chapter 8  S&B
Chapter 8  S&B Algebraic operations Matrix: The size of a matrix is indicated by the number of its rows and the number of its columns. A matrix with k rows and n columns is called a k n matrix. The number
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationReference: Davidson and MacKinnon Ch 2. In particular page
RNy, econ460 autumn 03 Lecture note Reference: Davidson and MacKinnon Ch. In particular page 578. Projection matrices The matrix M I X(X X) X () is often called the residual maker. That nickname is easy
More informationLinear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations.
POLI 7  Mathematical and Statistical Foundations Prof S Saiegh Fall Lecture Notes  Class 4 October 4, Linear Algebra The analysis of many models in the social sciences reduces to the study of systems
More informationChapter 4  MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in
Chapter 4  MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661  Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationLinear Algebra, Vectors and Matrices
Linear Algebra, Vectors and Matrices Prof. Manuela Pedio 20550 Quantitative Methods for Finance August 2018 Outline of the Course Lectures 1 and 2 (3 hours, in class): Linear and nonlinear functions on
More informationMultiplying matrices by diagonal matrices is faster than usual matrix multiplication.
76 Multiplying matrices by diagonal matrices is faster than usual matrix multiplication. The following equations generalize to matrices of any size. Multiplying a matrix from the left by a diagonal matrix
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationLecture 3 Linear Algebra Background
Lecture 3 Linear Algebra Background Dan Sheldon September 17, 2012 Motivation Preview of next class: y (1) w 0 + w 1 x (1) 1 + w 2 x (1) 2 +... + w d x (1) d y (2) w 0 + w 1 x (2) 1 + w 2 x (2) 2 +...
More informationLinear Algebra Review
Linear Algebra Review Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Linear Algebra Review 1 / 45 Definition of Matrix Rectangular array of elements arranged in rows and
More informationPOLI270  Linear Algebra
POLI7  Linear Algebra Septemer 8th Basics a x + a x +... + a n x n b () is the linear form where a, b are parameters and x n are variables. For a given equation such as x +x you only need a variable and
More informationMatrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix
Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Matrix Operations Matrix Addition and Matrix Scalar Multiply Matrix Multiply Matrix
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationVectors and matrices: matrices (Version 2) This is a very brief summary of my lecture notes.
Vectors and matrices: matrices (Version 2) This is a very brief summary of my lecture notes Matrices and linear equations A matrix is an mbyn array of numbers A = a 11 a 12 a 13 a 1n a 21 a 22 a 23 a
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 WeiTa Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationElementary Row Operations on Matrices
King Saud University September 17, 018 Table of contents 1 Definition A real matrix is a rectangular array whose entries are real numbers. These numbers are organized on rows and columns. An m n matrix
More informationAppendix A: Matrices
Appendix A: Matrices A matrix is a rectangular array of numbers Such arrays have rows and columns The numbers of rows and columns are referred to as the dimensions of a matrix A matrix with, say, 5 rows
More informationELEMENTS OF MATRIX ALGEBRA
ELEMENTS OF MATRIX ALGEBRA CHUNGMING KUAN Department of Finance National Taiwan University September 09, 2009 c ChungMing Kuan, 1996, 2001, 2009 Email: ckuan@ntuedutw; URL: homepagentuedutw/ ckuan CONTENTS
More information1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )
Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems PerOlof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical
More informationLecture 20: 6.1 Inner Products
Lecture 0: 6.1 Inner Products WeiTa Chu 011/1/5 Definition An inner product on a real vector space V is a function that associates a real number u, v with each pair of vectors u and v in V in such a way
More informationHomework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I)
CS 106 Spring 2004 Homework 1 Elena Davidson 8 April 2004 Problem 1.1 Let B be a 4 4 matrix to which we apply the following operations: 1. double column 1, 2. halve row 3, 3. add row 3 to row 1, 4. interchange
More informationMatrices and Vectors
Matrices and Vectors James K. Peterson Department of Biological Sciences and Department of Mathematical Sciences Clemson University November 11, 2013 Outline 1 Matrices and Vectors 2 Vector Details 3 Matrix
More informationChapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations
Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2
More informationLinear Algebra March 16, 2019
Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented
More informationELE/MCE 503 Linear Algebra Facts Fall 2018
ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2
More informationMathematics. EC / EE / IN / ME / CE. for
Mathematics for EC / EE / IN / ME / CE By www.thegateacademy.com Syllabus Syllabus for Mathematics Linear Algebra: Matrix Algebra, Systems of Linear Equations, Eigenvalues and Eigenvectors. Probability
More informationMatrices and Vectors. Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A =
30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J Olver 3 Review of Matrix Algebra Vectors and matrices are essential for modern analysis of systems of equations algebrai, differential, functional, etc In this
More informationMatrices Gaussian elimination Determinants. Graphics 2009/2010, period 1. Lecture 4: matrices
Graphics 2009/2010, period 1 Lecture 4 Matrices m n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in
More informationLecture 23: 6.1 Inner Products
Lecture 23: 6.1 Inner Products WeiTa Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such
More informationLecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,
2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for
More informationMAT Linear Algebra Collection of sample exams
MAT 342  Linear Algebra Collection of sample exams Ax. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationIndex. book 2009/5/27 page 121. (Page numbers set in bold type indicate the definition of an entry.)
page 121 Index (Page numbers set in bold type indicate the definition of an entry.) A absolute error...26 componentwise...31 in subtraction...27 normwise...31 angle in least squares problem...98,99 approximation
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationSection 12.4 Algebra of Matrices
244 Section 2.4 Algebra of Matrices Before we can discuss Matrix Algebra, we need to have a clear idea what it means to say that two matrices are equal. Let's start a definition. Equal Matrices Two matrices
More informationElementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
More informationMatrices. Chapter Definitions and Notations
Chapter 3 Matrices 3. Definitions and Notations Matrices are yet another mathematical object. Learning about matrices means learning what they are, how they are represented, the types of operations which
More informationLinear Algebra and Matrix Inversion
Jim Lambers MAT 46/56 Spring Semester 29 Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much
More informationn n matrices The system of m linear equations in n variables x 1, x 2,..., x n can be written as a matrix equation by Ax = b, or in full
n n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in n variables x 1, x 2,..., x n a 11 x 1 + a 12 x
More information10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )
c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.27.4) 1. When each of the functions F 1, F 2,..., F n in righthand side
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationINNER PRODUCT SPACE. Definition 1
INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of
More informationLINEAR ALGEBRA REVIEW
LINEAR ALGEBRA REVIEW JC Stuff you should know for the exam. 1. Basics on vector spaces (1) F n is the set of all ntuples (a 1,... a n ) with a i F. It forms a VS with the operations of + and scalar multiplication
More informationMatrix operations Linear Algebra with Computer Science Application
Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the
More informationLecture II: Linear Algebra Revisited
Lecture II: Linear Algebra Revisited Overview Vector spaces, Hilbert & Banach Spaces, etrics & Norms atrices, Eigenvalues, Orthogonal Transformations, Singular Values Operators, Operator Norms, Function
More informationMatrix Basic Concepts
Matrix Basic Concepts Topics: What is a matrix? Matrix terminology Elements or entries Diagonal entries Address/location of entries Rows and columns Size of a matrix A column matrix; vectors Special types
More informationMATH 304 Linear Algebra Lecture 18: Orthogonal projection (continued). Least squares problems. Normed vector spaces.
MATH 304 Linear Algebra Lecture 18: Orthogonal projection (continued). Least squares problems. Normed vector spaces. Orthogonality Definition 1. Vectors x,y R n are said to be orthogonal (denoted x y)
More informationORTHOGONALITY AND LEASTSQUARES [CHAP. 6]
ORTHOGONALITY AND LEASTSQUARES [CHAP. 6] Inner products and Norms Inner product or dot product of 2 vectors u and v in R n : u.v = u 1 v 1 + u 2 v 2 + + u n v n Calculate u.v when u = 1 2 2 0 v = 1 0
More informationMatrices and Determinants
Chapter1 Matrices and Determinants 11 INTRODUCTION Matrix means an arrangement or array Matrices (plural of matrix) were introduced by Cayley in 1860 A matrix A is rectangular array of m n numbers (or
More informationLecture 7: Vectors and Matrices II Introduction to Matrices (See Sections, 3.3, 3.6, 3.7 and 3.9 in Boas)
Lecture 7: Vectors and Matrices II Introduction to Matrices (See Sections 3.3 3.6 3.7 and 3.9 in Boas) Here we will continue our discussion of vectors and their transformations. In Lecture 6 we gained
More informationECS130 Scientific Computing. Lecture 1: Introduction. Monday, January 7, 10:00 10:50 am
ECS130 Scientific Computing Lecture 1: Introduction Monday, January 7, 10:00 10:50 am About Course: ECS130 Scientific Computing Professor: Zhaojun Bai Webpage: http://web.cs.ucdavis.edu/~bai/ecs130/ Today
More informationMATRIX ALGEBRA. or x = (x 1,..., x n ) R n. y 1 y 2. x 2. x m. y m. y = cos θ 1 = x 1 L x. sin θ 1 = x 2. cos θ 2 = y 1 L y.
as Basics Vectors MATRIX ALGEBRA An array of n real numbers x, x,, x n is called a vector and it is written x = x x n or x = x,, x n R n prime operation=transposing a column to a row Basic vector operations
More informationA matrix over a field F is a rectangular array of elements from F. The symbol
Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F ) denotes the collection of all m n matrices over F Matrices will usually be denoted
More information4 Linear Algebra Review
4 Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the course 41 Vectors and Matrices For the context of data analysis,
More informationRaphael Mrode. Training in quantitative genetics and genomics 30 May 10 June 2016 ILRI, Nairobi. Partner Logo. Partner Logo
Basic matrix algebra Raphael Mrode Training in quantitative genetics and genomics 3 May June 26 ILRI, Nairobi Partner Logo Partner Logo Matrix definition A matrix is a rectangular array of numbers set
More informationMATH2210 Notebook 2 Spring 2018
MATH2210 Notebook 2 Spring 2018 prepared by Professor Jenny Baglivo c Copyright 2009 2018 by Jenny A. Baglivo. All Rights Reserved. 2 MATH2210 Notebook 2 3 2.1 Matrices and Their Operations................................
More informationIntroduc)on to linear algebra
Introduc)on to linear algebra Vector A vector, v, of dimension n is an n 1 rectangular array of elements v 1 v v = 2 " v n % vectors will be column vectors. They may also be row vectors, when transposed
More informationorthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis,
5 Orthogonality Goals: We use scalar products to find the length of a vector, the angle between 2 vectors, projections, orthogonal relations between vectors and subspaces Then we study some applications
More informationGlossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB
Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the
More informationChapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form
Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1
More informationSystems of Linear Equations and Matrices
Chapter 1 Systems of Linear Equations and Matrices System of linear algebraic equations and their solution constitute one of the major topics studied in the course known as linear algebra. In the first
More informationLecture Notes in Linear Algebra
Lecture Notes in Linear Algebra Dr. Abdullah AlAzemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................
More informationIntroduction. Vectors and Matrices. Vectors [1] Vectors [2]
Introduction Vectors and Matrices Dr. TGI Fernando 1 2 Data is frequently arranged in arrays, that is, sets whose elements are indexed by one or more subscripts. Vector  one dimensional array Matrix 
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationDSGA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DSGA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationMatrix Operations: Determinant
Matrix Operations: Determinant Determinants Determinants are only applicable for square matrices. Determinant of the square matrix A is denoted as: det(a) or A Recall that the absolute value of the determinant
More informationReview Packet 1 B 11 B 12 B 13 B = B 21 B 22 B 23 B 31 B 32 B 33 B 41 B 42 B 43
Review Packet. For each of the following, write the vector or matrix that is specified: a. e 3 R 4 b. D = diag{, 3, } c. e R 3 d. I. For each of the following matrices and vectors, give their dimension.
More information(, ) : R n R n R. 1. It is bilinear, meaning it s linear in each argument: that is
17 Inner products Up until now, we have only examined the properties of vectors and matrices in R n. But normally, when we think of R n, we re really thinking of ndimensional Euclidean space  that is,
More informationELEMENTARY LINEAR ALGEBRA
ELEMENTARY LINEAR ALGEBRA K R MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND First Printing, 99 Chapter LINEAR EQUATIONS Introduction to linear equations A linear equation in n unknowns x,
More informationSection I. Define or explain the following terms (3 points each) 1. centered vs. uncentered 2 R  2. Frisch theorem 
First Exam: Economics 388, Econometrics Spring 006 in R. Butler s class YOUR NAME: Section I (30 points) Questions 110 (3 points each) Section II (40 points) Questions 1115 (10 points each) Section III
More informationIntroduction to Matrices
POLS 704 Introduction to Matrices Introduction to Matrices. The Cast of Characters A matrix is a rectangular array (i.e., a table) of numbers. For example, 2 3 X 4 5 6 (4 3) 7 8 9 0 0 0 Thismatrix,with4rowsand3columns,isoforder
More informationPOL 213: Research Methods
Brad 1 1 Department of Political Science University of California, Davis April 15, 2008 Some Matrix Basics What is a matrix? A rectangular array of elements arranged in rows and columns. 55 900 0 67 1112
More informationMath 360 Linear Algebra Fall Class Notes. a a a a a a. a a a
Math 360 Linear Algebra Fall 2008 91008 Class Notes Matrices As we have already seen, a matrix is a rectangular array of numbers. If a matrix A has m columns and n rows, we say that its dimensions are
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationIntroduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX
Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX September 2007 MSc Sep Intro QT 1 Who are these course for? The September
More informationNumerical Linear Algebra
Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an nvector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix
More informationMATH 106 LINEAR ALGEBRA LECTURE NOTES
MATH 6 LINEAR ALGEBRA LECTURE NOTES FALL  These Lecture Notes are not in a final form being still subject of improvement Contents Systems of linear equations and matrices 5 Introduction to systems of
More informationUndergraduate Mathematical Economics Lecture 1
Undergraduate Mathematical Economics Lecture 1 Yu Ren WISE, Xiamen University September 15, 2014 Outline 1 Courses Description and Requirement 2 Course Outline ematical techniques used in economics courses
More informationChapter 2. Linear Algebra. rather simple and learning them will eventually allow us to explain the strange results of
Chapter 2 Linear Algebra In this chapter, we study the formal structure that provides the background for quantum mechanics. The basic ideas of the mathematical machinery, linear algebra, are rather simple
More information