Linear Algebra Review
|
|
- Lester Murphy
- 5 years ago
- Views:
Transcription
1 Linear Algebra Review Yang Feng Yang Feng (Columbia University) Linear Algebra Review 1 / 45
2 Definition of Matrix Rectangular array of elements arranged in rows and columns A matrix has dimensions The dimension of a matrix is its number of rows and columns It is expressed as 3 2 (in this case) Yang Feng (Columbia University) Linear Algebra Review 2 / 45
3 Indexing a Matrix Rectangular array of elements arranged in rows and columns a11 a A = 12 a 13 a 21 a 22 a 23 A matrix can also be notated A = [a ij ], i = 1, 2; j = 1, 2, 3 Yang Feng (Columbia University) Linear Algebra Review 3 / 45
4 Square Matrix and Column Vector A square matrix has equal number of rows and columns a a 12 a 13 a a 22 a 23 a 31 a 32 a 33 A column vector is a matrix with a single column c 1 c 2 c 3 c 4 c 5 All vectors (row or column) are matrices, all scalars are 1 1 matrices. Yang Feng (Columbia University) Linear Algebra Review 4 / 45
5 Transpose The transpose of a matrix is another matrix in which the rows and columns have been interchanged 2 5 A = A = Yang Feng (Columbia University) Linear Algebra Review 5 / 45
6 Equality of Matrices Two matrices are the same if they have the same dimension and all the elements are equal a 1 4 A = a 2 B = 7 a 3 3 A = B implies a 1 = 4, a 2 = 7, a 3 = 3 Yang Feng (Columbia University) Linear Algebra Review 6 / 45
7 Matrix Addition and Substraction Then A = 2 5 B = A + B = Yang Feng (Columbia University) Linear Algebra Review 7 / 45
8 Multiplication of a Matrix by a Scalar 2 7 A = k ka = k = 9 3 9k 7k 3k Yang Feng (Columbia University) Linear Algebra Review 8 / 45
9 Multiplication of two Matrices where (AB) ij = for i = 1,, l and j = 1,, n. A l m B m n = (AB) l n, m A ik B jk, k=1 Yang Feng (Columbia University) Linear Algebra Review 9 / 45
10 Another Matrix Multiplication Example A = AB = B = = 41 2 Yang Feng (Columbia University) Linear Algebra Review 10 / 45
11 Special Matrices If A = A, then A is a symmetric matrix A = A = If the off-diagonal elements of a matrix are all zeros it is then called a diagonal matrix a A = 0 a 2 0 B = a Yang Feng (Columbia University) Linear Algebra Review 11 / 45
12 Identity Matrix A diagonal matrix whose diagonal entries are all ones is an identity matrix. Multiplication by an identity matrix leaves the pre or post multiplied matrix unchanged I = and AI = IA = A Yang Feng (Columbia University) Linear Algebra Review 12 / 45
13 Vector and matrix with all elements equal to one n =..... J n = n n n n 1 n = n 1 = n 1 n n = J n Yang Feng (Columbia University) Linear Algebra Review 13 / 45
14 Linear Dependence and Rank of Matrix Consider A = and think of this as a matrix of a collection of column vectors. Note that the third column vector is a multiple of the first column vector. Yang Feng (Columbia University) Linear Algebra Review 14 / 45
15 Linear Dependence When m scalars k 1,..., k m not all zero, can be found such that: k 1 A k m A m = 0 where 0 denotes the zero column vector and A i is the i th column of matrix A, the m column vectors are called linearly dependent. If the only set of scalars for which the equality holds is k 1 = 0,..., k m = 0, the set of m column vectors is linearly independent. In the previous example matrix the columns are linearly dependent = Yang Feng (Columbia University) Linear Algebra Review 15 / 45
16 Rank of Matrix The rank of a matrix is defined to be the maximum number of linearly independent columns in the matrix. Rank properties include The rank of a matrix is unique The rank of a matrix can equivalently be defined as the maximum number of linearly independent rows The rank of an r c matrix cannot exceed min(r, c) The row and column rank of a matrix are equal The rank of a matrix is preserved under nonsingular transformations., i.e. Let A (n n) and C (k k) be nonsingular matrices. Then for any n k matrix B we have rank(b) = rank(ab) = rank(bc) Yang Feng (Columbia University) Linear Algebra Review 16 / 45
17 Inverse of Matrix Like a reciprocal 6 1/6 = 1/6 6 = 1 x 1 x = 1 But for matrices AA 1 = A 1 A = I Yang Feng (Columbia University) Linear Algebra Review 17 / 45
18 Example More generally, where D = ad bc 2 4 A = 3 1 A =.3.2 A A = 0 1 a b A = c d A 1 = 1 d b D c a Yang Feng (Columbia University) Linear Algebra Review 18 / 45
19 Inverses of Diagonal Matrices are Easy then A = /3 0 0 A 1 = 0 1/ /2 Yang Feng (Columbia University) Linear Algebra Review 19 / 45
20 Finding the inverse Finding an inverse takes (for general matrices with no special structure) O(n 3 ) operations (when n is the number of rows in the matrix) We will assume that numerical packages can do this for us in R: solve(a) gives the inverse of matrix A Yang Feng (Columbia University) Linear Algebra Review 20 / 45
21 Example Solving a system of simultaneous equations 2y 1 + 4y 2 = 20 3y 1 + y 2 = 10 y1 = y1 y 2 = y Yang Feng (Columbia University) Linear Algebra Review 21 / 45
22 List of Useful Matrix Properties A + B = B + A (A + B) + C = A + (B + C) (AB)C = A(BC) C(A + B) = CA + CB k(a + B) = ka + kb (A ) = A (A + B) = A + B (AB) = B A (ABC) = C B A (AB) 1 = B 1 A 1 (ABC) 1 = C 1 B 1 A 1 (A 1 ) 1 = A, (A ) 1 = (A 1 ) Yang Feng (Columbia University) Linear Algebra Review 22 / 45
23 Random Vectors and Matrices Let s say we have a vector consisting of three random variables Y 1 Y = Y 2 Y 3 The expectation of a random vector is defined as E(Y 1 ) E(Y) = E(Y 2 ) E(Y 3 ) Yang Feng (Columbia University) Linear Algebra Review 23 / 45
24 Expectation of a Random Matrix The expectation of a random matrix is defined similarly E(Y) = [E(Y ij )] i = 1,...n; j = 1,..., p Yang Feng (Columbia University) Linear Algebra Review 24 / 45
25 Variance-covariance Matrix of a Random Vector The variances of three random variables σ 2 (Y i ) and the covariances between any two of the three random variables σ(y i, Y j ), are assembled in the variance-covariance matrix of Y σ 2 (Y 1 ) σ(y 1, Y 2 ) σ(y 1, Y 3 ) cov(y) = σ 2 {Y} = σ(y 2, Y 1 ) σ 2 (Y 2 ) σ(y 2, Y 3 ) σ(y 3, Y 1 ) σ(y 3, Y 2 ) σ 2 (Y 3 ) remember σ(y 2, Y 1 ) = σ(y 1, Y 2 ) so the covariance matrix is symmetric Yang Feng (Columbia University) Linear Algebra Review 25 / 45
26 Derivation of Covariance Matrix In vector terms the variance-covariance matrix is defined by σ 2 {Y} = E[(Y E(Y))(Y E(Y)) ] because Y 1 E(Y 1 ) σ 2 {Y} = E( Y 2 E(Y 2 ) Y 1 E(Y 1 ) Y 2 E(Y 2 ) Y 3 E(Y 3 ) ) Y 3 E(Y 3 ) Yang Feng (Columbia University) Linear Algebra Review 26 / 45
27 Regression Example Take a regression example with n = 3 with constant error terms σ 2 ( i ) and are uncorrelated so that σ 2 ( i, j ) = 0 for all i = j The variance-covariance matrix for the random vector is σ σ 2 () = 0 σ σ 2 which can be written as σ 2 {} = σ 2 I Yang Feng (Columbia University) Linear Algebra Review 27 / 45
28 Basic Results If A is a constant matrix and Y is a random matrix then W = AY is a random matrix E(A) = A E(W) = E(AY) = A E(Y) σ 2 {W} = σ 2 {AY} = Aσ 2 {Y}A Yang Feng (Columbia University) Linear Algebra Review 28 / 45
29 Multivariate Normal Density Let Y be a vector of p observations Y = (Y 1, Y 2,, Y p ) Let µ be a vector of the means of each of the p observations µ = (µ 1, µ 2,, µ p ) Yang Feng (Columbia University) Linear Algebra Review 29 / 45
30 Multivariate Normal Density let Σ be the variance-covariance matrix of Y σ 2 1 σ σ 1p σ 21 σ σ 2p Σ =... σ p1 σ p2... σp 2 Then the multivariate normal density is given by P(Y µ, Σ) = 1 (2π) p/2 Σ 1/2 exp[ 1 2 (Y µ) Σ 1 (Y µ)] Yang Feng (Columbia University) Linear Algebra Review 30 / 45
31 Matrix Simple Linear Regression Nothing new-only matrix formalism for previous results Remember the normal error regression model Y i = β 0 + β 1 X i + i, i N(0, σ 2 ), i = 1,..., n Expanded out this looks like Y 1 = β 0 + β 1 X Y 2 = β 0 + β 1 X Y n = β 0 + β 1 X n + n which points towards an obvious matrix formulation. Yang Feng (Columbia University) Linear Algebra Review 31 / 45
32 Regression Matrices If we identify the following matrices Y 1 Y 2 Y =... Y n 1 X 1 1 X 2 X =... 1 X n β = β0 β =... n We can write the linear regression equations in a compact form Y = Xβ + Yang Feng (Columbia University) Linear Algebra Review 32 / 45
33 Regression Matrices Of course, in the normal regression model the expected value of each of the s is zero, we can write E(Y) = Xβ This is because E() = 0 E( 1 ) E( 2 )... E( n ) = Yang Feng (Columbia University) Linear Algebra Review 33 / 45
34 Error Covariance Because the error terms are independent and have constant variance σ 2 σ σ 2 {} = 0 σ σ 2 σ 2 {} = σ 2 I Yang Feng (Columbia University) Linear Algebra Review 34 / 45
35 Matrix Normal Regression Model In matrix terms the normal regression model can be written as Y = Xβ + where N(0, σ 2 I) Yang Feng (Columbia University) Linear Algebra Review 35 / 45
36 Least Square Estimation If we remember both the starting normal equations that we derived and the fact that nb 0 + b 1 Xi = Y i b 0 Xi + b 1 X 2 i = X i Y i 1 X 1 X X 2 X = X 1 X 2... X n.... = n Xi Xi X 2 i 1 X n Y 1 X Y 2 Y = X 1 X 2... X n.. = Y n Yi Xi Y i Yang Feng (Columbia University) Linear Algebra Review 36 / 45
37 Least Square Estimation Then we can see that these equations are equivalent to the following matrix operations with X X b = X Y b = b0 b 1 with the solution to this equation given by when (X X) 1 exists. b = (X X) 1 X Y Yang Feng (Columbia University) Linear Algebra Review 37 / 45
38 Fitted Value Ŷ = Xb Because: Ŷ = Ŷ 1 Ŷ 2... Ŷ n 1 X 1 1 X 2 = X n b0 b 1 b 0 + b 1 X 1 b 0 + b 1 X 2 =... b 0 + b 1 X n Yang Feng (Columbia University) Linear Algebra Review 38 / 45
39 Fitted Values, Hat Matrix plug in b = (X X) 1 X Y We have Ŷ = Xb = X(X X) 1 X Y or where Ŷ = HY H = X(X X) 1 X is called the hat matrix. Property of hat matrix H: 1 symmetric 2 idempotent: HH = H. Yang Feng (Columbia University) Linear Algebra Review 39 / 45
40 Residuals Then e = Y Ŷ = Y HY = (I H)Y e = (I H)Y The matrix I H is also symmetric and idempotent. The variance-covariance matrix of e is And we can estimate it by σ 2 {e} = σ 2 (I H) s 2 {e} = MSE(I H) Yang Feng (Columbia University) Linear Algebra Review 40 / 45
41 Analysis of Variance Results We know SSTO = (Y i Ȳ )2 = Y 2 i ( Y i ) 2 n Y Y = Y 2 i and J is the matrix with entries all equal to 1. Then we have As a result: ( Y i ) 2 n = 1 n Y JY SSTO = Y Y 1 n Y JY Yang Feng (Columbia University) Linear Algebra Review 41 / 45
42 Analysis of Variance Results Also, SSE = e 2 i = (Y i Ŷi) 2 can be represented as SSE = e e = Y (I H) (I H)Y = Y (I H)Y Notice that H1 = 1, then (I H)J = 0 Finally by similarly reasoning, SSR = ([H 1 n J]Y) ([H 1 n J]Y) = Y [H 1 n J]Y Easy to check that SSTO = SSE + SSR Yang Feng (Columbia University) Linear Algebra Review 42 / 45
43 Sums of Squares as Quadratic Forms When n = 2, an example of quadratic forms: 5Y Y 1 Y 2 + 4Y 2 2 can be expressed as matrix term as 5 3 Y1 Y1 Y 2 = Y AY 3 4 In general, a quadratic term is defined as : Y 2 Y AY = n n A ij Y i Y j i=1 j=1 where A ij = A ji Here, A is a symmetric n n matrix, the matrix of the quadratic form. Yang Feng (Columbia University) Linear Algebra Review 43 / 45
44 Quadratic forms for ANOVA SSTO = Y [I 1 n J]Y SSE = Y [I H]Y SSR = Y [H 1 n J]Y Yang Feng (Columbia University) Linear Algebra Review 44 / 45
45 Inference in Regression Analysis Regression Coefficients: The variance-covariance matrix of b is σ 2 {b} = σ 2 (X X) 1 Mean Response: To estimate the mean response at X h, define 1 X h = Then X h Ŷ h = X h b And the variance-covariance matrix of Ŷh is σ 2 {Ŷ h } = X h σ2 {b}x h = σ 2 X h (X X) 1 X h Prediction of New Observation: s 2 {pred} = MSE(1 + X h (X X) 1 X h ) Yang Feng (Columbia University) Linear Algebra Review 45 / 45
Chapter 5 Matrix Approach to Simple Linear Regression
STAT 525 SPRING 2018 Chapter 5 Matrix Approach to Simple Linear Regression Professor Min Zhang Matrix Collection of elements arranged in rows and columns Elements will be numbers or symbols For example:
More informationTopic 7 - Matrix Approach to Simple Linear Regression. Outline. Matrix. Matrix. Review of Matrices. Regression model in matrix form
Topic 7 - Matrix Approach to Simple Linear Regression Review of Matrices Outline Regression model in matrix form - Fall 03 Calculations using matrices Topic 7 Matrix Collection of elements arranged in
More informationMatrix Approach to Simple Linear Regression: An Overview
Matrix Approach to Simple Linear Regression: An Overview Aspects of matrices that you should know: Definition of a matrix Addition/subtraction/multiplication of matrices Symmetric/diagonal/identity matrix
More informationOutline. Remedial Measures) Extra Sums of Squares Standardized Version of the Multiple Regression Model
Outline 1 Multiple Linear Regression (Estimation, Inference, Diagnostics and Remedial Measures) 2 Special Topics for Multiple Regression Extra Sums of Squares Standardized Version of the Multiple Regression
More informationLecture 9 SLR in Matrix Form
Lecture 9 SLR in Matrix Form STAT 51 Spring 011 Background Reading KNNL: Chapter 5 9-1 Topic Overview Matrix Equations for SLR Don t focus so much on the matrix arithmetic as on the form of the equations.
More informationNeed for Several Predictor Variables
Multiple regression One of the most widely used tools in statistical analysis Matrix expressions for multiple regression are the same as for simple linear regression Need for Several Predictor Variables
More informationFinal Review. Yang Feng. Yang Feng (Columbia University) Final Review 1 / 58
Final Review Yang Feng http://www.stat.columbia.edu/~yangfeng Yang Feng (Columbia University) Final Review 1 / 58 Outline 1 Multiple Linear Regression (Estimation, Inference) 2 Special Topics for Multiple
More information. a m1 a mn. a 1 a 2 a = a n
Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by
More informationInverse of a Square Matrix. For an N N square matrix A, the inverse of A, 1
Inverse of a Square Matrix For an N N square matrix A, the inverse of A, 1 A, exists if and only if A is of full rank, i.e., if and only if no column of A is a linear combination 1 of the others. A is
More informationSTAT5044: Regression and Anova. Inyoung Kim
STAT5044: Regression and Anova Inyoung Kim 2 / 51 Outline 1 Matrix Expression 2 Linear and quadratic forms 3 Properties of quadratic form 4 Properties of estimates 5 Distributional properties 3 / 51 Matrix
More informationMath Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
More informationLecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is
Lecture 15 Multiple regression I Chapter 6 Set 2 Least Square Estimation The quadratic form to be minimized is Q = (Y i β 0 β 1 X i1 β 2 X i2 β p 1 X i.p 1 ) 2, which in matrix notation is Q = (Y Xβ) (Y
More informationLecture 13: Simple Linear Regression in Matrix Format. 1 Expectations and Variances with Vectors and Matrices
Lecture 3: Simple Linear Regression in Matrix Format To move beyond simple regression we need to use matrix algebra We ll start by re-expressing simple linear regression in matrix form Linear algebra is
More informationPreliminaries. Copyright c 2018 Dan Nettleton (Iowa State University) Statistics / 38
Preliminaries Copyright c 2018 Dan Nettleton (Iowa State University) Statistics 510 1 / 38 Notation for Scalars, Vectors, and Matrices Lowercase letters = scalars: x, c, σ. Boldface, lowercase letters
More informationLINEAR REGRESSION MODELS W4315
LINEAR REGRESSION MODELS W431 HOMEWORK ANSWERS March 9, 2010 Due: 03/04/10 Instructor: Frank Wood 1. (20 points) In order to get a maximum likelihood estimate of the parameters of a Box-Cox transformed
More informationPOLI270 - Linear Algebra
POLI7 - Linear Algebra Septemer 8th Basics a x + a x +... + a n x n b () is the linear form where a, b are parameters and x n are variables. For a given equation such as x +x you only need a variable and
More informationLecture 6: Geometry of OLS Estimation of Linear Regession
Lecture 6: Geometry of OLS Estimation of Linear Regession Xuexin Wang WISE Oct 2013 1 / 22 Matrix Algebra An n m matrix A is a rectangular array that consists of nm elements arranged in n rows and m columns
More informationMatrix Algebra. Matrix Algebra. Chapter 8 - S&B
Chapter 8 - S&B Algebraic operations Matrix: The size of a matrix is indicated by the number of its rows and the number of its columns. A matrix with k rows and n columns is called a k n matrix. The number
More informationCS100: DISCRETE STRUCTURES. Lecture 3 Matrices Ch 3 Pages:
CS100: DISCRETE STRUCTURES Lecture 3 Matrices Ch 3 Pages: 246-262 Matrices 2 Introduction DEFINITION 1: A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called an m x n
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationAppendix A: Matrices
Appendix A: Matrices A matrix is a rectangular array of numbers Such arrays have rows and columns The numbers of rows and columns are referred to as the dimensions of a matrix A matrix with, say, 5 rows
More informationMatrix & Linear Algebra
Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84 Vectors Vectors Vector: A
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationLinear Algebra V = T = ( 4 3 ).
Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional
More informationPhys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
More informationMatrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices
Matrices A. Fabretti Mathematics 2 A.Y. 2015/2016 Table of contents Matrix Algebra Determinant Inverse Matrix Introduction A matrix is a rectangular array of numbers. The size of a matrix is indicated
More informationMatrix Arithmetic. j=1
An m n matrix is an array A = Matrix Arithmetic a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn of real numbers a ij An m n matrix has m rows and n columns a ij is the entry in the i-th row and j-th column
More informationElementary Row Operations on Matrices
King Saud University September 17, 018 Table of contents 1 Definition A real matrix is a rectangular array whose entries are real numbers. These numbers are organized on rows and columns. An m n matrix
More informationChapter 1 Simple Linear Regression (part 6: matrix version)
Chapter Simple Liear Regressio (part 6: matrix versio) Overview Simple liear regressio model: respose variable Y, a sigle idepedet variable X Y β 0 + β X + ε Multiple liear regressio model: respose Y,
More informationElementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
More informationLECTURE 2 LINEAR REGRESSION MODEL AND OLS
SEPTEMBER 29, 2014 LECTURE 2 LINEAR REGRESSION MODEL AND OLS Definitions A common question in econometrics is to study the effect of one group of variables X i, usually called the regressors, on another
More informationMatrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix
Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Matrix Operations Matrix Addition and Matrix Scalar Multiply Matrix Multiply Matrix
More information6. Multiple Linear Regression
6. Multiple Linear Regression SLR: 1 predictor X, MLR: more than 1 predictor Example data set: Y i = #points scored by UF football team in game i X i1 = #games won by opponent in their last 10 games X
More informationA FIRST COURSE IN LINEAR ALGEBRA. An Open Text by Ken Kuttler. Matrix Arithmetic
A FIRST COURSE IN LINEAR ALGEBRA An Open Text by Ken Kuttler Matrix Arithmetic Lecture Notes by Karen Seyffarth Adapted by LYRYX SERVICE COURSE SOLUTION Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)
More informationChapter 1. Matrix Algebra
ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface
More informationIntroduc)on to linear algebra
Introduc)on to linear algebra Vector A vector, v, of dimension n is an n 1 rectangular array of elements v 1 v v = 2 " v n % vectors will be column vectors. They may also be row vectors, when transposed
More informationApplied Matrix Algebra Lecture Notes Section 2.2. Gerald Höhn Department of Mathematics, Kansas State University
Applied Matrix Algebra Lecture Notes Section 22 Gerald Höhn Department of Mathematics, Kansas State University September, 216 Chapter 2 Matrices 22 Inverses Let (S) a 11 x 1 + a 12 x 2 + +a 1n x n = b
More informationCLASS 12 ALGEBRA OF MATRICES
CLASS 12 ALGEBRA OF MATRICES Deepak Sir 9811291604 SHRI SAI MASTERS TUITION CENTER CLASS 12 A matrix is an ordered rectangular array of numbers or functions. The numbers or functions are called the elements
More informationRegression. Oscar García
Regression Oscar García Regression methods are fundamental in Forest Mensuration For a more concise and general presentation, we shall first review some matrix concepts 1 Matrices An order n m matrix is
More informationLinear Algebra (Review) Volker Tresp 2018
Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationLinear Models Review
Linear Models Review Vectors in IR n will be written as ordered n-tuples which are understood to be column vectors, or n 1 matrices. A vector variable will be indicted with bold face, and the prime sign
More informationPOL 213: Research Methods
Brad 1 1 Department of Political Science University of California, Davis April 15, 2008 Some Matrix Basics What is a matrix? A rectangular array of elements arranged in rows and columns. 55 900 0 67 1112
More informationMultiple Regression. Dr. Frank Wood. Frank Wood, Linear Regression Models Lecture 12, Slide 1
Multiple Regression Dr. Frank Wood Frank Wood, fwood@stat.columbia.edu Linear Regression Models Lecture 12, Slide 1 Review: Matrix Regression Estimation We can solve this equation (if the inverse of X
More informationChapter 7. Linear Algebra: Matrices, Vectors,
Chapter 7. Linear Algebra: Matrices, Vectors, Determinants. Linear Systems Linear algebra includes the theory and application of linear systems of equations, linear transformations, and eigenvalue problems.
More informationMATRICES AND MATRIX OPERATIONS
SIZE OF THE MATRIX is defined by number of rows and columns in the matrix. For the matrix that have m rows and n columns we say the size of the matrix is m x n. If matrix have the same number of rows (n)
More informationMatrix operations Linear Algebra with Computer Science Application
Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the
More informationECON 186 Class Notes: Linear Algebra
ECON 86 Class Notes: Linear Algebra Jijian Fan Jijian Fan ECON 86 / 27 Singularity and Rank As discussed previously, squareness is a necessary condition for a matrix to be nonsingular (have an inverse).
More informationMAC Module 2 Systems of Linear Equations and Matrices II. Learning Objectives. Upon completing this module, you should be able to :
MAC 0 Module Systems of Linear Equations and Matrices II Learning Objectives Upon completing this module, you should be able to :. Find the inverse of a square matrix.. Determine whether a matrix is invertible..
More informationIntroduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX
Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX September 2007 MSc Sep Intro QT 1 Who are these course for? The September
More informationLinear Equations in Linear Algebra
1 Linear Equations in Linear Algebra 1.7 LINEAR INDEPENDENCE LINEAR INDEPENDENCE Definition: An indexed set of vectors {v 1,, v p } in n is said to be linearly independent if the vector equation x x x
More informationLecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,
2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for
More informationTopics. Vectors (column matrices): Vector addition and scalar multiplication The matrix of a linear function y Ax The elements of a matrix A : A ij
Topics Vectors (column matrices): Vector addition and scalar multiplication The matrix of a linear function y Ax The elements of a matrix A : A ij or a ij lives in row i and column j Definition of a matrix
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationMIT Spring 2015
Regression Analysis MIT 18.472 Dr. Kempthorne Spring 2015 1 Outline Regression Analysis 1 Regression Analysis 2 Multiple Linear Regression: Setup Data Set n cases i = 1, 2,..., n 1 Response (dependent)
More informationSTAT 540: Data Analysis and Regression
STAT 540: Data Analysis and Regression Wen Zhou http://www.stat.colostate.edu/~riczw/ Email: riczw@stat.colostate.edu Department of Statistics Colorado State University Fall 205 W. Zhou (Colorado State
More informationChapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations
Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2
More informationMath Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationSTAT 100C: Linear models
STAT 100C: Linear models Arash A. Amini June 9, 2018 1 / 56 Table of Contents Multiple linear regression Linear model setup Estimation of β Geometric interpretation Estimation of σ 2 Hat matrix Gram matrix
More informationMatrix Algebra, part 2
Matrix Algebra, part 2 Ming-Ching Luoh 2005.9.12 1 / 38 Diagonalization and Spectral Decomposition of a Matrix Optimization 2 / 38 Diagonalization and Spectral Decomposition of a Matrix Also called Eigenvalues
More informationPreliminary Linear Algebra 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 100
Preliminary Linear Algebra 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 100 Notation for all there exists such that therefore because end of proof (QED) Copyright c 2012
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2
MA 575 Linear Models: Cedric E Ginestet, Boston University Revision: Probability and Linear Algebra Week 1, Lecture 2 1 Revision: Probability Theory 11 Random Variables A real-valued random variable is
More informationMath 360 Linear Algebra Fall Class Notes. a a a a a a. a a a
Math 360 Linear Algebra Fall 2008 9-10-08 Class Notes Matrices As we have already seen, a matrix is a rectangular array of numbers. If a matrix A has m columns and n rows, we say that its dimensions are
More informationChapter 2. Linear Algebra. rather simple and learning them will eventually allow us to explain the strange results of
Chapter 2 Linear Algebra In this chapter, we study the formal structure that provides the background for quantum mechanics. The basic ideas of the mathematical machinery, linear algebra, are rather simple
More information2. Matrix Algebra and Random Vectors
2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns
More informationINSTITIÚID TEICNEOLAÍOCHTA CHEATHARLACH INSTITUTE OF TECHNOLOGY CARLOW MATRICES
1 CHAPTER 4 MATRICES 1 INSTITIÚID TEICNEOLAÍOCHTA CHEATHARLACH INSTITUTE OF TECHNOLOGY CARLOW MATRICES 1 Matrices Matrices are of fundamental importance in 2-dimensional and 3-dimensional graphics programming
More informationMA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7
MA 575 Linear Models: Cedric E. Ginestet, Boston University Midterm Review Week 7 1 Random Vectors Let a 0 and y be n 1 vectors, and let A be an n n matrix. Here, a 0 and A are non-random, whereas y is
More informationSection 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.
Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A
More informationMatrices. Math 240 Calculus III. Wednesday, July 10, Summer 2013, Session II. Matrices. Math 240. Definitions and Notation.
function Matrices Calculus III Summer 2013, Session II Wednesday, July 10, 2013 Agenda function 1. 2. function function Definition An m n matrix is a rectangular array of numbers arranged in m horizontal
More informationLecture Notes in Linear Algebra
Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................
More informationANOVA: Analysis of Variance - Part I
ANOVA: Analysis of Variance - Part I The purpose of these notes is to discuss the theory behind the analysis of variance. It is a summary of the definitions and results presented in class with a few exercises.
More informationSection 9.2: Matrices.. a m1 a m2 a mn
Section 9.2: Matrices Definition: A matrix is a rectangular array of numbers: a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn In general, a ij denotes the (i, j) entry of A. That is, the entry in
More information[ Here 21 is the dot product of (3, 1, 2, 5) with (2, 3, 1, 2), and 31 is the dot product of
. Matrices A matrix is any rectangular array of numbers. For example 3 5 6 4 8 3 3 is 3 4 matrix, i.e. a rectangular array of numbers with three rows four columns. We usually use capital letters for matrices,
More informationPrepared by: M. S. KumarSwamy, TGT(Maths) Page
Prepared by: M. S. KumarSwamy, TGT(Maths) Page - 50 - CHAPTER 3: MATRICES QUICK REVISION (Important Concepts & Formulae) MARKS WEIGHTAGE 03 marks Matrix A matrix is an ordered rectangular array of numbers
More informationNumerical Analysis Lecture Notes
Numerical Analysis Lecture Notes Peter J Olver 3 Review of Matrix Algebra Vectors and matrices are essential for modern analysis of systems of equations algebrai, differential, functional, etc In this
More informationIntroduction to Computational Finance and Financial Econometrics Matrix Algebra Review
You can t see this text! Introduction to Computational Finance and Financial Econometrics Matrix Algebra Review Eric Zivot Spring 2015 Eric Zivot (Copyright 2015) Matrix Algebra Review 1 / 54 Outline 1
More informationLecture 13: Simple Linear Regression in Matrix Format
See updates and corrections at http://www.stat.cmu.edu/~cshalizi/mreg/ Lecture 13: Simple Linear Regression in Matrix Format 36-401, Section B, Fall 2015 13 October 2015 Contents 1 Least Squares in Matrix
More informationLinear Algebra and Matrix Inversion
Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much
More informationSTAT200C: Review of Linear Algebra
Stat200C Instructor: Zhaoxia Yu STAT200C: Review of Linear Algebra 1 Review of Linear Algebra 1.1 Vector Spaces, Rank, Trace, and Linear Equations 1.1.1 Rank and Vector Spaces Definition A vector whose
More informationRegression Review. Statistics 149. Spring Copyright c 2006 by Mark E. Irwin
Regression Review Statistics 149 Spring 2006 Copyright c 2006 by Mark E. Irwin Matrix Approach to Regression Linear Model: Y i = β 0 + β 1 X i1 +... + β p X ip + ɛ i ; ɛ i iid N(0, σ 2 ), i = 1,..., n
More informationMatrices and Determinants
Chapter1 Matrices and Determinants 11 INTRODUCTION Matrix means an arrangement or array Matrices (plural of matrix) were introduced by Cayley in 1860 A matrix A is rectangular array of m n numbers (or
More informationRaphael Mrode. Training in quantitative genetics and genomics 30 May 10 June 2016 ILRI, Nairobi. Partner Logo. Partner Logo
Basic matrix algebra Raphael Mrode Training in quantitative genetics and genomics 3 May June 26 ILRI, Nairobi Partner Logo Partner Logo Matrix definition A matrix is a rectangular array of numbers set
More informationLecture 3: Matrix and Matrix Operations
Lecture 3: Matrix and Matrix Operations Representation, row vector, column vector, element of a matrix. Examples of matrix representations Tables and spreadsheets Scalar-Matrix operation: Scaling a matrix
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationM. Matrices and Linear Algebra
M. Matrices and Linear Algebra. Matrix algebra. In section D we calculated the determinants of square arrays of numbers. Such arrays are important in mathematics and its applications; they are called matrices.
More informationPractical Linear Algebra: A Geometry Toolbox
Practical Linear Algebra: A Geometry Toolbox Third edition Chapter 4: Changing Shapes: Linear Maps in 2D Gerald Farin & Dianne Hansford CRC Press, Taylor & Francis Group, An A K Peters Book www.farinhansford.com/books/pla
More informationMATH2210 Notebook 2 Spring 2018
MATH2210 Notebook 2 Spring 2018 prepared by Professor Jenny Baglivo c Copyright 2009 2018 by Jenny A. Baglivo. All Rights Reserved. 2 MATH2210 Notebook 2 3 2.1 Matrices and Their Operations................................
More informationReview Packet 1 B 11 B 12 B 13 B = B 21 B 22 B 23 B 31 B 32 B 33 B 41 B 42 B 43
Review Packet. For each of the following, write the vector or matrix that is specified: a. e 3 R 4 b. D = diag{, 3, } c. e R 3 d. I. For each of the following matrices and vectors, give their dimension.
More informationBiostatistics 380 Multiple Regression 1. Multiple Regression
Biostatistics 0 Multiple Regression ORIGIN 0 Multiple Regression Multiple Regression is an extension of the technique of linear regression to describe the relationship between a single dependent (response)
More informationPeter Hoff Linear and multilinear models April 3, GLS for multivariate regression 5. 3 Covariance estimation for the GLM 8
Contents 1 Linear model 1 2 GLS for multivariate regression 5 3 Covariance estimation for the GLM 8 4 Testing the GLH 11 A reference for some of this material can be found somewhere. 1 Linear model Recall
More informationFormal Statement of Simple Linear Regression Model
Formal Statement of Simple Linear Regression Model Y i = β 0 + β 1 X i + ɛ i Y i value of the response variable in the i th trial β 0 and β 1 are parameters X i is a known constant, the value of the predictor
More informationMultiple Linear Regression
Multiple Linear Regression Simple linear regression tries to fit a simple line between two variables Y and X. If X is linearly related to Y this explains some of the variability in Y. In most cases, there
More informationIntroduction. Vectors and Matrices. Vectors [1] Vectors [2]
Introduction Vectors and Matrices Dr. TGI Fernando 1 2 Data is frequently arranged in arrays, that is, sets whose elements are indexed by one or more subscripts. Vector - one dimensional array Matrix -
More informationICS 6N Computational Linear Algebra Matrix Algebra
ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24 Matrix Consider an m n matrix
More informationAddition and subtraction: element by element, and dimensions must match.
Matrix Essentials review: ) Matrix: Rectangular array of numbers. ) ranspose: Rows become columns and vice-versa ) single row or column is called a row or column) Vector ) R ddition and subtraction: element
More informationLinear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
More informationRecall the convention that, for us, all vectors are column vectors.
Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists
More informationMatrix Algebra, Class Notes (part 2) by Hrishikesh D. Vinod Copyright 1998 by Prof. H. D. Vinod, Fordham University, New York. All rights reserved.
Matrix Algebra, Class Notes (part 2) by Hrishikesh D. Vinod Copyright 1998 by Prof. H. D. Vinod, Fordham University, New York. All rights reserved. 1 Converting Matrices Into (Long) Vectors Convention:
More informationEstimation of the Response Mean. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 27
Estimation of the Response Mean Copyright c 202 Dan Nettleton (Iowa State University) Statistics 5 / 27 The Gauss-Markov Linear Model y = Xβ + ɛ y is an n random vector of responses. X is an n p matrix
More information