Matrix Differentiation
|
|
- Piers Griffith
- 5 years ago
- Views:
Transcription
1 Matrix Differentiation CS5240 Theoretical Foundations in Multimedia Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (NUS) Matrix Differentiation 1 / 35
2 Matrix Derivatives There are 6 common types of matrix derivatives: Type Scalar Vector Matrix Scalar Y Vector Matrix Leow Wee Kheng (NUS) Matrix Differentiation 2 / 35
3 Derivatives by Scalar Numerator Layout Notation Y = = 11. m1 1. m 1n.... mn Denominator Layout Notation [ ] = 1 m Leow Wee Kheng (NUS) Matrix Differentiation 3 / 35
4 Derivatives by Vector Numerator Layout Notation [ = 1 = ] n n..... m m 1 n Denominator Layout Notation = 1. n 1 m = m n n Leow Wee Kheng (NUS) Matrix Differentiation 4 / 35
5 Derivative by Matrix Numerator Layout Notation = 11 m n mn Denominator Layout Notation = 11 1n..... m1 mn Leow Wee Kheng (NUS) Matrix Differentiation 5 / 35
6 Pictorial Representation numerator layout denominator layout.. Leow Wee Kheng (NUS) Matrix Differentiation 6 / 35
7 Caution Most books and papers don t state which convention they use. Reference [2] uses both conventions but clearly differentiate them. [ ] = 1 n = 1. n m = 1 n..... m m = m 1 n n n It is best not to mix the two conventions in your equations. We adopt numerator layout notation. Leow Wee Kheng (NUS) Matrix Differentiation 7 / 35
8 Commonly Used Derivatives Commonly Used Derivatives Here, scalar a, vector a and matrix A are not functions of x and x. da (C1) = 0 (column matrix) dx (C2) (C3) (C4) (C5) da dx = 0 (row matrix) da dx = 0 (matrix) da dx = 0 (matrix) dx dx = I Leow Wee Kheng (NUS) Matrix Differentiation 8 / 35
9 Commonly Used Derivatives (C6) (C7) da x dx = dx a dx = a dx x dx = 2x (C8) d(x a) 2 dx = 2x aa (C9) (C10) dax dx = A dx A dx = A (C11) dx Ax dx = x (A+A ) Leow Wee Kheng (NUS) Matrix Differentiation 9 / 35
10 Derivatives of Scalar by Scalar Derivatives of Scalar by Scalar (SS1) (u+v) = u + v (SS2) uv = u v +v u (product rule) (SS3) g(u) = g(u) u u (chain rule) (SS4) f(g(u)) = f(g) g g(u) u u (chain rule) Leow Wee Kheng (NUS) Matrix Differentiation 10 / 35
11 Derivatives of Vector by Scalar Derivatives of Vector by Scalar (VS1) (VS2) (VS3) au = a u where a is not a function of x. Au = A u where A is not a function of x. u = ( ) u (VS4) (u+v) = u + v Leow Wee Kheng (NUS) Matrix Differentiation 11 / 35
12 Derivatives of Vector by Scalar (VS5) g(u) = g(u) u u (chain rule) with consistent matrix layout. (VS6) f(g(u)) = f(g) g g(u) u u (chain rule) with consistent matrix layout. Leow Wee Kheng (NUS) Matrix Differentiation 12 / 35
13 Derivatives of Matrix by Scalar Derivatives of Matrix by Scalar (MS1) (MS2) au = a U where a is not a function of x. AUB = A U B where A and B are not functions of x. (MS3) (U+V) = U + V (MS4) UV = U V + U V (product rule) Leow Wee Kheng (NUS) Matrix Differentiation 13 / 35
14 Derivatives of Scalar by Vector Derivatives of Scalar by Vector (SV1) au = a u where a is not a function of x. (SV2) (u+v) = u + v (SV3) (SV4) uv = u v +v u g(u) = g(u) u u (product rule) (chain rule) (SV5) f(g(u)) = f(g) g g(u) u u (chain rule) Leow Wee Kheng (NUS) Matrix Differentiation 14 / 35
15 Derivatives of Scalar by Vector (SV6) (SV7) u v = u v +v u where u u Av where u and v = u A v +v A u and v and A is not a function of x. (product rule) are in numerator layout. (product rule) are in numerator layout, Leow Wee Kheng (NUS) Matrix Differentiation 15 / 35
16 Derivatives of Scalar by Matrix Derivatives of Scalar by Matrix (SM1) (SM2) au = a u where a is not a function of X. (u+v) = u + v (SM3) (SM4) uv = u v u +v g(u) = g(u) u u (product rule) (chain rule) (SM5) f(g(u)) = f(g) g g(u) u u (chain rule) Leow Wee Kheng (NUS) Matrix Differentiation 16 / 35
17 Derivatives of Vector by Vector Derivatives of Vector by Vector (VV1) au = a u +u a (product rule) (VV2) Au = A u where A is not a function of x. (VV3) (u+v) = u + v (VV4) g(u) = g(u) u u (chain rule) (VV5) f(g(u)) = f(g) g g(u) u u (chain rule) Leow Wee Kheng (NUS) Matrix Differentiation 17 / 35
18 Notes on Denominator Layout Notes on Denominator Layout In some cases, the results of denominator layout are the transpose of those of numerator layout. Moreover, the chain rule for denominator layout goes from right to left instead of left to right. Numerator Layout Notation Denominator Layout Notation (C7) (C11) (VV5) dx Ax dx f(g(u)) da x dx = a = x (A+A ) = f(g) g g(u) u u dx Ax dx f(g(u)) da x dx = a = (A+A )x = u g(u) f(g) u g Leow Wee Kheng (NUS) Matrix Differentiation 18 / 35
19 Derivations of Derivatives Derivations of Derivatives (C6) da x dx = dx a dx = a (The not-so-hard way) Let s = a x = a 1 x 1 + +a n x n. Then, (The easier way) Let s = a x = i a i x i. Then, s i = a i. So, ds dx = a. s i = a i. So, ds dx = a. (C7) dx x dx = 2x Let s = x x = i x 2 i. Then, s = 2x i. So, ds i dx = 2x. Leow Wee Kheng (NUS) Matrix Differentiation 19 / 35
20 Derivations of Derivatives (C8) d(x a) 2 dx = 2x aa Let s = x a. Then, s2 i = 2s s i = 2sa i. So, ds2 dx = 2x aa. (C9) dax dx = A (The hard way) a 11 a 1n Ax =..... a n1 a nn x 1. x n = a 11 x 1 + +a 1n x n. a n1 x 1 + +a nn x n. (The easy way) Let s = Ax. Then, s i = j a ij x j, and s i j = a ij. So, ds dx = A. Leow Wee Kheng (NUS) Matrix Differentiation 20 / 35
21 Derivations of Derivatives (C10) dx A dx = A Let y = x A, and a j denote the j-th column of A. Then, y i = x a j. Applying (C6) yields dy i dx = a dy j. So, dx = A. (C11) dx Ax dx = x (A+A ) Apply (SV6) to dx Ax dx and obtain x dax dx +(Ax) dx dx, Next, apply (C9) to the first part of the sum, and obtain x A+(Ax), which is x (A+A ). (Need to prove SV6 Homework.) Leow Wee Kheng (NUS) Matrix Differentiation 21 / 35
22 Derivatives of Trace Derivatives of Trace For variable matrix X and constant matrices A, B, C, (DT1) tr(x) = I (DT2) tr(x k ) = kxk 1 (DT3) tr(ax) = tr(xa) = A (DT4) tr(ax ) = tr(x A) = A (DT5) tr(x AX) = X (A+A ) Leow Wee Kheng (NUS) Matrix Differentiation 22 / 35
23 Derivatives of Trace (DT6) tr(x 1 A) = X AX (DT7) tr(axb) = tr(bax) = BA (DT8) tr(axbx C) = BX CA+B X A C Leow Wee Kheng (NUS) Matrix Differentiation 23 / 35
24 Derivatives of Determinant Derivatives of Determinant For variable matrix X and constant matrices A, B, (DD1) X = X X 1 (DD2) (DD3) X k = k Xk X 1 log X = X 1 (DD4) AXB = AXB X 1 (DD5) X AX = 2 X AX X 1 Leow Wee Kheng (NUS) Matrix Differentiation 24 / 35
25 Derivations of Derivatives Derivations of Derivatives (DD1) X = X X 1 For an n n matrix X, Laplace s formula expresses X as X = n ( 1) i+j x ij M ij, i=1 where M ij is a minor, which is the determinant of the sub-matrix of X obtained by removing the ith row and jth column. The adjugate adj(x) is the transpose of the matrix consisting of the cofactors ( 1) i+j M ij : adj(x) ji = ( 1) i+j M ij, which is independent of x ij, element (i,j) of X. Leow Wee Kheng (NUS) Matrix Differentiation 25 / 35
26 Derivations of Derivatives So, Then, and, in numerator layout, X = n x ij adj(x) ji. i=1 X and adj(x) are also related by X ij = adj(x) ji, X = adj(x). X I = adj(x)x. So, X = X X 1. Leow Wee Kheng (NUS) Matrix Differentiation 26 / 35
27 Derivations of Derivatives X k (DD2) = k Xk X 1 Note that X k = X k. So, X k = X k = k X k 1 X X 1 = k X k X 1. (DD3) log X = X 1 log X = 1 X X = X 1. AXB (DD4) = AXB X 1 Note that AXB = A X B. So, AXB = A X X 1 B = AXB X 1. Leow Wee Kheng (NUS) Matrix Differentiation 27 / 35
28 Derivations of Derivatives (DD5) X AX = 2 X AX X 1 Note that X = X. So, X AX = X X 1 A X + X A X X 1 = 2 X AX X 1. Leow Wee Kheng (NUS) Matrix Differentiation 28 / 35
29 Derivations of Derivatives (DT1) tr(x) = I For an n n matrix X, Then, So, tr(x) = n x ii. i=1 { tr(x) 1 if i = j, = ij 0 if i j. tr(x) = I. Leow Wee Kheng (NUS) Matrix Differentiation 29 / 35
30 Derivations of Derivatives (DT2) tr(x k ) ij tr(x k ) = kxk 1 So, in numerator layout, (X k ) ij = l 1 x il1 x l1 l 2 x lk 1 j l k 1 tr(x k ) = x l0 l 1 x l1 l 2 x lk 1 l 0 l 0 l 1 l k 1 = k 1 factors {}}{ x jl2 x l2 l 3 x lk 1 i+ (l 0 = i,l 1 = j) l 2 l k 1 + (k 2 terms) x jl1 x l1 l 2 x lk 2 i (l k 1 = i,l 0 = j) l k 2 l tr(x k ) = kxk 1. Leow Wee Kheng (NUS) Matrix Differentiation 30 / 35
31 Derivations of Derivatives (DT3) tr(ax) = tr(xa) = A (AX) ij = k tr(ax) = l tr(ax) = a ji, ij a ik x kj a lk x kl k So, in numerator layout, tr(ax) = A. Caution: tr(ab) tr(a) tr(b). Leow Wee Kheng (NUS) Matrix Differentiation 31 / 35
32 Derivations of Derivatives (DT4) tr(ax ) = tr(x A) Similar to the derivation for DT3, = A (AX ) ij = k a ik x jk tr(ax) = l tr(ax ) ij = a ij a lk x lk k So, in numerator layout, tr(ax ) = A. Leow Wee Kheng (NUS) Matrix Differentiation 32 / 35
33 Derivations of Derivatives (DT5) tr(x AX) = X (A+A ) (X AX) ij = k tr(x AX) = r tr(x AX) ij = l x ki a kl x lj l x kr a kl x lr k l a il x lj + k x kj a ki = k x kj a ki + l x lj a il. So, in numerator layout, tr(x AX) = X A+X A = X (A+A ). Leow Wee Kheng (NUS) Matrix Differentiation 33 / 35
34 Derivations of Derivatives (DT7) DT7 is a direct result of DT3. tr(axbx C) (DT8) = BX CA+B X A C (AXBX C) ij = a ip x pq b qr x sr c sj p q r s tr(axbx C) = a kp x pq b qr x sr c sk. k p q r s tr(axbx C) = a ki b jr x sr c sk + a kp x pq b qj c ik ij k r s k p q = b jr x sr c sk a ki + b qj x pq a kp c ik k r s k p q So, in numerator layout, tr(axbx C) = BX CA+B X A C. Leow Wee Kheng (NUS) Matrix Differentiation 34 / 35
35 References References 1. J. E. Gentle, Matrix Algebra: Theory, Computations, and Applications in Statistics, Springer, H. Lütkepohl, Handbook of Matrices, John Wiley & Sons, K. B. Petersen and M. S. Pedersen, The Matrix Cookbook, Wikipedia, Matrix Calculus. en.wikipedia.org/wiki/matrix_calculus Leow Wee Kheng (NUS) Matrix Differentiation 35 / 35
Math 4377/6308 Advanced Linear Algebra
2.3 Composition Math 4377/6308 Advanced Linear Algebra 2.3 Composition of Linear Transformations Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/ jiwenhe/math4377
More informationMatrices and Determinants
Chapter1 Matrices and Determinants 11 INTRODUCTION Matrix means an arrangement or array Matrices (plural of matrix) were introduced by Cayley in 1860 A matrix A is rectangular array of m n numbers (or
More informationLinear Algebra Solutions 1
Math Camp 1 Do the following: Linear Algebra Solutions 1 1. Let A = and B = 3 8 5 A B = 3 5 9 A + B = 9 11 14 4 AB = 69 3 16 BA = 1 4 ( 1 3. Let v = and u = 5 uv = 13 u v = 13 v u = 13 Math Camp 1 ( 7
More informationPhys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
More informationICS 6N Computational Linear Algebra Matrix Algebra
ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24 Matrix Consider an m n matrix
More informationMATH 431: FIRST MIDTERM. Thursday, October 3, 2013.
MATH 431: FIRST MIDTERM Thursday, October 3, 213. (1) An inner product on the space of matrices. Let V be the vector space of 2 2 real matrices (that is, the algebra Mat 2 (R), but without the mulitiplicative
More informationChapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in
Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column
More informationMatrices A brief introduction
Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 41 Definitions Definition A matrix is a set of N real or complex
More informationOn Expected Gaussian Random Determinants
On Expected Gaussian Random Determinants Moo K. Chung 1 Department of Statistics University of Wisconsin-Madison 1210 West Dayton St. Madison, WI 53706 Abstract The expectation of random determinants whose
More informationKevin James. MTHSC 3110 Section 2.1 Matrix Operations
MTHSC 3110 Section 2.1 Matrix Operations Notation Let A be an m n matrix, that is, m rows and n columns. We ll refer to the entries of A by their row and column indices. The entry in the i th row and j
More informationAnnouncements Wednesday, October 10
Announcements Wednesday, October 10 The second midterm is on Friday, October 19 That is one week from this Friday The exam covers 35, 36, 37, 39, 41, 42, 43, 44 (through today s material) WeBWorK 42, 43
More informationAnnouncements Monday, October 02
Announcements Monday, October 02 Please fill out the mid-semester survey under Quizzes on Canvas WeBWorK 18, 19 are due Wednesday at 11:59pm The quiz on Friday covers 17, 18, and 19 My office is Skiles
More informationMA 1B ANALYTIC - HOMEWORK SET 7 SOLUTIONS
MA 1B ANALYTIC - HOMEWORK SET 7 SOLUTIONS 1. (7 pts)[apostol IV.8., 13, 14] (.) Let A be an n n matrix with characteristic polynomial f(λ). Prove (by induction) that the coefficient of λ n 1 in f(λ) is
More informationCartesian Tensors. e 2. e 1. General vector (formal definition to follow) denoted by components
Cartesian Tensors Reference: Jeffreys Cartesian Tensors 1 Coordinates and Vectors z x 3 e 3 y x 2 e 2 e 1 x x 1 Coordinates x i, i 123,, Unit vectors: e i, i 123,, General vector (formal definition to
More informationLinear Equations and Matrix
1/60 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear
More informationChapter 1 Matrices and Systems of Equations
Chapter 1 Matrices and Systems of Equations System of Linear Equations 1. A linear equation in n unknowns is an equation of the form n i=1 a i x i = b where a 1,..., a n, b R and x 1,..., x n are variables.
More informationCS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
More informationMatrix Arithmetic. j=1
An m n matrix is an array A = Matrix Arithmetic a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn of real numbers a ij An m n matrix has m rows and n columns a ij is the entry in the i-th row and j-th column
More informationExercise Set Suppose that A, B, C, D, and E are matrices with the following sizes: A B C D E
Determine the size of a given matrix. Identify the row vectors and column vectors of a given matrix. Perform the arithmetic operations of matrix addition, subtraction, scalar multiplication, and multiplication.
More informationFundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
More information6.1 Matrices. Definition: A Matrix A is a rectangular array of the form. A 11 A 12 A 1n A 21. A 2n. A m1 A m2 A mn A 22.
61 Matrices Definition: A Matrix A is a rectangular array of the form A 11 A 12 A 1n A 21 A 22 A 2n A m1 A m2 A mn The size of A is m n, where m is the number of rows and n is the number of columns The
More informationII. Determinant Functions
Supplemental Materials for EE203001 Students II Determinant Functions Chung-Chin Lu Department of Electrical Engineering National Tsing Hua University May 22, 2003 1 Three Axioms for a Determinant Function
More informationTOPIC III LINEAR ALGEBRA
[1] Linear Equations TOPIC III LINEAR ALGEBRA (1) Case of Two Endogenous Variables 1) Linear vs. Nonlinear Equations Linear equation: ax + by = c, where a, b and c are constants. 2 Nonlinear equation:
More informationIntroduction. Vectors and Matrices. Vectors [1] Vectors [2]
Introduction Vectors and Matrices Dr. TGI Fernando 1 2 Data is frequently arranged in arrays, that is, sets whose elements are indexed by one or more subscripts. Vector - one dimensional array Matrix -
More information. D Matrix Calculus D 1
D Matrix Calculus D 1 Appendix D: MATRIX CALCULUS D 2 In this Appendix we collect some useful formulas of matrix calculus that often appear in finite element derivations D1 THE DERIVATIVES OF VECTOR FUNCTIONS
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationMassachusetts Institute of Technology Department of Economics Statistics. Lecture Notes on Matrix Algebra
Massachusetts Institute of Technology Department of Economics 14.381 Statistics Guido Kuersteiner Lecture Notes on Matrix Algebra These lecture notes summarize some basic results on matrix algebra used
More informationPrincipal Component Analysis
Principal Component Analysis CS5240 Theoretical Foundations in Multimedia Leow Wee Kheng Department of Computer Science School of Computing National University of Singapore Leow Wee Kheng (NUS) Principal
More informationMatrices: 2.1 Operations with Matrices
Goals In this chapter and section we study matrix operations: Define matrix addition Define multiplication of matrix by a scalar, to be called scalar multiplication. Define multiplication of two matrices,
More informationLecture 4: Products of Matrices
Lecture 4: Products of Matrices Winfried Just, Ohio University January 22 24, 2018 Matrix multiplication has a few surprises up its sleeve Let A = [a ij ] m n, B = [b ij ] m n be two matrices. The sum
More informationMath Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationChapter 5: Matrices. Daniel Chan. Semester UNSW. Daniel Chan (UNSW) Chapter 5: Matrices Semester / 33
Chapter 5: Matrices Daniel Chan UNSW Semester 1 2018 Daniel Chan (UNSW) Chapter 5: Matrices Semester 1 2018 1 / 33 In this chapter Matrices were first introduced in the Chinese Nine Chapters on the Mathematical
More informationElementary Row Operations on Matrices
King Saud University September 17, 018 Table of contents 1 Definition A real matrix is a rectangular array whose entries are real numbers. These numbers are organized on rows and columns. An m n matrix
More informationELEMENTARY LINEAR ALGEBRA
ELEMENTARY LINEAR ALGEBRA K R MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND First Printing, 99 Chapter LINEAR EQUATIONS Introduction to linear equations A linear equation in n unknowns x,
More informationMatrix Representation
Matrix Representation Matrix Rep. Same basics as introduced already. Convenient method of working with vectors. Superposition Complete set of vectors can be used to express any other vector. Complete set
More informationMATRICES AND MATRIX OPERATIONS
SIZE OF THE MATRIX is defined by number of rows and columns in the matrix. For the matrix that have m rows and n columns we say the size of the matrix is m x n. If matrix have the same number of rows (n)
More information2 Constructions of manifolds. (Solutions)
2 Constructions of manifolds. (Solutions) Last updated: February 16, 2012. Problem 1. The state of a double pendulum is entirely defined by the positions of the moving ends of the two simple pendula of
More informationMathematical Methods wk 2: Linear Operators
John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm
More informationMATRIX ALGEBRA. or x = (x 1,..., x n ) R n. y 1 y 2. x 2. x m. y m. y = cos θ 1 = x 1 L x. sin θ 1 = x 2. cos θ 2 = y 1 L y.
as Basics Vectors MATRIX ALGEBRA An array of n real numbers x, x,, x n is called a vector and it is written x = x x n or x = x,, x n R n prime operation=transposing a column to a row Basic vector operations
More informationLinear Systems and Matrices
Department of Mathematics The Chinese University of Hong Kong 1 System of m linear equations in n unknowns (linear system) a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.......
More informationEA = I 3 = E = i=1, i k
MTH5 Spring 7 HW Assignment : Sec.., # (a) and (c), 5,, 8; Sec.., #, 5; Sec.., #7 (a), 8; Sec.., # (a), 5 The due date for this assignment is //7. Sec.., # (a) and (c). Use the proof of Theorem. to obtain
More informationSection 9.2: Matrices.. a m1 a m2 a mn
Section 9.2: Matrices Definition: A matrix is a rectangular array of numbers: a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn In general, a ij denotes the (i, j) entry of A. That is, the entry in
More informationElementary maths for GMT
Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1
More informationIntroduction to Matrices
214 Analysis and Design of Feedback Control Systems Introduction to Matrices Derek Rowell October 2002 Modern system dynamics is based upon a matrix representation of the dynamic equations governing the
More information1 Matrices and Systems of Linear Equations
March 3, 203 6-6. Systems of Linear Equations Matrices and Systems of Linear Equations An m n matrix is an array A = a ij of the form a a n a 2 a 2n... a m a mn where each a ij is a real or complex number.
More informationa 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 11 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,, a n, b are given real
More informationCP3 REVISION LECTURES VECTORS AND MATRICES Lecture 1. Prof. N. Harnew University of Oxford TT 2013
CP3 REVISION LECTURES VECTORS AND MATRICES Lecture 1 Prof. N. Harnew University of Oxford TT 2013 1 OUTLINE 1. Vector Algebra 2. Vector Geometry 3. Types of Matrices and Matrix Operations 4. Determinants
More informationChapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations
Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2
More informationMath113: Linear Algebra. Beifang Chen
Math3: Linear Algebra Beifang Chen Spring 26 Contents Systems of Linear Equations 3 Systems of Linear Equations 3 Linear Systems 3 2 Geometric Interpretation 3 3 Matrices of Linear Systems 4 4 Elementary
More informationRank, Trace, Determinant, Transpose an Inverse of a Matrix Let A be an n n square matrix: A = a11 a1 a1n a1 a an a n1 a n a nn nn where is the jth col
Review of Linear Algebra { E18 Hanout Vectors an Their Inner Proucts Let X an Y be two vectors: an Their inner prouct is ene as X =[x1; ;x n ] T Y =[y1; ;y n ] T (X; Y ) = X T Y = x k y k k=1 where T an
More information3 (Maths) Linear Algebra
3 (Maths) Linear Algebra References: Simon and Blume, chapters 6 to 11, 16 and 23; Pemberton and Rau, chapters 11 to 13 and 25; Sundaram, sections 1.3 and 1.5. The methods and concepts of linear algebra
More informationComputational Foundations of Cognitive Science. Addition and Scalar Multiplication. Matrix Multiplication
Computational Foundations of Cognitive Science Lecture 1: Algebraic ; Transpose; Inner and Outer Product Frank Keller School of Informatics University of Edinburgh keller@inf.ed.ac.uk February 23, 21 1
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationMatrix Algebra, Class Notes (part 2) by Hrishikesh D. Vinod Copyright 1998 by Prof. H. D. Vinod, Fordham University, New York. All rights reserved.
Matrix Algebra, Class Notes (part 2) by Hrishikesh D. Vinod Copyright 1998 by Prof. H. D. Vinod, Fordham University, New York. All rights reserved. 1 Converting Matrices Into (Long) Vectors Convention:
More information1 Matrices and Systems of Linear Equations. a 1n a 2n
March 31, 2013 16-1 16. Systems of Linear Equations 1 Matrices and Systems of Linear Equations An m n matrix is an array A = (a ij ) of the form a 11 a 21 a m1 a 1n a 2n... a mn where each a ij is a real
More information. a m1 a mn. a 1 a 2 a = a n
Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by
More informationBare minimum on matrix algebra. Psychology 588: Covariance structure and factor models
Bare minimum on matrix algebra Psychology 588: Covariance structure and factor models Matrix multiplication 2 Consider three notations for linear combinations y11 y1 m x11 x 1p b11 b 1m y y x x b b n1
More informationA matrix over a field F is a rectangular array of elements from F. The symbol
Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F ) denotes the collection of all m n matrices over F Matrices will usually be denoted
More information10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )
c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.2-7.4) 1. When each of the functions F 1, F 2,..., F n in right-hand side
More informationSection 9.2: Matrices. Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns.
Section 9.2: Matrices Definition: A matrix A consists of a rectangular array of numbers, or elements, arranged in m rows and n columns. That is, a 11 a 12 a 1n a 21 a 22 a 2n A =...... a m1 a m2 a mn A
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationJim Lambers MAT 610 Summer Session Lecture 1 Notes
Jim Lambers MAT 60 Summer Session 2009-0 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra
More informationLecture 3 Linear Algebra Background
Lecture 3 Linear Algebra Background Dan Sheldon September 17, 2012 Motivation Preview of next class: y (1) w 0 + w 1 x (1) 1 + w 2 x (1) 2 +... + w d x (1) d y (2) w 0 + w 1 x (2) 1 + w 2 x (2) 2 +...
More informationWilliam Stallings Copyright 2010
A PPENDIX E B ASIC C ONCEPTS FROM L INEAR A LGEBRA William Stallings Copyright 2010 E.1 OPERATIONS ON VECTORS AND MATRICES...2 Arithmetic...2 Determinants...4 Inverse of a Matrix...5 E.2 LINEAR ALGEBRA
More informationM. Matrices and Linear Algebra
M. Matrices and Linear Algebra. Matrix algebra. In section D we calculated the determinants of square arrays of numbers. Such arrays are important in mathematics and its applications; they are called matrices.
More informationA FIRST COURSE IN LINEAR ALGEBRA. An Open Text by Ken Kuttler. Matrix Arithmetic
A FIRST COURSE IN LINEAR ALGEBRA An Open Text by Ken Kuttler Matrix Arithmetic Lecture Notes by Karen Seyffarth Adapted by LYRYX SERVICE COURSE SOLUTION Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)
More informationMATRICES. a m,1 a m,n A =
MATRICES Matrices are rectangular arrays of real or complex numbers With them, we define arithmetic operations that are generalizations of those for real and complex numbers The general form a matrix of
More informationELEMENTARY LINEAR ALGEBRA
ELEMENTARY LINEAR ALGEBRA K. R. MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND Corrected Version, 7th April 013 Comments to the author at keithmatt@gmail.com Chapter 1 LINEAR EQUATIONS 1.1
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationMatrix operations Linear Algebra with Computer Science Application
Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the
More informationLinear Algebra and Matrix Inversion
Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much
More informationMatrix Algebra Determinant, Inverse matrix. Matrices. A. Fabretti. Mathematics 2 A.Y. 2015/2016. A. Fabretti Matrices
Matrices A. Fabretti Mathematics 2 A.Y. 2015/2016 Table of contents Matrix Algebra Determinant Inverse Matrix Introduction A matrix is a rectangular array of numbers. The size of a matrix is indicated
More informationMatrix & Linear Algebra
Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84 Vectors Vectors Vector: A
More informationLinear Algebra. Linear Equations and Matrices. Copyright 2005, W.R. Winfrey
Copyright 2005, W.R. Winfrey Topics Preliminaries Systems of Linear Equations Matrices Algebraic Properties of Matrix Operations Special Types of Matrices and Partitioned Matrices Matrix Transformations
More informationDeterminants. Samy Tindel. Purdue University. Differential equations and linear algebra - MA 262
Determinants Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T. Determinants Differential equations
More informationMatrices A brief introduction
Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 44 Definitions Definition A matrix is a set of N real or complex
More informationSTAT200C: Review of Linear Algebra
Stat200C Instructor: Zhaoxia Yu STAT200C: Review of Linear Algebra 1 Review of Linear Algebra 1.1 Vector Spaces, Rank, Trace, and Linear Equations 1.1.1 Rank and Vector Spaces Definition A vector whose
More informationMathematics 13: Lecture 10
Mathematics 13: Lecture 10 Matrices Dan Sloughter Furman University January 25, 2008 Dan Sloughter (Furman University) Mathematics 13: Lecture 10 January 25, 2008 1 / 19 Matrices Recall: A matrix is a
More informationMatrix Algebra: Summary
May, 27 Appendix E Matrix Algebra: Summary ontents E. Vectors and Matrtices.......................... 2 E.. Notation.................................. 2 E..2 Special Types of Vectors.........................
More informationLecture Notes in Linear Algebra
Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................
More informationMatrices. Math 240 Calculus III. Wednesday, July 10, Summer 2013, Session II. Matrices. Math 240. Definitions and Notation.
function Matrices Calculus III Summer 2013, Session II Wednesday, July 10, 2013 Agenda function 1. 2. function function Definition An m n matrix is a rectangular array of numbers arranged in m horizontal
More informationCS100: DISCRETE STRUCTURES. Lecture 3 Matrices Ch 3 Pages:
CS100: DISCRETE STRUCTURES Lecture 3 Matrices Ch 3 Pages: 246-262 Matrices 2 Introduction DEFINITION 1: A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called an m x n
More informationI = i 0,
Special Types of Matrices Certain matrices, such as the identity matrix 0 0 0 0 0 0 I = 0 0 0, 0 0 0 have a special shape, which endows the matrix with helpful properties The identity matrix is an example
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More information7.5 Operations with Matrices. Copyright Cengage Learning. All rights reserved.
7.5 Operations with Matrices Copyright Cengage Learning. All rights reserved. What You Should Learn Decide whether two matrices are equal. Add and subtract matrices and multiply matrices by scalars. Multiply
More informationMatrix Arithmetic. a 11 a. A + B = + a m1 a mn. + b. a 11 + b 11 a 1n + b 1n = a m1. b m1 b mn. and scalar multiplication for matrices via.
Matrix Arithmetic There is an arithmetic for matrices that can be viewed as extending the arithmetic we have developed for vectors to the more general setting of rectangular arrays: if A and B are m n
More informationIntroduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX
Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX September 2007 MSc Sep Intro QT 1 Who are these course for? The September
More informationMaterials engineering Collage \\ Ceramic & construction materials department Numerical Analysis \\Third stage by \\ Dalya Hekmat
Materials engineering Collage \\ Ceramic & construction materials department Numerical Analysis \\Third stage by \\ Dalya Hekmat Linear Algebra Lecture 2 1.3.7 Matrix Matrix multiplication using Falk s
More informationELEMENTARY LINEAR ALGEBRA
ELEMENTARY LINEAR ALGEBRA K R MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND Second Online Version, December 998 Comments to the author at krm@mathsuqeduau All contents copyright c 99 Keith
More information7 Matrix Operations. 7.0 Matrix Multiplication + 3 = 3 = 4
7 Matrix Operations Copyright 017, Gregory G. Smith 9 October 017 The product of two matrices is a sophisticated operations with a wide range of applications. In this chapter, we defined this binary operation,
More informationMath Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
More information2 b 3 b 4. c c 2 c 3 c 4
OHSx XM511 Linear Algebra: Multiple Choice Questions for Chapter 4 a a 2 a 3 a 4 b b 1. What is the determinant of 2 b 3 b 4 c c 2 c 3 c 4? d d 2 d 3 d 4 (a) abcd (b) abcd(a b)(b c)(c d)(d a) (c) abcd(a
More informationTensor Analysis in Euclidean Space
Tensor Analysis in Euclidean Space James Emery Edited: 8/5/2016 Contents 1 Classical Tensor Notation 2 2 Multilinear Functionals 4 3 Operations With Tensors 5 4 The Directional Derivative 5 5 Curvilinear
More informationLinear Algebra: Lecture notes from Kolman and Hill 9th edition.
Linear Algebra: Lecture notes from Kolman and Hill 9th edition Taylan Şengül March 20, 2019 Please let me know of any mistakes in these notes Contents Week 1 1 11 Systems of Linear Equations 1 12 Matrices
More informationComputational Linear Algebra
Computational Linear Algebra PD Dr. rer. nat. habil. Ralf Peter Mundani Computation in Engineering / BGU Scientific Computing in Computer Science / INF Winter Term 2017/18 Part 2: Direct Methods PD Dr.
More informationDeterminants - Uniqueness and Properties
Determinants - Uniqueness and Properties 2-2-2008 In order to show that there s only one determinant function on M(n, R), I m going to derive another formula for the determinant It involves permutations
More informationMAT 2037 LINEAR ALGEBRA I web:
MAT 237 LINEAR ALGEBRA I 2625 Dokuz Eylül University, Faculty of Science, Department of Mathematics web: Instructor: Engin Mermut http://kisideuedutr/enginmermut/ HOMEWORK 2 MATRIX ALGEBRA Textbook: Linear
More informationMath Linear Algebra Final Exam Review Sheet
Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationMatrix Algebra. Matrix Algebra. Chapter 8 - S&B
Chapter 8 - S&B Algebraic operations Matrix: The size of a matrix is indicated by the number of its rows and the number of its columns. A matrix with k rows and n columns is called a k n matrix. The number
More informationLecture 9: Submatrices and Some Special Matrices
Lecture 9: Submatrices and Some Special Matrices Winfried Just Department of Mathematics, Ohio University February 2, 2018 Submatrices A submatrix B of a given matrix A is any matrix that can be obtained
More information