# MATRIX ALGEBRA. or x = (x 1,..., x n ) R n. y 1 y 2. x 2. x m. y m. y = cos θ 1 = x 1 L x. sin θ 1 = x 2. cos θ 2 = y 1 L y.

Save this PDF as:

Size: px
Start display at page:

Download "MATRIX ALGEBRA. or x = (x 1,..., x n ) R n. y 1 y 2. x 2. x m. y m. y = cos θ 1 = x 1 L x. sin θ 1 = x 2. cos θ 2 = y 1 L y."

## Transcription

1 as Basics Vectors MATRIX ALGEBRA An array of n real numbers x, x,, x n is called a vector and it is written x = x x n or x = x,, x n R n prime operation=transposing a column to a row Basic vector operations Multiplication with a constant c cx { cx cx = if c < c > cx m contraction expantion Addition of x, y R n x + y = x x + y y = x + y x + y x m y m x m + y m Inner Product or Scalar Product x = x y = y x n y n x y = x y + x y + + x n y n L x = length of x = x x Let x, y R and θ i denotes the angle between the vector and the x axis Then cos θ = x L x sin θ = x L x cos θ = y L y sin θ = y L y θ = angle between x and y θ = θ θ cos θ = cosθ θ = cos θ cos θ + sin θ sin θ

2 In general for x, y R n cos θ = x L x y L y + x L x y L y = x y L x Ly cos θ = x y L x L y or x y = L x L y cos θ Remark: cos θ = x and y are perpendicular Length of a vector x R n : L x = x + x + + x n How multiplication with a constant c changes the length? L x = c x + c x + + c x n = c L x Remarks: Let c = L x, then is a vector with unit length and with direction of x L x x If ax + bx = then x = b a x and x have the same direction but different length Unit vectors in R : e = e = Unit vectors in R n : Let e = x = x e = e n = = x e + x e + + x m e n x n Definition: The space of all n-tuples with scalar multiplication and addition as defined above, is called a vector space Definition: by = a x + a x + + a k x k is a linear combination of the vectors x,, x k The zero vector is defined as =

3 Definition: The vectors x, x,, x k are said to be linearly dependent if there exist k numbers a,, a k not all zero, such that a x + + a k x k = Otherwise x, x,, x k is said to be linearly independent Examples: i e = and e = are linearly independent, because if a e + a e = then a a = a = a = ii Similarly you can prove in R n that e,, e n are linearly independent iii Let x = x = 5 x 3 = Then x, x and x 3 are linearly dependent since x x + 3x 3 = Definition: A set of m linearly independent vectors in R m called a basis for the vector space of m-tuples Theorem: basis Every vector in R m can be expressed as a unique linear combination of a fixed Example Let e,, e m be a basis in R m Then x = x e + x e + + x m e m Definition: The length of a vector x is L x = x + x + + x m Definition: The inner product or dot product of two vectors x, y R m is x y = y x = x y + x y + + x m y m Remark: ilength of a vector x: L x = x x 3

4 ii Let us denote θ the angle between two vectors x, y R m Then cosθ = x y x x y y Definition: When the angle between two vectors x and y is θ = 9 or 7 we say that x and y are perpendicular or orthogonal Since cos 9 = cos 7 = x and y are perpendicular if x y = Notation Example x y The basis vectors e,, e m are mutually perpendicular e ie j =, i j bf Remark: For any vector x, the unit vector with direction x is fracx x x Results: a z is perpendicular to every vector x R m z = b If z is perpendicular to each vector x,, x k then z is perpendicular to their linear span If z x i =, i =,, k then z a x + a x + + a k x k = c Mutually perpendicular vectors are linearly independent Definition: The projection of x on y P y x = x y y L y If L y = then P y x = x yy Lemma: Let z = x y L y y Then x z is orthogonal to y Proof: y x x y y = y x x y y y L y L y = 4

5 GRAM-SCHMIDT ORTHOGONALIZATION PROCESS: Given linearly independent vectors x,, x k, there exists mutually perpendicular vectors u,, u k with the same linear span This may be constructed sequentially by setting u = x u = x x u u u u u 3 = x 3 x 3 u u u u x 3 u u u u u k = x k x k u u u u x k u k u u k u k k We can use unit length vectors z,, z k instead of u,, u k ; z j = u j u j u j j =,, k Example Compute u, u, z, z for x = 4 x = 3 5

6 MATRICES A matrix is a rectangular array of real numbers An m k columns A, R, Σ boldface letters, denote matrices a a a k a A m k = a a k a m a m a mk matrix has m rows and k A = {a ij } i =,, m, j =,, k For example an m matrix has m rows and column, it is an m dimensional vector or column matrix m matrix is a column vector, m matrix is a row vector Definition: Let A = a ij B = b ij be two m k matrices We say that the two matrices are equal A = B a ij = b ij i, j I e two matrices are equal if i their dimensionality is the same, ii every corresponding element is the same MATRIX OPERATIONS Let A, B be two m k matrices Matrix Addition: C = A + B is an m k matrix with elements c ij = a ij + b ij i =,, m j =,, k Scalar Multiplication: c arbitrary scalar A = a ij ca = Ac = B = b ij where b ij = ca ij = a ij c i =,, m j =,, k Matrix Substraction A = a ij B = b ij i =,, m j =,, k A B = A + B = C where C = c ij c ij = a ij b ij i =,, m, j =,, k Definition: Transpose of a matrix A = a ij, i =,, m j =,, k, A is defined as a k m matrix with element a ji j =,, k i =,, m That is the transpose of a matrix A is obtained from A by interchanging the rows and columns 6

7 Example: A = 3 A = 3 Theorem: Let A, B, C be m k matrices, c, d are scalars a A + B + C = A + B + C b A + B = B + A c ca + B = ca + cb d c + da = ca + da e A + B = A + B f cda = cda g ca = ca Definition:If numbers of rows are equal to the numbers columns of a matrix A, then it is called square matrix Definition: Let A be a k k square matrix The matrix A is said to be symmetric if A = A That is A is symmetric a ij = a ji i, j =,, k Examples: i Let I denotes the m m matrix with -s in the main diagonal and zeros elswhere Then I is symmetric ii 4 A = 4 Definition: I the k k matrix is defined as the identity matrix if it has ones only in the main diagonal and zeros elsewhere Definition: Matrix Product Suppose that A = a ij is an m n matrix B = b ij is an n k matrix Then AB = C = c ij, where C is and m k matrix with elements c ij, i =,, m j =,, k, where c ij is the scalar product of ith row of A with jth column of B, n c ij = a il b lj i =,, m j =,, k l= Example: A = 3 = 3 B = 3 = AB = = 4 4

8 Theorem: Properties of matrix multiplication A, B, C defined such that the indicated products are defined and a scalar c is given acab = cab babc = ABC cab + C = AB + AC db + CA = BA + CA eab = B A Important Remarks: AB BA AB = does not imply that A =, or B = Example: 3 4 = However, it is true, that if A = or B = then A B = Definition: The row rank of a matrix A is the maximum number of linearly independent rows, considered as vectors The column rank of A is the maximum number of independent columns considered as vectors Theorem: For any matrix A, the number of independent rows is equal to the number of independent columns Definition: Let A be a k k square matrix The matrix A is nonsingular if Ax = x = where x is a k-dimensional vector A is singular if there exists x such that Ax = Remark: Let a x + a x + + a k x k a Ax = x + a x + + a k x k a k x + a k x + + a kk x k = x a + x a + + x k a k where a,, a k are the column vectors of A Thus, for a nonsingular matrix A x a + + x k a k = x = x = = x k = i e nonsingularty is equivalent that the columns of A are linearly independent 8

9 Theorem: Let A be a k k nonsingular matrix Then there exists one and only one k k matrix B such that AB = BA = I where I is the identity Definition: The matrix B, such that AB = BA = I is called the inverse of A and it is denoted by A In fact if BA = I or AB = I then B = A 4 3 Example: A = A 3 = AA = = = I Theorem: a Inverse of a matrix A = a a a a A = A is given by a a a a b In general A is given the following way a ij = A A ji i+j where A ij is the value of the determinant of A ji obtained from A deleting the j-th row and i-th column of A Definition: The determinant of a square matrix A is a scalar denoted by A defined recursively if k = A = a, k A = a ij A ij i+j j= where A ij is the determinant of the k k matrix, obtained as deleting the ith row and jth column of A We can use the first row: A = k a j A j +j j= Theorem: The following statements are equivalent for a k k square matrix A; a Ax = x = ie A nonsingular 9

10 b A c there exists a matrix A such that AA = A A = I Theorem: Let A and B be k k square matrices, and suppose that both have inverse matrix Then a A = A b AB = B A Proof: a I = A A = A A b ABAB = ABB A = I Theorem: Let A and B be k k matrices a A = A b If each element of a row column of A is zero then A = c If any two rows or columns of A are identical, then A = d If A is nonsingular, then A = ie A A = A e AB = A B f ca = c k A Definition: Let A = a ij be a k k square matrix The trace of the matrix A is tra = k a ii i= Theorem: Let A, B be k k matrices, c scalar Then a trca = ctra b tra ± B = tra ± trb ctrab = trba d trb AB = tra e traa = k k a ij i= j= Definition: A square matrix A is said to be orthogonal if its rows considered as vectors are mutually perpendicular and have unit length, which means that AA = I Theorem: A matrix A is orthogonal if and only if A = A For an orthogonal matrix AA = A A = I Example: A = A A = is orthogonal A = = = I

11 Definiton: Let A be a k k matrix and I be the k k identity Then the scalars λ, λ,, λ k satisfy the polynomial equation A λi = called eigenvalues of A A λi = called characteristic equation λ Example: A = A λi = λ A λi = λ4 λ = λ = λ = 4 are the eigenvalues Definiton: Let A be a k k matrix Let λ be an eigenvalue of A If x such that Ax = λx then x is said to be an eigenvector of A associated with the eigenvalue λ Ax = λx = A λix = x col A λi + x col A λi + + x k col k A λi If x it means that the columns of A λi are linearly dependent Example: A = λ = λ = From the first expression, x x x x = = x x 4x 4x x = x 3x + 4x = x x = x = From the second expression x = 4x 3x + 4x = 4x x = x = Usual practise to determine an eigenvector with length one In the example: e i = x i x i x i i =, e = x e = x

12 Definition: Quadratic form Qx in k variables x,, x k, where x = x,, x k R k, is defined as Qx = x Ax where A is a fix k k matrix The quadratic form is a quadratic function of x, x,, x k Example: x, x x x = x, x x + x x x x + x x x = x + x x + x = x + x Theorem Let A be a k k symmetric matrix, i e A = A then A has k pairs of eigenvalues and eigenvectors λ, e, λ, e,, λ k, e k The eigenvectors can be chosen to satisfy = e e = = e ke k and be mutually perpendicular The eigenvectors are unique unless two or more eigenvalues are equal Definition: Positive Definite Matrices Let A be a symmetric matrix k k A is said to be positive definite if x Ax > for all x R k x A is positive semi-definite if x Ax for all x R k Theorem: Spectral decomposition of a symmetric k k matrix A is given: A = λ e e + λ e e + + λ k e k e k λ,, λ k are eigenvalues of A and e,, e k are the associated normalized eigenvectors Theorem A, k k symmetric matrix is positive definite if and only if every eigenvalue of A is positive A is positive semi-definite if and only if every eigenvalue of A is nonnegative Proof: Trivial from the spectral decomposition theorem A = λ e e + λ e e + + λ k e k e k x Ax = λ x e e x + λ x e e x + + λ k x ke k e kx = λ y + λ y + + λ k y k where y i = x e i Choosing x = e j j =,, k follows the theorem

13 Another form of the spectral decomposition: λ A = e e k e = PΛP λ k e k INVERZE AND SQUARE ROOT OF A POSITIVE DEFINIT MATRIX By the spectral decomposition, if A is positive definit: A = PΛP Inverse of A If A is positive definite, then the eigenvaluse of A are λ λ, λ λ k > i =,, k The inverse of A is A = PΛ P where Λ = Because; A A = PΛ P PΛP = PΛ ΛP = PP = I Square Root Matrix defined for positive semi-definite A Notice: A A = PΛ P PΛ P = A A = A k A = λ i e i e i = PΛ P i= λ k MATRIX INEQUALITIES AND MAXIMIZATION Cauchy-Schwarz Inequality: Let b, d R p, then b d b bd d, with equality if and only if b = cd Proof: The inequality is obvious when one of the vectors is the zero one For b xd we have that, < b xd b xd = b b xb d xd b + x d d = b b xb d + x d d Adding and substratcting b d /d d, we have that < b b b d d d + d d x b d d d 3

14 Choosing x = b d/d d follows the statement Extended Cauchy Schwartz Inequality: Let b, d R p and let B be a positive definite p p matrix Then b d b Bbd B d, with equality if and only if b = cb d for some constant c Maximization of Quadratic forms on the Unit Sphere: Let B be a p p positive definite matrix with eigenvectors e,, e p and associated eigenvalues λ,, λ p eigenvectors, where λ λ λ p Then Morover max x x Bx x x = λ x Bx min x x x x Bx max x e,,e k x x and the equality is attained when x = e k+ = λ k+, = λ p 4

### 2. Matrix Algebra and Random Vectors

2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns

### Basic Concepts in Matrix Algebra

Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1

### Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

### ELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS. 1. Linear Equations and Matrices

ELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS KOLMAN & HILL NOTES BY OTTO MUTZBAUER 11 Systems of Linear Equations 1 Linear Equations and Matrices Numbers in our context are either real numbers or complex

### Linear Algebra Massoud Malek

CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

### Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

### Def. The euclidian distance between two points x = (x 1,...,x p ) t and y = (y 1,...,y p ) t in the p-dimensional space R p is defined as

MAHALANOBIS DISTANCE Def. The euclidian distance between two points x = (x 1,...,x p ) t and y = (y 1,...,y p ) t in the p-dimensional space R p is defined as d E (x, y) = (x 1 y 1 ) 2 + +(x p y p ) 2

### Matrices and Vectors. Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A =

30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,

2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for

### HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)

HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe

### ELEMENTARY LINEAR ALGEBRA

ELEMENTARY LINEAR ALGEBRA K. R. MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND Corrected Version, 7th April 013 Comments to the author at keithmatt@gmail.com Chapter 1 LINEAR EQUATIONS 1.1

### Math Linear Algebra Final Exam Review Sheet

Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

### 4. Matrix inverses. left and right inverse. linear independence. nonsingular matrices. matrices with linearly independent columns

L. Vandenberghe ECE133A (Winter 2018) 4. Matrix inverses left and right inverse linear independence nonsingular matrices matrices with linearly independent columns matrices with linearly independent rows

### Problem Set 1. Homeworks will graded based on content and clarity. Please show your work clearly for full credit.

CSE 151: Introduction to Machine Learning Winter 2017 Problem Set 1 Instructor: Kamalika Chaudhuri Due on: Jan 28 Instructions This is a 40 point homework Homeworks will graded based on content and clarity

### Introduction to Matrices

214 Analysis and Design of Feedback Control Systems Introduction to Matrices Derek Rowell October 2002 Modern system dynamics is based upon a matrix representation of the dynamic equations governing the

### ICS 6N Computational Linear Algebra Symmetric Matrices and Orthogonal Diagonalization

ICS 6N Computational Linear Algebra Symmetric Matrices and Orthogonal Diagonalization Xiaohui Xie University of California, Irvine xhx@uci.edu Xiaohui Xie (UCI) ICS 6N 1 / 21 Symmetric matrices An n n

### MATH 369 Linear Algebra

Assignment # Problem # A father and his two sons are together 00 years old. The father is twice as old as his older son and 30 years older than his younger son. How old is each person? Problem # 2 Determine

### Linear Algebra Primer

Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

### Computational math: Assignment 1

Computational math: Assignment 1 Thanks Ting Gao for her Latex file 11 Let B be a 4 4 matrix to which we apply the following operations: 1double column 1, halve row 3, 3add row 3 to row 1, 4interchange

### Preliminary Linear Algebra 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 100

Preliminary Linear Algebra 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 100 Notation for all there exists such that therefore because end of proof (QED) Copyright c 2012

### Matrices and Determinants

Chapter1 Matrices and Determinants 11 INTRODUCTION Matrix means an arrangement or array Matrices (plural of matrix) were introduced by Cayley in 1860 A matrix A is rectangular array of m n numbers (or

### 1. General Vector Spaces

1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

### MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

### Chapter 3. Matrices. 3.1 Matrices

40 Chapter 3 Matrices 3.1 Matrices Definition 3.1 Matrix) A matrix A is a rectangular array of m n real numbers {a ij } written as a 11 a 12 a 1n a 21 a 22 a 2n A =.... a m1 a m2 a mn The array has m rows

### A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

### Mathematical Methods wk 2: Linear Operators

John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm

### Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

### MSc Quantitative Techniques Mathematics Weeks 1 and 2

MSc Quantitative Techniques Mathematics Weeks 1 and 2 DEPARTMENT OF ECONOMICS, MATHEMATICS AND STATISTICS LONDON WC1E 7HX September 2014 MSc Economics / Finance / Financial Economics / Financial Risk Management

### Recall the convention that, for us, all vectors are column vectors.

Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

### Notes on Mathematics

Notes on Mathematics - 12 1 Peeyush Chandra, A. K. Lal, V. Raghavendra, G. Santhanam 1 Supported by a grant from MHRD 2 Contents I Linear Algebra 7 1 Matrices 9 1.1 Definition of a Matrix......................................

### MA 265 FINAL EXAM Fall 2012

MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators

### Mathematical Foundations

Chapter 1 Mathematical Foundations 1.1 Big-O Notations In the description of algorithmic complexity, we often have to use the order notations, often in terms of big O and small o. Loosely speaking, for

### Math 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.

Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses

### Review of Linear Algebra

Review of Linear Algebra Dr Gerhard Roth COMP 40A Winter 05 Version Linear algebra Is an important area of mathematics It is the basis of computer vision Is very widely taught, and there are many resources

### Extra Problems for Math 2050 Linear Algebra I

Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as

Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous

### Image Registration Lecture 2: Vectors and Matrices

Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES

AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES JOEL A. TROPP Abstract. We present an elementary proof that the spectral radius of a matrix A may be obtained using the formula ρ(a) lim

### B553 Lecture 5: Matrix Algebra Review

B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations

### Linear Algebra for Machine Learning. Sargur N. Srihari

Linear Algebra for Machine Learning Sargur N. srihari@cedar.buffalo.edu 1 Overview Linear Algebra is based on continuous math rather than discrete math Computer scientists have little experience with it

### STAT 8260 Theory of Linear Models Lecture Notes

STAT 8260 Theory of Linear Models Lecture Notes Classical linear models are at the core of the field of statistics, and are probably the most commonly used set of statistical techniques in practice. For

### L3: Review of linear algebra and MATLAB

L3: Review of linear algebra and MATLAB Vector and matrix notation Vectors Matrices Vector spaces Linear transformations Eigenvalues and eigenvectors MATLAB primer CSCE 666 Pattern Analysis Ricardo Gutierrez-Osuna

### 1 Positive definiteness and semidefiniteness

Positive definiteness and semidefiniteness Zdeněk Dvořák May 9, 205 For integers a, b, and c, let D(a, b, c) be the diagonal matrix with + for i =,..., a, D i,i = for i = a +,..., a + b,. 0 for i = a +

### Chapter 7 Iterative Techniques in Matrix Algebra

Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition

### Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

### Optimization Theory. A Concise Introduction. Jiongmin Yong

October 11, 017 16:5 ws-book9x6 Book Title Optimization Theory 017-08-Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 017-08-Lecture Notes page Optimization

### JUST THE MATHS UNIT NUMBER 9.8. MATRICES 8 (Characteristic properties) & (Similarity transformations) A.J.Hobson

JUST THE MATHS UNIT NUMBER 9.8 MATRICES 8 (Characteristic properties) & (Similarity transformations) by A.J.Hobson 9.8. Properties of eigenvalues and eigenvectors 9.8. Similar matrices 9.8.3 Exercises

### 2. Review of Linear Algebra

2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

### 22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices

m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix

### Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v

### CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33

Linear Algebra 1/33 Vectors A vector is a magnitude and a direction Magnitude = v Direction Also known as norm, length Represented by unit vectors (vectors with a length of 1 that point along distinct

### 18.06SC Final Exam Solutions

18.06SC Final Exam Solutions 1 (4+7=11 pts.) Suppose A is 3 by 4, and Ax = 0 has exactly 2 special solutions: 1 2 x 1 = 1 and x 2 = 1 1 0 0 1 (a) Remembering that A is 3 by 4, find its row reduced echelon

### Review of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem

Review of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem Steven J. Miller June 19, 2004 Abstract Matrices can be thought of as rectangular (often square) arrays of numbers, or as

### Chapter Two Elements of Linear Algebra

Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to

### Linear Equations and Matrix

1/60 Chia-Ping Chen Professor Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear

### Stat 159/259: Linear Algebra Notes

Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the

### Linear Algebra. Workbook

Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx

### Linear Algebra Practice Problems

Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,

### Maths for Signals and Systems Linear Algebra in Engineering

Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE

### Inverses. Stephen Boyd. EE103 Stanford University. October 28, 2017

Inverses Stephen Boyd EE103 Stanford University October 28, 2017 Outline Left and right inverses Inverse Solving linear equations Examples Pseudo-inverse Left and right inverses 2 Left inverses a number

### 5.) For each of the given sets of vectors, determine whether or not the set spans R 3. Give reasons for your answers.

Linear Algebra - Test File - Spring Test # For problems - consider the following system of equations. x + y - z = x + y + 4z = x + y + 6z =.) Solve the system without using your calculator..) Find the

MTH50 Spring 07 HW Assignment 7 {From [FIS0]}: Sec 44 #4a h 6; Sec 5 #ad ac 4ae 4 7 The due date for this assignment is 04/05/7 Sec 44 #4a h Evaluate the erminant of the following matrices by any legitimate

### Linear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization

Linear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization January 9, 2015 1 Notation 1. Book pages without explicit citation refer to [1]. 2. R n denotes the set of real n-column vectors. 3.

### EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

### Systems of Algebraic Equations and Systems of Differential Equations

Systems of Algebraic Equations and Systems of Differential Equations Topics: 2 by 2 systems of linear equations Matrix expression; Ax = b Solving 2 by 2 homogeneous systems Functions defined on matrices

### Lecture 23: 6.1 Inner Products

Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such

### 1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)

Notes for 2017-01-30 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty

### Review of some mathematical tools

MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical

### Matrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix

Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Matrix Operations Matrix Addition and Matrix Scalar Multiply Matrix Multiply Matrix

### Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007

Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007 You have 1 hour and 20 minutes. No notes, books, or other references. You are permitted to use Maple during this exam, but you must start with a blank

### c c c c c c c c c c a 3x3 matrix C= has a determinant determined by

Linear Algebra Determinants and Eigenvalues Introduction: Many important geometric and algebraic properties of square matrices are associated with a single real number revealed by what s known as the determinant.

### Matrices and Deformation

ES 111 Mathematical Methods in the Earth Sciences Matrices and Deformation Lecture Outline 13 - Thurs 9th Nov 2017 Strain Ellipse and Eigenvectors One way of thinking about a matrix is that it operates

### Math 225 Linear Algebra II Lecture Notes. John C. Bowman University of Alberta Edmonton, Canada

Math 225 Linear Algebra II Lecture Notes John C Bowman University of Alberta Edmonton, Canada March 23, 2017 c 2010 John C Bowman ALL RIGHTS RESERVED Reproduction of these lecture notes in any form, in

### 7 : APPENDIX. Vectors and Matrices

7 : APPENDIX Vectors and Matrices An n-tuple vector x is defined as an ordered set of n numbers. Usually we write these numbers x 1,...,x n in a column in the order indicated by their subscripts. The transpose

### Preliminaries. Copyright c 2018 Dan Nettleton (Iowa State University) Statistics / 38

Preliminaries Copyright c 2018 Dan Nettleton (Iowa State University) Statistics 510 1 / 38 Notation for Scalars, Vectors, and Matrices Lowercase letters = scalars: x, c, σ. Boldface, lowercase letters

### MSc Quantitative Techniques Mathematics Weeks 1 and 2

MSc Quantitative Techniques Mathematics Weeks 1 and 2 DEPARTMENT OF ECONOMICS, MATHEMATICS AND STATISTICS LONDON WC1E 7HX September 2017 For MSc Programmes in Economics and Finance Who is this course for?

### CS100: DISCRETE STRUCTURES. Lecture 3 Matrices Ch 3 Pages:

CS100: DISCRETE STRUCTURES Lecture 3 Matrices Ch 3 Pages: 246-262 Matrices 2 Introduction DEFINITION 1: A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called an m x n

### Linear Algebra in Actuarial Science: Slides to the lecture

Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations

### ICS 6N Computational Linear Algebra Matrix Algebra

ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24 Matrix Consider an m n matrix

### Lecture 3: Review of Linear Algebra

ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,

### Elementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th.

Elementary Linear Algebra Review for Exam Exam is Monday, November 6th. The exam will cover sections:.4,..4, 5. 5., 7., the class notes on Markov Models. You must be able to do each of the following. Section.4

### av 1 x 2 + 4y 2 + xy + 4z 2 = 16.

74 85 Eigenanalysis The subject of eigenanalysis seeks to find a coordinate system, in which the solution to an applied problem has a simple expression Therefore, eigenanalysis might be called the method

### AM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition

AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized

### Fall Inverse of a matrix. Institute: UC San Diego. Authors: Alexander Knop

Fall 2017 Inverse of a matrix Authors: Alexander Knop Institute: UC San Diego Row-Column Rule If the product AB is defined, then the entry in row i and column j of AB is the sum of the products of corresponding

### Symmetric matrices and dot products

Symmetric matrices and dot products Proposition An n n matrix A is symmetric iff, for all x, y in R n, (Ax) y = x (Ay). Proof. If A is symmetric, then (Ax) y = x T A T y = x T Ay = x (Ay). If equality

### 2.1 Matrices. 3 5 Solve for the variables in the following matrix equation.

2.1 Matrices Reminder: A matrix with m rows and n columns has size m x n. (This is also sometimes referred to as the order of the matrix.) The entry in the ith row and jth column of a matrix A is denoted

### Lecture 3: Review of Linear Algebra

ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,

### Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam

Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system

### Appendix C Vector and matrix algebra

Appendix C Vector and matrix algebra Concepts Scalars Vectors, rows and columns, matrices Adding and subtracting vectors and matrices Multiplying them by scalars Products of vectors and matrices, scalar

### MAT 242 CHAPTER 4: SUBSPACES OF R n

MAT 242 CHAPTER 4: SUBSPACES OF R n JOHN QUIGG 1. Subspaces Recall that R n is the set of n 1 matrices, also called vectors, and satisfies the following properties: x + y = y + x x + (y + z) = (x + y)

### Advanced Digital Signal Processing -Introduction

Advanced Digital Signal Processing -Introduction LECTURE-2 1 AP9211- ADVANCED DIGITAL SIGNAL PROCESSING UNIT I DISCRETE RANDOM SIGNAL PROCESSING Discrete Random Processes- Ensemble Averages, Stationary

### Symmetric Matrices and Eigendecomposition

Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions

### Linear vector spaces and subspaces.

Math 2051 W2008 Margo Kondratieva Week 1 Linear vector spaces and subspaces. Section 1.1 The notion of a linear vector space. For the purpose of these notes we regard (m 1)-matrices as m-dimensional vectors,

### Theorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form. k k n-k. m-k k m-k n-k

I. REVIEW OF LINEAR ALGEBRA A. Equivalence Definition A1. If A and B are two m x n matrices, then A is equivalent to B if we can obtain B from A by a finite sequence of elementary row or elementary column

### Math 215 HW #9 Solutions

Math 5 HW #9 Solutions. Problem 4.4.. If A is a 5 by 5 matrix with all a ij, then det A. Volumes or the big formula or pivots should give some upper bound on the determinant. Answer: Let v i be the ith