MATRIX ALGEBRA. or x = (x 1,..., x n ) R n. y 1 y 2. x 2. x m. y m. y = cos θ 1 = x 1 L x. sin θ 1 = x 2. cos θ 2 = y 1 L y.


 Elaine Hall
 1 years ago
 Views:
Transcription
1 as Basics Vectors MATRIX ALGEBRA An array of n real numbers x, x,, x n is called a vector and it is written x = x x n or x = x,, x n R n prime operation=transposing a column to a row Basic vector operations Multiplication with a constant c cx { cx cx = if c < c > cx m contraction expantion Addition of x, y R n x + y = x x + y y = x + y x + y x m y m x m + y m Inner Product or Scalar Product x = x y = y x n y n x y = x y + x y + + x n y n L x = length of x = x x Let x, y R and θ i denotes the angle between the vector and the x axis Then cos θ = x L x sin θ = x L x cos θ = y L y sin θ = y L y θ = angle between x and y θ = θ θ cos θ = cosθ θ = cos θ cos θ + sin θ sin θ
2 In general for x, y R n cos θ = x L x y L y + x L x y L y = x y L x Ly cos θ = x y L x L y or x y = L x L y cos θ Remark: cos θ = x and y are perpendicular Length of a vector x R n : L x = x + x + + x n How multiplication with a constant c changes the length? L x = c x + c x + + c x n = c L x Remarks: Let c = L x, then is a vector with unit length and with direction of x L x x If ax + bx = then x = b a x and x have the same direction but different length Unit vectors in R : e = e = Unit vectors in R n : Let e = x = x e = e n = = x e + x e + + x m e n x n Definition: The space of all ntuples with scalar multiplication and addition as defined above, is called a vector space Definition: by = a x + a x + + a k x k is a linear combination of the vectors x,, x k The zero vector is defined as =
3 Definition: The vectors x, x,, x k are said to be linearly dependent if there exist k numbers a,, a k not all zero, such that a x + + a k x k = Otherwise x, x,, x k is said to be linearly independent Examples: i e = and e = are linearly independent, because if a e + a e = then a a = a = a = ii Similarly you can prove in R n that e,, e n are linearly independent iii Let x = x = 5 x 3 = Then x, x and x 3 are linearly dependent since x x + 3x 3 = Definition: A set of m linearly independent vectors in R m called a basis for the vector space of mtuples Theorem: basis Every vector in R m can be expressed as a unique linear combination of a fixed Example Let e,, e m be a basis in R m Then x = x e + x e + + x m e m Definition: The length of a vector x is L x = x + x + + x m Definition: The inner product or dot product of two vectors x, y R m is x y = y x = x y + x y + + x m y m Remark: ilength of a vector x: L x = x x 3
4 ii Let us denote θ the angle between two vectors x, y R m Then cosθ = x y x x y y Definition: When the angle between two vectors x and y is θ = 9 or 7 we say that x and y are perpendicular or orthogonal Since cos 9 = cos 7 = x and y are perpendicular if x y = Notation Example x y The basis vectors e,, e m are mutually perpendicular e ie j =, i j bf Remark: For any vector x, the unit vector with direction x is fracx x x Results: a z is perpendicular to every vector x R m z = b If z is perpendicular to each vector x,, x k then z is perpendicular to their linear span If z x i =, i =,, k then z a x + a x + + a k x k = c Mutually perpendicular vectors are linearly independent Definition: The projection of x on y P y x = x y y L y If L y = then P y x = x yy Lemma: Let z = x y L y y Then x z is orthogonal to y Proof: y x x y y = y x x y y y L y L y = 4
5 GRAMSCHMIDT ORTHOGONALIZATION PROCESS: Given linearly independent vectors x,, x k, there exists mutually perpendicular vectors u,, u k with the same linear span This may be constructed sequentially by setting u = x u = x x u u u u u 3 = x 3 x 3 u u u u x 3 u u u u u k = x k x k u u u u x k u k u u k u k k We can use unit length vectors z,, z k instead of u,, u k ; z j = u j u j u j j =,, k Example Compute u, u, z, z for x = 4 x = 3 5
6 MATRICES A matrix is a rectangular array of real numbers An m k columns A, R, Σ boldface letters, denote matrices a a a k a A m k = a a k a m a m a mk matrix has m rows and k A = {a ij } i =,, m, j =,, k For example an m matrix has m rows and column, it is an m dimensional vector or column matrix m matrix is a column vector, m matrix is a row vector Definition: Let A = a ij B = b ij be two m k matrices We say that the two matrices are equal A = B a ij = b ij i, j I e two matrices are equal if i their dimensionality is the same, ii every corresponding element is the same MATRIX OPERATIONS Let A, B be two m k matrices Matrix Addition: C = A + B is an m k matrix with elements c ij = a ij + b ij i =,, m j =,, k Scalar Multiplication: c arbitrary scalar A = a ij ca = Ac = B = b ij where b ij = ca ij = a ij c i =,, m j =,, k Matrix Substraction A = a ij B = b ij i =,, m j =,, k A B = A + B = C where C = c ij c ij = a ij b ij i =,, m, j =,, k Definition: Transpose of a matrix A = a ij, i =,, m j =,, k, A is defined as a k m matrix with element a ji j =,, k i =,, m That is the transpose of a matrix A is obtained from A by interchanging the rows and columns 6
7 Example: A = 3 A = 3 Theorem: Let A, B, C be m k matrices, c, d are scalars a A + B + C = A + B + C b A + B = B + A c ca + B = ca + cb d c + da = ca + da e A + B = A + B f cda = cda g ca = ca Definition:If numbers of rows are equal to the numbers columns of a matrix A, then it is called square matrix Definition: Let A be a k k square matrix The matrix A is said to be symmetric if A = A That is A is symmetric a ij = a ji i, j =,, k Examples: i Let I denotes the m m matrix with s in the main diagonal and zeros elswhere Then I is symmetric ii 4 A = 4 Definition: I the k k matrix is defined as the identity matrix if it has ones only in the main diagonal and zeros elsewhere Definition: Matrix Product Suppose that A = a ij is an m n matrix B = b ij is an n k matrix Then AB = C = c ij, where C is and m k matrix with elements c ij, i =,, m j =,, k, where c ij is the scalar product of ith row of A with jth column of B, n c ij = a il b lj i =,, m j =,, k l= Example: A = 3 = 3 B = 3 = AB = = 4 4
8 Theorem: Properties of matrix multiplication A, B, C defined such that the indicated products are defined and a scalar c is given acab = cab babc = ABC cab + C = AB + AC db + CA = BA + CA eab = B A Important Remarks: AB BA AB = does not imply that A =, or B = Example: 3 4 = However, it is true, that if A = or B = then A B = Definition: The row rank of a matrix A is the maximum number of linearly independent rows, considered as vectors The column rank of A is the maximum number of independent columns considered as vectors Theorem: For any matrix A, the number of independent rows is equal to the number of independent columns Definition: Let A be a k k square matrix The matrix A is nonsingular if Ax = x = where x is a kdimensional vector A is singular if there exists x such that Ax = Remark: Let a x + a x + + a k x k a Ax = x + a x + + a k x k a k x + a k x + + a kk x k = x a + x a + + x k a k where a,, a k are the column vectors of A Thus, for a nonsingular matrix A x a + + x k a k = x = x = = x k = i e nonsingularty is equivalent that the columns of A are linearly independent 8
9 Theorem: Let A be a k k nonsingular matrix Then there exists one and only one k k matrix B such that AB = BA = I where I is the identity Definition: The matrix B, such that AB = BA = I is called the inverse of A and it is denoted by A In fact if BA = I or AB = I then B = A 4 3 Example: A = A 3 = AA = = = I Theorem: a Inverse of a matrix A = a a a a A = A is given by a a a a b In general A is given the following way a ij = A A ji i+j where A ij is the value of the determinant of A ji obtained from A deleting the jth row and ith column of A Definition: The determinant of a square matrix A is a scalar denoted by A defined recursively if k = A = a, k A = a ij A ij i+j j= where A ij is the determinant of the k k matrix, obtained as deleting the ith row and jth column of A We can use the first row: A = k a j A j +j j= Theorem: The following statements are equivalent for a k k square matrix A; a Ax = x = ie A nonsingular 9
10 b A c there exists a matrix A such that AA = A A = I Theorem: Let A and B be k k square matrices, and suppose that both have inverse matrix Then a A = A b AB = B A Proof: a I = A A = A A b ABAB = ABB A = I Theorem: Let A and B be k k matrices a A = A b If each element of a row column of A is zero then A = c If any two rows or columns of A are identical, then A = d If A is nonsingular, then A = ie A A = A e AB = A B f ca = c k A Definition: Let A = a ij be a k k square matrix The trace of the matrix A is tra = k a ii i= Theorem: Let A, B be k k matrices, c scalar Then a trca = ctra b tra ± B = tra ± trb ctrab = trba d trb AB = tra e traa = k k a ij i= j= Definition: A square matrix A is said to be orthogonal if its rows considered as vectors are mutually perpendicular and have unit length, which means that AA = I Theorem: A matrix A is orthogonal if and only if A = A For an orthogonal matrix AA = A A = I Example: A = A A = is orthogonal A = = = I
11 Definiton: Let A be a k k matrix and I be the k k identity Then the scalars λ, λ,, λ k satisfy the polynomial equation A λi = called eigenvalues of A A λi = called characteristic equation λ Example: A = A λi = λ A λi = λ4 λ = λ = λ = 4 are the eigenvalues Definiton: Let A be a k k matrix Let λ be an eigenvalue of A If x such that Ax = λx then x is said to be an eigenvector of A associated with the eigenvalue λ Ax = λx = A λix = x col A λi + x col A λi + + x k col k A λi If x it means that the columns of A λi are linearly dependent Example: A = λ = λ = From the first expression, x x x x = = x x 4x 4x x = x 3x + 4x = x x = x = From the second expression x = 4x 3x + 4x = 4x x = x = Usual practise to determine an eigenvector with length one In the example: e i = x i x i x i i =, e = x e = x
12 Definition: Quadratic form Qx in k variables x,, x k, where x = x,, x k R k, is defined as Qx = x Ax where A is a fix k k matrix The quadratic form is a quadratic function of x, x,, x k Example: x, x x x = x, x x + x x x x + x x x = x + x x + x = x + x Theorem Let A be a k k symmetric matrix, i e A = A then A has k pairs of eigenvalues and eigenvectors λ, e, λ, e,, λ k, e k The eigenvectors can be chosen to satisfy = e e = = e ke k and be mutually perpendicular The eigenvectors are unique unless two or more eigenvalues are equal Definition: Positive Definite Matrices Let A be a symmetric matrix k k A is said to be positive definite if x Ax > for all x R k x A is positive semidefinite if x Ax for all x R k Theorem: Spectral decomposition of a symmetric k k matrix A is given: A = λ e e + λ e e + + λ k e k e k λ,, λ k are eigenvalues of A and e,, e k are the associated normalized eigenvectors Theorem A, k k symmetric matrix is positive definite if and only if every eigenvalue of A is positive A is positive semidefinite if and only if every eigenvalue of A is nonnegative Proof: Trivial from the spectral decomposition theorem A = λ e e + λ e e + + λ k e k e k x Ax = λ x e e x + λ x e e x + + λ k x ke k e kx = λ y + λ y + + λ k y k where y i = x e i Choosing x = e j j =,, k follows the theorem
13 Another form of the spectral decomposition: λ A = e e k e = PΛP λ k e k INVERZE AND SQUARE ROOT OF A POSITIVE DEFINIT MATRIX By the spectral decomposition, if A is positive definit: A = PΛP Inverse of A If A is positive definite, then the eigenvaluse of A are λ λ, λ λ k > i =,, k The inverse of A is A = PΛ P where Λ = Because; A A = PΛ P PΛP = PΛ ΛP = PP = I Square Root Matrix defined for positive semidefinite A Notice: A A = PΛ P PΛ P = A A = A k A = λ i e i e i = PΛ P i= λ k MATRIX INEQUALITIES AND MAXIMIZATION CauchySchwarz Inequality: Let b, d R p, then b d b bd d, with equality if and only if b = cd Proof: The inequality is obvious when one of the vectors is the zero one For b xd we have that, < b xd b xd = b b xb d xd b + x d d = b b xb d + x d d Adding and substratcting b d /d d, we have that < b b b d d d + d d x b d d d 3
14 Choosing x = b d/d d follows the statement Extended Cauchy Schwartz Inequality: Let b, d R p and let B be a positive definite p p matrix Then b d b Bbd B d, with equality if and only if b = cb d for some constant c Maximization of Quadratic forms on the Unit Sphere: Let B be a p p positive definite matrix with eigenvectors e,, e p and associated eigenvalues λ,, λ p eigenvectors, where λ λ λ p Then Morover max x x Bx x x = λ x Bx min x x x x Bx max x e,,e k x x and the equality is attained when x = e k+ = λ k+, = λ p 4
2. Matrix Algebra and Random Vectors
2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns
More informationBasic Concepts in Matrix Algebra
Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS. 1. Linear Equations and Matrices
ELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS KOLMAN & HILL NOTES BY OTTO MUTZBAUER 11 Systems of Linear Equations 1 Linear Equations and Matrices Numbers in our context are either real numbers or complex
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationDef. The euclidian distance between two points x = (x 1,...,x p ) t and y = (y 1,...,y p ) t in the pdimensional space R p is defined as
MAHALANOBIS DISTANCE Def. The euclidian distance between two points x = (x 1,...,x p ) t and y = (y 1,...,y p ) t in the pdimensional space R p is defined as d E (x, y) = (x 1 y 1 ) 2 + +(x p y p ) 2
More informationMatrices and Vectors. Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A =
30 MATHEMATICS REVIEW G A.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a twodimensional array of numbers A = a 11 a 12... a 1N a 21 a 22... a 2N...... a M1 a M2... a MN A matrix can
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationLecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,
2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for
More informationHOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)
HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe
More informationELEMENTARY LINEAR ALGEBRA
ELEMENTARY LINEAR ALGEBRA K. R. MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND Corrected Version, 7th April 013 Comments to the author at keithmatt@gmail.com Chapter 1 LINEAR EQUATIONS 1.1
More informationMath Linear Algebra Final Exam Review Sheet
Math 151 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a componentwise operation. Two vectors v and w may be added together as long as they contain the same number n of
More information4. Matrix inverses. left and right inverse. linear independence. nonsingular matrices. matrices with linearly independent columns
L. Vandenberghe ECE133A (Winter 2018) 4. Matrix inverses left and right inverse linear independence nonsingular matrices matrices with linearly independent columns matrices with linearly independent rows
More informationProblem Set 1. Homeworks will graded based on content and clarity. Please show your work clearly for full credit.
CSE 151: Introduction to Machine Learning Winter 2017 Problem Set 1 Instructor: Kamalika Chaudhuri Due on: Jan 28 Instructions This is a 40 point homework Homeworks will graded based on content and clarity
More informationIntroduction to Matrices
214 Analysis and Design of Feedback Control Systems Introduction to Matrices Derek Rowell October 2002 Modern system dynamics is based upon a matrix representation of the dynamic equations governing the
More informationICS 6N Computational Linear Algebra Symmetric Matrices and Orthogonal Diagonalization
ICS 6N Computational Linear Algebra Symmetric Matrices and Orthogonal Diagonalization Xiaohui Xie University of California, Irvine xhx@uci.edu Xiaohui Xie (UCI) ICS 6N 1 / 21 Symmetric matrices An n n
More informationMATH 369 Linear Algebra
Assignment # Problem # A father and his two sons are together 00 years old. The father is twice as old as his older son and 30 years older than his younger son. How old is each person? Problem # 2 Determine
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationComputational math: Assignment 1
Computational math: Assignment 1 Thanks Ting Gao for her Latex file 11 Let B be a 4 4 matrix to which we apply the following operations: 1double column 1, halve row 3, 3add row 3 to row 1, 4interchange
More informationPreliminary Linear Algebra 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 100
Preliminary Linear Algebra 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 100 Notation for all there exists such that therefore because end of proof (QED) Copyright c 2012
More informationMatrices and Determinants
Chapter1 Matrices and Determinants 11 INTRODUCTION Matrix means an arrangement or array Matrices (plural of matrix) were introduced by Cayley in 1860 A matrix A is rectangular array of m n numbers (or
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationMATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)
MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m
More informationChapter 3. Matrices. 3.1 Matrices
40 Chapter 3 Matrices 3.1 Matrices Definition 3.1 Matrix) A matrix A is a rectangular array of m n real numbers {a ij } written as a 11 a 12 a 1n a 21 a 22 a 2n A =.... a m1 a m2 a mn The array has m rows
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationMathematical Methods wk 2: Linear Operators
John Magorrian, magog@thphysoxacuk These are workinprogress notes for the secondyear course on mathematical methods The most uptodate version is available from http://wwwthphysphysicsoxacuk/people/johnmagorrian/mm
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationMSc Quantitative Techniques Mathematics Weeks 1 and 2
MSc Quantitative Techniques Mathematics Weeks 1 and 2 DEPARTMENT OF ECONOMICS, MATHEMATICS AND STATISTICS LONDON WC1E 7HX September 2014 MSc Economics / Finance / Financial Economics / Financial Risk Management
More informationRecall the convention that, for us, all vectors are column vectors.
Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists
More informationNotes on Mathematics
Notes on Mathematics  12 1 Peeyush Chandra, A. K. Lal, V. Raghavendra, G. Santhanam 1 Supported by a grant from MHRD 2 Contents I Linear Algebra 7 1 Matrices 9 1.1 Definition of a Matrix......................................
More informationMA 265 FINAL EXAM Fall 2012
MA 265 FINAL EXAM Fall 22 NAME: INSTRUCTOR S NAME:. There are a total of 25 problems. You should show work on the exam sheet, and pencil in the correct answer on the scantron. 2. No books, notes, or calculators
More informationMathematical Foundations
Chapter 1 Mathematical Foundations 1.1 BigO Notations In the description of algorithmic complexity, we often have to use the order notations, often in terms of big O and small o. Loosely speaking, for
More informationMath 443 Differential Geometry Spring Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook.
Math 443 Differential Geometry Spring 2013 Handout 3: Bilinear and Quadratic Forms This handout should be read just before Chapter 4 of the textbook. Endomorphisms of a Vector Space This handout discusses
More informationReview of Linear Algebra
Review of Linear Algebra Dr Gerhard Roth COMP 40A Winter 05 Version Linear algebra Is an important area of mathematics It is the basis of computer vision Is very widely taught, and there are many resources
More informationExtra Problems for Math 2050 Linear Algebra I
Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as
More informationConjugate Gradient (CG) Method
Conjugate Gradient (CG) Method by K. Ozawa 1 Introduction In the series of this lecture, I will introduce the conjugate gradient method, which solves efficiently large scale sparse linear simultaneous
More informationImage Registration Lecture 2: Vectors and Matrices
Image Registration Lecture 2: Vectors and Matrices Prof. Charlene Tsai Lecture Overview Vectors Matrices Basics Orthogonal matrices Singular Value Decomposition (SVD) 2 1 Preliminary Comments Some of this
More informationMATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2
MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a
More informationAN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES
AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES JOEL A. TROPP Abstract. We present an elementary proof that the spectral radius of a matrix A may be obtained using the formula ρ(a) lim
More informationB553 Lecture 5: Matrix Algebra Review
B553 Lecture 5: Matrix Algebra Review Kris Hauser January 19, 2012 We have seen in prior lectures how vectors represent points in R n and gradients of functions. Matrices represent linear transformations
More informationLinear Algebra for Machine Learning. Sargur N. Srihari
Linear Algebra for Machine Learning Sargur N. srihari@cedar.buffalo.edu 1 Overview Linear Algebra is based on continuous math rather than discrete math Computer scientists have little experience with it
More informationSTAT 8260 Theory of Linear Models Lecture Notes
STAT 8260 Theory of Linear Models Lecture Notes Classical linear models are at the core of the field of statistics, and are probably the most commonly used set of statistical techniques in practice. For
More informationL3: Review of linear algebra and MATLAB
L3: Review of linear algebra and MATLAB Vector and matrix notation Vectors Matrices Vector spaces Linear transformations Eigenvalues and eigenvectors MATLAB primer CSCE 666 Pattern Analysis Ricardo GutierrezOsuna
More information1 Positive definiteness and semidefiniteness
Positive definiteness and semidefiniteness Zdeněk Dvořák May 9, 205 For integers a, b, and c, let D(a, b, c) be the diagonal matrix with + for i =,..., a, D i,i = for i = a +,..., a + b,. 0 for i = a +
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra PerOlof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationChapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015
Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal
More informationOptimization Theory. A Concise Introduction. Jiongmin Yong
October 11, 017 16:5 wsbook9x6 Book Title Optimization Theory 01708Lecture Notes page 1 1 Optimization Theory A Concise Introduction Jiongmin Yong Optimization Theory 01708Lecture Notes page Optimization
More informationJUST THE MATHS UNIT NUMBER 9.8. MATRICES 8 (Characteristic properties) & (Similarity transformations) A.J.Hobson
JUST THE MATHS UNIT NUMBER 9.8 MATRICES 8 (Characteristic properties) & (Similarity transformations) by A.J.Hobson 9.8. Properties of eigenvalues and eigenvectors 9.8. Similar matrices 9.8.3 Exercises
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More information22m:033 Notes: 7.1 Diagonalization of Symmetric Matrices
m:33 Notes: 7. Diagonalization of Symmetric Matrices Dennis Roseman University of Iowa Iowa City, IA http://www.math.uiowa.edu/ roseman May 3, Symmetric matrices Definition. A symmetric matrix is a matrix
More informationInner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:
Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v
More informationCS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33
Linear Algebra 1/33 Vectors A vector is a magnitude and a direction Magnitude = v Direction Also known as norm, length Represented by unit vectors (vectors with a length of 1 that point along distinct
More information18.06SC Final Exam Solutions
18.06SC Final Exam Solutions 1 (4+7=11 pts.) Suppose A is 3 by 4, and Ax = 0 has exactly 2 special solutions: 1 2 x 1 = 1 and x 2 = 1 1 0 0 1 (a) Remembering that A is 3 by 4, find its row reduced echelon
More informationReview of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem
Review of Linear Algebra Definitions, Change of Basis, Trace, Spectral Theorem Steven J. Miller June 19, 2004 Abstract Matrices can be thought of as rectangular (often square) arrays of numbers, or as
More informationChapter Two Elements of Linear Algebra
Chapter Two Elements of Linear Algebra Previously, in chapter one, we have considered single first order differential equations involving a single unknown function. In the next chapter we will begin to
More informationLinear Equations and Matrix
1/60 ChiaPing Chen Professor Department of Computer Science and Engineering National Sun Yatsen University Linear Algebra Gaussian Elimination 2/60 Alpha Go Linear algebra begins with a system of linear
More informationStat 159/259: Linear Algebra Notes
Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the
More informationLinear Algebra. Workbook
Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx
More informationLinear Algebra Practice Problems
Linear Algebra Practice Problems Math 24 Calculus III Summer 25, Session II. Determine whether the given set is a vector space. If not, give at least one axiom that is not satisfied. Unless otherwise stated,
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE
More informationInverses. Stephen Boyd. EE103 Stanford University. October 28, 2017
Inverses Stephen Boyd EE103 Stanford University October 28, 2017 Outline Left and right inverses Inverse Solving linear equations Examples Pseudoinverse Left and right inverses 2 Left inverses a number
More information5.) For each of the given sets of vectors, determine whether or not the set spans R 3. Give reasons for your answers.
Linear Algebra  Test File  Spring Test # For problems  consider the following system of equations. x + y  z = x + y + 4z = x + y + 6z =.) Solve the system without using your calculator..) Find the
More informationMTH50 Spring 07 HW Assignment 7 {From [FIS0]}: Sec 44 #4a h 6; Sec 5 #ad ac 4ae 4 7 The due date for this assignment is 04/05/7 Sec 44 #4a h Evaluate the erminant of the following matrices by any legitimate
More informationLinear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization
Linear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization January 9, 2015 1 Notation 1. Book pages without explicit citation refer to [1]. 2. R n denotes the set of real ncolumn vectors. 3.
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150  Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationSystems of Algebraic Equations and Systems of Differential Equations
Systems of Algebraic Equations and Systems of Differential Equations Topics: 2 by 2 systems of linear equations Matrix expression; Ax = b Solving 2 by 2 homogeneous systems Functions defined on matrices
More informationLecture 23: 6.1 Inner Products
Lecture 23: 6.1 Inner Products WeiTa Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such
More information1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)
Notes for 20170130 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty
More informationReview of some mathematical tools
MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical
More informationMatrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix
Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Matrix Operations Matrix Addition and Matrix Scalar Multiply Matrix Multiply Matrix
More informationMath 224, Fall 2007 Exam 3 Thursday, December 6, 2007
Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007 You have 1 hour and 20 minutes. No notes, books, or other references. You are permitted to use Maple during this exam, but you must start with a blank
More informationc c c c c c c c c c a 3x3 matrix C= has a determinant determined by
Linear Algebra Determinants and Eigenvalues Introduction: Many important geometric and algebraic properties of square matrices are associated with a single real number revealed by what s known as the determinant.
More informationMatrices and Deformation
ES 111 Mathematical Methods in the Earth Sciences Matrices and Deformation Lecture Outline 13  Thurs 9th Nov 2017 Strain Ellipse and Eigenvectors One way of thinking about a matrix is that it operates
More informationMath 225 Linear Algebra II Lecture Notes. John C. Bowman University of Alberta Edmonton, Canada
Math 225 Linear Algebra II Lecture Notes John C Bowman University of Alberta Edmonton, Canada March 23, 2017 c 2010 John C Bowman ALL RIGHTS RESERVED Reproduction of these lecture notes in any form, in
More information7 : APPENDIX. Vectors and Matrices
7 : APPENDIX Vectors and Matrices An ntuple vector x is defined as an ordered set of n numbers. Usually we write these numbers x 1,...,x n in a column in the order indicated by their subscripts. The transpose
More informationPreliminaries. Copyright c 2018 Dan Nettleton (Iowa State University) Statistics / 38
Preliminaries Copyright c 2018 Dan Nettleton (Iowa State University) Statistics 510 1 / 38 Notation for Scalars, Vectors, and Matrices Lowercase letters = scalars: x, c, σ. Boldface, lowercase letters
More informationMSc Quantitative Techniques Mathematics Weeks 1 and 2
MSc Quantitative Techniques Mathematics Weeks 1 and 2 DEPARTMENT OF ECONOMICS, MATHEMATICS AND STATISTICS LONDON WC1E 7HX September 2017 For MSc Programmes in Economics and Finance Who is this course for?
More informationCS100: DISCRETE STRUCTURES. Lecture 3 Matrices Ch 3 Pages:
CS100: DISCRETE STRUCTURES Lecture 3 Matrices Ch 3 Pages: 246262 Matrices 2 Introduction DEFINITION 1: A matrix is a rectangular array of numbers. A matrix with m rows and n columns is called an m x n
More informationLinear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a ToolBox Linear Equation Systems Discretization of differential equations: solving linear equations
More informationICS 6N Computational Linear Algebra Matrix Algebra
ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24 Matrix Consider an m n matrix
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,
More informationElementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th.
Elementary Linear Algebra Review for Exam Exam is Monday, November 6th. The exam will cover sections:.4,..4, 5. 5., 7., the class notes on Markov Models. You must be able to do each of the following. Section.4
More informationav 1 x 2 + 4y 2 + xy + 4z 2 = 16.
74 85 Eigenanalysis The subject of eigenanalysis seeks to find a coordinate system, in which the solution to an applied problem has a simple expression Therefore, eigenanalysis might be called the method
More informationAM 205: lecture 8. Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition
AM 205: lecture 8 Last time: Cholesky factorization, QR factorization Today: how to compute the QR factorization, the Singular Value Decomposition QR Factorization A matrix A R m n, m n, can be factorized
More informationFall Inverse of a matrix. Institute: UC San Diego. Authors: Alexander Knop
Fall 2017 Inverse of a matrix Authors: Alexander Knop Institute: UC San Diego RowColumn Rule If the product AB is defined, then the entry in row i and column j of AB is the sum of the products of corresponding
More informationSymmetric matrices and dot products
Symmetric matrices and dot products Proposition An n n matrix A is symmetric iff, for all x, y in R n, (Ax) y = x (Ay). Proof. If A is symmetric, then (Ax) y = x T A T y = x T Ay = x (Ay). If equality
More information2.1 Matrices. 3 5 Solve for the variables in the following matrix equation.
2.1 Matrices Reminder: A matrix with m rows and n columns has size m x n. (This is also sometimes referred to as the order of the matrix.) The entry in the ith row and jth column of a matrix A is denoted
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,
More informationMath 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam
Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system
More informationAppendix C Vector and matrix algebra
Appendix C Vector and matrix algebra Concepts Scalars Vectors, rows and columns, matrices Adding and subtracting vectors and matrices Multiplying them by scalars Products of vectors and matrices, scalar
More informationMAT 242 CHAPTER 4: SUBSPACES OF R n
MAT 242 CHAPTER 4: SUBSPACES OF R n JOHN QUIGG 1. Subspaces Recall that R n is the set of n 1 matrices, also called vectors, and satisfies the following properties: x + y = y + x x + (y + z) = (x + y)
More informationAdvanced Digital Signal Processing Introduction
Advanced Digital Signal Processing Introduction LECTURE2 1 AP9211 ADVANCED DIGITAL SIGNAL PROCESSING UNIT I DISCRETE RANDOM SIGNAL PROCESSING Discrete Random Processes Ensemble Averages, Stationary
More informationSymmetric Matrices and Eigendecomposition
Symmetric Matrices and Eigendecomposition Robert M. Freund January, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 2 1 Symmetric Matrices and Convexity of Quadratic Functions
More informationLinear vector spaces and subspaces.
Math 2051 W2008 Margo Kondratieva Week 1 Linear vector spaces and subspaces. Section 1.1 The notion of a linear vector space. For the purpose of these notes we regard (m 1)matrices as mdimensional vectors,
More informationTheorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form. k k nk. mk k mk nk
I. REVIEW OF LINEAR ALGEBRA A. Equivalence Definition A1. If A and B are two m x n matrices, then A is equivalent to B if we can obtain B from A by a finite sequence of elementary row or elementary column
More informationMath 215 HW #9 Solutions
Math 5 HW #9 Solutions. Problem 4.4.. If A is a 5 by 5 matrix with all a ij, then det A. Volumes or the big formula or pivots should give some upper bound on the determinant. Answer: Let v i be the ith
More informationHomework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I)
CS 106 Spring 2004 Homework 1 Elena Davidson 8 April 2004 Problem 1.1 Let B be a 4 4 matrix to which we apply the following operations: 1. double column 1, 2. halve row 3, 3. add row 3 to row 1, 4. interchange
More informationChapter 3: Theory Review: Solutions Math 308 F Spring 2015
Chapter : Theory Review: Solutions Math 08 F Spring 05. What two properties must a function T : R m R n satisfy to be a linear transformation? (a) For all vectors u and v in R m, T (u + v) T (u) + T (v)
More information