2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
|
|
- Kory Sullivan
- 5 years ago
- Views:
Transcription
1 FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian 2-1
2 Vector notation n-vector x: x = x 1 x n x 2. also written as x = (x 1, x 2,..., x n ) set of n-vectors is denoted R n (Euclidean space) x i : ith element or component or entry of x x is also called a column vector y = [ ] y 1 y 2 y n is called a row vector unless stated otherwise, a vector typically means a column vector Linear algebra 2-2
3 zero vectors: x = (0, 0,..., 0) Special vectors all-ones vectors: x = (1, 1,, 1) (we will denote it by 1) standard unit vectors: e k has only 1 at the kth entry and zero otherwise (standard unit vectors in R 3 ) e 1 = 1 0, e 2 = 0 1, e 3 = unit vectors: any vector u whose norm (magnitude) is 1, i.e., u example: u = (1/ 2, 2/ 6, 1/ 2) u u u2 n = 1 Linear algebra 2-3
4 Inner products definition: the inner product of two n-vectors x, y is x 1 y 1 + x 2 y x n y n also known as the dot product of vectors x, y notation: x T y properties (αx) T y = α(x T y) for scalar α (x + y) T z = x T z + y T z x T y = y T x Linear algebra 2-4
5 properties x = Euclidean norm x x x2 n = x T x also written x 2 to distinguish from other norms αx = α x for scalar α x + y x + y (triangle inequality) x 0 and x = 0 only if x = 0 interpretation x measures the magnitude or length of x x y measures the distance between x and y Linear algebra 2-5
6 Matrix notation an m n matrix A is defined as A = a 11 a a 1n a 21 a a 2n a m1 a m2... a mn, or A = [a ij] m n a ij are the elements, or coefficients, or entries of A set of m n-matrices is denoted R m n A has m rows and n columns (m, n are the dimensions) the (i, j) entry of A is also commonly denoted by A ij A is called a square matrix if m = n Linear algebra 2-6
7 Special matrices zero matrix: A = 0 A = a ij = 0, for i = 1,..., m, j = 1,..., n identity matrix: A = I A = a square matrix with a ii = 1, a ij = 0 for i j Linear algebra 2-7
8 diagonal matrix: a square matrix with a ij = 0 for i j A = a a a n triangular matrix: a square matrix with zero entries in a triangular part upper triangular a 11 a 12 a 1n A = 0 a 22 a 2n..... A = 0 0 a nn a ij = 0 for i j lower triangular a a 21 a a n1 a n2 a nn a ij = 0 for i j Linear algebra 2-8
9 Block matrix notation example: 2 2-block matrix A A = [ ] B C D E for example, if B, C, D, E are defined as B = [ ] 2 1, C = 3 8 then A is the matrix [ ] 0 1 7, D = [ 0 1 ], E = [ ] A = note: dimensions of the blocks must be compatible Linear algebra 2-9
10 Column and Row partitions write an m n-matrix A in terms of its columns or its rows A = [ a 1 a 2 a n ] = b T 1 b T 2. b T m a j for j = 1, 2,..., n are the columns of A b T i for i = 1, 2,..., m are the rows of A example: A = [ 1 2 ] [ [ [ a 1 =, a 4] 2 =, a 9] 3 =, b 0] T 1 = [ ], b T 2 = [ ] Linear algebra 2-10
11 Matrix-vector product product of m n-matrix A with n-vector x Ax = a 11 x 1 + a 12 x a 1n x n a 21 x 1 + a 22 x a 2n x n. a m1 x 1 + a m2 x a mn x n dimensions must be compatible: # columns in A = # elements in x if A is partitioned as A = [ a 1 a 2 a n ], then Ax = a 1 x 1 + a 2 x a n x n Ax is a linear combination of the column vectors of A the coefficients are the entries of x Linear algebra 2-11
12 Product with standard unit vectors post-multiply with a column vector 0 a 11 a a 1n Ae k = a 21 a a 2n a m1 a m2... a mn 1. 0 = a 1k a 2k. a mk = the kth column of A pre-multiply with a row vector e T k A = [ ] a 11 a a 1n a 21 a a 2n a m1 a m2... a mn = [ a k1 a k2 a kn ] = the kth row of A Linear algebra 2-12
13 Trace Definition: trace of a square matrix A is the sum of the diagonal entries in A tr(a) = a 11 + a a nn example: trace of A is = 7 properties tr(a T ) = tr(a) A = tr(αa + B) = α tr(a) + tr(b) tr(ab) = tr(ba) Linear algebra 2-13
14 Inverse of matrices Definition: a square matrix A is called invertible or nonsingular if there exists B s.t. AB = BA = I B is called an inverse of A it is also true that B is invertible and A is an inverse of B if no such B can be found A is said to be singular assume A is invertible an inverse of A is unique the inverse of A is denoted by A 1 Linear algebra 2-14
15 assume A, B are invertible Facts (αa) 1 = α 1 A 1 for nonzero α A T is also invertible and (A T ) 1 = (A 1 ) T AB is invertible and (AB) 1 = B 1 A 1 (A + B) 1 A 1 + B 1 Linear algebra 2-15
16 Inverse of 2 2 matrices the matrix A = [ ] a b c d is invertible if and only if and its inverse is given by A 1 = ad bc 0 1 ad bc [ d ] b c a example: A = [ ] 2 1, A 1 = [ ] Linear algebra 2-16
17 Invertible matrices Theorem: for a square matrix A, the following statements are equivalent 1. A is invertible 2. Ax = 0 has only the trivial solution (x = 0) 3. the reduced echelon form of A is I 4. A is invertible if and only if det(a) 0 Linear algebra 2-17
18 Inverse of special matrices diagonal matrix A = a a a n a diagonal matrix is invertible iff the diagonal entries are all nonzero a ii 0, i = 1, 2,..., n the inverse of A is given by A 1 = 1/a /a /a n the diagonal entries in A 1 are the inverse of the diagonal entries in A Linear algebra 2-18
19 triangular matrix: upper triangular a 11 a 12 a 1n A = 0 a 22 a 2n..... A = 0 0 a nn a ij = 0 for i j lower triangular a a 21 a a n1 a n2 a nn a ij = 0 for i j a triangular matrix is invertible iff the diagonal entries are all nonzero a ii 0, i = 1, 2,..., n product of lower (upper) triangular matrices is lower (upper) triangular the inverse of a lower (upper) triangular matrix is lower (upper) triangular Linear algebra 2-19
20 symmetric matrix: A = A T for any square matrix A, AA T and A T A are always symmetric if A is symmetric and invertible, then A 1 is symmetric if A is invertible, then AA T and A T A are also invertible Linear algebra 2-20
21 Symmetric matrix A R n n is called symmetric if A = A T Facts: if A is symmetric all eigenvalues of A are real all eigenvectors of A are orthogonal A admits a decomposition A = UDU T where U T U = UU T = I (U is unitary) and D is diagonal (of course, the diagonals of D are eigenvalues of A) Linear algebra 2-21
22 a matrix U R n n is called unitary if Unitary matrix U T U = UU T = I [ ] 1 example: 2 1 1, 1 1 Facts: [ cos θ ] sin θ sin θ cos θ a real unitary matrix is also called orthogonal a unitary matrix is always invertible and U 1 = U T columns vectors of U are mutually orthogonal norm is preserved under a unitary transformation: y = Ux = y = x Linear algebra 2-22
23 Orthogonal projection matrix P is said to be an orthogonal projection if P = P T and P 2 = P examples: P = P is bounded, i.e., P x x [ ] 1 0, P = 0 0 [ 1/2 ] 1/2 1/2 1/2 y = (y P y)+p y, and P y (y P y) (by using P = P T and P 2 = P ) hence, y 2 = P y 2 and that the norm of P y must be less than y if P is an orthogonal projection onto a line spanned by a unit vector u, P = uu T (we see that rank(p ) = 1 as the dimension of a line is 1) another example: P = A(A T A) 1 A T for any matrix A Linear algebra 2-23
24 Positive definite matrix a symmetric matrix A is positive semidefinite, written as A 0 if x T Ax 0, x R n and positive definite, written as A 0 if x T Ax > 0, for all nonzero x R n Facts: A 0 if and only if all eigenvalues of A are non-negative all principle minors of A are non-negative Linear algebra 2-24
25 example: A = [ 1 ] or we can check from 0 because x T Ax = [ ] [ ] [ ] 1 1 x1 x 1 x x 2 = x x 2 2 2x 1 x 2 = (x 1 x 2 ) 2 + x eigenvalues of A are 0.38 and 2.61 (real and positive) the principle minors are 1 and = 1 (all positive) note: A 0 does not mean all entries of A are positive! Linear algebra 2-25
26 Schur complement a consider a symmetric matrix X partitioned as [ ] A B X = B T C Schur complement of A in X is defined as S 1 = C B T A 1 B, if det A 0 Schur complement of C in X is defined as S 2 = A BC 1 B T, if det C 0 we can show that det X = det A det S 1 = det C det S 2 Linear algebra 2-26
27 Schur complement of positive definite matrix Facts: X 0 if and only if A 0 and S 1 0 if A 0 then X 0 if and only if S 1 0 analogous results for S 2 X 0 if and only if C 0 and S 2 0 if C 0 then X 0 if and only if S 2 0 Linear algebra 2-27
28 Linear equations a general linear system of m equations with n variables is described by a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2. =. a m1 x 1 + a m2 x a mn x n = b m where a ij, b j are constants and x 1, x 2,..., x n are unknowns equations are linear in x 1, x 2,..., x n existence and uniqueness of a solution depend on a ij and b j Linear algebra 2-28
29 Linear equation in matrix form the linear system of m equations in n variables a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2. =. a m1 x 1 + a m2 x a mn x n = b m in matrix form: Ax = b where A = a 11 a a 1n a 21 a a 2n a m1 a m2... a mn, x = x 1 x 2. x n, b = b 1 b 2. b m Linear algebra 2-29
30 Three types of linear equations square if m = n [ ] [ ] a11 a 12 x1 = a 22 x 2 a 21 [ b1 b 2 ] (A is square) underdetermined if m < n [ ] a11 a 12 a 13 x 1 x a 21 a 22 a 2 = 23 x 3 [ b1 b 2 ] (A is fat) overdetermined if m > n a 11 a 12 [ ] a 21 a 22 x1 = x a 31 a 2 32 b 1 b 2 b 3 (A is skinny) Linear algebra 2-30
31 Existence and uniqueness of solutions existence: no solution a solution exists uniqueness: the solution is unique there are infinitely many solutions every system of linear equations has zero, one, or infinitely many solutions there are no other possibilities Linear algebra 2-31
32 Nullspace the nullspace of an m n matrix is defined as N (A) = {x R n Ax = 0} the set of all vectors that are mapped to zero by f(x) = Ax the set of all vectors that are orthogonal to the rows of A if Ax = b then A(x + z) = b for all z N (A) also known as kernel of A N (A) is a subspace of R n Linear algebra 2-32
33 Zero nullspace matrix A has a zero nullspace if N (A) = {0} if A has a zero nullspace and Ax = b is solvable, the solution is unique columns of A are independent equivalent conditions: A R n n A has a zero nullspace A is invertible or nonsingular columns of A are a basis for R n Linear algebra 2-33
34 Range space the range of an m n matrix A is defined as R(A) = {y R m y = Ax for some x R n } the set of all m-vectors that can be expressed as Ax the set of all linear combinations of the columns of A = [ ] a 1 a n R(A) = {y y = x 1 a 1 + x 2 a x n a n, x R n } the set of all vectors b for which Ax = b is solvable also known as the column space of A R(A) is a subspace of R m Linear algebra 2-34
35 Full range matrices A has a full range if R(A) = R m equivalent conditions: A has a full range columns of A span R m Ax = b is solvable for every b N (A T ) = {0} Linear algebra 2-35
36 Rank and Nullity rank of a matrix A R m n is defined as rank(a) = dim R(A) nullity of a matrix A R m n is nullity(a) = dim N (A) Facts rank(a) is maximum number of independent columns (or rows) of A rank(a) min(m, n) rank(a) = rank(a T ) Linear algebra 2-36
37 Full rank matrices for A R m n we always have rank(a) min(m, n) we say A is full rank if rank(a) = min(m, n) for square matrices, full rank means nonsingular (invertible) for skinny matrices (m n), full rank means columns are independent for fat matrices (m n), full rank means rows are independent Linear algebra 2-37
38 Theorems Rank-Nullity Theorem: for any A R m n, rank(a) + dim N (A) = n the system Ax = b has a solution if and only if b R(A) the system Ax = b has a unique solution if and only if b R(A), and N (A) = {0} Linear algebra 2-38
39 Derivative and Gradient Suppose f : R n R m and x int dom f the derivative (or Jacobian) of f at x is the matrix Df(x) R m n : Df(x) ij = f i(x) x j, i = 1,..., m, j = 1,..., n when f is scalar-valued (i.e., f : R n R), the derivative Df(x) is a row vector its transpose is called the gradient of the function: f(x) = Df(x) T, f(x) i = f(x) x i, i = 1,..., n which is a column vector in R n Linear algebra 2-39
40 Second Derivative suppose f is a scalar-valued function (i.e., f : R n R) the second derivative or Hessian matrix of f at x, denoted 2 f(x) is 2 f(x) ij = 2 f(x) x i x j, i = 1,..., n, j = 1,..., n example: the quadratic function f : R n R where P S n, q R n, and r R f(x) = P x + q 2 f(x) = P f(x) = (1/2)x T P x + q T x + r, Linear algebra 2-40
41 Chain rule assumptions: f : R n R m is differentiable at x int dom f g : R m R p is differentiable at f(x) int dom g define the composition h : R n R p by h(z) = g(f(z)) then h is differentiable at x, with derivative Dh(x) = Dg(f(x))Df(x) special case: f : R n R, g : R R, and h(x) = g(f(x)) h(x) = g (f(x)) f(x) Linear algebra 2-41
42 example: h(x) = f(ax + b) Dh(x) = Df(Ax + b)a h(x) = A T f(ax + b) example: h(x) = (1/2)(Ax b) T P (Ax b) h(x) = A T P (Ax b) Linear algebra 2-42
43 References Chapter 1 in H. Anton, Elementary Linear Algebra, 10th edition, Wiley, 2010 R.A. Horn and C.R. Johnson, Matrix analysis, Cambridge press, 2012 K.B. Petersen, M.S. Pedersen, et.al., The Matrix Cookbook, Technical University of Denmark, 2008 Linear algebra 2-43
Chapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationMAT Linear Algebra Collection of sample exams
MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationLinear Algebra Formulas. Ben Lee
Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries
More informationELE/MCE 503 Linear Algebra Facts Fall 2018
ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More informationGlossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB
Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the
More informationMATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.
MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Row-column rule: i-j-th entry of AB:
More informationMath Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
More informationBasic Concepts in Linear Algebra
Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationMath Linear Algebra Final Exam Review Sheet
Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationLinear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization
Linear Algebra & Analysis Review UW EE/AA/ME 578 Convex Optimization January 9, 2015 1 Notation 1. Book pages without explicit citation refer to [1]. 2. R n denotes the set of real n-column vectors. 3.
More informationReview of Basic Concepts in Linear Algebra
Review of Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University September 7, 2017 Math 565 Linear Algebra Review September 7, 2017 1 / 40 Numerical Linear Algebra
More informationMath 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam
Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationEE202 - EE MATH II 1. Mathematical Proofs Jitkomut Songsiri conditional statements sufficient and necessary conditions methods of proofs disproving statements proofs of quantified statements 1-1 Statements
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationLinear Algebra Highlights
Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationLINEAR ALGEBRA REVIEW
LINEAR ALGEBRA REVIEW SPENCER BECKER-KAHN Basic Definitions Domain and Codomain. Let f : X Y be any function. This notation means that X is the domain of f and Y is the codomain of f. This means that for
More information2. Every linear system with the same number of equations as unknowns has a unique solution.
1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations
More informationMath 4A Notes. Written by Victoria Kala Last updated June 11, 2017
Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationElementary Linear Algebra Review for Exam 2 Exam is Monday, November 16th.
Elementary Linear Algebra Review for Exam Exam is Monday, November 6th. The exam will cover sections:.4,..4, 5. 5., 7., the class notes on Markov Models. You must be able to do each of the following. Section.4
More informationLecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,
2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for
More informationChapter 5 Eigenvalues and Eigenvectors
Chapter 5 Eigenvalues and Eigenvectors Outline 5.1 Eigenvalues and Eigenvectors 5.2 Diagonalization 5.3 Complex Vector Spaces 2 5.1 Eigenvalues and Eigenvectors Eigenvalue and Eigenvector If A is a n n
More informationRow Space and Column Space of a Matrix
Row Space and Column Space of a Matrix 1/18 Summary: To a m n matrix A = (a ij ), we can naturally associate subspaces of K n and of K m, called the row space of A and the column space of A, respectively.
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationLINEAR ALGEBRA SUMMARY SHEET.
LINEAR ALGEBRA SUMMARY SHEET RADON ROSBOROUGH https://intuitiveexplanationscom/linear-algebra-summary-sheet/ This document is a concise collection of many of the important theorems of linear algebra, organized
More informationReview of Some Concepts from Linear Algebra: Part 2
Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set
More informationRank and Nullity. MATH 322, Linear Algebra I. J. Robert Buchanan. Spring Department of Mathematics
Rank and Nullity MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 2015 Objectives We have defined and studied the important vector spaces associated with matrices (row space,
More informationMath 313 Chapter 5 Review
Math 313 Chapter 5 Review Howard Anton, 9th Edition May 2010 Do NOT write on me! Contents 1 5.1 Real Vector Spaces 2 2 5.2 Subspaces 3 3 5.3 Linear Independence 4 4 5.4 Basis and Dimension 5 5 5.5 Row
More informationSingular Value Decomposition (SVD)
School of Computing National University of Singapore CS CS524 Theoretical Foundations of Multimedia More Linear Algebra Singular Value Decomposition (SVD) The highpoint of linear algebra Gilbert Strang
More informationChapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in
Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column
More informationSolutions to Final Practice Problems Written by Victoria Kala Last updated 12/5/2015
Solutions to Final Practice Problems Written by Victoria Kala vtkala@math.ucsb.edu Last updated /5/05 Answers This page contains answers only. See the following pages for detailed solutions. (. (a x. See
More informationNOTES on LINEAR ALGEBRA 1
School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura
More informationMATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)
MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m
More informationMath Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationCS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
More informationCSL361 Problem set 4: Basic linear algebra
CSL361 Problem set 4: Basic linear algebra February 21, 2017 [Note:] If the numerical matrix computations turn out to be tedious, you may use the function rref in Matlab. 1 Row-reduced echelon matrices
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationFoundations of Matrix Analysis
1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationFinal Review Written by Victoria Kala SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015
Final Review Written by Victoria Kala vtkala@mathucsbedu SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015 Summary This review contains notes on sections 44 47, 51 53, 61, 62, 65 For your final,
More informationDefinitions for Quizzes
Definitions for Quizzes Italicized text (or something close to it) will be given to you. Plain text is (an example of) what you should write as a definition. [Bracketed text will not be given, nor does
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More informationLecture 1: Review of linear algebra
Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, least-squares and least-norm solutions Subspaces, basis, and dimension Change of basis and similarity transformations
More information1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det
What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix
More informationMatrix Algebra for Engineers Jeffrey R. Chasnov
Matrix Algebra for Engineers Jeffrey R. Chasnov The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Department of Mathematics Clear Water Bay, Kowloon
More informationLecture Summaries for Linear Algebra M51A
These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture
More information1. Select the unique answer (choice) for each problem. Write only the answer.
MATH 5 Practice Problem Set Spring 7. Select the unique answer (choice) for each problem. Write only the answer. () Determine all the values of a for which the system has infinitely many solutions: x +
More informationAlgebra II. Paulius Drungilas and Jonas Jankauskas
Algebra II Paulius Drungilas and Jonas Jankauskas Contents 1. Quadratic forms 3 What is quadratic form? 3 Change of variables. 3 Equivalence of quadratic forms. 4 Canonical form. 4 Normal form. 7 Positive
More informationMatrices A brief introduction
Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 44 Definitions Definition A matrix is a set of N real or complex
More informationJim Lambers MAT 610 Summer Session Lecture 1 Notes
Jim Lambers MAT 60 Summer Session 2009-0 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra
More informationLecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
More informationChapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015
Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal
More informationCheat Sheet for MATH461
Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A
More informationLecture Notes in Linear Algebra
Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................
More informationLinear Algebra March 16, 2019
Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented
More informationMath 407: Linear Optimization
Math 407: Linear Optimization Lecture 16: The Linear Least Squares Problem II Math Dept, University of Washington February 28, 2018 Lecture 16: The Linear Least Squares Problem II (Math Dept, University
More information1. Linear systems of equations. Chapters 7-8: Linear Algebra. Solution(s) of a linear system of equations (continued)
1 A linear system of equations of the form Sections 75, 78 & 81 a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2 a m1 x 1 + a m2 x 2 + + a mn x n = b m can be written in matrix
More informationReview of some mathematical tools
MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical
More informationLecture 23: 6.1 Inner Products
Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such
More information10-701/ Recitation : Linear Algebra Review (based on notes written by Jing Xiang)
10-701/15-781 Recitation : Linear Algebra Review (based on notes written by Jing Xiang) Manojit Nandi February 1, 2014 Outline Linear Algebra General Properties Matrix Operations Inner Products and Orthogonal
More informationLecture: Linear algebra. 4. Solutions of linear equation systems The fundamental theorem of linear algebra
Lecture: Linear algebra. 1. Subspaces. 2. Orthogonal complement. 3. The four fundamental subspaces 4. Solutions of linear equation systems The fundamental theorem of linear algebra 5. Determining the fundamental
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationLecture 21: 5.6 Rank and Nullity
Lecture 21: 5.6 Rank and Nullity Wei-Ta Chu 2008/12/5 Rank and Nullity Definition The common dimension of the row and column space of a matrix A is called the rank ( 秩 ) of A and is denoted by rank(a);
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More information1. Let m 1 and n 1 be two natural numbers such that m > n. Which of the following is/are true?
. Let m and n be two natural numbers such that m > n. Which of the following is/are true? (i) A linear system of m equations in n variables is always consistent. (ii) A linear system of n equations in
More informationMath 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008
Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures
More informationNumerical Linear Algebra Homework Assignment - Week 2
Numerical Linear Algebra Homework Assignment - Week 2 Đoàn Trần Nguyên Tùng Student ID: 1411352 8th October 2016 Exercise 2.1: Show that if a matrix A is both triangular and unitary, then it is diagonal.
More informationOnline Exercises for Linear Algebra XM511
This document lists the online exercises for XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Lecture 02 ( 1.1) Online Exercises for Linear Algebra XM511 1) The matrix [3 2
More informationMATH 323 Linear Algebra Lecture 12: Basis of a vector space (continued). Rank and nullity of a matrix.
MATH 323 Linear Algebra Lecture 12: Basis of a vector space (continued). Rank and nullity of a matrix. Basis Definition. Let V be a vector space. A linearly independent spanning set for V is called a basis.
More informationMath 315: Linear Algebra Solutions to Assignment 7
Math 5: Linear Algebra s to Assignment 7 # Find the eigenvalues of the following matrices. (a.) 4 0 0 0 (b.) 0 0 9 5 4. (a.) The characteristic polynomial det(λi A) = (λ )(λ )(λ ), so the eigenvalues are
More informationMatrices A brief introduction
Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino September 2013 Basilio Bona (DAUIN) Matrices September 2013 1 / 74 Definitions Definition A matrix is a set of N real or complex numbers
More informationLecture 2: Linear Algebra Review
EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationHOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)
HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe
More informationMATH 369 Linear Algebra
Assignment # Problem # A father and his two sons are together 00 years old. The father is twice as old as his older son and 30 years older than his younger son. How old is each person? Problem # 2 Determine
More informationA Quick Tour of Linear Algebra and Optimization for Machine Learning
A Quick Tour of Linear Algebra and Optimization for Machine Learning Masoud Farivar January 8, 2015 1 / 28 Outline of Part I: Review of Basic Linear Algebra Matrices and Vectors Matrix Multiplication Operators
More informationMatrices A brief introduction
Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 41 Definitions Definition A matrix is a set of N real or complex
More informationMATH 235. Final ANSWERS May 5, 2015
MATH 235 Final ANSWERS May 5, 25. ( points) Fix positive integers m, n and consider the vector space V of all m n matrices with entries in the real numbers R. (a) Find the dimension of V and prove your
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS. 1. Linear Equations and Matrices
ELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS KOLMAN & HILL NOTES BY OTTO MUTZBAUER 11 Systems of Linear Equations 1 Linear Equations and Matrices Numbers in our context are either real numbers or complex
More informationA matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and
Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.
More informationTBP MATH33A Review Sheet. November 24, 2018
TBP MATH33A Review Sheet November 24, 2018 General Transformation Matrices: Function Scaling by k Orthogonal projection onto line L Implementation If we want to scale I 2 by k, we use the following: [
More informationMODULE 8 Topics: Null space, range, column space, row space and rank of a matrix
MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix Definition: Let L : V 1 V 2 be a linear operator. The null space N (L) of L is the subspace of V 1 defined by N (L) = {x
More informationA Review of Linear Algebra
A Review of Linear Algebra Mohammad Emtiyaz Khan CS,UBC A Review of Linear Algebra p.1/13 Basics Column vector x R n, Row vector x T, Matrix A R m n. Matrix Multiplication, (m n)(n k) m k, AB BA. Transpose
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationOHSx XM511 Linear Algebra: Solutions to Online True/False Exercises
This document gives the solutions to all of the online exercises for OHSx XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Answers are in square brackets [. Lecture 02 ( 1.1)
More informationDuke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014
Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Linear Algebra A Brief Reminder Purpose. The purpose of this document
More informationLecture notes on Quantum Computing. Chapter 1 Mathematical Background
Lecture notes on Quantum Computing Chapter 1 Mathematical Background Vector states of a quantum system with n physical states are represented by unique vectors in C n, the set of n 1 column vectors 1 For
More information