Linear Algebra and Graphs
|
|
- Joella Eaton
- 6 years ago
- Views:
Transcription
1 Linear Algebra and Graphs IGERT Data and Network Science Bootcamp Victor Amelkin UC Santa Barbara September 11, 2015
2 Table of Contents Linear Algebra: Review of Fundamentals Matrix Arithmetic Inversion and Linear Systems Vector Spaces Geometry Eigenproblem Linear Algebra and Graphs Graphs: Definitions, Properties, Representation Spectral Graph Theory
3 Materials Copy of These Slides MATLAB Source Code for Examples
4 Vectors and Matrices Notation x C scalar (often, R is good enough) v C n n-dimensional (column-)vector A C n m matrix with n rows and m columns A ij = A i,j element of A in i th row and j th column A T transpose of A (A T ij = A ji) A H Hermitian transpose of A (A H ij = s A ji) 1 Examples A = [ ] R 2 3 A T = R a + i b = a i b C
5 Block Matrices A = m 1... m p n 1 A A 1p n q A q1... A qp Definition Block matrix a matrix of matrices. A ij block at i th row and j th column of partitioned matrix A. Blocks of the same row (column) have the same number of rows (columns).
6 Matrix Arithmetic (1) Multiplication by scalar Performed elementwise: α A n m = {α A ij} n m Addition Performed elementwise: A n m + B n m = {A ij + B ij} n m Examples [ ] [ ] = [ ]
7 Matrix Arithmetic (2) Multiplication A n mb m k = C n k [ A11 A 12 A 21 A 22 [ ] 1 0[ ] 0 1 = [0] 1 1 [a] 1 1[b] 1 1 = [a b] 1 1 (a, b C) ] [ ] [ B11 B 12 A11B 11 + A 12B 21 A 11B 12 + A 12B 22 = B 21 B 22 A 21B 11 + A 22B 21 A 21B 12 + A 22B 22 ] Corollary { m } A n mb m k = A il B lj l=1 n k
8 Matrix Arithmetic (2) Examples Example 1 (matrix-matrix (MM) multiplication): [ ] 0 1 [ = Example 2 (matrix-vector (MV) multiplication): [ ] 1 [ = Example 3 (MV block multiplication): ] ] [ ] = [ [ 1 4 ] [ 2 [1] + 5 ] [ 3 [3] + 6 ] [2] ] 1 1
9 Matrix Arithmetic (2) More Examples Example 4 (MV block multiplication): [ ] = [ ] Example 5 (row scaling 2 ): d d n n n [ A 1. A n Example 6 (permutation of rows 3 ): A A A 3 ] n n 3 m 2 For column scaling, apply diagonal matrix from the right. 3 For permutation of columns, apply P T from the right. = = 1 2 [ [ 13 = 25 d 1A 1. d na n A 3 A 1 A 2 ] [ m n n ] ]
10 Inversion Left Inverse A L is a left inverse of A if A L A = I, where I is the identity matrix (I ii = 1, I ij = 0 if i j). Right Inverse A R is a right inverse of A if AA R = I. Inverse A 1 is the inverse of A if A 1 = A L = A R. If inverse exists, it is unique; if it does not, then the (Moore-Penrose) pseudoinverse is the closest substitute.
11 Inversion Examples (matrix inversion.m) Left Inverse (no right inverse for skinny matrices): L [ Right Inverse (no left inverse for fat matrices): [ ] R Inverse (may exist only for square matrices): ]
12 Linear Systems Here, a linear system = a system of linear algebraic equations. Solving Ax = b w.r.t. x R n is one of two fundamental problems of linear algebra (the other one is eigenproblem). Unique solution exists iff A is non-singular (det(a) 0). Problem is related to matrix inversion (i.e., x = A 1 b). A system with singular A either has no or infinitely many solutions.
13 LU Factorization The method to directly solve linear systems (a kind of) Gaussian elimination. Bad ideas: Cramer s rule; inversion followed by multiplication. One kind of Gaussian elimination LU factorization / decomposition (a.k.a. Gaussian elimination with partial pivoting). Theorem (LU factorization) For any n-by-m matrix A, there exist a permutation matrix P such that P A = LU, where L is lower-triangular with units on the main diagonal and U is a block-matrix of the form r (m r) [ ] r U 11 U 12 U = (n r) 0 0 where U 11 is upper-triangular with non-zero diagonal entries. Integer r is the rank of A.
14 Solving Linear Systems with LU (linear systems.m) Problem: solve Ax = b w.r.t. x R n 1. Step 1: decompose A = P 1 LU (O(n 3 )). Step 2: (P 1 LU)x = b (LU)x = b, where b = P b (O(n)). Step 3: solve Ly = b w.r.t. y using forward substitution (O(n 2 )). Step 4: solve Ux = y w.r.t. x using back substitution (O(n 2 )).
15 Vector Space A vector space consists of a set (a field) of scalars F (often, C or R), a set of vectors V (sequences, matrices, functions,... ) and a pair of operations, vector addition + : V V V and scalar multiplication : F V V, such that α, β F x, y, z V: x + y = y + x (commutativity of addition), x + (y + z) = (x + y) + z (associativity of addition), 0 V : x + 0 = x (existence of additive identity), x V : x + ( x) = 0 (existence of additive inverse), α(βx) = (αβ)x (multiplicative associativity), 1x = x (unit scaling), α(x + y) = αx + αy (distributivity), (α + β)x = αx + βx (distributivity).
16 Vector Space A vector space consists of a set (a field) of scalars F (often, C or R), a set of vectors V (sequences, matrices, functions,... ) and a pair of operations, vector addition + : V V V and scalar multiplication : F V V, such that α, β F x, y, z V: x + y = y + x (commutativity of addition), x + (y + z) = (x + y) + z (associativity of addition), 0 V : x + 0 = x (existence of additive identity), x V : x + ( x) = 0 (existence of additive inverse), α(βx) = (αβ)x (multiplicative associativity), 1x = x (unit scaling), α(x + y) = αx + αy (distributivity), (α + β)x = αx + βx (distributivity). (Notation abuse: instead of V, F, +,, we usually refer to a vector space simply as V or V over, when we want to emphasize how the operations + and are defined.)
17 Vector Space A vector space consists of a set (a field) of scalars F (often, C or R), a set of vectors V (sequences, matrices, functions,... ) and a pair of operations, vector addition + : V V V and scalar multiplication : F V V, such that α, β F x, y, z V: x + y = y + x (commutativity of addition), x + (y + z) = (x + y) + z (associativity of addition), 0 V : x + 0 = x (existence of additive identity), x V : x + ( x) = 0 (existence of additive inverse), α(βx) = (αβ)x (multiplicative associativity), 1x = x (unit scaling), α(x + y) = αx + αy (distributivity), (α + β)x = αx + βx (distributivity). (Notation abuse: instead of V, F, +,, we usually refer to a vector space simply as V or V over, when we want to emphasize how the operations + and are defined.) A subset W of V is a subspace of V if W is a vector space on its own. Alternatively, W is a subspace iff it is closed under +,.
18 Linear Independence Linear combination: a 1x a nx n (where a i F, x j V) linear combination of vectors {x 1,..., x n} with coefficients {a 1,..., a n}.
19 Linear Independence Linear combination: a 1x a nx n (where a i F, x j V) linear combination of vectors {x 1,..., x n} with coefficients {a 1,..., a n}. Important observation: a 1x a nx n = [ ] x 1... x n a 1. a n If we think about x i not as abstract elements of V, but as (column-)vectors, it becomes clear that the result of a matrix-vector multiplication is a linear combination of the matrix columns.
20 Linear Independence Linear combination: a 1x a nx n (where a i F, x j V) linear combination of vectors {x 1,..., x n} with coefficients {a 1,..., a n}. Important observation: a 1x a nx n = [ ] x 1... x n a 1. a n If we think about x i not as abstract elements of V, but as (column-)vectors, it becomes clear that the result of a matrix-vector multiplication is a linear combination of the matrix columns. span(x 1,..., x n) = {a 1x a nx n a i F} a span of a set of vectors is the set of all their possible linear combinations.
21 Linear Independence Linear combination: a 1x a nx n (where a i F, x j V) linear combination of vectors {x 1,..., x n} with coefficients {a 1,..., a n}. Important observation: a 1x a nx n = [ ] x 1... x n a 1. a n If we think about x i not as abstract elements of V, but as (column-)vectors, it becomes clear that the result of a matrix-vector multiplication is a linear combination of the matrix columns. span(x 1,..., x n) = {a 1x a nx n a i F} a span of a set of vectors is the set of all their possible linear combinations. {x 1,..., x n} are linearly independent if a 1x a nx n = 0 iff [ a1... a n ] = 0.
22 Linear Independence Linear combination: a 1x a nx n (where a i F, x j V) linear combination of vectors {x 1,..., x n} with coefficients {a 1,..., a n}. Important observation: a 1x a nx n = [ ] x 1... x n a 1. a n If we think about x i not as abstract elements of V, but as (column-)vectors, it becomes clear that the result of a matrix-vector multiplication is a linear combination of the matrix columns. span(x 1,..., x n) = {a 1x a nx n a i F} a span of a set of vectors is the set of all their possible linear combinations. {x 1,..., x n} are linearly independent if a 1x a nx n = 0 iff [ a1... a n ] = 0. A set {x 1,..., x n} of linearly independent vectors is a basis of subspace W if W = span(x 1,..., x n). W s dimension dim W is n.
23 Fundamental Subspaces of a Matrix The nullspace (kernel) of A n m is N (A) = {x Ax = 0}. The range (column space, image) of A n m is N (A H ) left nullspace of A. R(A H ) row space of A. R(A) = colspan(a) = {y y = Ax}
24 Fundamental Subspaces of a Matrix The Big Picture row space column space nullspace left nullspace (from Gilbert Strang s Introduction to Linear Algebra, 4 th edition, 2009)
25 Norm Lengths in a vector space are measured using a norm. A vector space augmented with a norm is a normed (vector) space. A norm defined on a vector space V over field F is a mapping : V R, such that α F x, y V the following norm axioms hold x V : x 0 (non-negativity 4 ), x = 0 x = 0 (positive definiteness), αx = α x (homogeneity), x + y x + y (subadditivity / triangle inequality). Examples: l p-norm: x p = ( i x i p) 1/p, ( 1/p L p-norm: f p = f dµ) p <. D 4 Non-negativity axiom is redundant, as it can be derived from other axioms of a norm.
26 Convexity and Norm A unit sphere is defined as {x x = 1}. A set S V is convex if x, y S 0 λ 1 : λx + (1 λ)y S. In other words, for any two points of S, all the points on the line between x and y are also in S. A function f : V R is convex if x, y V 0 λ 1 : f(λx + (1 λ)y) λf(x) + (1 λ)f(y) Norms are convex.
27 Convexity and Norm l p (norms and convexity.m) ( ) 1/p Function f p(x) = x i p mapping R n to R is a norm iff 1 p. i Alternatively, f p is a norm iff the unit sphere {x f p(x) = 1} induced by f p is convex. 1 Unit spheres induced by fp(x) = ( P i jxijp ) 1=p p=5.0 p= p=1.0 p= p=0.5 x2 0 p= x1
28 Common Vector Norms Euclidean norm x 2 = x x2 n Taxicab / Manhattan norm x 1 = x x n Chebyshev norm x = max{ x 1,..., x n }
29 Norm Equivalence Norm Equivalence Two norms α and β are equivalent if there exist two positive constants c 1, c 2 < such that x V: c 1 x α x β c 2 x α. Theorem In finite-dimensional vector spaces, all norms are equivalent. Examples for x C n x 1 n x 2, x 1 n x, x 2 n x, and, more generally, for 0 < p < q x q x p n (1/p 1/q) x q.
30 Matrix Norms (operator norm.m) Frobenius Norm: A F = n i,j=1 A 2 ij = trace(ah A), where trace() of a matrix is the sum of the elements on its main diagonal. Ax Operator Norm (Induced p-norm): A = sup p x p = sup Ax p x 0 x p=1 An operator norm measures the maximum degree of distortion / amount of stretch of a unit sphere under transformation by A :0 3:0.... : 3:696465!2:0 0:5 x original unit sphere distorted unit sphere x 1 For A, B C n n, AB A B (submultiplicativity).
31 Inner Product, Angle, Projection Inner product (scalar product) of x, y V: x, y = x H y = sx iy i. i
32 Inner Product, Angle, Projection Inner product (scalar product) of x, y V: x, y = x H y = i sx iy i. Hölder s inequality: x, y x p y q for 1 p + 1 q = 1. Cauchy-Bunyakovsky-Schwarz (CBS) inequality: x, y x 2 y 2
33 Inner Product, Angle, Projection Inner product (scalar product) of x, y V: x, y = x H y = i sx iy i. Hölder s inequality: x, y x p y q for 1 p + 1 q = 1. Cauchy-Bunyakovsky-Schwarz (CBS) inequality: x, y x 2 y 2 CBS inequality inspires the following definition of an angle θ between vectors x, y R n : x, y cos θ = x y Vectors x and y are orthogonal (x y) if the cosine of the angle between them is 0.
34 Inner Product, Angle, Projection Inner product (scalar product) of x, y V: x, y = x H y = i sx iy i. Hölder s inequality: x, y x p y q for 1 p + 1 q = 1. Cauchy-Bunyakovsky-Schwarz (CBS) inequality: x, y x 2 y 2 CBS inequality inspires the following definition of an angle θ between vectors x, y R n : x, y cos θ = x y Vectors x and y are orthogonal (x y) if the cosine of the angle between them is 0. The length of the orthogonal projection of x upon y is x, y y
35 Inner Product, Angle, Projection Example Problem: Given a matrix A R n n, find the amount of stretch caused by A to vectors along a direction defined by a vector x in l 2. relative length?
36 Inner Product, Angle, Projection Example Problem: Given a matrix A R n n, find the amount of stretch caused by A to vectors along a direction defined by a vector x in l 2. relative length? Given an arbitrary x, normalize it, so that its length is 1: ( x = x/ x x = x x = x ) x = 1
37 Inner Product, Angle, Projection Example Problem: Given a matrix A R n n, find the amount of stretch caused by A to vectors along a direction defined by a vector x in l 2. relative length? Given an arbitrary x, normalize it, so that its length is 1: ( x = x/ x x = x x = x ) x = 1 The amount of stretch caused by A along x: new size original size = proj x A x x, Ax = proj x A x = x, A x = = xt Ax x x 2 x T x
38 Inner Product, Angle, Projection Example Problem: Given a matrix A R n n, find the amount of stretch caused by A to vectors along a direction defined by a vector x in l 2. relative length? Given an arbitrary x, normalize it, so that its length is 1: ( x = x/ x x = x x = x ) x = 1 The amount of stretch caused by A along x: new size original size = proj x A x x, Ax = proj x A x = x, A x = = xt Ax x x 2 x T x Derived Rayleigh quotient (generally, x H is used instead of x T ).
39 Linear Systems (Revisited) If the columns of matrix A of a linear system Ax = b span the entire space, then b can be uniquely explained in terms of these columns (b has a unique representation in this basis). If the columns of A span a subspace, then either b has infinitely many representations (if it belongs to the column (sub)space) or it has no precise representation in terms of A s columns. Even if b is out of A s range, we can replace b by the next best thing its projection upon the column (sub)space. UU is a projector upon subspace spanned by U s columns, where U = (U H U) 1 U H is U s pseudo-inverse.
40 Determinants Definition Permutation p 1,..., p n of numbers 1,..., n is their rearrangement. Sign σ(p) of permutation p is 1 if p has an even number of element interchanges; otherwise, it is 1 (e.g., σ( 1, 3, 2 = 1), σ( 3, 1, 2 = 1)). Determinant det(a n n) = A = σ(p)a 1,p1... A n,pn C (Leibniz). p Simple determinants: det(a 1 1) = A 11, det(a 2 2) = A 11A 22 A 12A 21 Expansion along a row (similarly, along a column): = (+1) ( 1) (+1) = 1 ( ) 2 ( ) + 0 ( ) = 5. =
41 Determinants Properties, Computation Properties det(a T ) = det(a), Adding rows to each other does not change det(a). Multiplying any row by α 0 scales det(a) by α. Even # of row swaps does not change det(); odd number changes sign. det(ab) = det(a) det(b) For (block-)triangular matrices A 11 A A 1n 0 A A 2n n = det(a.. ii). i= A nn Computing det(a) for large A det(a) = det(p LU) = det(p ) det(l) det(u) }{{}}{{}}{{} σ(p ) 1 ni=1 U ii
42 Determinants Important Facts Matrix Singularity An invertible A n n is called non-singular. Otherwise, it is singular. A is singular iff det(a) = 0. (det(a) 0 does not mean almost singular.)
43 Determinants Important Facts Matrix Singularity Thus, An invertible A n n is called non-singular. Otherwise, it is singular. A is singular iff det(a) = 0. (det(a) 0 does not mean almost singular.) All the columns (rows) of A are linearly independent iff det(a) 0. (In this case, we say that matrix A is full-rank, i.e., rank(a n n) = n). Linear system Ax = b has a unique solution iff det(a) 0. Homogeneous linear system Ax = 0 has non-trivial solutions iff det(a) = 0.
44 Eigenproblem Definition For a square matrix A n n, we are interested in those non-trivial vectors x 0 that do not change their direction under transformation by A: Ax = λx, λ C. These x are eigenvectors 5 of A, and the corresponding scaling factors λ are eigenvalues of A. Pairs λ, x of corresponding eigenvalues and eigenvectors are eigenpairs. Distinct eigenvalues of matrix A comprise its spectrum σ(a). Spectral radius ρ(a) = max{ σ i(a) } 5 If x is an eigenvector of A, then α x is also an eigenvector. Thus, an eigenvector defines an entire direction or, more generally, a subspace, referred to as eigenspace, whose elements do not change direction when transformed by A. Thus, an eigenspace is an invariant subspace of its matrix.
45 Characteristic Polynomial, Its Roots and Coefficients Definition Goal: find non-trivial solutions of A n nx = λx (A λi)x = 0, where I is the n-by-n identity matrix (I ii = 1, I ij = 0(i j).
46 Characteristic Polynomial, Its Roots and Coefficients Definition Goal: find non-trivial solutions of A n nx = λx (A λi)x = 0, where I is the n-by-n identity matrix (I ii = 1, I ij = 0(i j). Homogeneous system (A λi)x = 0 has non-trivial solutions iff its matrix is singular, that is, if det(a λi) = 0.
47 Characteristic Polynomial, Its Roots and Coefficients Definition Goal: find non-trivial solutions of A n nx = λx (A λi)x = 0, where I is the n-by-n identity matrix (I ii = 1, I ij = 0(i j). Homogeneous system (A λi)x = 0 has non-trivial solutions iff its matrix is singular, that is, if det(a λi) = 0. p(λ) = det(a λi) is the characteristic polynomial (in λ, of degree n) of matrix A; its roots are A s eigenvalues, and multiplicity of each root is the algebraic multiplicity of the corresponding eigenvalue. p(λ) = 0 is the characteristic equation for A.
48 Characteristic Polynomial, Its Roots and Coefficients Definition Goal: find non-trivial solutions of A n nx = λx (A λi)x = 0, where I is the n-by-n identity matrix (I ii = 1, I ij = 0(i j). Homogeneous system (A λi)x = 0 has non-trivial solutions iff its matrix is singular, that is, if det(a λi) = 0. p(λ) = det(a λi) is the characteristic polynomial (in λ, of degree n) of matrix A; its roots are A s eigenvalues, and multiplicity of each root is the algebraic multiplicity of the corresponding eigenvalue. p(λ) = 0 is the characteristic equation for A. Useful Facts From the fundamental theorem of algebra, any n-by-n square matrix always has n (not necessarily distinct) complex eigenvalues. From the complex conjugate root theorem, if a + i b C is an eigenvalue of A R n n, then a i b is also its eigenvalue. From Vieta s theorem applied to p(λ) = λ n + c 1λ n c n 1λ + c n, λ 1 + λ λ n = c 1 = trace(a) λ 1 λ 2... λ n = ( 1) n c n = det(a)
49 Resolving Eigenproblem Directly Algorithm Step 0: estimate where eigenvalues are located. Step 1: solve the characteristic equation det(a λi) = 0 using the estimates from Step 0, and find eigenvalues. Step 2: for each eigenvalue λ i, find the corresponding eigenspace by solving homogeneous system (A λ ii)x = 0. This eigenspace is comprised of non-trivial members of N (A λ ii).
50 Resolving Eigenproblem Directly Example (eigen direct.m) A = , p(λ) = det(a λi) = λ 3 + 6λ 2 11λ + 6, λ 1,2,3 = 1, 2, 3, λ 1 = 1 : N (A λ 1I) = span( [ ] T ), }{{} e 1 λ 2 = 2 : N (A λ 2I) = span( [ ] T ), }{{} e 2 λ 3 = 3 : N (A λ 3I) = span( [ ] T ). }{{} e 3
51 Resolving Eigenproblem What Actually Works (eigen arnoldi.m) Bad news: solving an equation p(λ) = 0 for high-degree p is very hard.
52 Resolving Eigenproblem What Actually Works (eigen arnoldi.m) Bad news: solving an equation p(λ) = 0 for high-degree p is very hard. Most real-world eigensolvers use the idea of Krylov sequences {x, Ax, A 2 x,... } and subspaces spanned by them.
53 Resolving Eigenproblem What Actually Works (eigen arnoldi.m) Bad news: solving an equation p(λ) = 0 for high-degree p is very hard. Most real-world eigensolvers use the idea of Krylov sequences {x, Ax, A 2 x,... } and subspaces spanned by them. A popular eigensolver for sparse matrices Arnoldi/Lancsoz iteration (an advanced version of the power method). It allows to quickly compute several (largest, smallest, closest to a given value) eigenvalues and the corresponding eigenvectors of a sparse matrix, mostly, using matrix-vector multiplication. This method is used by MATLAB s eigs and by Python s scipy.sparse.linalg.eigs. For dense matrices, eigensolvers based on Schur or Cholesky decomposition may be used.
54 Eigenvalue Localization Sometimes, it may be enough to have a good estimate of where eigenvalues are, without actually computing them. That estimation is referred to as eigenvalue localization. Tools Crude bound: λ(a) < A. Cauchy s [ Interlacing ] Theorem: for real symmetric n-by-n matrix B c A = c T, where δ R, δ λ n(a) λ n 1(B)... λ k (B) λ k (A) λ k 1 (B) λ 1(B) λ 1(A). Gerschgorin Circles The eigenvalues of A C n n are trapped inside the union of Gerschgorin circles z A ii < r i, where r i = min { n n A ij, A ji, }, i = 1,..., n. A k Gerschgorin circles j=1 j i j=1 j i disjoint from others contain exactly k eigenvalues.
55 Gerschgorin Circles Example * * * *
56 Eigendecomposition (eigendecomp.m) A C n n is diagonalizable if there is an invertible P such that P 1 AP is diagonal. If a real-valued matrix is symmetric, then it is diagonalizable. (Though, invertible matrices do not have to be symmetric in general.)
57 Eigendecomposition (eigendecomp.m) A C n n is diagonalizable if there is an invertible P such that P 1 AP is diagonal. If a real-valued matrix is symmetric, then it is diagonalizable. (Though, invertible matrices do not have to be symmetric in general.) Each diagonalizable A permits (eigen)decomposition: A = q 1... q n } {{ } Q λ λ λ n } {{ } Λ q 1... q n = QΛQ 1. Analog for non-diagonalizable matrices Jordan normal form.
58 Eigendecomposition MV Multiplication for Real Symmetric Matrices Assume A R n n symmetric. A is diagonalizable, and its eigenvectors are orthogonal. For orthogonal matrix A, A 1 = A T. ( n ) Ax = (QΛQ 1 )x = λ iq i q T i x = i=1 n λ iq i (q T i x) = i=1 n i=1 λ i x, q i }{{} q i proj qi x
59 Singular Value Decomposition (SVD) Theorem For every (rectangular) matrix A C n m, there are two unitary 6 matrices U C m m and V C n n, as well as a matrix Σ R m n of the form σ Σ =. 0 σ 2....,.... with σ 1 σ 2 σ min (n,m) 0, such that A = UΣV H. Diagonal values of Σ singular values of A, columns of U and V left and right singular vectors of A, respectively. (Notice redundant columns in U and rows (or columns) in Σ.) 6 A matrix A C n n is unitary if A H A = AA H = I. Unitary matrices play a role similar to the role a scalar 1 plays ( a size-preserving transform.)
60 Table of Contents Linear Algebra: Review of Fundamentals Matrix Arithmetic Inversion and Linear Systems Vector Spaces Geometry Eigenproblem Linear Algebra and Graphs Graphs: Definitions, Properties, Representation Spectral Graph Theory
61 Graphs: definitions, properties An (edge-weighted) graph is a tuple G = V, E, w, where V is a set of nodes, E V V is a set of edges between the nodes, and w : E R defined edge weights. If w is not specified, then edge weights can assumed to be equal 1. If E is a symmetric relation (and w, if specified, is a symmetric function), then G is said to be undirected; otherwise, it is directed. A graph may have weights on its nodes rather than the edges (or on both). A node-weighted graph can be transformed into an edge-weighted graph, or vice versa. A graph G = V, E, w is (weakly) connected if for any v 1, v 2 V, v 1 v 2, we can reach v 2 from v 1 by walking along the adjacent edges E (ignoring their direction). A (weakly) disconnected graph G consists of connected components (CC), which are maximal (weakly) connected subgraphs. When we take into account direction of edges, the notion of connectedness extends to the notion of strong connectedness (strongly connected components (SCC) are, then, defined similarly to weakly connected components).
62 Graphs Examples
63 Graphs Special Graphs
64 Representation of Graphs Adjacency Matrix All definitions are given for a graph G = V, E, w having V = n nodes and E = m edges. The most popular representation of a graph is its adjacency matrix A: { w( i, j ) = w ij if i, j E, A ij = 0 otherwise. If weights w are not specified, then A is a binary matrix.
65 Representation of Graphs Adjacency Matrices of Special Graphs
66 Representation of Graphs Degree Matrix The in-degree d in i of node v i is the sum of the weights on all of its incoming edges. The out-degree d out i of node v i is similarly defined via v i s outgoing edges. For undirected graphs, both in-degree and out-degree are equal d i referred to as simply the degree of node v i. For unweighted graphs, the degree measures the number of a node s (in-, our-, or all) neighbors. The degree matrix D = diag({d i} 1...n) of a graph G is a diagonal matrix with node degrees on its main diagonal. For directed graphs, in-degree and out-degree matrices can be similarly defined using the appropriate degree definitions.
67 Laplacian of Undirected Graphs (Combinatorial) Laplacian Weighted graph: L ij = Unweighted graph: Other Laplacians L = D A { Dii = d i = i,l E w i,l if i = j, w ij if i, j E, 0 otherwise; L ij = { di = i,l E 1 if i = j, 1 if i, j E, 0 otherwise. L sym = D 1/2 LD 1/2 = I D 1/2 AD 1/2, L rw = D 1 A.
68 Spectral Graph Theory Spectral Graph Theory Graphs are usually represented with matrices 7. Spectral graph theory attempts to connect spectral properties of these matrices with the corresponding graphs structural properties. Limitations Most results of spectral graph theory are obtained for undirected and unweighted graphs, i.e., graphs having binary symmetric adjacency matrices. If a result applies to weighted graphs, it will be explicitly stated. 7 Some may even go as far as to claim that graphs and matrices are the same thing.
69 Spectrum of Adjacency Matrix Walks in Graphs A {0, 1} n n adjacency matrix of an undirected unweighted graph G. A ij number of walks of length 1 in G between nodes v i and v j. (A k ) ij number of walks of length k in G between nodes v i and v j. (A k 1) i number of walks of length k ending at v i. 1 T A k 1 number of walks of length k in G.
70 Largest 8 Eigenvalue of Adjacency Matrix µ 1 = µ max Connection to µ max (undirected, unweighted, connected G): 1 T A k 1 = (since A is real and symmetric) = 1 T (Q diag (µ i)q 1 ) k 1 = = (since Q is orthogonal) = 1 T Q diag (µ k i )Q 1 1 = ( n ) = 1 T µ k i q i q T i 1 = i=1 n µ k i (1 T q i )(1 T q i ) T = i=1 = (1 = α 1q α nq n ) = k µmax n i=1 i max µ k i n µ k i q i, 1 2 = i=1 ( n ) 2 α j qi, q j = j=1 ( ( n ) 1/k 1/k lim 1 T A 1) k = lim µ k i αi 2 = k k i=1 = lim αmax 2 + ( ) k µi α 2 µ max i 1/k n µ k i (α i) 2 ; i=1 = µ max(= A ). Thus, µ k max = A(G) k is the number of walks of length k in G. 8 The largest eigenvalue is such w.r.t. its absolute value.
71 Largest Eigenvalue of Adjacency Matrix µ 1 = µ max Summary Derived µ k max = A(G) k is the number of walks of length k in G. For directed A, the meaning of lim k A k 1 = q max is close to the one of PageRank.
72 Largest Eigenvalue of Adjacency Matrix µ 1 = µ max Summary Derived µ k max = A(G) k is the number of walks of length k in G. For directed A, the meaning of lim k A k 1 = q max is close to the one of PageRank. Beyond Walks (( ) applies to weighted graphs) ( ) If graph G is connected, µ max has multiplicity 1, and its eigenvector is positive (all its entries are strictly positive). ( ) d avg µ 1 d max (d avg, d max mean and maximum node degrees). max {d avg, d max} µ max d max. ( ) If G is connected, and µ max = d max, then i : d i = d max. ( ) A connected graph is bipartite iff µ min = µ max. A graph is bipartite iff its spectrum is symmetric about 0. χ(g) 1 + µ min/µ max.
73 Smallest Eigenvalues of Combinatorial Laplacian L = D A. Eigenvalues of L are non-negative: 0 λ 1 λ 2 λ n.
74 Smallest Eigenvalues of Combinatorial Laplacian L = D A. Eigenvalues of L are non-negative: 0 λ 1 λ 2 λ n. L is singular 0 is its eigenvalue corresponding to eigenvector 1. If G is connected, λ 1 = 0 has multiplicity 1.
75 Smallest Eigenvalues of Combinatorial Laplacian L = D A. Eigenvalues of L are non-negative: 0 λ 1 λ 2 λ n. L is singular 0 is its eigenvalue corresponding to eigenvector 1. If G is connected, λ 1 = 0 has multiplicity 1. If G has k connected components, λ 1 = 0 s multiplicity is k.
76 Smallest Eigenvalues of Combinatorial Laplacian L = D A. Eigenvalues of L are non-negative: 0 λ 1 λ 2 λ n. L is singular 0 is its eigenvalue corresponding to eigenvector 1. If G is connected, λ 1 = 0 has multiplicity 1. If G has k connected components, λ 1 = 0 s multiplicity is k. The harder it is to disconnect G by removing its edges, the larger the gap between λ 1 = 0 and λ 2 > 0 is.
77 Smallest Eigenvalues of Combinatorial Laplacian L = D A. Eigenvalues of L are non-negative: 0 λ 1 λ 2 λ n. L is singular 0 is its eigenvalue corresponding to eigenvector 1. If G is connected, λ 1 = 0 has multiplicity 1. If G has k connected components, λ 1 = 0 s multiplicity is k. The harder it is to disconnect G by removing its edges, the larger the gap between λ 1 = 0 and λ 2 > 0 is. λ 2 algebraic connectivity (a.k.a. Fiedler value, spectral gap) a measure of graph connectedness. q 2 Fiedler vector eigenvector associated with λ 2 solution to a relaxed min-cut (sparsest cut) in G. The same eigenvector of a normalized Laplacian L sym solution to a relaxed normalized min-cut ( edge-balanced sparsest cut ) in G.
78 Spectral Bisection (spectral bisection.m) Spectral bisection using 7 2 of Combinatorial Laplacian (6 2 = ) 18 cut edges Figure : Example of spectral bisection with Fiedler vector. (The Tapir graph as well as the plotting functions come from meshpart toolbox by John R. Gilbert and Shang-Hua Teng.)
79 Spectral Clustering (spectral clustering.m) Partitioning into 2 clusters Partitioning into 3 clusters Partitioning into 4 clusters Partitioning into 5 clusters Figure : Example of spectral clustering using normalized Laplacian and k-means. (The Tapir graph as well as the plotting functions come from meshpart toolbox by John R. Gilbert and Shang-Hua Teng.)
80 What is next Relevant Courses at UCSB ECE/CS211A Matrix Analysis a decent overview of most fundamentals of linear algebra, from the definition of block-matrix arithmetic to spectral theory. CS290H Graph Laplacians and Spectra this course is focused on the study of spectra of graph Laplacians as well as on the accompanying computational problems (extracting eigenpairs of Laplacians, solving Laplacian linear systems). Reading Linear Algebra Core Matrix Analysis by Shiv Chandrasekaran a textbook for ECE/CS211A. Provides an overview of most necessary fundamentals. Introduction to Linear Algebra (any edition) by Gilbert Strang an entry-level book about fundamentals of linear algebra; great exposition. Matrix Analysis and Applied Linear Algebra by Carl Meyer an advanced linear algebra textbook; pick this one if Strang s textbook feels too easy to read.
81 What is next Reading Linear Algebra of Graphs Spectral Graph Theory by Fan Chung (1997). Complex Graphs and Networks by Fan Chung (2006). Dan Spielman s course on spectral graph theory. Eigenvalues of Graphs by László Lovász (2007). Luca Trevisan s course on spectral graph theory. Algebraic connectivity of graphs by Miroslav Fiedler (1973). A tutorial on spectral clustering by Ulrike von Luxburg (2007).
82 Thanks Copyright (c) Victor Amelkin, 2015
Networks and Their Spectra
Networks and Their Spectra Victor Amelkin University of California, Santa Barbara Department of Computer Science victor@cs.ucsb.edu December 4, 2017 1 / 18 Introduction Networks (= graphs) are everywhere.
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationReview of Some Concepts from Linear Algebra: Part 2
Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationFoundations of Matrix Analysis
1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationOHSx XM511 Linear Algebra: Solutions to Online True/False Exercises
This document gives the solutions to all of the online exercises for OHSx XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Answers are in square brackets [. Lecture 02 ( 1.1)
More informationLinear algebra and applications to graphs Part 1
Linear algebra and applications to graphs Part 1 Written up by Mikhail Belkin and Moon Duchin Instructor: Laszlo Babai June 17, 2001 1 Basic Linear Algebra Exercise 1.1 Let V and W be linear subspaces
More information1 9/5 Matrices, vectors, and their applications
1 9/5 Matrices, vectors, and their applications Algebra: study of objects and operations on them. Linear algebra: object: matrices and vectors. operations: addition, multiplication etc. Algorithms/Geometric
More informationEigenvalues and Eigenvectors
Chapter 1 Eigenvalues and Eigenvectors Among problems in numerical linear algebra, the determination of the eigenvalues and eigenvectors of matrices is second in importance only to the solution of linear
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationLinear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationLecture 2: Linear Algebra Review
EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1
More informationA Brief Outline of Math 355
A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting
More informationLinear Algebra Highlights
Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to
More informationReview of some mathematical tools
MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical
More informationLinGloss. A glossary of linear algebra
LinGloss A glossary of linear algebra Contents: Decompositions Types of Matrices Theorems Other objects? Quasi-triangular A matrix A is quasi-triangular iff it is a triangular matrix except its diagonal
More informationThe Eigenvalue Problem: Perturbation Theory
Jim Lambers MAT 610 Summer Session 2009-10 Lecture 13 Notes These notes correspond to Sections 7.2 and 8.1 in the text. The Eigenvalue Problem: Perturbation Theory The Unsymmetric Eigenvalue Problem Just
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationOnline Exercises for Linear Algebra XM511
This document lists the online exercises for XM511. The section ( ) numbers refer to the textbook. TYPE I are True/False. Lecture 02 ( 1.1) Online Exercises for Linear Algebra XM511 1) The matrix [3 2
More informationMAT 610: Numerical Linear Algebra. James V. Lambers
MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic
More informationLecture Summaries for Linear Algebra M51A
These lecture summaries may also be viewed online by clicking the L icon at the top right of any lecture screen. Lecture Summaries for Linear Algebra M51A refers to the section in the textbook. Lecture
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationMATH 583A REVIEW SESSION #1
MATH 583A REVIEW SESSION #1 BOJAN DURICKOVIC 1. Vector Spaces Very quick review of the basic linear algebra concepts (see any linear algebra textbook): (finite dimensional) vector space (or linear space),
More informationA Review of Linear Algebra
A Review of Linear Algebra Mohammad Emtiyaz Khan CS,UBC A Review of Linear Algebra p.1/13 Basics Column vector x R n, Row vector x T, Matrix A R m n. Matrix Multiplication, (m n)(n k) m k, AB BA. Transpose
More informationANSWERS. E k E 2 E 1 A = B
MATH 7- Final Exam Spring ANSWERS Essay Questions points Define an Elementary Matrix Display the fundamental matrix multiply equation which summarizes a sequence of swap, combination and multiply operations,
More informationBasic Elements of Linear Algebra
A Basic Review of Linear Algebra Nick West nickwest@stanfordedu September 16, 2010 Part I Basic Elements of Linear Algebra Although the subject of linear algebra is much broader than just vectors and matrices,
More informationMath Linear Algebra Final Exam Review Sheet
Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationChapter 5 Eigenvalues and Eigenvectors
Chapter 5 Eigenvalues and Eigenvectors Outline 5.1 Eigenvalues and Eigenvectors 5.2 Diagonalization 5.3 Complex Vector Spaces 2 5.1 Eigenvalues and Eigenvectors Eigenvalue and Eigenvector If A is a n n
More informationLarge Scale Data Analysis Using Deep Learning
Large Scale Data Analysis Using Deep Learning Linear Algebra U Kang Seoul National University U Kang 1 In This Lecture Overview of linear algebra (but, not a comprehensive survey) Focused on the subset
More informationNumerical Methods - Numerical Linear Algebra
Numerical Methods - Numerical Linear Algebra Y. K. Goh Universiti Tunku Abdul Rahman 2013 Y. K. Goh (UTAR) Numerical Methods - Numerical Linear Algebra I 2013 1 / 62 Outline 1 Motivation 2 Solving Linear
More informationMAT Linear Algebra Collection of sample exams
MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationMATH 532: Linear Algebra
MATH 532: Linear Algebra Chapter 5: Norms, Inner Products and Orthogonality Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2015 fasshauer@iit.edu MATH 532 1 Outline
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationHOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)
HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe
More informationA Review of Linear Algebra
A Review of Linear Algebra Gerald Recktenwald Portland State University Mechanical Engineering Department gerry@me.pdx.edu These slides are a supplement to the book Numerical Methods with Matlab: Implementations
More information5.6. PSEUDOINVERSES 101. A H w.
5.6. PSEUDOINVERSES 0 Corollary 5.6.4. If A is a matrix such that A H A is invertible, then the least-squares solution to Av = w is v = A H A ) A H w. The matrix A H A ) A H is the left inverse of A and
More informationMath Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationLINEAR ALGEBRA SUMMARY SHEET.
LINEAR ALGEBRA SUMMARY SHEET RADON ROSBOROUGH https://intuitiveexplanationscom/linear-algebra-summary-sheet/ This document is a concise collection of many of the important theorems of linear algebra, organized
More informationLinear Algebra Formulas. Ben Lee
Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationStat 159/259: Linear Algebra Notes
Stat 159/259: Linear Algebra Notes Jarrod Millman November 16, 2015 Abstract These notes assume you ve taken a semester of undergraduate linear algebra. In particular, I assume you are familiar with the
More informationELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS. 1. Linear Equations and Matrices
ELEMENTARY LINEAR ALGEBRA WITH APPLICATIONS KOLMAN & HILL NOTES BY OTTO MUTZBAUER 11 Systems of Linear Equations 1 Linear Equations and Matrices Numbers in our context are either real numbers or complex
More informationLinear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
More informationEcon Slides from Lecture 7
Econ 205 Sobel Econ 205 - Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for
More informationComputational Methods. Eigenvalues and Singular Values
Computational Methods Eigenvalues and Singular Values Manfred Huber 2010 1 Eigenvalues and Singular Values Eigenvalues and singular values describe important aspects of transformations and of data relations
More informationLinear Algebra: Matrix Eigenvalue Problems
CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationEigenvalues and Eigenvectors
/88 Chia-Ping Chen Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Eigenvalue Problem /88 Eigenvalue Equation By definition, the eigenvalue equation for matrix
More informationThe following definition is fundamental.
1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic
More informationMath 4A Notes. Written by Victoria Kala Last updated June 11, 2017
Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...
More informationLINEAR ALGEBRA REVIEW
LINEAR ALGEBRA REVIEW SPENCER BECKER-KAHN Basic Definitions Domain and Codomain. Let f : X Y be any function. This notation means that X is the domain of f and Y is the codomain of f. This means that for
More informationCS 246 Review of Linear Algebra 01/17/19
1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE
More informationSome notes on Linear Algebra. Mark Schmidt September 10, 2009
Some notes on Linear Algebra Mark Schmidt September 10, 2009 References Linear Algebra and Its Applications. Strang, 1988. Practical Optimization. Gill, Murray, Wright, 1982. Matrix Computations. Golub
More informationNotes on Linear Algebra and Matrix Theory
Massimo Franceschet featuring Enrico Bozzo Scalar product The scalar product (a.k.a. dot product or inner product) of two real vectors x = (x 1,..., x n ) and y = (y 1,..., y n ) is not a vector but a
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More informationCS 143 Linear Algebra Review
CS 143 Linear Algebra Review Stefan Roth September 29, 2003 Introductory Remarks This review does not aim at mathematical rigor very much, but instead at ease of understanding and conciseness. Please see
More informationLinear Algebra. Session 12
Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)
More informationNotes on Eigenvalues, Singular Values and QR
Notes on Eigenvalues, Singular Values and QR Michael Overton, Numerical Computing, Spring 2017 March 30, 2017 1 Eigenvalues Everyone who has studied linear algebra knows the definition: given a square
More informationNumerical Linear Algebra
Numerical Linear Algebra The two principal problems in linear algebra are: Linear system Given an n n matrix A and an n-vector b, determine x IR n such that A x = b Eigenvalue problem Given an n n matrix
More informationMATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)
MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m
More informationANSWERS (5 points) Let A be a 2 2 matrix such that A =. Compute A. 2
MATH 7- Final Exam Sample Problems Spring 7 ANSWERS ) ) ). 5 points) Let A be a matrix such that A =. Compute A. ) A = A ) = ) = ). 5 points) State ) the definition of norm, ) the Cauchy-Schwartz inequality
More informationChapter 0 Miscellaneous Preliminaries
EE 520: Topics Compressed Sensing Linear Algebra Review Notes scribed by Kevin Palmowski, Spring 2013, for Namrata Vaswani s course Notes on matrix spark courtesy of Brian Lois More notes added by Namrata
More informationMTH 464: Computational Linear Algebra
MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University March 2, 2018 Linear Algebra (MTH 464)
More informationGlossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB
Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the
More informationLinear Algebra - Part II
Linear Algebra - Part II Projection, Eigendecomposition, SVD (Adapted from Sargur Srihari s slides) Brief Review from Part 1 Symmetric Matrix: A = A T Orthogonal Matrix: A T A = AA T = I and A 1 = A T
More informationG1110 & 852G1 Numerical Linear Algebra
The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the
More informationDuke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014
Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Linear Algebra A Brief Reminder Purpose. The purpose of this document
More informationQuantum Computing Lecture 2. Review of Linear Algebra
Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces
More informationLinear Algebra review Powers of a diagonalizable matrix Spectral decomposition
Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing
More informationand let s calculate the image of some vectors under the transformation T.
Chapter 5 Eigenvalues and Eigenvectors 5. Eigenvalues and Eigenvectors Let T : R n R n be a linear transformation. Then T can be represented by a matrix (the standard matrix), and we can write T ( v) =
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More informationLinear Algebra for Machine Learning. Sargur N. Srihari
Linear Algebra for Machine Learning Sargur N. srihari@cedar.buffalo.edu 1 Overview Linear Algebra is based on continuous math rather than discrete math Computer scientists have little experience with it
More informationforms Christopher Engström November 14, 2014 MAA704: Matrix factorization and canonical forms Matrix properties Matrix factorization Canonical forms
Christopher Engström November 14, 2014 Hermitian LU QR echelon Contents of todays lecture Some interesting / useful / important of matrices Hermitian LU QR echelon Rewriting a as a product of several matrices.
More informationNumerical Methods I Eigenvalue Problems
Numerical Methods I Eigenvalue Problems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 October 2nd, 2014 A. Donev (Courant Institute) Lecture
More informationMATH 581D FINAL EXAM Autumn December 12, 2016
MATH 58D FINAL EXAM Autumn 206 December 2, 206 NAME: SIGNATURE: Instructions: there are 6 problems on the final. Aim for solving 4 problems, but do as much as you can. Partial credit will be given on all
More informationMath Introduction to Numerical Analysis - Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620 - Introduction to Numerical Analysis - Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More informationSUMMARY OF MATH 1600
SUMMARY OF MATH 1600 Note: The following list is intended as a study guide for the final exam. It is a continuation of the study guide for the midterm. It does not claim to be a comprehensive list. You
More informationMatrix Factorization and Analysis
Chapter 7 Matrix Factorization and Analysis Matrix factorizations are an important part of the practice and analysis of signal processing. They are at the heart of many signal-processing algorithms. Their
More informationMath 302 Outcome Statements Winter 2013
Math 302 Outcome Statements Winter 2013 1 Rectangular Space Coordinates; Vectors in the Three-Dimensional Space (a) Cartesian coordinates of a point (b) sphere (c) symmetry about a point, a line, and a
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Logistics Notes for 2016-08-29 General announcement: we are switching from weekly to bi-weekly homeworks (mostly because the course is much bigger than planned). If you want to do HW but are not formally
More information4. Determinants.
4. Determinants 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 2 2 determinant 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 3 3 determinant 4.1.
More informationLecture 1: Review of linear algebra
Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, least-squares and least-norm solutions Subspaces, basis, and dimension Change of basis and similarity transformations
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationLinear Algebra M1 - FIB. Contents: 5. Matrices, systems of linear equations and determinants 6. Vector space 7. Linear maps 8.
Linear Algebra M1 - FIB Contents: 5 Matrices, systems of linear equations and determinants 6 Vector space 7 Linear maps 8 Diagonalization Anna de Mier Montserrat Maureso Dept Matemàtica Aplicada II Translation:
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationAPPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.
APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationIMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET
IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More information1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)
Notes for 2017-01-30 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty
More informationLinear Algebra Review
Chapter 1 Linear Algebra Review It is assumed that you have had a course in linear algebra, and are familiar with matrix multiplication, eigenvectors, etc. I will review some of these terms here, but quite
More information