Matrices A brief introduction
|
|
- Garey Ford
- 6 years ago
- Views:
Transcription
1 Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino September 2013 Basilio Bona (DAUIN) Matrices September / 74
2 Definitions Definition A matrix is a set of N real or complex numbers organized in m rows and n columns, with N = mn a 11 a 12 a 1n a A = 21 a 22 a 2n a ij [ ] a ij i = 1,...,m j = 1,...,n a m1 a m2 a mn A matrix is always written as a boldface capital letter viene as in A. To indicate matrix dimensions we use the following symbols A m n A m n A F m n A F m n where F = R for real elements and F = C for complex elements. Basilio Bona (DAUIN) Matrices September / 74
3 Transpose matrix Given a matrix A m n we define a transpose matrix the matrix obtained exchanging rows and columns a 11 a 21 a m1 A T n m = a 12 a 22 a m a 1n a 2n a mn The following property holds (A T ) T = A Basilio Bona (DAUIN) Matrices September / 74
4 Square matrix A matrix is said to be square when m = n A square n n matrix is upper triangular when a ij = 0, i > j a 11 a 12 a 1n 0 a A n n = 22 a 2n a nn If a square matrix is upper triangular its transpose is lower triangular and viceversa a A T n n = a 12 a a 1n a 2n a nn Basilio Bona (DAUIN) Matrices September / 74
5 Symmetric matrix A real square matrix is said to be symmetric if A = A T, or A A T = O In a real symmetric matrix there are at least n(n+1) independent 2 elements. If a matrix K has complex elements k ij = a ij +jb ij (where j = 1) its conjugate is K with elements k ij = a ij jb ij. Given a complex matrix K, an adjoint matrix K is defined, as the conjugate transpose K = K T = K T A complex matrix is called self-adjoint or hermitian when K = K. Some textbooks indicate this matrix as K or K H Basilio Bona (DAUIN) Matrices September / 74
6 Diagonal matrix A square matrix is diagonal if a ij = 0 for i j a a A n n = diag(a i ) = a n A diagonal matrix is always symmetric. Basilio Bona (DAUIN) Matrices September / 74
7 Skew-symmetric matrix Skew-symmetric matrix A square matrix is skew-symmetric or antisymmetric if A+A T = 0 A = A T Given the constraints of the above relation, a generic skew-symmetric matrix has the following structure 0 a 12 a 1n a A n n = 12 0 a 2n a 1n a 2n 0 In a skew-symmetric matrix there are at most n(n 1) non zero 2 independent elements. We will see in the following some important properties of the skew-symmetric 3 3 matrices. Basilio Bona (DAUIN) Matrices September / 74
8 Block matrix It is possible to represent a matrix with blocks as A = A 11 A 1n A ij A m1 A mn where the blocks A ij have suitable dimensions. Given the following matrices A 1 = A 11 A 1n O A ij A 2 = A 11 O O A ij O A 3 = A 11 O O O A ij O O O A mn A m1 A mn O O A mn A 1 is upper block triangular, A 2 is lower block triangular, and A 3 is block diagonal Basilio Bona (DAUIN) Matrices September / 74
9 Matrix algebra Matrices are elements of an algebra, i.e., a vector space together with a product operator. The main operations of this algebra are: product by a scalar, sum, and matrix product Product by a scalar a 11 a 12 a 1n αa 11 αa 12 αa 1n a αa = α 21 a 22 a 2n = αa 21 αa 22 αa 2n a m1 a m2 a mn αa m1 αa m2 αa mn Sum a 11 +b 11 a 12 +b 12 a 1n +b 1n a A+B = 21 +b 21 a 22 +b 22 a 2n +b 2n a m1 +b m1 a m2 +b m2 a mn +b mn Basilio Bona (DAUIN) Matrices September / 74
10 Matrix sum Sum properties A+O = A A+B = B+A (A+B)+C = A+(B+C) (A+B) T = A T +B T The null (neutral, zero) element O takes the name of null matrix. The subtraction (difference) operation is defined using the scalar α = 1: A B = A+( 1)B Basilio Bona (DAUIN) Matrices September / 74
11 Matrix product Matrix product The operation is performed using the well-known rule rows by columns : the generic element c ij of the matrix product C m p = A m n B n p is c ij = n a ik b kj k=1 The bi-linearity of the matrix product is guaranteed, since it is immediate to verify that, given a generic scalar α, the following identity holds: α(a B) = (αa) B = A (αb) Basilio Bona (DAUIN) Matrices September / 74
12 Product Product properties In general: A B C = (A B) C = A (B C) A (B+C) = A B+A C (A+B) C = A C+B C (A B) T = B T A T the matrix product is non-commutative: A B B A, apart from particular cases; A B = A C does not imply B = C, apart from particular cases; A B = O does not imply A = O or B = O, apart from particular cases. Basilio Bona (DAUIN) Matrices September / 74
13 Identity matrix A neutral element wrt product exists and is called identity matrix, written as I n or simply I when no ambiguity arises; given a rectangular matrix A m n the following identities hold A m n = I m A m n = A m n I n Identity matrix I = Basilio Bona (DAUIN) Matrices September / 74
14 Idempotent matrix Given a square matrix A R n n, the k-th power is A k = A matrix is said to be idempotent if k A l=1 A 2 = A A n = A Basilio Bona (DAUIN) Matrices September / 74
15 Trace Trace The trace of a square matrix A n n is the sum of its diagonal elements tr(a) = n k=1 a kk The matrix traces satisfies the following properties tr(αa+βb) = αtr(a)+β tr(b) tr(ab) = tr(ba) tr(a) = tr(a T ) tr(a) = tr(t 1 AT) for non singular T (see below) Basilio Bona (DAUIN) Matrices September / 74
16 Minor A minor of order p of a matrix A m n is the determinant D p of a square sub-matrix obtained selecting any p rows and p columns of A m n The formal definition of determinant will be presented below There are as many minors as there are possible choices of p on m rows and of p on n columns. Given a matrix A m n, the principal minors of order k are the determinants D k, with k = 1,,min{m,n}, obtained selecting the first k rows an kd columns of A m n. Basilio Bona (DAUIN) Matrices September / 74
17 Minor and cofactor Given A R n n, we indicate with A (ij) R (n 1) (n 1) the matrix obtained taking out the i-th row and the j-th column of A. We define the minor D rc of a generic element a rc of a square matrix A n n, the determinant of the matrix obtained taking out the r-th row and the c-th column, i.e., deta (rc) D rc = deta (rc). We define the cofactor of an element a rc of a square matrix A n n the product A rc = ( 1) r+c D rc Basilio Bona (DAUIN) Matrices September / 74
18 Determinant Once defined the cofactor, the determinant of a square matrix A can be defined by row, i.e., choosing a generic row i, det(a) = n a ik ( 1) i+k det(a (ik) ) = k=1 n a ik A ik or, choosing a generic column j, we have the definition by column : det(a) = n a kj ( 1) k+j det(a (kj) ) = k=1 k=1 n a kj A kj Since these definition are recursive and assume the computation of determinants of smaller order minors, it is necessary to define the determinant of a matrix 1 1 (scalar), that is simply det(a ij ) = a ij. k=1 Basilio Bona (DAUIN) Matrices September / 74
19 Properties of determinant det(a B) = det(a)det(b) det(a T ) = det(a) det(ka) = k n det(a) if one makes a number of s exchanges between rows or columns of A, obtaining a new matrix A s, we have det(a s ) = ( 1) s det(a) if A has two equal or proportional rows/columns, we have det(a) = 0 if A has a row or a column that is a linear combination of other rows or columns, we have det(a) = 0 if A è upper or lower triangular, we have det(a) = n i=1 a ii if A is block triangular, with p blocks A ii on the diagonal, we have det(a) = p i=1 deta ii Basilio Bona (DAUIN) Matrices September / 74
20 Singular matrix and rank A matrix A is singular if det(a) = 0. We define the rank of matrix A m n, the number ρ(a m n ), computed as the maximum integer such that at least a non zero minor D p exists. The following properties hold: ρ(a) min{m,n} if ρ(a) = min{m,n}, A is said to have full rank if ρ(a) < min{m,n}, the matrix does not have full rank and one says that there is a fall of rank ρ(ab) min{ρ(a),ρ(b)} ρ(a) = ρ(a T ) ρ(aa T ) = ρ(a T A) = ρ(a) if A n n and deta < n then it has no full rank Basilio Bona (DAUIN) Matrices September / 74
21 Invertible matrix Given a square matrix A R n n, it is invertible of nonsingular if an inverse matrix A 1 n n exists, such that AA 1 = A 1 A = I n The matrix is invertible iff ρ(a) = n, or rather it has full rank; this implies det(a) 0. The inverse matrix can be computed as A 1 = 1 det(a) Adj(A) The following properties hold: (A 1 ) 1 = A; (A T ) 1 = (A 1 ) T. The inverse matrix, if exists, allows to compute the following matrix equation y = Ax obtaining the unknown x as x = A 1 y. Basilio Bona (DAUIN) Matrices September / 74
22 Orthonormal matrix A square matrix is orthonormal if A 1 = A T. the following identity holds A T A = AA T = I Given two square matrices A and B of equal dimension n n, the following identity holds (AB) 1 = B 1 A 1 An important results, called Inversion lemma, establish what follows: if A,C are square invertible matrices and B,D are matrices of of suitable dimensions, then (A+BCD) 1 = A 1 A 1 B(DA 1 B+C 1 ) 1 DA 1 Matrix (DA 1 B+C 1 ) must be invertible. The inversion lemma is useful to compute the inverse of a sum of matrices A 1 +A 2, when A 2 is decomposable into the product BCD and C is easily invertible, for instance diagonal or triangular. Basilio Bona (DAUIN) Matrices September / 74
23 Matrix derivative If a matrix A(t) is composed of elements a ij (t) that are all differentiable wrt (t), then the matrix derivative is [ ] d d A(t) = Ȧ(t) = dt dt a ij(t) = [ȧ ij (t)] If a square matrix A(t) has rank ρ(a(t)) = n for any time (t), then the derivative of its inverse is d dt A(t) 1 1 = A (t)ȧ(t)a(t) 1 Since the inverse operator is non linear, in general it results [ da(t) dt ] 1 d [ A(t) 1 ] dt Basilio Bona (DAUIN) Matrices September / 74
24 Symmetric Skew-symmetric decomposition Given a real matrix A R m n, the two matrices are both symmetric. A T A R n n AA T R m m Given a square matrix A, it is always possible to factor it in a sum of two matrices, as follows: A = A s +A a where A s = 1 2 (A+AT ) symmetric matrix A a = 1 2 (A AT ) skew-symmetric matrix Basilio Bona (DAUIN) Matrices September / 74
25 Similarity transformation Given a square matrix A R n n and a non singular square matrix T R n n, the new matrix B R n n, obtained as B = T 1 AT oppure B = TAT 1 is said to be similar to A, and the transformation T is called similarity transformation. Basilio Bona (DAUIN) Matrices September / 74
26 Eigenvalues and eigenvectors Considering the similarity transformation between A and Λ, where the latter is diagonal Λ = diag(λ i ) and A = UΛU 1 U = [ u 1 u 2 u n ] Multiplying to the right A by U one obtains and then AU = UΛ Au i = λ i u i This identity is the well-known formula that relates the matrix eigenvalues to eigenvectors; the constant quantities λ i are the eigenvalues of A, while vectors u i are the eigenvectors of A, usually with non-unit norm. Basilio Bona (DAUIN) Matrices September / 74
27 Eigenvalues and eigenvectors Given a square matrix A n n, the solutions λ i (real or complex) of the characteristic equation det(λi A) = 0 are the eigenvalues of A. det(λi A) is a polynomial in λ, called characteristic polynomial. If the eigenvalues arre all distinct, the vectors u i that satisfy the identity are the eigenvectors of A. Au i = λ i u i Basilio Bona (DAUIN) Matrices September / 74
28 Generalized eigenvectors If the eigenvalues are not all distinct, one obtains the so-called generalized eigenvalues, whose characterization goes beyond the scope of these notes. From a geometrical point of view, the eigenvectors define those directions in R n (domain of the linear transformation represented by A) that are invariant wrt the transformation A, while the eigenvalues provide the related scale factors along these directions. The set of eigenvalues of a matrix A will be indicated as Λ(A), or rather {λ i (A)}; the set of eigenvectors of A will be indicated as {u i (A)}. In general, since the eigenvectors represent the invariant directions of the transformation, they are represented up to a constant factor, so they are usually normalized; this is a tacit assumption that will be considered here, unless otherwise stated. Basilio Bona (DAUIN) Matrices September / 74
29 Eigenvalues Properties Given a matrix A and its eigenvalues {λ i (A)}, the following holds true {λ i (A+cI)} = {(λ i (A)+c)} Given a matrix A and its eigenvalues {λ i (A)}, the following holds true {λ i (ca)} = {(cλ i (A)} Given an upper or lower triangular matrix a 11 a 12 a 1n 0 a 22 a 2n......, 0 0 a nn a a 21 a a n1 a n2 a nn its eigenvalues are the terms on the diagonal {λ i (A)} = {a ii }; the same applies for a diagonal matrix. Basilio Bona (DAUIN) Matrices September / 74
30 Invariance of the eigenvalues Given a matrix A n n and its eigenvalues {λ i (A)}, the following holds true and det(a) = tr(a) = Given a general invertible transformation, represented by the matrix T, the eigenvalues of A are invariant to the similarity transformation n i=1 n i=1 λ i λ i B = T 1 AT or rather {λ i (B)} = {λ i (A)} Basilio Bona (DAUIN) Matrices September / 74
31 Modal matrix If we build a matrix M, whose columns are the unit eigenvalues u i (A) of A M = [ u 1 u n ] then the similarity transformation wrt M results i a diagonal matrix M takes the name of modal matrix. λ λ Λ = = M 1 AM 0 0 λ n If A is symmetric, its eigenvalues are all real and the following identity holds Λ = M T AM In this particular case M is orthonormal. Basilio Bona (DAUIN) Matrices September / 74
32 Singular value decomposition SVD Given a generic matrix A R m n, having rank r = ρ(a) s, with s = min{m,n}, it can be factorized according to the in the following way: Singular value decomposition (SVD) A = UΣV T = s σ i u i v T i i=1 the important elements of this decomposition are σ i, u i and v i Basilio Bona (DAUIN) Matrices September / 74
33 SVD σ i (A) 0 are called singular values and are equal to the non-negative square roots of the eigenvalues of the symmetric matrix A T A: {σ i (A)} = { λ i (A T A)} σ i 0 listed in decreasing order σ 1 σ 2 σ s 0 if rank r < s there are only r positive singular values; the remaining ones are zero σ 1 σ 2 σ r > 0; σ r+1 = = σ s = 0 U is an orthonormal square matrix (m m) U = [ u 1 u 2 u m ] whose columns are the eigenvectors u i of AA T Basilio Bona (DAUIN) Matrices September / 74
34 SVD V is a orthonormal square matrix (n n) V = [ v 1 v 2 v n ] whose columns are the eigenvectors v i of A T A Σ is a rectangular matrix (m n) with the following structure if m < n Σ = [ Σ s O ] if m = n Σ = Σ s if m > n Σ = [ Σs O Σ s = diag(σ i ) is s s diagonal, and its diagonal terms are the singular values. ]. Basilio Bona (DAUIN) Matrices September / 74
35 SVD Otherwise we can decompose A in a way that puts in evidence the positive singular values alone: where A = [ P P ] [ ] Σr O }{{} O O U }{{} Σ [ Q T Q T ] }{{} V T = PΣ r Q T P is a m r orthonormal matrix; P is a m (m r) orthonormal matrix; Q is a n r orthonormal matrix, Q T ia a n (n r) orthonormal matrix; Σ r is a r r diagonal matrix with diagonal elements the positive singular values σ i, i = 1,,r. Basilio Bona (DAUIN) Matrices September / 74
36 SVD and rank The rank r of A is equal to the number r s of nonzero singular values. Given a generic matrix A R m n, the two matrices A T A and AA T are symmetrical, have the same positive singular values, and differ only for the number of zero singular values. Basilio Bona (DAUIN) Matrices September / 74
37 Linear operators representation Given two vector spaces X R n and Y R m, with dimensions n and m, and given two generic vectors x X and y Y, the generic linear transformation between the two spaces can be represented by the matrix operator A R m n, as follows: y = Ax; x R n ; y R m. Therefore a matrix can be always interpreted as an operator that transforms a vector from the domain in X to the range Y. Conversely, a linear operator has at least one matrix that represents it. Basilio Bona (DAUIN) Matrices September / 74
38 Image space and null space The image space or range of a transformation A is the subspace Y defined by the following property: R(A) = {y y = Ax, x X}; R(A) Y The null space or kernel of a transformation A is the subspace of X defined by the following property: N(A) = {x 0 = Ax, x X}; N(A) X The null space represents all the vectors in X that are trasformed into the origin of Y. The dimensions of the range and kernel are called, respectively, rank ρ(a) and nullity ν(a): ρ(a) = dim(r(a)); ν(a) = dim(n(a)). Basilio Bona (DAUIN) Matrices September / 74
39 Image space and null space If X and Y have finite dimensions, the the following equalities hold: N(A) = R(A T ) R(A) = N(A T ) N(A) = R(A T ) R(A) = N(A T ) where indicates the orthogonal complement to the corresponding (sub-)space. We recall that {0} = R. The following orthogonal decomposition of subspaces X and Y hold X = N(A) N(A) = N(A) R(A T ) Y = R(A) R(A) = R(A) N(A T ) where the symbol represents the direct sum operator between subspaces. Basilio Bona (DAUIN) Matrices September / 74
40 Generalized inverse Given a generic real matrix A R m n, with m n, the inverse matrix is not defined. Nevertheless, it is possible to define a class of matrices A, called pseudo-inverses or generalized inverses, that satisfy the following relation: AA A = A If A has full rank, i.e., ρ(a) = min{m,n}, it is possible to define two classes of generalized inverses if m < n (i.e., ρ(a) = m), the right inverse of A is a matrix A r R n m such that AA r = I m m is n < m (i.e., ρ(a) = n), the left inverse of A is a matrix A l R n m such that A l A = I n n Basilio Bona (DAUIN) Matrices September / 74
41 Pseudo-inverse matrix Among the possible left- or right- inverses, two classes are important: right pseudo-inverse (m < n): A + r = A T (AA T ) 1 When ρ(a) = m, then (AA T ) 1 exists. left pseudo-inverse (n < m): A + l = (AT A) 1 A T When ρ(a) = n, then (A T A) 1 exists; this particular left pseudo-inverse (A T A) 1 A T is also known as the Moore-Penrose pseudo-inverse. Basilio Bona (DAUIN) Matrices September / 74
42 Moore-Penrose pseudo-inverse In general, also if A T A is non invertible, it is always possible to define a Moore-Penrose pseudo-inverse A + that satistfies the following relations: AA + A = A A + AA + = A + (AA + ) T = AA + (A + A) T = A + A (1) Basilio Bona (DAUIN) Matrices September / 74
43 Left and right pseudo-inverses The two pseudo-inverses A + r and A + l coincide with the traditional inverse matrix A 1 when A is square and full-rank: A 1 = A + r = A + l = A+ The linear transformation associated to A R m n y = Ax, with x R n and y R m, is equivalent to a system of m linear equations in n unknowns, whose coefficients are the elements of A; this linear system can admit one solution, no solution or an infinite number of solutions. If we use the pseudo-inverses to solve the linear system y = Ax, we must distinguish two cases, assuming that A has full rank. Basilio Bona (DAUIN) Matrices September / 74
44 Linear systems solution 1 When n > m there are more unknowns than equations; among the infinite possible solutions x R n, we choose the one with minimum norm x, given by x = A + d y = AT (AA T ) 1 y All the other possible solutions of y = Ax are obtained as x = x +v = A + d y+v where v N(A) is a vector belonging to the null space of A, with dimensions n m. These other possible solutions can be expressed also as x = A + d y+(i A+ d A)w where w R n is a n 1 generic vector. The matrix I A + d A projects w on the null space of A, transforming w in v N(A); this matrix is called projection matrix. Basilio Bona (DAUIN) Matrices September / 74
45 Figure: Solution of y = Ax when n > m. Basilio Bona (DAUIN) Matrices September / 74
46 Linear systems solution 2 When m > n there are more equations than unknowns; no exact solutions exist for y = Ax, but only approximate solutions, with an error e = y Ax 0. Among these possible approximate solutions we choose that minimizing the norm of the error, i.e., The solution is ˆx = arg min x R n y Ax ˆx = A + l y = (AT A) 1 A T y Geometrically it is the orthogonal projection of y on the orthogonal complement of N(A), i.e., on the subspace N(A) = R(A T ). The approximation error, also called projection error, is ê = (I AA + s )y and its norm is the lowest, as said above. Basilio Bona (DAUIN) Matrices September / 74
47 Figure: Solution of y = Ax when m > n. Basilio Bona (DAUIN) Matrices September / 74
48 Linear systems solution 3 The similarity between the projection matrix I A + da and the matrix that gives the projection error I AA + s is important and will be studied when projection matrices will be treated. In order to compute the generalized inverses, one can use the SVD. In particular, the pseudo-inverse is computed as [ ] A + Σ 1 = V r O U T = QΣ 1 O O r P T. Basilio Bona (DAUIN) Matrices September / 74
49 Projections and projection matrices The geometrical concept of a projection of a segment on a plane can be extended and generalized to the elements of a vector space. This concept is important for the solution of a large number of problems, as approximation, estimation, prediction and filtering problems. Give a n-dimensional real vector space V(R n ) with dimensions, endowed with the scalar product, and a k n dimensional subspace W(R k ), it is possible to define the projection operator of vectors v V on the subspace W. The projection operator is the square projection matrix P R n n, whose columns are the projections of the base elements of V in W. A matrix is a projection matrix iff P 2 = P i.e., is idempotent. The projection can be orthogonal or non orthogonal; in the first case P is symmetrical, in the second case it is generic. If P is a projection matrix, also I P is a projection matrix. Basilio Bona (DAUIN) Matrices September / 74
50 Projection matrices Some examples of projection matrices are those associated to the left pseudo-inverse P 1 = AA + s e P 2 = I AA + s and to the right pseudo-inverse P 3 = A + d A e P 4 = I A + d A From a geometrical point of view, P 1 projects every vector v V in the range space R(A), while P 2 projects v in its orthogonal complement R(A) = N(A T ). Basilio Bona (DAUIN) Matrices September / 74
51 Matrix norm 1 Similarly to what can be established for a vector, it is possible to provide a measure of the matrix, i.e., give its magnitude, defining the matrix norm. Since a matrix represents a linear transformation between vectors, the matrix norm measures how big this transformation is, but in some way, must normalize the result, to avoid that the magnitude of the transformed vector affects the norm; hence the following definition: A def Ax = sup x x = sup Ax. x =1 Basilio Bona (DAUIN) Matrices September / 74
52 Matrix norm 2 Given a square matrix A R n n, its norm must satisfy the following general (norm) axioms: 1 A > 0 for every A O; 2 A = 0 iff A = O; 3 A+B A + B (triangular inequality); 4 αa = α A for any scalar α and any matrix A; 5 AB A B. Given A R n n and its eigenvalues {λ i (A)}, the following inequality holds true 1 A 1 λ i A i = 1,...,n Basilio Bona (DAUIN) Matrices September / 74
53 Matrix norm 3 Taking only into account real matrices, the most used norms are: Spectral norm: Frobenius norm Max singular value: A 2 = max{λ i (A T A)} i A F = aij tra 2 = T A i j A σ = max{σ i (A)} i Basilio Bona (DAUIN) Matrices September / 74
54 Matrix norm 4 1-norm or max-norm: -norm: In general, and A 1 = max j A = max i A 2 = A σ n a ij i=1 n a ij j=1 A 2 2 A 1 A Basilio Bona (DAUIN) Matrices September / 74
55 Skew-symmetric matrices Skew-symmetric matrix A square matrix S is called skew-symmetric or antisymmetric when S+S T = O or S = S T A skew-symmetric matrix has the following structure 0 s 12 s 1n s A n n = 12 0 s 2n s 1n s 2n 0 Therefore there it has at most n(n 1) 2 independent elements. Basilio Bona (DAUIN) Matrices September / 74
56 Skew-symmetric matrices For n = 3 it results n(n 1) = 3, hence an skew-symmetric matrix has as 2 many element as a 3D vector v. Given a vector v = [ v 1 v 2 v 3 ] T it is possible to build S, and given a matrix S it is possible to extract the associated vector v. We indicate this fact using the symbol S(v), where, by convention S(v) = 0 v 3 v 2 v 3 0 v 1 v 2 v 1 0 Basilio Bona (DAUIN) Matrices September / 74
57 Skew-symmetric matrices Some properties: Given any vector v R 3 : Given two scalars λ 1,λ 2 R: Given any two vectors v,u R 3 : S T (v) = S(v) = S( v) S(λ 1 u+λ 2 v) = λ 1 S(u)+λ 2 S(v) S(u)v = u v = v u = S( v)u = S T (v)u Therefore S(u) is the representation of the operator (u ) and viceversa. Basilio Bona (DAUIN) Matrices September / 74
58 Skew-symmetric matrices The matrix S(u)S(u) = S 2 (u) is symmetrical and S 2 (u) = uu T u 2 I Hence the dyadic product D(u,u) = uu T = S 2 (u)+ u 2 I Basilio Bona (DAUIN) Matrices September / 74
59 Eigenvalues and eigenvectors of skew-symmetric matrices Given an skew-symmetric matrix S(v), its eigenvalues are imaginary or zero. λ 1 = 0, λ 2,3 = ±j v The eigenvalue related to the eigenvector λ 1 = 0 is v; the other two are complex conjugate. The set of skew-symmetric matrices is a vector space, denoted as so(3). Given two skew-symmetric matrices S 1 and S 2, we call commutator or Lie bracket the following operator that is itself skew-symmetric. [S 1,S 2 ] def = S 1 S 2 S 2 S 1 Skew-symmetric matrices form a Lie algebra, which is related to the Lie group of orthogonal matrices. Basilio Bona (DAUIN) Matrices September / 74
60 Orthogonal matrices A square matrix A R n is called orthogonal when with α i 0. α A T 0 α A = α n A square orthogonal matrix U R n is called orthonormal when all the constants α i are 1: U T U = UU T = I Therefore U 1 = U T Basilio Bona (DAUIN) Matrices September / 74
61 Orthonormal matrices Other properties: The columns, as well as the rows, of U or orthogonal to each other and have unit norm. U = 1; The determinant of U has unit module: det(u) = 1 therefore it can be +1 or 1. Given a vector x, its orthonormal transformation is y = Ux. Basilio Bona (DAUIN) Matrices September / 74
62 Orthonormal matrices If U is an orthonormal matrix, then AU = UA = A. Property in general valid also for unitary matrices, i.e., U U = I. When U R 3 3, only 3 out of 9 elements are independent. Scalar product is invariant to orthonormal transformations, (Ux) (Uy) = (Ux) T (Uy) = x T U T Uy = x T y = x y This means that vector lengths are invariant wrt orthonormal trasformations Ux = (Ux) T (Ux) = x T U T Ux = x T Ix = x T x = x Basilio Bona (DAUIN) Matrices September / 74
63 Orthonormal matrices When considering orthonormal transformations, it is important to distinguish the two cases: When det(u) = +1, U represents a proper rotation or simply a rotation, when det(u) = 1, U represents an improper rotation or reflection. The set of rotations forms a continuous non-commutative (wrt product) group; the set of reflections do not have this quality. Intuitively this means that infinitesimal rotations exist, while infinitesimal reflections do not have any meaning. Reflections are the most basic transformation in 3D spaces, in the sense that translations, rotations and roto-reflections (slidings) are obtained from the composition of two or three reflections Basilio Bona (DAUIN) Matrices September / 74
64 Figure: Reflections. Basilio Bona (DAUIN) Matrices September / 74
65 Orthonormal matrices If U is an orthonormal matrix, the distributive property wrt the cross product holds: U(x y) = (Ux) (Uy) (with general A matrices this is not true). For any proper rotation matrix U and a generic vector x the following holds US(x)U T y = U ( x (U T y) ) = (Ux) (UU T y) = (Ux) y = S(Ux)y where S(x) is the skew-symmetric matrix associated with x; therefore: US(x)U T = S(Ux) US(x) = S(Ux)U Basilio Bona (DAUIN) Matrices September / 74
66 Bilinear and quadratic forms A bilinear form associated to the matrix A R m n is the scalar quantity defined as b(x,y) def = x T Ay = y T A T x A quadratic form associated to the square matrix A R n n is the scalar quantity defined as q(x) def = x T Ax = x T A T x Every quadratic form associated to a skew-symmetric matrix S(y) is identically zero x T S(y)x 0 x Indeed, assuming w = S(y)x = y x, one obtains x T S(y)x = x T w, but since, by definition, w is orthogonal to both y and x, the scalar product x T w will be always zero, and also the quadratic form at the left hand side. Basilio Bona (DAUIN) Matrices September / 74
67 Definite positive matrices 1 Recalling the standard decomposition of a generic square matrix A in symmetric term A s and an skew-symmetric one A a, one concludes that the quadratic form depends only on the symmetric part of the matrix: q(x) = x T Ax = x T (A s +A a )x = x T A s x A square matrix A is said to be positive definite if the associated quadratic form x T Ax satisfies to the following conditions x T Ax > 0 x 0 x T Ax = 0 x = 0 A square matrix A is said to be positive semidefinite if the associated quadratic form x T Ax satisfies to the following conditions x T Ax 0 x A square matrix A is said to be negative definite if A is positive definite; similarly, a square matrix A is semidefinite negative if A è semidefinite positive. Basilio Bona (DAUIN) Matrices September / 74
68 Definite positive matrices 2 Often we use the following notations: definite positive matrix: A 0 semidefinite positive matrix: A 0 definite negative matrix: A 0 semidefinite negative matrix: A 0 A necessary but not sufficient condition for a square matrix A to bepositive definite is that the elements on its diagonal are all strictly positive. A necessary and sufficient condition for a square matrix A to be definite positive is that all its eigenvalues are strictly positive. Basilio Bona (DAUIN) Matrices September / 74
69 Sylvester criterion The Sylvester criterion states that a square matrix A is positive definite iff all its principal minors are strictly positive. A definite positive matrix has full rank and is always invertible The associated quadratic form x T Ax satisfies the following identity λ min (A) x 2 x T Ax λ max (A) x 2 where λ min (A) and λ max (A) are, respectively, the minimum and the maximum eigenvalues. Basilio Bona (DAUIN) Matrices September / 74
70 Semidefinite matrix and rank A semidefinite positive matrix A n n has rank ρ(a) = r < n, i.e., it has r strictly positive eigenvalues and n r zero eigenvalues. The quadratic form sgoes to zero for every vector x N(A). Given a real matrix of generic dimensions A m n, we have seen that both A T A and AA T are symmetrical; in addition we know that ρ(a T A) = ρ(aa T ) = ρ(a) These matrices have all real, non negative eigenvalues, and therefore they are definite or semidefinite positive: in particular, if A m n has full rank, then if m < n, A T A 0 and AA T 0, if m = n, A T A 0 and AA T 0, if m > n, A T A 0 and AA T 0. Basilio Bona (DAUIN) Matrices September / 74
71 Matrix derivatives 1 If matrix A has its elements that are functions of a quantity x, one can define the matrix derivative wrt x as [ ] d daij A(x) := dx dx If x is the time t, one writes d A(t) Ȧ(t) := dt [ ] daij (t) [ ] ȧ ij dt If A is a time function through the variable x(t), then [ ] [ ] d aij (x) dx(t) aij (x) A(x(t)) Ȧ(x(t)) := ẋ(t) dt x dt x Basilio Bona (DAUIN) Matrices September / 74
72 Matrix derivatives 2 Given a vector-values scalar function φ(x) defined as φ( ) : R n R 1, the gradient of the function φ wrt to x is a column vector x φ = φ x := φ(x) x 1 φ(x) x n i.e., x := If x(t) is a differentiable time function, then x 1 x n = grad x dφ(x) dt φ(x) = T x φ(x)ẋ (Notice the convention: the gradient for us is a column vector, although many textbooks assume it is a row vector) Basilio Bona (DAUIN) Matrices September / 74
73 Jacobian matrix Given a m 1 vector function f(x) = [ f 1 (x) f m (x) ] T, x R n, the Jacobian matrix (or simply the jacobian) is a m n matrix defined as ( f1 (x) x J f (x) = ( fm (x) x ) T ) T f 1 (x) x 1 = f m (x) x 1 f 1 (x) x n f i (x) x j and if x(t) is a differentiable time function, then ḟ(x) df(x) dt f m (x) x n = df(x) dx ẋ(t) = J f(x)ẋ(t) = (grad xf 1 ) T (grad x f m ) T Notice that the rows of J f are the transpose of the gradients of the various functions. Basilio Bona (DAUIN) Matrices September / 74
74 Gradient Given a bilinear form b(x,y) = x T Ay, we call gradients the following vectors: gradient wrt x: grad x b(x,y) def = b(x,y) = Ay x gradient wrt y: grad y b(x,y) def = b(x,y) = A T x y Given the quadratic form q(x) = x T Ax, we call gradient wrt x the following vector: x q(x) grad x q(x) def = q(x) = 2Ax x Basilio Bona (DAUIN) Matrices September / 74
Matrices A brief introduction
Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 44 Definitions Definition A matrix is a set of N real or complex
More informationMatrices A brief introduction
Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 41 Definitions Definition A matrix is a set of N real or complex
More informationMSMS Vectors and Matrices
MSMS Vectors and Matrices Basilio Bona DAUIN Politecnico di Torino Semester 1, 2015-2016 B. Bona (DAUIN) MSMS-Vectors and matrices Semester 1, 2015-2016 1 / 39 Introduction Most of the topics introduced
More informationFoundations of Matrix Analysis
1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationp 2 p 3 p y p z It will not be considered in the present context; the interested reader can find more details in [05].
1. Geometrical vectors A geometrical vector p represents a point P in space. The point P is an abstraction that often, but not always, requires a representation. Vector representations are given wrt a
More informationReview of some mathematical tools
MATHEMATICAL FOUNDATIONS OF SIGNAL PROCESSING Fall 2016 Benjamín Béjar Haro, Mihailo Kolundžija, Reza Parhizkar, Adam Scholefield Teaching assistants: Golnoosh Elhami, Hanjie Pan Review of some mathematical
More informationLecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,
2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko
More informationMath Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT
Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationProblem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show
MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,
More informationA Brief Outline of Math 355
A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting
More informationSAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra
1.1. Introduction SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear algebra is a specific branch of mathematics dealing with the study of vectors, vector spaces with functions that
More informationPhys 201. Matrices and Determinants
Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1
More informationKnowledge Discovery and Data Mining 1 (VO) ( )
Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory
More informationLecture 2: Linear Algebra Review
EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1
More informationLinGloss. A glossary of linear algebra
LinGloss A glossary of linear algebra Contents: Decompositions Types of Matrices Theorems Other objects? Quasi-triangular A matrix A is quasi-triangular iff it is a triangular matrix except its diagonal
More informationChapter 1. Matrix Algebra
ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2017 LECTURE 5
STAT 39: MATHEMATICAL COMPUTATIONS I FALL 17 LECTURE 5 1 existence of svd Theorem 1 (Existence of SVD) Every matrix has a singular value decomposition (condensed version) Proof Let A C m n and for simplicity
More informationMassachusetts Institute of Technology Department of Economics Statistics. Lecture Notes on Matrix Algebra
Massachusetts Institute of Technology Department of Economics 14.381 Statistics Guido Kuersteiner Lecture Notes on Matrix Algebra These lecture notes summarize some basic results on matrix algebra used
More informationMAT Linear Algebra Collection of sample exams
MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationMaths for Signals and Systems Linear Algebra in Engineering
Maths for Signals and Systems Linear Algebra in Engineering Lectures 13 15, Tuesday 8 th and Friday 11 th November 016 DR TANIA STATHAKI READER (ASSOCIATE PROFFESOR) IN SIGNAL PROCESSING IMPERIAL COLLEGE
More informationMathematical Foundations of Applied Statistics: Matrix Algebra
Mathematical Foundations of Applied Statistics: Matrix Algebra Steffen Unkel Department of Medical Statistics University Medical Center Göttingen, Germany Winter term 2018/19 1/105 Literature Seber, G.
More informationProperties of Matrices and Operations on Matrices
Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationIntroduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX
Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX September 2007 MSc Sep Intro QT 1 Who are these course for? The September
More informationLinear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.
Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationMathematics. EC / EE / IN / ME / CE. for
Mathematics for EC / EE / IN / ME / CE By www.thegateacademy.com Syllabus Syllabus for Mathematics Linear Algebra: Matrix Algebra, Systems of Linear Equations, Eigenvalues and Eigenvectors. Probability
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationEE731 Lecture Notes: Matrix Computations for Signal Processing
EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten
More informationSymmetric and anti symmetric matrices
Symmetric and anti symmetric matrices In linear algebra, a symmetric matrix is a square matrix that is equal to its transpose. Formally, matrix A is symmetric if. A = A Because equal matrices have equal
More informationSAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to Linear Algebra
SAMPLE OF THE STUDY MATERIAL PART OF CHAPTER 1 Introduction to 1.1. Introduction Linear algebra is a specific branch of mathematics dealing with the study of vectors, vector spaces with functions that
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More informationMATRICES ARE SIMILAR TO TRIANGULAR MATRICES
MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,
More informationLinear Algebra Primer
Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................
More informationChapter 3. Matrices. 3.1 Matrices
40 Chapter 3 Matrices 3.1 Matrices Definition 3.1 Matrix) A matrix A is a rectangular array of m n real numbers {a ij } written as a 11 a 12 a 1n a 21 a 22 a 2n A =.... a m1 a m2 a mn The array has m rows
More informationTaxonomy of n n Matrices. Complex. Integer. Real. diagonalizable. Real. Doubly stochastic. Unimodular. Invertible. Permutation. Orthogonal.
Doubly stochastic Taxonomy of n n Matrices Each rectangle represents one class of complex n n matrices. Arrows indicate subset relations. Classes in green are closed under multiplication. Classes in blue
More informationThere are six more problems on the next two pages
Math 435 bg & bu: Topics in linear algebra Summer 25 Final exam Wed., 8/3/5. Justify all your work to receive full credit. Name:. Let A 3 2 5 Find a permutation matrix P, a lower triangular matrix L with
More informationQuantum Computing Lecture 2. Review of Linear Algebra
Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces
More informationMATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.
MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation
More informationThroughout these notes we assume V, W are finite dimensional inner product spaces over C.
Math 342 - Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationMATH 583A REVIEW SESSION #1
MATH 583A REVIEW SESSION #1 BOJAN DURICKOVIC 1. Vector Spaces Very quick review of the basic linear algebra concepts (see any linear algebra textbook): (finite dimensional) vector space (or linear space),
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationIntroduction to Matrix Algebra
Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p
More informationFundamentals of Engineering Analysis (650163)
Philadelphia University Faculty of Engineering Communications and Electronics Engineering Fundamentals of Engineering Analysis (6563) Part Dr. Omar R Daoud Matrices: Introduction DEFINITION A matrix is
More informationECE 275A Homework #3 Solutions
ECE 75A Homework #3 Solutions. Proof of (a). Obviously Ax = 0 y, Ax = 0 for all y. To show sufficiency, note that if y, Ax = 0 for all y, then it must certainly be true for the particular value of y =
More informationLinear Algebra Formulas. Ben Lee
Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationEquality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.
Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read
More informationSTAT200C: Review of Linear Algebra
Stat200C Instructor: Zhaoxia Yu STAT200C: Review of Linear Algebra 1 Review of Linear Algebra 1.1 Vector Spaces, Rank, Trace, and Linear Equations 1.1.1 Rank and Vector Spaces Definition A vector whose
More informationLecture Notes in Linear Algebra
Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................
More informationMathematical Methods wk 2: Linear Operators
John Magorrian, magog@thphysoxacuk These are work-in-progress notes for the second-year course on mathematical methods The most up-to-date version is available from http://www-thphysphysicsoxacuk/people/johnmagorrian/mm
More informationBasic Elements of Linear Algebra
A Basic Review of Linear Algebra Nick West nickwest@stanfordedu September 16, 2010 Part I Basic Elements of Linear Algebra Although the subject of linear algebra is much broader than just vectors and matrices,
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationACM 104. Homework Set 5 Solutions. February 21, 2001
ACM 04 Homework Set 5 Solutions February, 00 Franklin Chapter 4, Problem 4, page 0 Let A be an n n non-hermitian matrix Suppose that A has distinct eigenvalues λ,, λ n Show that A has the eigenvalues λ,,
More informationELE/MCE 503 Linear Algebra Facts Fall 2018
ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2
More informationComputational math: Assignment 1
Computational math: Assignment 1 Thanks Ting Gao for her Latex file 11 Let B be a 4 4 matrix to which we apply the following operations: 1double column 1, halve row 3, 3add row 3 to row 1, 4interchange
More informationTopic 1: Matrix diagonalization
Topic : Matrix diagonalization Review of Matrices and Determinants Definition A matrix is a rectangular array of real numbers a a a m a A = a a m a n a n a nm The matrix is said to be of order n m if it
More informationMath Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p
Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the
More informationMATH 315 Linear Algebra Homework #1 Assigned: August 20, 2018
Homework #1 Assigned: August 20, 2018 Review the following subjects involving systems of equations and matrices from Calculus II. Linear systems of equations Converting systems to matrix form Pivot entry
More informationLinear Algebra in Actuarial Science: Slides to the lecture
Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations
More informationLinear Algebra Review
Linear Algebra Review Contents 1 Basic Concepts and Notations 2 2 Matrix Operations and Properties 3 21 Matrix Multiplication 3 211 Vector-Vector Products 3 212 Matrix-Vector Products 4 213 Matrix-Matrix
More informationQueens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane.
Queens College, CUNY, Department of Computer Science Numerical Methods CSCI 361 / 761 Spring 2018 Instructor: Dr. Sateesh Mane c Sateesh R. Mane 2018 8 Lecture 8 8.1 Matrices July 22, 2018 We shall study
More informationJim Lambers MAT 610 Summer Session Lecture 1 Notes
Jim Lambers MAT 60 Summer Session 2009-0 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra
More informationLinear Algebra and Matrix Inversion
Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much
More information(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =
. (5 points) (a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? dim N(A), since rank(a) 3. (b) If we also know that Ax = has no solution, what do we know about the rank of A? C(A)
More informationChapter 7. Linear Algebra: Matrices, Vectors,
Chapter 7. Linear Algebra: Matrices, Vectors, Determinants. Linear Systems Linear algebra includes the theory and application of linear systems of equations, linear transformations, and eigenvalue problems.
More informationChapter 7. Canonical Forms. 7.1 Eigenvalues and Eigenvectors
Chapter 7 Canonical Forms 7.1 Eigenvalues and Eigenvectors Definition 7.1.1. Let V be a vector space over the field F and let T be a linear operator on V. An eigenvalue of T is a scalar λ F such that there
More informationLinear Algebra Lecture Notes-II
Linear Algebra Lecture Notes-II Vikas Bist Department of Mathematics Panjab University, Chandigarh-64 email: bistvikas@gmail.com Last revised on March 5, 8 This text is based on the lectures delivered
More informationReview of Some Concepts from Linear Algebra: Part 2
Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set
More informationLINEAR ALGEBRA REVIEW
LINEAR ALGEBRA REVIEW JC Stuff you should know for the exam. 1. Basics on vector spaces (1) F n is the set of all n-tuples (a 1,... a n ) with a i F. It forms a VS with the operations of + and scalar multiplication
More information18.06SC Final Exam Solutions
18.06SC Final Exam Solutions 1 (4+7=11 pts.) Suppose A is 3 by 4, and Ax = 0 has exactly 2 special solutions: 1 2 x 1 = 1 and x 2 = 1 1 0 0 1 (a) Remembering that A is 3 by 4, find its row reduced echelon
More informationLinear Algebra: Matrix Eigenvalue Problems
CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given
More informationLinear Algebra Highlights
Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to
More informationANSWERS (5 points) Let A be a 2 2 matrix such that A =. Compute A. 2
MATH 7- Final Exam Sample Problems Spring 7 ANSWERS ) ) ). 5 points) Let A be a matrix such that A =. Compute A. ) A = A ) = ) = ). 5 points) State ) the definition of norm, ) the Cauchy-Schwartz inequality
More informationAlgebra C Numerical Linear Algebra Sample Exam Problems
Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric
More informationPseudoinverse & Orthogonal Projection Operators
Pseudoinverse & Orthogonal Projection Operators ECE 174 Linear & Nonlinear Optimization Ken Kreutz-Delgado ECE Department, UC San Diego Ken Kreutz-Delgado (UC San Diego) ECE 174 Fall 2016 1 / 48 The Four
More informationRecall the convention that, for us, all vectors are column vectors.
Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists
More informationMAT 610: Numerical Linear Algebra. James V. Lambers
MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic
More informationG1110 & 852G1 Numerical Linear Algebra
The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the
More informationLinear Algebra. Workbook
Linear Algebra Workbook Paul Yiu Department of Mathematics Florida Atlantic University Last Update: November 21 Student: Fall 2011 Checklist Name: A B C D E F F G H I J 1 2 3 4 5 6 7 8 9 10 xxx xxx xxx
More informationEigenvalues and Eigenvectors
/88 Chia-Ping Chen Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra Eigenvalue Problem /88 Eigenvalue Equation By definition, the eigenvalue equation for matrix
More informationChapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in
Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column
More informationSingular Value Decomposition (SVD)
School of Computing National University of Singapore CS CS524 Theoretical Foundations of Multimedia More Linear Algebra Singular Value Decomposition (SVD) The highpoint of linear algebra Gilbert Strang
More informationDuke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014
Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Linear Algebra A Brief Reminder Purpose. The purpose of this document
More informationNOTES on LINEAR ALGEBRA 1
School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura
More informationAPPENDIX A. Background Mathematics. A.1 Linear Algebra. Vector algebra. Let x denote the n-dimensional column vector with components x 1 x 2.
APPENDIX A Background Mathematics A. Linear Algebra A.. Vector algebra Let x denote the n-dimensional column vector with components 0 x x 2 B C @. A x n Definition 6 (scalar product). The scalar product
More informationNumerical Methods I Solving Square Linear Systems: GEM and LU factorization
Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 18th,
More informationMatrix Representation
Matrix Representation Matrix Rep. Same basics as introduced already. Convenient method of working with vectors. Superposition Complete set of vectors can be used to express any other vector. Complete set
More informationEigenvalues, Eigenvectors. Eigenvalues and eigenvector will be fundamentally related to the nature of the solutions of state space systems.
Chapter 3 Linear Algebra In this Chapter we provide a review of some basic concepts from Linear Algebra which will be required in order to compute solutions of LTI systems in state space form, discuss
More informationNotes on Linear Algebra
1 Notes on Linear Algebra Jean Walrand August 2005 I INTRODUCTION Linear Algebra is the theory of linear transformations Applications abound in estimation control and Markov chains You should be familiar
More informationChapter 5 Eigenvalues and Eigenvectors
Chapter 5 Eigenvalues and Eigenvectors Outline 5.1 Eigenvalues and Eigenvectors 5.2 Diagonalization 5.3 Complex Vector Spaces 2 5.1 Eigenvalues and Eigenvectors Eigenvalue and Eigenvector If A is a n n
More informationA Review of Linear Algebra
A Review of Linear Algebra Mohammad Emtiyaz Khan CS,UBC A Review of Linear Algebra p.1/13 Basics Column vector x R n, Row vector x T, Matrix A R m n. Matrix Multiplication, (m n)(n k) m k, AB BA. Transpose
More informationIntroduction to Mobile Robotics Compact Course on Linear Algebra. Wolfram Burgard, Cyrill Stachniss, Kai Arras, Maren Bennewitz
Introduction to Mobile Robotics Compact Course on Linear Algebra Wolfram Burgard, Cyrill Stachniss, Kai Arras, Maren Bennewitz Vectors Arrays of numbers Vectors represent a point in a n dimensional space
More informationMath 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008
Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures
More informationMath Linear Algebra II. 1. Inner Products and Norms
Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,
More information