Knowledge Discovery and Data Mining 1 (VO) ( )

Size: px
Start display at page:

Download "Knowledge Discovery and Data Mining 1 (VO) ( )"

Transcription

1 Knowledge Discovery and Data Mining 1 (VO) ( ) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

2 Big picture: KDDM Probability Theory Linear Algebra Hardware & Programming Model Information Theory Statistical Inference Mathematical Tools Infrastructure Knowledge Discovery Process Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

3 Outline 1 Matrices and Vectors 2 Matrix Multiplication 3 Operations and Properties 4 Eigenvalues and Eigenvectors Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

4 Matrices and Vectors Matrix Matrix Matrix is a rectangular array of numbers, i.e. A R m n a 11 a a 1n a 21 a a 2n A = a m1 a m2... a mn Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

5 Matrices and Vectors Matrix: Examples A = A R 2 3 B R 4 2 ( ) m and n are matrix dimensions B = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

6 Matrices and Vectors Multiplication with a scalar Scalar multiplication Given a matrix A R m n and a scalar α R the scalar multiplication of A with α is the matrix B = αa R m n, where B ij = αa ij αa 11 αa αa 1n αa 21 αa αa 2n αa = αa m1 αa m2... αa mn Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

7 Matrices and Vectors Multiplication with a scalar: example 2 ( ) = ( ) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

8 Matrices and Vectors Vector Vector Vector is a matrix consisting of one column, i.e. x R n. A row vector is a matrix consisting of one row. x 1 x 2 x =. x n Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

9 Matrices and Vectors Vector: Example x = x R 4 n is the vector dimension Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

10 Matrix Multiplication Matrix Multiplication Matrix Multiplication The product of two matrices A R m n and B R n p is the matrix C = AB R m p, where n C ij = A ik B kj (1) k=1 For the product to exist: the number of columns in A must equal the number of rows in B Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

11 Matrix Multiplication Matrix Multiplication: Example ( ) 4 5 ( ) a11 a = a a 22 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

12 Matrix Multiplication Matrix Multiplication: Example ( ) 4 5 ( ) a11 a = a a 22 a 11 = 1 4+ Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

13 Matrix Multiplication Matrix Multiplication: Example ( ) 4 5 ( ) a11 a = a a 22 a 11 = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

14 Matrix Multiplication Matrix Multiplication: Example ( ) 4 5 ( ) a12 = a a 22 a 11 = = 30 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

15 Matrix Multiplication Matrix Multiplication: Example ( ) = ( ) a 21 a 22 a 12 = = 39 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

16 Matrix Multiplication Matrix Multiplication: Example ( ) = ( ) a 22 a 21 = = 34 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

17 Matrix Multiplication Matrix Multiplication: Example ( ) = ( 30 ) a 22 = = 29 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

18 Matrix Multiplication Vector-Vector products Inner product Given two vectors x, y R n the quantity x T y, sometimes called the inner product or dot product, is a real number given by x T y = ( y 1 ) x 1 x 2... x n y 2 n = x i y i.y i=1 n The inner products are just special cases of matrix multiplication Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

19 Matrix Multiplication Inner product: Example ( ) = x T y = = 29 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

20 Matrix Multiplication Vector-Vector products Outer product Given two vectors x R m and y R n the matrix xy T is called the outer product of the vectors. The elements of the matrix are given by (xy T ) ij = x i y j, i.e. x 1 x 1 y 1 x 1 y 2... x 1 y n xy T x 2 ( ) x 2 y 1 x 2 y 2... x 2 y n = y1 y 2... y n = x m x m y 1 x m y 2... x m y n Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

21 Matrix Multiplication Matrix-Vector products Matrix-vector product Given a matrix A R m n and a vector x R n, their product is a vector y = Ax R m, i.e. a 11 a a 1n x n 1 i=1 a 21 a a 2n x 2 a n 1ix i a T Ax = = i=1 a 1 2ix i x. = a T 2 x. a m1 a m2... a mn x n n i=1 a mix i a T mx Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

22 Matrix Multiplication Matrix-Vector products: Example ( ) = ( ) 30 y 2 y 1 = = 30 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

23 Matrix Multiplication Matrix-Vector products: Example ( ) = ( ) y 2 = = 34 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

24 Matrix Multiplication Matrix-Vector products Matrix-vector product Given a matrix A R m n and a vector x R m, their (left row) product is a vector y t = x T A R n, i.e. x T A = = a 11 a a 1n a ( ) 21 a a 2n x1 x 2... x m a m1 a m2... a mn ( mi=1 a i1 x mi=1 i a i2 x i... mi=1 ) ) a in x i = (x T a 1 x T a 2... x T a m Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

25 Matrix Multiplication Matrix-Vector products: Example ( ) ( ) = ( ) 34 y y 3 y 1 = = 34 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

26 Matrix Multiplication Matrix-Vector products: Example ( ) ( ) = ( ) y y 2 = = 28 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

27 Matrix Multiplication Matrix-Vector products: Example ( ) ( ) = ( ) y 3 = = 22 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

28 Matrix Multiplication Properties of matrix multiplication Matrix multiplication is associative: (AB)C = A(BC) Matrix multiplication is distributive: A(B + C) = AB + AC Matrix multiplication is not commutative: AB BA Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

29 Matrix Multiplication Matrix Sum Matrix Sum The sum of two matrices A R m n and B R m n is the matrix C = A + B R m n, where C ij = A ij + B ij (2) For the sum to exist: the dimension of A and B must be equal Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

30 Matrix Multiplication Matrix Sum: Example ( ) ( ) = ( 5 7 ) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

31 Matrix Multiplication Linear combinations of vectors Linear combination Given are k vectors x i R n and k scalars α i R. Then the linear combination of those vectors with those scalars is defined as: α 1 x 1 + α 2 x α k x k = k α i x i (3) i=1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

32 Matrix Multiplication Linear combinations of vectors: example = (4) In fact, any vector in R 3 is a linear combination of those three vectors Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

33 Operations and Properties Identity matrix Identity matrix The identity matrix, denoted I R n n is a square matrix with ones on the diagonal and zeros everywhere else, that is { 1, i = j I ij = 0, i j The identity matrix has the property that for all A R m n AI = A = IA (5) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

34 Operations and Properties Identity matrix I = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

35 Operations and Properties Diagonal matrix Diagonal matrix A diagonal matrix, denoted D R n n is a square matrix with all non-diagonal elements zero. This is typically denoted D = diag(d 1, d 2,..., d n ), with { d i, i = j D ij = 0, i j The identity matrix is then I = diag(1, 1,..., 1) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

36 Operations and Properties Diagonal matrix D = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

37 Operations and Properties The Transpose Transpose The transpose of a matrix results from flipping the rows and columns. Given a matrix A R m n, its transpose A T R n m, is the n m matrix whose entries are given = A ji A T ij We have been using the transpose for describing row vectors Transpose of a column vector is a row vector Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

38 Operations and Properties Properties of transpose (A T ) T = A (AB) T = B T A T (A + B) T = A T + B T Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

39 Operations and Properties Transpose: Example ( ) T = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

40 Operations and Properties Symmetric Matrices Symmetric matrices A square matrix A R n n is symmetric if A = A T. For any matrix A R n n, the matrix AA T is symmetric Proof left for exercise ;) Symmetric matrices occur very often in practice and they have many nice properties Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

41 Operations and Properties Symmetric Matrices: Example A = (6) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

42 Operations and Properties Trace Trace The trace of a square matrix A R n n, denoted by tr(a) is the sum of diagonal elements in the matrix: tr(a) = n A ii (7) i=1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

43 Operations and Properties Trace: example tr(a) = tr( ) = 2 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

44 Operations and Properties Norms Norm A norm of a vector x is informally a measure of the length of the vector. For example, we have the commonly used Euclidean or l 2 norm, Note that: x 2 2 = xt x x 2 = n xi 2 (8) i=1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

45 Operations and Properties Norms Norm A norm is any function f : R n R that satisfies 4 properties: 1 For all x R n, f (x) 0 (non-negativity) 2 f (x) = 0, if and only if x = 0 (definiteness) 3 For all x R n, t R, f (tx) = t f (x) (homogeneity) 4 For all x, y R n, f (x + y) f (x) + f (y) (triangle inequality) Norms are used to normalize vectors and obtain unit vectors (vectors of length 1) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

46 Operations and Properties Norms Other examples of norms include l 1, l, or more generally l p, with p 1 x 1 = n x i i=1 x = max i x i ( n ) 1/p x p = x i p i=1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

47 Operations and Properties Norms: example 4 x = 10 2 x 1 = 16 x 2 = x = 10 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

48 Operations and Properties Linear independence Linear independence A set of vectors {x 1, x 2,..., x k } R n is said to be linearly independent if no vector can be represented as a linear combination of the remaining vectors. Linear independence A set of vectors {x 1, x 2,..., x k } R n is said to be linearly independent if no scalars {α 1, α 2,..., α k } exist such that k α i x i = 0 i=1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

49 Operations and Properties Linear independence: example x 1 = 0 x 2 = 1 x 3 = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

50 Operations and Properties Linear dependence Conversly, if one vector belonging to the set can be represented as a linear combination of the remaining vectors the vectors are linearly dependent x 1 = 2 x 2 = 1 x 3 = Dependent because: x 3 = 2x 1 + x 2 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

51 Operations and Properties Vector bases Span A set of n vectors {x 1, x 2,..., x n } R n is said to span R n, i.e. they form a basis for the space. Any vector v R n can be written as a linear combination of x 1 through x n. Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

52 Operations and Properties Vector bases Orthogonal basis An orthogonal basis {x 1, x 2,..., x n } R n satisfies x T i x j = 0, if i j Orthonormal basis An orthonormal basis {x 1, x 2,..., x n } R n satisfies x T i x j = 0, if i j x T i x j = 1, if i = j Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

53 Operations and Properties Orthonormal basis: example x 1 = 0 x 2 = 1 x 3 = Orthonormal basis for R 3 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

54 Operations and Properties Rank Rank The column rank of a matrix A R m n is the size of the largest subset of columns that constitute a linearly independent set. The row rank is the largest subset of rows that constitute a linearly independent set. For any matrix A R m n the columns rank is equal to row rank and both quantities are referred to as the rank of A, denoted as rank(a) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

55 Operations and Properties Rank: example rank( ) = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

56 Operations and Properties Properties of the matrix rank Some basic properties of the rank: 1 For A R m n, rank(a) min(m, n). If rank(a) = min(m, n) then A is said to be full rank. 2 For A R m n, rank(a) = rank(a T ) 3 For A R m n, B R n p, rank(ab) = min(rank(a), rank(b)) 4 For A, B R m n, rank(a + B) rank(a) + rank(b) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

57 Operations and Properties The Inverse Inverse The inverse of a square matrix A R n n is denoted A 1 and is a unique matrix such that A 1 A = I = AA 1 Not all matrices have inverses If A 1 exists we say that A is invertible or non-singular If A 1 does not exist we say that A is non-invertible or singular Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

58 Operations and Properties The Inverse For A to be invertible it must be full rank There are many alternative conditions for invertability The inverse is used in e.g. solving the system of linear equations: Ax = b With A R n n and x, b R n Then x = A 1 b Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

59 Operations and Properties Properties of inverse (A 1 ) 1 = A (AB) 1 = B 1 A 1 (A 1 ) T = (A T ) 1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

60 Operations and Properties Inverse: Example ( ) 1 a b = c d 1 ad bc ( ) = ( d ) b c a ( 5 ) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

61 Operations and Properties Orthogonal matrices Inverse A square matrix U R n n is orthogonal if all its columns are orthonormal (orthogonal to each other and normalized). It follows from the definition: U T U = I = UU T In other words, the inverse of an orthogonal matrix is its transpose Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

62 Operations and Properties Orthogonal matrices: Example ( ) T ( ) = ( ) 1 0 = 0 1 ( ) ( ) T Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

63 Operations and Properties The determinant Determinant The determinant is a real number associated with a square matrix A R n n, denoted det(a). Depending on n it can be calculated by different arithmetic expressions. ( ) a b det( ) = ad bc c d ( ) 6 2 det( ) = Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

64 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors Eigenvalues and eigenvectors Given a square matrix A R n n, we say that λ C is an eigenvalue of A and x C n is the corresponding eigenvector if Ax = λx, x 0 Intuitively, multiplying A by the vector x does not change the direction of x The new vector points in the same direction but is scaled by factor λ Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

65 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors Also, for any eigenvector x C n, and scalar c C: A(cx) = cax = cλx = λ(cx) Thus, cx is also an eigenvector with eigenvalue λ For this reason, we usually normalize the eigenvector x to have length 1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

66 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors We can rewrite the equation from above (it holds: x 0): Ax = λx Ax = λix λix Ax = 0 (λi A)x = 0 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

67 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors (λi A)x = 0, x 0 has a non-zero solution only if the matrix (λi A) is singular, i.e.: det(λi A) = 0 This can be expanded in a very large polynomial in λ, with maximal degree n We then find the roots of that polynomial and obtain the eigenvalues λ 1,..., λ n Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

68 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors To find the eigenvector corresponding to the eigenvalue λ i we simply solve the linear equation: (λ i I A)x = 0 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

69 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors: example ( ) 2 4 A = 1 1 ( ) λ 2 4 det(λi A) = det( ) = (λ 2)(λ + 1) 4 1 λ + 1 = λ 2 2λ + λ 2 4 = λ 2 λ 6 = (λ 3)(λ + 2) Thus, λ 1 = 3, and λ 2 = 2 are eigenvalues of A We now solve (λ i I A)x = 0 for each eigenvalue to find the corresponding eigenvectors Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

70 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors: example For λ 1 = 3 ( ) ( ) 1 4 x1 = 1 4 x 2 ( ) 0 0 x 1 + 4x 2 = 0 x 1 + 4x 2 = 0 Thus, x 1 = 4x 2, and we might pick x = And we normalize to: ( ) 4/ 17 1/ 17 ( ) 4 1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

71 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors: example For λ 1 = 2 ( ) ( ) 4 4 x1 = 1 1 x 2 ( ) 0 0 4x 1 + 4x 2 = 0 x 1 x 2 = 0 Thus, x 1 = x 2, and we might pick x = And we normalize to: ( ) 1/ 2 1/ 2 ( ) 1 1 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

72 Eigenvalues and Eigenvectors Properties of eigenvalues and eigenvectors For A R n n, eigenvalues λ 1,..., λ n, and associated eigenvectors x 1,..., x n the following are properties: 1 The trace of A is equal to the sum of its eigenvalues: tr(a) = 2 The determinant of A is equal to the product of its eigenvalues: det(a) = n i=1 n i=1 λ i λ i Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

73 Eigenvalues and Eigenvectors Properties of eigenvalues and eigenvectors For A R n n, eigenvalues λ 1,..., λ n, and associated eigenvectors x 1,..., x n the following are properties: 3 The rank of A is equal to the number of non-zero eigenvalues of A 4 If A is non-singular then 1/λ i is an eigenvalue of A 1 with associated vector x i, i.e. A 1 x i = (1/λ i )x i 5 The eigenvalues of a diagonal matrix D = diag(d 1,..., d n ) are just the diagonal entries d 1,..., d n Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

74 Eigenvalues and Eigenvectors Eigenvalues and eigenvectors of symmetric matrices There are two important properties of eigenvalues and eigenvectors of symmetric matrices All eigenvalues are real The eigenvectors are orthonormal If the eigenvectors are linearly independent then we can decompose A as UΛA T U is an orthogonal matrix where colums are the eigenvectors Λ = diag(λ 1,..., λ n ) Denis Helic (KTI, TU Graz) KDDM1 Oct 9, / 74

CS 246 Review of Linear Algebra 01/17/19

CS 246 Review of Linear Algebra 01/17/19 1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

More information

Quick Tour of Linear Algebra and Graph Theory

Quick Tour of Linear Algebra and Graph Theory Quick Tour of Linear Algebra and Graph Theory CS224W: Social and Information Network Analysis Fall 2014 David Hallac Based on Peter Lofgren, Yu Wayne Wu, and Borja Pelato s previous versions Matrices and

More information

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the

More information

Matrices and Linear Algebra

Matrices and Linear Algebra Contents Quantitative methods for Economics and Business University of Ferrara Academic year 2017-2018 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2 3 4 5 Contents 1 Basics 2

More information

Quick Tour of Linear Algebra and Graph Theory

Quick Tour of Linear Algebra and Graph Theory Quick Tour of Linear Algebra and Graph Theory CS224w: Social and Information Network Analysis Fall 2012 Yu Wayne Wu Based on Borja Pelato s version in Fall 2011 Matrices and Vectors Matrix: A rectangular

More information

A Quick Tour of Linear Algebra and Optimization for Machine Learning

A Quick Tour of Linear Algebra and Optimization for Machine Learning A Quick Tour of Linear Algebra and Optimization for Machine Learning Masoud Farivar January 8, 2015 1 / 28 Outline of Part I: Review of Basic Linear Algebra Matrices and Vectors Matrix Multiplication Operators

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

10-701/ Recitation : Linear Algebra Review (based on notes written by Jing Xiang)

10-701/ Recitation : Linear Algebra Review (based on notes written by Jing Xiang) 10-701/15-781 Recitation : Linear Algebra Review (based on notes written by Jing Xiang) Manojit Nandi February 1, 2014 Outline Linear Algebra General Properties Matrix Operations Inner Products and Orthogonal

More information

Preliminary Linear Algebra 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 100

Preliminary Linear Algebra 1. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 100 Preliminary Linear Algebra 1 Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 100 Notation for all there exists such that therefore because end of proof (QED) Copyright c 2012

More information

Linear Algebra Formulas. Ben Lee

Linear Algebra Formulas. Ben Lee Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries

More information

Linear Algebra Review. Vectors

Linear Algebra Review. Vectors Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors

More information

Introduction to Matrix Algebra

Introduction to Matrix Algebra Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

More information

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0. Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the

More information

Jim Lambers MAT 610 Summer Session Lecture 1 Notes

Jim Lambers MAT 610 Summer Session Lecture 1 Notes Jim Lambers MAT 60 Summer Session 2009-0 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra

More information

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n Lectures -2: Linear Algebra Background Almost all linear and nonlinear problems in scientific computation require the use of linear algebra These lectures review basic concepts in a way that has proven

More information

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the

More information

Quantum Computing Lecture 2. Review of Linear Algebra

Quantum Computing Lecture 2. Review of Linear Algebra Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces

More information

. a m1 a mn. a 1 a 2 a = a n

. a m1 a mn. a 1 a 2 a = a n Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by

More information

MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, Chapter 1: Linear Equations and Matrices MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

More information

NOTES on LINEAR ALGEBRA 1

NOTES on LINEAR ALGEBRA 1 School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

More information

Econ Slides from Lecture 7

Econ Slides from Lecture 7 Econ 205 Sobel Econ 205 - Slides from Lecture 7 Joel Sobel August 31, 2010 Linear Algebra: Main Theory A linear combination of a collection of vectors {x 1,..., x k } is a vector of the form k λ ix i for

More information

ELE/MCE 503 Linear Algebra Facts Fall 2018

ELE/MCE 503 Linear Algebra Facts Fall 2018 ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2

More information

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

More information

Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,

Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, 2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for

More information

Review of Some Concepts from Linear Algebra: Part 2

Review of Some Concepts from Linear Algebra: Part 2 Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set

More information

Math 315: Linear Algebra Solutions to Assignment 7

Math 315: Linear Algebra Solutions to Assignment 7 Math 5: Linear Algebra s to Assignment 7 # Find the eigenvalues of the following matrices. (a.) 4 0 0 0 (b.) 0 0 9 5 4. (a.) The characteristic polynomial det(λi A) = (λ )(λ )(λ ), so the eigenvalues are

More information

Linear Algebra Highlights

Linear Algebra Highlights Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to

More information

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Dr Gerhard Roth COMP 40A Winter 05 Version Linear algebra Is an important area of mathematics It is the basis of computer vision Is very widely taught, and there are many resources

More information

4 Linear Algebra Review

4 Linear Algebra Review 4 Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the course 41 Vectors and Matrices For the context of data analysis,

More information

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88 Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

More information

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian

More information

Chapter 2. Matrix Arithmetic. Chapter 2

Chapter 2. Matrix Arithmetic. Chapter 2 Matrix Arithmetic Matrix Addition and Subtraction Addition and subtraction act element-wise on matrices. In order for the addition/subtraction (A B) to be possible, the two matrices A and B must have the

More information

Elementary maths for GMT

Elementary maths for GMT Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

More information

Phys 201. Matrices and Determinants

Phys 201. Matrices and Determinants Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1

More information

STAT200C: Review of Linear Algebra

STAT200C: Review of Linear Algebra Stat200C Instructor: Zhaoxia Yu STAT200C: Review of Linear Algebra 1 Review of Linear Algebra 1.1 Vector Spaces, Rank, Trace, and Linear Equations 1.1.1 Rank and Vector Spaces Definition A vector whose

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

Linear Algebra Review

Linear Algebra Review Linear Algebra Review ORIE 4741 September 1, 2017 Linear Algebra Review September 1, 2017 1 / 33 Outline 1 Linear Independence and Dependence 2 Matrix Rank 3 Invertible Matrices 4 Norms 5 Projection Matrix

More information

1. Linear systems of equations. Chapters 7-8: Linear Algebra. Solution(s) of a linear system of equations (continued)

1. Linear systems of equations. Chapters 7-8: Linear Algebra. Solution(s) of a linear system of equations (continued) 1 A linear system of equations of the form Sections 75, 78 & 81 a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2 a m1 x 1 + a m2 x 2 + + a mn x n = b m can be written in matrix

More information

Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX

Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX Introduction to Quantitative Techniques for MSc Programmes SCHOOL OF ECONOMICS, MATHEMATICS AND STATISTICS MALET STREET LONDON WC1E 7HX September 2007 MSc Sep Intro QT 1 Who are these course for? The September

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 2 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 2 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory April 5, 2012 Andre Tkacenko

More information

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015

Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 2015 Chapters 5 & 6: Theory Review: Solutions Math 308 F Spring 205. If A is a 3 3 triangular matrix, explain why det(a) is equal to the product of entries on the diagonal. If A is a lower triangular or diagonal

More information

Math Linear Algebra Final Exam Review Sheet

Math Linear Algebra Final Exam Review Sheet Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

More information

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

More information

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear

More information

Review of linear algebra

Review of linear algebra Review of linear algebra 1 Vectors and matrices We will just touch very briefly on certain aspects of linear algebra, most of which should be familiar. Recall that we deal with vectors, i.e. elements of

More information

1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det

1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

Problem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show

Problem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,

More information

Matrices and systems of linear equations

Matrices and systems of linear equations Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T.

More information

Chapter 2 Notes, Linear Algebra 5e Lay

Chapter 2 Notes, Linear Algebra 5e Lay Contents.1 Operations with Matrices..................................1.1 Addition and Subtraction.............................1. Multiplication by a scalar............................ 3.1.3 Multiplication

More information

Conceptual Questions for Review

Conceptual Questions for Review Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.

More information

Linear Algebra (Review) Volker Tresp 2017

Linear Algebra (Review) Volker Tresp 2017 Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.

More information

3 (Maths) Linear Algebra

3 (Maths) Linear Algebra 3 (Maths) Linear Algebra References: Simon and Blume, chapters 6 to 11, 16 and 23; Pemberton and Rau, chapters 11 to 13 and 25; Sundaram, sections 1.3 and 1.5. The methods and concepts of linear algebra

More information

Basic Concepts in Linear Algebra

Basic Concepts in Linear Algebra Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University February 2, 2015 Grady B Wright Linear Algebra Basics February 2, 2015 1 / 39 Numerical Linear Algebra Linear

More information

CS 143 Linear Algebra Review

CS 143 Linear Algebra Review CS 143 Linear Algebra Review Stefan Roth September 29, 2003 Introductory Remarks This review does not aim at mathematical rigor very much, but instead at ease of understanding and conciseness. Please see

More information

MAT 610: Numerical Linear Algebra. James V. Lambers

MAT 610: Numerical Linear Algebra. James V. Lambers MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic

More information

Linear Algebra Review and Reference

Linear Algebra Review and Reference Linear Algebra Review and Reference Zico Kolter (updated by Chuong Do) October 7, 2008 Contents 1 Basic Concepts and Notation 2 11 BasicNotation 2 2 Matrix Multiplication 3 21 Vector-VectorProducts 4 22

More information

Introduction. Vectors and Matrices. Vectors [1] Vectors [2]

Introduction. Vectors and Matrices. Vectors [1] Vectors [2] Introduction Vectors and Matrices Dr. TGI Fernando 1 2 Data is frequently arranged in arrays, that is, sets whose elements are indexed by one or more subscripts. Vector - one dimensional array Matrix -

More information

Chapter 1. Matrix Algebra

Chapter 1. Matrix Algebra ST4233, Linear Models, Semester 1 2008-2009 Chapter 1. Matrix Algebra 1 Matrix and vector notation Definition 1.1 A matrix is a rectangular or square array of numbers of variables. We use uppercase boldface

More information

4 Linear Algebra Review

4 Linear Algebra Review Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the text 1 Vectors and Matrices For the context of data analysis, the

More information

Recall the convention that, for us, all vectors are column vectors.

Recall the convention that, for us, all vectors are column vectors. Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

More information

Linear Algebra: Matrix Eigenvalue Problems

Linear Algebra: Matrix Eigenvalue Problems CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given

More information

Review of Basic Concepts in Linear Algebra

Review of Basic Concepts in Linear Algebra Review of Basic Concepts in Linear Algebra Grady B Wright Department of Mathematics Boise State University September 7, 2017 Math 565 Linear Algebra Review September 7, 2017 1 / 40 Numerical Linear Algebra

More information

Linear Algebra (Review) Volker Tresp 2018

Linear Algebra (Review) Volker Tresp 2018 Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c

More information

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form

Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1

More information

Properties of Matrices and Operations on Matrices

Properties of Matrices and Operations on Matrices Properties of Matrices and Operations on Matrices A common data structure for statistical analysis is a rectangular array or matris. Rows represent individual observational units, or just observations,

More information

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v ) Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

More information

Basic Concepts in Matrix Algebra

Basic Concepts in Matrix Algebra Basic Concepts in Matrix Algebra An column array of p elements is called a vector of dimension p and is written as x p 1 = x 1 x 2. x p. The transpose of the column vector x p 1 is row vector x = [x 1

More information

Matrices. Chapter Definitions and Notations

Matrices. Chapter Definitions and Notations Chapter 3 Matrices 3. Definitions and Notations Matrices are yet another mathematical object. Learning about matrices means learning what they are, how they are represented, the types of operations which

More information

Foundations of Matrix Analysis

Foundations of Matrix Analysis 1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the

More information

and let s calculate the image of some vectors under the transformation T.

and let s calculate the image of some vectors under the transformation T. Chapter 5 Eigenvalues and Eigenvectors 5. Eigenvalues and Eigenvectors Let T : R n R n be a linear transformation. Then T can be represented by a matrix (the standard matrix), and we can write T ( v) =

More information

Math 360 Linear Algebra Fall Class Notes. a a a a a a. a a a

Math 360 Linear Algebra Fall Class Notes. a a a a a a. a a a Math 360 Linear Algebra Fall 2008 9-10-08 Class Notes Matrices As we have already seen, a matrix is a rectangular array of numbers. If a matrix A has m columns and n rows, we say that its dimensions are

More information

Massachusetts Institute of Technology Department of Economics Statistics. Lecture Notes on Matrix Algebra

Massachusetts Institute of Technology Department of Economics Statistics. Lecture Notes on Matrix Algebra Massachusetts Institute of Technology Department of Economics 14.381 Statistics Guido Kuersteiner Lecture Notes on Matrix Algebra These lecture notes summarize some basic results on matrix algebra used

More information

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations

Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2

More information

Introduction Eigen Values and Eigen Vectors An Application Matrix Calculus Optimal Portfolio. Portfolios. Christopher Ting.

Introduction Eigen Values and Eigen Vectors An Application Matrix Calculus Optimal Portfolio. Portfolios. Christopher Ting. Portfolios Christopher Ting Christopher Ting http://www.mysmu.edu/faculty/christophert/ : christopherting@smu.edu.sg : 6828 0364 : LKCSB 5036 November 4, 2016 Christopher Ting QF 101 Week 12 November 4,

More information

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures

More information

Chapter 3. Matrices. 3.1 Matrices

Chapter 3. Matrices. 3.1 Matrices 40 Chapter 3 Matrices 3.1 Matrices Definition 3.1 Matrix) A matrix A is a rectangular array of m n real numbers {a ij } written as a 11 a 12 a 1n a 21 a 22 a 2n A =.... a m1 a m2 a mn The array has m rows

More information

EE731 Lecture Notes: Matrix Computations for Signal Processing

EE731 Lecture Notes: Matrix Computations for Signal Processing EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten

More information

[3] (b) Find a reduced row-echelon matrix row-equivalent to ,1 2 2

[3] (b) Find a reduced row-echelon matrix row-equivalent to ,1 2 2 MATH Key for sample nal exam, August 998 []. (a) Dene the term \reduced row-echelon matrix". A matrix is reduced row-echelon if the following conditions are satised. every zero row lies below every nonzero

More information

Linear Algebra. Session 12

Linear Algebra. Session 12 Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

More information

Linear Algebra March 16, 2019

Linear Algebra March 16, 2019 Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented

More information

Linear Algebra and Matrices

Linear Algebra and Matrices Linear Algebra and Matrices 4 Overview In this chapter we studying true matrix operations, not element operations as was done in earlier chapters. Working with MAT- LAB functions should now be fairly routine.

More information

Linear Algebra. Christos Michalopoulos. September 24, NTU, Department of Economics

Linear Algebra. Christos Michalopoulos. September 24, NTU, Department of Economics Linear Algebra Christos Michalopoulos NTU, Department of Economics September 24, 2011 Christos Michalopoulos Linear Algebra September 24, 2011 1 / 93 Linear Equations Denition A linear equation in n-variables

More information

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017

Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017 Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...

More information

Final Review Written by Victoria Kala SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015

Final Review Written by Victoria Kala SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015 Final Review Written by Victoria Kala vtkala@mathucsbedu SH 6432u Office Hours R 12:30 1:30pm Last Updated 11/30/2015 Summary This review contains notes on sections 44 47, 51 53, 61, 62, 65 For your final,

More information

Lecture 2: Linear operators

Lecture 2: Linear operators Lecture 2: Linear operators Rajat Mittal IIT Kanpur The mathematical formulation of Quantum computing requires vector spaces and linear operators So, we need to be comfortable with linear algebra to study

More information

Basic Elements of Linear Algebra

Basic Elements of Linear Algebra A Basic Review of Linear Algebra Nick West nickwest@stanfordedu September 16, 2010 Part I Basic Elements of Linear Algebra Although the subject of linear algebra is much broader than just vectors and matrices,

More information

Homework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I)

Homework 1 Elena Davidson (B) (C) (D) (E) (F) (G) (H) (I) CS 106 Spring 2004 Homework 1 Elena Davidson 8 April 2004 Problem 1.1 Let B be a 4 4 matrix to which we apply the following operations: 1. double column 1, 2. halve row 3, 3. add row 3 to row 1, 4. interchange

More information

Study Notes on Matrices & Determinants for GATE 2017

Study Notes on Matrices & Determinants for GATE 2017 Study Notes on Matrices & Determinants for GATE 2017 Matrices and Determinates are undoubtedly one of the most scoring and high yielding topics in GATE. At least 3-4 questions are always anticipated from

More information

Linear Algebra and Matrix Inversion

Linear Algebra and Matrix Inversion Jim Lambers MAT 46/56 Spring Semester 29- Lecture 2 Notes These notes correspond to Section 63 in the text Linear Algebra and Matrix Inversion Vector Spaces and Linear Transformations Matrices are much

More information

Matrix Algebra: Summary

Matrix Algebra: Summary May, 27 Appendix E Matrix Algebra: Summary ontents E. Vectors and Matrtices.......................... 2 E.. Notation.................................. 2 E..2 Special Types of Vectors.........................

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections ) c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.2-7.4) 1. When each of the functions F 1, F 2,..., F n in right-hand side

More information

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP) MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

More information

Linear Algebra Review

Linear Algebra Review Linear Algebra Review Contents 1 Basic Concepts and Notations 2 2 Matrix Operations and Properties 3 21 Matrix Multiplication 3 211 Vector-Vector Products 3 212 Matrix-Vector Products 4 213 Matrix-Matrix

More information

Lecture 2: Linear Algebra Review

Lecture 2: Linear Algebra Review EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1

More information

3 Matrix Algebra. 3.1 Operations on matrices

3 Matrix Algebra. 3.1 Operations on matrices 3 Matrix Algebra A matrix is a rectangular array of numbers; it is of size m n if it has m rows and n columns. A 1 n matrix is a row vector; an m 1 matrix is a column vector. For example: 1 5 3 5 3 5 8

More information

Notes on Linear Algebra and Matrix Theory

Notes on Linear Algebra and Matrix Theory Massimo Franceschet featuring Enrico Bozzo Scalar product The scalar product (a.k.a. dot product or inner product) of two real vectors x = (x 1,..., x n ) and y = (y 1,..., y n ) is not a vector but a

More information

Chapter 5 Eigenvalues and Eigenvectors

Chapter 5 Eigenvalues and Eigenvectors Chapter 5 Eigenvalues and Eigenvectors Outline 5.1 Eigenvalues and Eigenvectors 5.2 Diagonalization 5.3 Complex Vector Spaces 2 5.1 Eigenvalues and Eigenvectors Eigenvalue and Eigenvector If A is a n n

More information