# Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT

Save this PDF as:
Size: px
Start display at page:

Download "Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT"

## Transcription

1 Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014

2 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares

3 Systems of Linear Equations Definition A linear equation in the variables x 1,... x n is an equation that can be written in the form a 1 x 1 + a 2 x a n x n = b where b and the coefficients a 1,... a n are real or complex numbers. The subscript n may be any positive integer. Definition A system of linear equations (or a linear system) is a collection of one or more linear equations involving the same variables say, x 1,..., x n. Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

4 Properties of systems of linear equations Systems of linear equations fall into one of three camps A single solution (the two lines intersect at one point) No solution (the two lines are exactly parallel but are shifted by some constant) An infinite number of solutions Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

5 Graphical View of the Solutions x + y + z = 3 x + 2y 3z = 2 x + y + z = 5 Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

6 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares

7 Definition Properties of Vector An ordered list of number in R: v 1 v = ( ) v 2 v 1 v 2... v n, v =. Properties Vector Addition v n (u1 ) + v 1 u 2 + v 2 u k + v n Scalar Multiplication cv = ( ) cv 1 cv 2... cv n Vector Inner Product u v = u 1 v 1 + u 2 v u n v n = Vector Norm n u i v i i=1 v = v v = v 1 v 1 + v 2 v v n v n Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

8 Functions of Vectors C P G I V C = β 1 P + β 2 G + β 3 I + β 4 V Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

9 Functions of Vectors Definition (Linear combinations) The vector u is a linear combination of the vectors v 1, v 2,, v k if Definition (Linear independence) u = c 1 v 1 + c 2 v c k v k A set of vectors v 1, v 2,, v k is linearly independent if the only solution to the equation c 1 v 1 + c 2 v c k v k = 0 is c 1 = c 2 = = c k = 0. If another solution exists, the set of vectors is linearly dependent. Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

10 Systems of Linear Equations We might have a system of m equations with n unknowns a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2... a m1 x 1 + a m2 x a mn x n = b m A solution to a linear system of m equations in n unknowns is a set of n numbers x 1, x 2,, x n that satisfy each of the m equations. Can we rewrite it some way? Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

11 Elementary row (column) operations Definition Elementary row (column) operations on an m n matrix A includes the following. 1 Interchange rows (columns) r and s of A. 2 Multiply row (column) r of A by a nonzero scalar k 0. 3 Add k times row (column) r of A to row (column) s of A where r s. An m n matrix A is said to be row (column) equivalent to an m n matrix B if B can be obtained by applying a finite sequence of elementary row (column) operations to A. Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

12 Elementary equation operations Elementary equation operations transform the equations of a linear system while maintaining an equivalent linear system equivalent in that the same values of x j solve the original and transformed systems, i.e., transformed systems look different but have the same properties. These operations are: 1 Interchanging two equations, 2 Multiplying two sides of an equation by a constant, and 3 Adding equations to each other Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

13 Vector Space Definition A real vector space is a set of vectors together with rules for vector addition and for multiplication by real numbers It s a set of vectors There are rules Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

14 Subspace Definition A subspace of a vector space is a set of vectors (including 0) that satisfies two requirements: If V and W are vectors in the subspace and c is any scalar, then 1 V + W is in the subspace 2 cv is in the subspace All linear combinations stay in the subspace Subspace must go through the origin Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

15 Questions Is a identity matrix a subspace by itself? Is the union of a line and a plane (the line is not on the plane) a subspace? Is the intersection of two subspaces a subspace? Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

16 Column Space Suppose A is a m n matrix. Definition The column space consists of all linear combinations of the columns. The combinations are all possible vectors Ax. They fill the column space of A, denoted by C(A). The system Ax = b is solvable iff b is in C(A). C(A) is a subspace of R m (why C(A) is a subspace?) Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

17 Spanning Definition Subspace SS is a span of S if it contains all combinations of vectors in S V is a vector space, e.g., R 3 S is a set of vectors in V (probably not a subspace) SS is the span of S, a subspace of V Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

18 Null Space Suppose A is a m n matrix. Definition The null space of A, denoted by N(A), consists of all solutions to Ax = 0. N(A) is subspace of R n. Why? Remember, C(A) is a subspace of R m Any null space must include the origin Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

19 Reduced Row Echelon Form Definition A rectangular matrix is in echelon form (or row echelon form) if it has the following three properties: 1 All nonzero rows are above any rows of all zeros 2 Each leading entry of a row is in a column to the right of the leading entry of the row above it. 3 All entries in a column below a leading entry are zeros. If a matrix in echelon form satisfies the following additional conditions, then it is in reduced echelon form (or reduced row echelon form) 4 The leading entry in each nonzero row is 1. 5 Each leading 1 is the only nonzero entry in its column Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

20 Pivots Definition A pivot position in a matrix A is a location in A that corresponds to a leading 1 in the reduced echelon form of A. A pivot column is a column of A that contains a pivot position A = Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

21 Special Solutions Every free column of an echelon matrix leads to a special solution. The free variable equals 1 and the other free variables equal 0. Back substitution solves Ax = 0 The complete solution to Ax = 0 is a combination of the special solutions If n > m then A has at least one column without pivots, giving a special solution. So there are nonzero vectors x in N(A) Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

22 Rank The rank r of A is the number of pivots The r pivot columns are not combinations of earlier columns The n r free columns are combinations of the pivot columns Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

23 Augmented Matrices Instead of writing out all of the x terms of each matrix, we can write the augmented matrix which is just the matrix representation of Ax = b where x is a vector (x 1, x 2,, x n ). Often times we ll just work with the augmented matrix in RREF, [R d], e.g., Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

24 One Particular Solution 1 Choose the free variable(s) to be zero (e.g., x 3 = 0) 2 Nonzero equations give the values of the pivot variables (e.g., x 1 = 2, x 2 = 4) 3 The particular solution in the following case is ( 2, 4, 0) Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

25 The Complete Solution of Ax = b The Complete Solution of Ax = b is One particular solution Add the null space x p + x n Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

26 Rank and the Complete Solution of Ax = b Ax = b is solvable iff the last m r equations reduce to 0 = Full column rank r = n means no free variables: one solution or none Full row rank r = m means one solution if m = n or infinitely many if m < n [ ] Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

27 Full Column Rank Every matrix A with full column rank (r = n) has the following properties All columns of A are pivot columns No free variables or special solutions N(A) contains only the zero vector x = 0 Ax = b has zero or only one solution A few variables, many many equations [R d] = = Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

28 Full Row Rank Every matrix A with full column rank (r = m) has the following properties All row have pivots Ax = b has a solution for every right side b The column space is the whole space of R m There are n r = n m special solutions in N(A) A few equations, many many variables [ ] [R d] = Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

29 Four Cases Ranks RREF A Ax = b r = m = n [I ] Square and invertible 1 solution r = m < n [I F ] Short and wide solutions r = n < m r < m, r < n [ ] I 0 [ ] I F 0 0 Tall and thin Not full rank 0 or 1 solution 0 or solutions Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

30 Linear Independence and Spanning, Again Definition The columns of a matrix A are linearly independent when the only solutions to Ax = 0 is x = 0. No other combination x of the columns gives the zero vector. Definition A set of vectors spans a space if their linear combinations fill the space A = Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

31 Basis Definition A basis for a vector space is a sequence of vectors with two properties: The basis vectors are linearly independent They span the space The columns of every invertible n by n matrix give a basis for R n, why? Can you list one basis for the space of all 2 by 2 matrix? Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

32 Dimensions Definition The dimension of a space is the number of vectors in very basis The dimension of the whole n by n matrix space is n 2 The dimension of the subspace of upper triangular matrices is (n 2 + n)/2 The dimension of the subspace of diagonal matrices is n The dimension of the subspace of symmetric matrices is (n 2 + n)/2 Dimension of column space + dimension of null space = dimension of R n, why? Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

33 In Summary 1 Independent vectors: no extra vectors 2 Spanning a space: enough vectors to produce the rest 3 Basis for a space: not too many, not too few 4 Dimension of a space: the number of vectors in a basis Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

34 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares

35 Matrix A matrix is an array of mn real numbers arranged in m rows by n columns. a 11 a 12 a 1n a 21 a 22 a 2n A = a m1 a m2 a mn Vectors are special cases of matrices; a column vector of length k is a k 1 matrix, while a row vector of the same length is a 1 k matrix. You can also think of larger matrices as being made up of a collection of row or column vectors. For example, A = ( a 1 a 2 a m ) Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

36 Matrix Operations Let A and B be m n matrices. 1 (equality) A = B if a ij = b ij. 2 (addition) C = A + B if c ij = a ij + b ij and C is an m n matrix. 3 (scalar multiplication) Given k R, C = ka if c ij = ka ij where C is an m n matrix. 4 (product) Let C be an n l matrix. D = AC if d ij = n k=1 a ikc kj and D is an m l matrix. 5 (transpose) C = A if c ij = a ji and C is an n m matrix. Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

37 Matrix Multiplication a c e b ( ) aa + bc d A B = ca + dc C D f ea + fc ab + bd cb + dd eb + fd The number of columns of the first matrix must equal the number of rows of the second matrix. If so they are conformable for multiplication. The sizes of the matrices (including the resulting product) must be (m k)(k n) = (m n) Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

38 Matrix Algebra Laws 1 Associative (A + B) + C = A + (B + C); (AB)C = A(BC) 2 Commutative A + B = B + A 3 Distributive A(B + C) = AB + AC; (A + B)C = AC + BC Note Commutative law for multiplication does not hold the order of multiplication matters: AB BA Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

39 Rules of Matrix Transpose 1 (A + B) T = A T + B T 2 (A T ) T = A 3 (sa) T = sa T 4 (AB) T = B T A T Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

40 Square Matrix Square matrices have the same number of rows and columns; a k k square matrix is referred to as a matrix of order k. Definition Let A be an m n matrix. 1 A is called a square matrix if n = m. 2 A is called symmetric if A = A. 3 A square matrix A is called a diagonal matrix if a ij = 0 for i j. A is called upper triangular if a ij = 0 for i > j and called lower triangular if a ij = 0 for i < j. 4 A diagonal matrix A is called an identity matrix if a ij = 1 for i = j and is denoted by I n. Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

41 Examples of the Square Matrix 1 Symmetric Matrix: A matrix A is symmetric if A = A T ; this implies that a ij = a ji for all i and j. ( ) A = = A T, B = = B Diagonal Matrix: A matrix A is diagonal if all of its non-diagonal entries are zero; formally, if a ij = 0 for all i j ( ) A =, B = Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

42 Examples of the Square Matrix 1 Triangular Matrix: A matrix is triangular one of two cases. If all entries below the diagonal are zero (a ij = 0 for all i > j), it is upper triangular. Conversely, if all entries above the diagonal are zero (a ij = 0 for all i < j), it is lower triangular A LT = 4 2 0, A UT = Identity Matrix: The n n identity matrix I n is the matrix whose diagonal elements are 1 and all off-diagonal elements are 0. Examples: ( ) I 2 =, I = Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

43 Trace Definition If A is an n n matrix, then the trace of A denoted by tr(a) is defined as the sum of all the main diagonal elements of A. That is, tr(a) = n i=1 a ii. Properties of the trace operator: If A and B are square matrices of order k, then 1 tr(a + B) = tr(a) + tr(b) 2 tr(a T ) = tr(a) 3 tr(sa) = str(a) 4 tr(ab) = tr(ba) Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

44 Matrix Inverse Definition Inverse Matrix: An n n matrix A is invertible if there exists an n n matrix A 1 such that AA 1 = A 1 A = I n A 1 is the inverse of A. If there is no such A 1, then A is noninvertible (another term for this is singular ). Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

45 Matrix Inverse Theorem (Uniqueness of Inverse) The inverse of a matrix, if it exists, is unique. We denote the unique inverse of A by A 1. Theorem (Properties of Inverse) Let A and B be nonsingular n n matrices. 1 AB is nonsingular and (AB) 1 = B 1 A 1. 2 A 1 is nonsingular and (A 1 ) 1 = A. 3 (A ) 1 = (A 1 ). Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

46 Shortcut for 2 2 It turns out that there is a shortcut for 2x2 matrices that are invertible: [ ] [ ] a b A =, then A 1 1 d b = c d ad bc c a Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

47 Solving Linear Systems Matrix representation of a linear system Ax = b If A is an n n matrix,then Ax = b is a system of n equations in n unknowns. If A is invertible then A 1 exists. To solve this system, we can left multiply each side by A 1. A 1 (Ax) = A 1 b (A 1 A)x = A 1 b I n x x = A 1 b = A 1 b Hence, given A and b and given that A is nonsingular, then x = A 1 b is a unique solution to this system. Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

48 Determinants If the determinant is non-zero then the matrix is invertible. For a 1 1 matrix A = a the determinant is simply a. We want the determinant to equal zero when the inverse does not exist. Since the inverse of a, 1/a, does not exist when a = 0. a = a ( ) a11 a For a 2 2 matrix A = 12, the determinant is defined as a 21 a 22 a 11 a 12 a 21 a 22 = det A = a 11a 22 a 12 a 21 == a 11 a 22 a 12 a 21 Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

49 Properties of determinants 1 A = A T 2 Interchanging rows changes the sign of the determinant 3 If two rows of A are exactly the same, then A = 0 4 If a row of A has all 0 then A = 0 5 If c 0 is some constant then A = ca 6 A square matrix is nonsingular iff its determinant is 0 7 AB = A B Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

50 Bringing It Together Theorem Let A be an n n matrix. Then the following statements are equivalent to each other. A is an invertible matrix. A ir row equivalent to the n n identity matrix. The equation Ax = 0 has only the trivial solution. The equation Ax = b has at least one solution for each b in R n. There is an n n matrix C such that AC=I. There is an n n matrix D such that DA=I. A T is an invertible matrix. rank(a)=n. det(a) 0 Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

51 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares

52 Geometrics of Ordinary Least Square One way is to think about the OLS is to find (or approximate) solutions to Ax = b, or X b = y X is a n by k matrix When n > k, a solution does not exist Two ways to think about it 1 Find a vector ˆb such that X ˆb is closest to y 2 Project y R n onto subspace C(X ), a k-dimensional subspace in R n x 11 x 12 x 1k b 1 y 1 x 21 x 22 x 2k b = y 2. x n1 x n2 x nk b k y k Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

53 First Interpretation Find a vector ˆb such that X ˆb is closest to y X = [1 income ethnic] X ˆb is on a plane (why?) 3D e Income EthnicHeterogeneity Democracy Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

54 Projection If is not super intuitive because humans do not have intuitions for high-dimensional spaces We denote the projection of a vector as p = Pb, P is called the projection matrix b p is orthogonal to p Definition Vectors V and W are orthogonal if V W = 0 Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

55 Projection onto a Subspace Problem: Given x 1, x 2,, x k are n 1 vectors (k variables), find the combination ˆb 1 x 1 + ˆb 1 x ˆb k x k closest to a given vector y We know that the error vector y X ˆb is orthogonal to C(X ) x 1(y X ˆb) = 0. x k (y X ˆb) = 0 or X (y X ˆb) = 0 ˆb = (X X ) 1 X y P = X (X X ) 1 X Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

56 Residual Maker and OLS Mechanics The residual maker: M = I P = I X (X X ) 1 X M is both symmetric and idempotent M 2 = M (why?) MX = 0 (why?) ŷ and residual e are orthogonal (why?) y = Py + My y y = y P Py + y M My = ŷ ŷ + e e e e = e y = y e (why?) y P Py = ˆb X X ˆb = ˆb X y = y X ˆb (why?) Yiqing Xu (MIT) Linear Algebra Aug 26, / 51

### Matrix & Linear Algebra

Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84 Vectors Vectors Vector: A

### Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

### Matrix Algebra. Matrix Algebra. Chapter 8 - S&B

Chapter 8 - S&B Algebraic operations Matrix: The size of a matrix is indicated by the number of its rows and the number of its columns. A matrix with k rows and n columns is called a k n matrix. The number

### Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in

Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column

### Elementary maths for GMT

Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

### Math Linear Algebra Final Exam Review Sheet

Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

### 2. Every linear system with the same number of equations as unknowns has a unique solution.

1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations

### Chapter 1: Systems of linear equations and matrices. Section 1.1: Introduction to systems of linear equations

Chapter 1: Systems of linear equations and matrices Section 1.1: Introduction to systems of linear equations Definition: A linear equation in n variables can be expressed in the form a 1 x 1 + a 2 x 2

### Linear Algebra March 16, 2019

Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented

### MATH 2331 Linear Algebra. Section 2.1 Matrix Operations. Definition: A : m n, B : n p. Example: Compute AB, if possible.

MATH 2331 Linear Algebra Section 2.1 Matrix Operations Definition: A : m n, B : n p ( 1 2 p ) ( 1 2 p ) AB = A b b b = Ab Ab Ab Example: Compute AB, if possible. 1 Row-column rule: i-j-th entry of AB:

### Elementary Row Operations on Matrices

King Saud University September 17, 018 Table of contents 1 Definition A real matrix is a rectangular array whose entries are real numbers. These numbers are organized on rows and columns. An m n matrix

### Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

### MTH501- Linear Algebra MCQS MIDTERM EXAMINATION ~ LIBRIANSMINE ~

MTH501- Linear Algebra MCQS MIDTERM EXAMINATION ~ LIBRIANSMINE ~ Question No: 1 (Marks: 1) If for a linear transformation the equation T(x) =0 has only the trivial solution then T is One-to-one Onto Question

### MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS. + + x 1 x 2. x n 8 (4) 3 4 2

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS SYSTEMS OF EQUATIONS AND MATRICES Representation of a linear system The general system of m equations in n unknowns can be written a x + a 2 x 2 + + a n x n b a

### MATH2210 Notebook 2 Spring 2018

MATH2210 Notebook 2 Spring 2018 prepared by Professor Jenny Baglivo c Copyright 2009 2018 by Jenny A. Baglivo. All Rights Reserved. 2 MATH2210 Notebook 2 3 2.1 Matrices and Their Operations................................

### MATH 2331 Linear Algebra. Section 1.1 Systems of Linear Equations. Finding the solution to a set of two equations in two variables: Example 1: Solve:

MATH 2331 Linear Algebra Section 1.1 Systems of Linear Equations Finding the solution to a set of two equations in two variables: Example 1: Solve: x x = 3 1 2 2x + 4x = 12 1 2 Geometric meaning: Do these

### Phys 201. Matrices and Determinants

Phys 201 Matrices and Determinants 1 1.1 Matrices 1.2 Operations of matrices 1.3 Types of matrices 1.4 Properties of matrices 1.5 Determinants 1.6 Inverse of a 3 3 matrix 2 1.1 Matrices A 2 3 7 =! " 1

### MATH 213 Linear Algebra and ODEs Spring 2015 Study Sheet for Midterm Exam. Topics

MATH 213 Linear Algebra and ODEs Spring 2015 Study Sheet for Midterm Exam This study sheet will not be allowed during the test Books and notes will not be allowed during the test Calculators and cell phones

### Finite Mathematics Chapter 2. where a, b, c, d, h, and k are real numbers and neither a and b nor c and d are both zero.

Finite Mathematics Chapter 2 Section 2.1 Systems of Linear Equations: An Introduction Systems of Equations Recall that a system of two linear equations in two variables may be written in the general form

### Review of Linear Algebra

Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

### Chapter 5. Linear Algebra. A linear (algebraic) equation in. unknowns, x 1, x 2,..., x n, is. an equation of the form

Chapter 5. Linear Algebra A linear (algebraic) equation in n unknowns, x 1, x 2,..., x n, is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b where a 1, a 2,..., a n and b are real numbers. 1

### A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

### Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB

Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the

### Problem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show

MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,

### A matrix is a rectangular array of. objects arranged in rows and columns. The objects are called the entries. is called the size of the matrix, and

Section 5.5. Matrices and Vectors A matrix is a rectangular array of objects arranged in rows and columns. The objects are called the entries. A matrix with m rows and n columns is called an m n matrix.

### POLI270 - Linear Algebra

POLI7 - Linear Algebra Septemer 8th Basics a x + a x +... + a n x n b () is the linear form where a, b are parameters and x n are variables. For a given equation such as x +x you only need a variable and

### Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.

Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the

### 2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian

FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian

### February 20 Math 3260 sec. 56 Spring 2018

February 20 Math 3260 sec. 56 Spring 2018 Section 2.2: Inverse of a Matrix Consider the scalar equation ax = b. Provided a 0, we can solve this explicity x = a 1 b where a 1 is the unique number such that

### Math 313 Chapter 1 Review

Math 313 Chapter 1 Review Howard Anton, 9th Edition May 2010 Do NOT write on me! Contents 1 1.1 Introduction to Systems of Linear Equations 2 2 1.2 Gaussian Elimination 3 3 1.3 Matrices and Matrix Operations

### 3.4 Elementary Matrices and Matrix Inverse

Math 220: Summer 2015 3.4 Elementary Matrices and Matrix Inverse A n n elementary matrix is a matrix which is obtained from the n n identity matrix I n n by a single elementary row operation. Elementary

### MAT 2037 LINEAR ALGEBRA I web:

MAT 237 LINEAR ALGEBRA I 2625 Dokuz Eylül University, Faculty of Science, Department of Mathematics web: Instructor: Engin Mermut http://kisideuedutr/enginmermut/ HOMEWORK 2 MATRIX ALGEBRA Textbook: Linear

### Linear Algebra. The analysis of many models in the social sciences reduces to the study of systems of equations.

POLI 7 - Mathematical and Statistical Foundations Prof S Saiegh Fall Lecture Notes - Class 4 October 4, Linear Algebra The analysis of many models in the social sciences reduces to the study of systems

### MATH 1210 Assignment 4 Solutions 16R-T1

MATH 1210 Assignment 4 Solutions 16R-T1 Attempt all questions and show all your work. Due November 13, 2015. 1. Prove using mathematical induction that for any n 2, and collection of n m m matrices A 1,

### Chapter 7. Linear Algebra: Matrices, Vectors,

Chapter 7. Linear Algebra: Matrices, Vectors, Determinants. Linear Systems Linear algebra includes the theory and application of linear systems of equations, linear transformations, and eigenvalue problems.

### Applied Matrix Algebra Lecture Notes Section 2.2. Gerald Höhn Department of Mathematics, Kansas State University

Applied Matrix Algebra Lecture Notes Section 22 Gerald Höhn Department of Mathematics, Kansas State University September, 216 Chapter 2 Matrices 22 Inverses Let (S) a 11 x 1 + a 12 x 2 + +a 1n x n = b

### Introduction to Matrices

POLS 704 Introduction to Matrices Introduction to Matrices. The Cast of Characters A matrix is a rectangular array (i.e., a table) of numbers. For example, 2 3 X 4 5 6 (4 3) 7 8 9 0 0 0 Thismatrix,with4rowsand3columns,isoforder

### CSL361 Problem set 4: Basic linear algebra

CSL361 Problem set 4: Basic linear algebra February 21, 2017 [Note:] If the numerical matrix computations turn out to be tedious, you may use the function rref in Matlab. 1 Row-reduced echelon matrices

### We could express the left side as a sum of vectors and obtain the Vector Form of a Linear System: a 12 a x n. a m2

Week 22 Equations, Matrices and Transformations Coefficient Matrix and Vector Forms of a Linear System Suppose we have a system of m linear equations in n unknowns a 11 x 1 + a 12 x 2 + + a 1n x n b 1

### MAT Linear Algebra Collection of sample exams

MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### Introduction. Vectors and Matrices. Vectors [1] Vectors [2]

Introduction Vectors and Matrices Dr. TGI Fernando 1 2 Data is frequently arranged in arrays, that is, sets whose elements are indexed by one or more subscripts. Vector - one dimensional array Matrix -

### Math 4A Notes. Written by Victoria Kala Last updated June 11, 2017

Math 4A Notes Written by Victoria Kala vtkala@math.ucsb.edu Last updated June 11, 2017 Systems of Linear Equations A linear equation is an equation that can be written in the form a 1 x 1 + a 2 x 2 +...

### Matrix operations Linear Algebra with Computer Science Application

Linear Algebra with Computer Science Application February 14, 2018 1 Matrix operations 11 Matrix operations If A is an m n matrix that is, a matrix with m rows and n columns then the scalar entry in the

### Lecture 6: Geometry of OLS Estimation of Linear Regession

Lecture 6: Geometry of OLS Estimation of Linear Regession Xuexin Wang WISE Oct 2013 1 / 22 Matrix Algebra An n m matrix A is a rectangular array that consists of nm elements arranged in n rows and m columns

### Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.

Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read

### Linear Algebra Highlights

Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to

### Matrices Gaussian elimination Determinants. Graphics 2009/2010, period 1. Lecture 4: matrices

Graphics 2009/2010, period 1 Lecture 4 Matrices m n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in

### Chapter 2 Notes, Linear Algebra 5e Lay

Contents.1 Operations with Matrices..................................1.1 Addition and Subtraction.............................1. Multiplication by a scalar............................ 3.1.3 Multiplication

### MATH 315 Linear Algebra Homework #1 Assigned: August 20, 2018

Homework #1 Assigned: August 20, 2018 Review the following subjects involving systems of equations and matrices from Calculus II. Linear systems of equations Converting systems to matrix form Pivot entry

### IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

### MAC Module 2 Systems of Linear Equations and Matrices II. Learning Objectives. Upon completing this module, you should be able to :

MAC 0 Module Systems of Linear Equations and Matrices II Learning Objectives Upon completing this module, you should be able to :. Find the inverse of a square matrix.. Determine whether a matrix is invertible..

### ECON 186 Class Notes: Linear Algebra

ECON 86 Class Notes: Linear Algebra Jijian Fan Jijian Fan ECON 86 / 27 Singularity and Rank As discussed previously, squareness is a necessary condition for a matrix to be nonsingular (have an inverse).

### . a m1 a mn. a 1 a 2 a = a n

Biostat 140655, 2008: Matrix Algebra Review 1 Definition: An m n matrix, A m n, is a rectangular array of real numbers with m rows and n columns Element in the i th row and the j th column is denoted by

### MATH 106 LINEAR ALGEBRA LECTURE NOTES

MATH 6 LINEAR ALGEBRA LECTURE NOTES FALL - These Lecture Notes are not in a final form being still subject of improvement Contents Systems of linear equations and matrices 5 Introduction to systems of

### Matrices and Matrix Algebra.

Matrices and Matrix Algebra 3.1. Operations on Matrices Matrix Notation and Terminology Matrix: a rectangular array of numbers, called entries. A matrix with m rows and n columns m n A n n matrix : a square

### MATH 152 Exam 1-Solutions 135 pts. Write your answers on separate paper. You do not need to copy the questions. Show your work!!!

MATH Exam -Solutions pts Write your answers on separate paper. You do not need to copy the questions. Show your work!!!. ( pts) Find the reduced row echelon form of the matrix Solution : 4 4 6 4 4 R R

### NOTES on LINEAR ALGEBRA 1

School of Economics, Management and Statistics University of Bologna Academic Year 207/8 NOTES on LINEAR ALGEBRA for the students of Stats and Maths This is a modified version of the notes by Prof Laura

### MTH 464: Computational Linear Algebra

MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University February 6, 2018 Linear Algebra (MTH

### Knowledge Discovery and Data Mining 1 (VO) ( )

Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory

### Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p

Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the

### n n matrices The system of m linear equations in n variables x 1, x 2,..., x n can be written as a matrix equation by Ax = b, or in full

n n matrices Matrices Definitions Diagonal, Identity, and zero matrices Addition Multiplication Transpose and inverse The system of m linear equations in n variables x 1, x 2,..., x n a 11 x 1 + a 12 x

### Matrices and systems of linear equations

Matrices and systems of linear equations Samy Tindel Purdue University Differential equations and linear algebra - MA 262 Taken from Differential equations and linear algebra by Goode and Annin Samy T.

### Matrices A brief introduction

Matrices A brief introduction Basilio Bona DAUIN Politecnico di Torino Semester 1, 2014-15 B. Bona (DAUIN) Matrices Semester 1, 2014-15 1 / 41 Definitions Definition A matrix is a set of N real or complex

### CHAPTER 6. Direct Methods for Solving Linear Systems

CHAPTER 6 Direct Methods for Solving Linear Systems. Introduction A direct method for approximating the solution of a system of n linear equations in n unknowns is one that gives the exact solution to

### Linear Algebra. James Je Heon Kim

Linear lgebra James Je Heon Kim (jjk9columbia.edu) If you are unfamiliar with linear or matrix algebra, you will nd that it is very di erent from basic algebra or calculus. For the duration of this session,

### 1 Last time: determinants

1 Last time: determinants Let n be a positive integer If A is an n n matrix, then its determinant is the number det A = Π(X, A)( 1) inv(x) X S n where S n is the set of n n permutation matrices Π(X, A)

### Review Let A, B, and C be matrices of the same size, and let r and s be scalars. Then

1 Sec 21 Matrix Operations Review Let A, B, and C be matrices of the same size, and let r and s be scalars Then (i) A + B = B + A (iv) r(a + B) = ra + rb (ii) (A + B) + C = A + (B + C) (v) (r + s)a = ra

### Linear Algebra V = T = ( 4 3 ).

Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional

### Matrix Algebra for Engineers Jeffrey R. Chasnov

Matrix Algebra for Engineers Jeffrey R. Chasnov The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Department of Mathematics Clear Water Bay, Kowloon

### Chapter 2: Matrix Algebra

Chapter 2: Matrix Algebra (Last Updated: October 12, 2016) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). Write A = 1. Matrix operations [a 1 a n. Then entry

### Linear Algebra Review. Vectors

Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors

### INSTITIÚID TEICNEOLAÍOCHTA CHEATHARLACH INSTITUTE OF TECHNOLOGY CARLOW MATRICES

1 CHAPTER 4 MATRICES 1 INSTITIÚID TEICNEOLAÍOCHTA CHEATHARLACH INSTITUTE OF TECHNOLOGY CARLOW MATRICES 1 Matrices Matrices are of fundamental importance in 2-dimensional and 3-dimensional graphics programming

### 1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det

What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix

### Kevin James. MTHSC 3110 Section 2.2 Inverses of Matrices

MTHSC 3110 Section 2.2 Inverses of Matrices Definition Suppose that T : R n R m is linear. We will say that T is invertible if for every b R m there is exactly one x R n so that T ( x) = b. Note If T is

### Linear Algebra Homework and Study Guide

Linear Algebra Homework and Study Guide Phil R. Smith, Ph.D. February 28, 20 Homework Problem Sets Organized by Learning Outcomes Test I: Systems of Linear Equations; Matrices Lesson. Give examples of

### LECTURES 14/15: LINEAR INDEPENDENCE AND BASES

LECTURES 14/15: LINEAR INDEPENDENCE AND BASES MA1111: LINEAR ALGEBRA I, MICHAELMAS 2016 1. Linear Independence We have seen in examples of span sets of vectors that sometimes adding additional vectors

### 1 Multiply Eq. E i by λ 0: (λe i ) (E i ) 2 Multiply Eq. E j by λ and add to Eq. E i : (E i + λe j ) (E i )

Direct Methods for Linear Systems Chapter Direct Methods for Solving Linear Systems Per-Olof Persson persson@berkeleyedu Department of Mathematics University of California, Berkeley Math 18A Numerical

### Introduction to Matrix Algebra

Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

### Chapter 3. Vector spaces

Chapter 3. Vector spaces Lecture notes for MA1111 P. Karageorgis pete@maths.tcd.ie 1/22 Linear combinations Suppose that v 1,v 2,...,v n and v are vectors in R m. Definition 3.1 Linear combination We say

### Inverting Matrices. 1 Properties of Transpose. 2 Matrix Algebra. P. Danziger 3.2, 3.3

3., 3.3 Inverting Matrices P. Danziger 1 Properties of Transpose Transpose has higher precedence than multiplication and addition, so AB T A ( B T and A + B T A + ( B T As opposed to the bracketed expressions

### GEOMETRY OF MATRICES x 1

GEOMETRY OF MATRICES. SPACES OF VECTORS.. Definition of R n. The space R n consists of all column vectors with n components. The components are real numbers... Representation of Vectors in R n.... R. The

### MATH 2360 REVIEW PROBLEMS

MATH 2360 REVIEW PROBLEMS Problem 1: In (a) (d) below, either compute the matrix product or indicate why it does not exist: ( )( ) 1 2 2 1 (a) 0 1 1 2 ( ) 0 1 2 (b) 0 3 1 4 3 4 5 2 5 (c) 0 3 ) 1 4 ( 1

### Digital Workbook for GRA 6035 Mathematics

Eivind Eriksen Digital Workbook for GRA 6035 Mathematics November 10, 2014 BI Norwegian Business School Contents Part I Lectures in GRA6035 Mathematics 1 Linear Systems and Gaussian Elimination........................

### Vectors and matrices: matrices (Version 2) This is a very brief summary of my lecture notes.

Vectors and matrices: matrices (Version 2) This is a very brief summary of my lecture notes Matrices and linear equations A matrix is an m-by-n array of numbers A = a 11 a 12 a 13 a 1n a 21 a 22 a 23 a

### CS 246 Review of Linear Algebra 01/17/19

1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

### 1 Last time: inverses

MATH Linear algebra (Fall 8) Lecture 8 Last time: inverses The following all mean the same thing for a function f : X Y : f is invertible f is one-to-one and onto 3 For each b Y there is exactly one a

### Math 54 HW 4 solutions

Math 54 HW 4 solutions 2.2. Section 2.2 (a) False: Recall that performing a series of elementary row operations A is equivalent to multiplying A by a series of elementary matrices. Suppose that E,...,

### Linear Systems and Matrices

Department of Mathematics The Chinese University of Hong Kong 1 System of m linear equations in n unknowns (linear system) a 11 x 1 + a 12 x 2 + + a 1n x n = b 1 a 21 x 1 + a 22 x 2 + + a 2n x n = b 2.......

### Linear Algebra: Lecture notes from Kolman and Hill 9th edition.

Linear Algebra: Lecture notes from Kolman and Hill 9th edition Taylan Şengül March 20, 2019 Please let me know of any mistakes in these notes Contents Week 1 1 11 Systems of Linear Equations 1 12 Matrices

### A FIRST COURSE IN LINEAR ALGEBRA. An Open Text by Ken Kuttler. Matrix Arithmetic

A FIRST COURSE IN LINEAR ALGEBRA An Open Text by Ken Kuttler Matrix Arithmetic Lecture Notes by Karen Seyffarth Adapted by LYRYX SERVICE COURSE SOLUTION Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)

### Systems of Linear Equations. By: Tri Atmojo Kusmayadi and Mardiyana Mathematics Education Sebelas Maret University

Systems of Linear Equations By: Tri Atmojo Kusmayadi and Mardiyana Mathematics Education Sebelas Maret University Standard of Competency: Understanding the properties of systems of linear equations, matrices,

### MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

### 3. Vector spaces 3.1 Linear dependence and independence 3.2 Basis and dimension. 5. Extreme points and basic feasible solutions

A. LINEAR ALGEBRA. CONVEX SETS 1. Matrices and vectors 1.1 Matrix operations 1.2 The rank of a matrix 2. Systems of linear equations 2.1 Basic solutions 3. Vector spaces 3.1 Linear dependence and independence

### Linear Algebra Summary. Based on Linear Algebra and its applications by David C. Lay

Linear Algebra Summary Based on Linear Algebra and its applications by David C. Lay Preface The goal of this summary is to offer a complete overview of all theorems and definitions introduced in the chapters

### LINEAR ALGEBRA SUMMARY SHEET.

LINEAR ALGEBRA SUMMARY SHEET RADON ROSBOROUGH https://intuitiveexplanationscom/linear-algebra-summary-sheet/ This document is a concise collection of many of the important theorems of linear algebra, organized

### Chapter 2: Matrices and Linear Systems

Chapter 2: Matrices and Linear Systems Paul Pearson Outline Matrices Linear systems Row operations Inverses Determinants Matrices Definition An m n matrix A = (a ij ) is a rectangular array of real numbers

### ELEMENTARY LINEAR ALGEBRA

ELEMENTARY LINEAR ALGEBRA K R MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND First Printing, 99 Chapter LINEAR EQUATIONS Introduction to linear equations A linear equation in n unknowns x,