Mathematical Optimisation, Chpt 2: Linear Equations and inequalities

Size: px
Start display at page:

Download "Mathematical Optimisation, Chpt 2: Linear Equations and inequalities"

Transcription

1 Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson version: 12/02/18 Monday 5th February 2018 Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 1/40

2 Table of Contents 1 Introduction 2 Gauss-elimination [MO, 2.1] 3 Orthogonal projection, Least Squares, [MO, 2.2] 4 Linear Inequalities [MO, 2.4] 5 Integer Solutions of Linear Equations [MO, 2.3] Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 2/40

3 Organization and Material Lectures (hoor-colleges) and 3 exercise classes for motivation, geometric illustration, and proofs. Script Mathematical Optimization, cited as: [MO,?] Sheets of the course (on Blackboard) Mark based on written exam Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 3/40

4 Chapter 1: Real vector spaces A collection of facts from Analysis and Linear Algebra, self-instruction Chapter 2: Linear equations, linear inequalities Gauss elimination and application Gauss elimination provides constructive proofs of main theorems in matrix theory. Fourier-Motzkin elimination This method provides constructive proofs of the Farkas Lemma (which is strong LP duality in disguise). Least square approximation, Fourier approximation Integer solutions of linear equations (discrete optimization) Chapter 3: Linear programs and applications LP duality, sensitivity analysis, matrix games. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 4/40

5 Chapter 4: Convex analysis Properties of convex sets and convex functions. Applications in optimization Chapter 5: Unconstrained optimization optimality conditions algorithms: descent methods, Newton s method, Gauss-Newton method, Quasi-Newton methods, minimization of nondifferentiable functions. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 5/40

6 Chapter 2. Linear equations, inequalities We start with some definitions: Definitions in matrix theory M = (m ij ) is said to be lower triangular: if m ij = 0 for i < j, upper triangular: if m ij = 0 for i > j. P = (p ij ) R m m is a permutation matrix if p ij {0, 1} and each row and each column of P contains exactly one coefficient 1. Note that P T P = I, implying that P 1 = P T. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 6/40

7 The set of symmetric matrices in R n n is denoted by S n. Q S n, is called Positive Semidefinite (denoted Q O or Q PSD n ) if x T Qx 0 for all x R n, Positive Definite (denoted: Q O) if x T Qx > 0 for all x R n \ {0}, Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 7/40

8 Table of Contents 1 Introduction 2 Gauss-elimination [MO, 2.1] Gauss-elimination (for solving Ax = b) Explicit Gauss Algorithm Implications of the Gauss algorithm Gauss-Algorithm for symmmetric A 3 Orthogonal projection, Least Squares, [MO, 2.2] 4 Linear Inequalities [MO, 2.4] 5 Integer Solutions of Linear Equations [MO, 2.3] Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 8/40

9 Gauss-elimination (for solving Ax = b) In this Section 2.1 We shortly survey the Gauss algorithm and use the Gauss algorithm to give a constructive proof of important theorems in Matrix Theory Motivation: We give a simple example which shows that successive elimination is equivalent with the Gauss algorithm. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 9/40

10 General Idea: To eliminate x 1, x 2... is equivalent with transforming Ax = b or (A b) to triangular normal form (Ã b) (with same solution set). Then solve Ãx = b, recursively: a 11 a a 1n b 1 a 21 a a 2n b 2.. a m1 a m2... a mn b m Transformation into form (Ã b): ã 1 j1 ã 1 j2 ã 1 jr 1 ã 1 jr b1 ã 2 j2 ã 2 jr 1 ã 2 jr b2.... ã r 1 jr 1 ã r 1 jr b r 1 ã r jr b r. b m Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 10/40

11 This Gauss elimination uses 2 types of row operations: (G1) (i, j)-pivot: k > i add λ k = a kj a ij times row i to row k. (G2) interchange row i with row k The matrix form of these operations are: [MO, Ex.2.3] The matrix form of (G1), (A b) (Ã b), is given by (Ã b) = M (A b) with a nonsingular lower triangular M R m m. [MO, Ex.2.4] The matrix form of (G2), (A b) (Ã b), is given by (Ã b) = P (A b) with a permutation matrix P R m m. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 11/40

12 Explicit Gauss Algorithm 1 Set i = 1 and j = 1. 2 While i m and j n do 3 If k i such that a kj 0 then 4 Interchange row i and row k. (G2) 5 Apply (i, j)-pivot. (G1) 6 Update i i + 1 and j j else 8 Update j j end if 10 end while Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 12/40

13 Implications of the Gauss algorithm Lemma 2.1 The set of lower triangular matrices is closed under addition, multiplication and inversion (provided square and nonsingular). Theorem 2.2 ([MO, Thm. 2.1]) For every A R m n, there exists an (m m)-permutation matrix P and an invertible lower triangular matrix M R m m such that U = MPA is upper triangular. Corollary 2.3 (Cor. 2.1, LU-factorization) For A R m n, there exists an (m m)-permutation matrix P, an invertible lower triangular L R m m and an upper triangular U R m n such that LU = PA. Rem.: Solve Ax = b by using the decomposition PA = LU! (How?) Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 13/40

14 Corollary 2.4 ([MO, Cor. 2.2], Gale s Theorem) Exactly one of the following statements is true for A R m n : (a) The system Ax = b has a solution x R n. (b) There exists y R m such that: y T A = 0 T and y T b 0. Remark: In normal form A Ã, the number r gives dimension of the space spanned by the rows of A. This equals the dimension of the space spanned by columns of A. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 14/40

15 Modified Gauss-Algorithm for symmmetric A Perform same row and column operators: 1 Set i = 1. 2 While i n do 3 If a kk 0 for some k i 4 Apply (G2 ): Interchange row i and row k. Interchange col. i and col. k, A PAP T. 5 Apply (G1 ): For all j > i add λ j = a ij a ii times row i to row j, and λ j times col. i to col. j, A MAM T. 6 Update i i Else if a ik 0 for some k > i then 8 Add row k to row i and add col. k to col. i, A BAB T. 9 Else 10 Update i i End if 12 End while. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 15/40

16 Implications of the symmetric Gauss algorithm Note: By symmetric Gauss the solution set of Ax = b is destroyed!!! But it is useful to get the following results. Theorem 2.5 ([MO, Thm. 2.2]) A R n n symmetric. Then with some nonsingular Q R n n QAQ T = D = diag(d 1,..., d n ), d R n. Look out: The d i s are in general not the eigenvalues of A. Recall: Q PSD n is positive semidefinite (notation: Q O) if: x T Qx 0 for all x R n. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 16/40

17 Corollary 2.6 ([MO, Cor. 2.3]) Let A S n, Q R n n nonsingular s.t. QAQ T = diag(d 1,..., d n ). Then (a) A O d i 0, i = 1,..., n (b) A O d i > 0, i = 1,..., n Implication: Checking A O can be done by the Gauss-algorithm. Corollary 2.7 ([MO, Cor. 2.4]) Let A S n. Then (a) A O A = BB T for some B R n m (b) A O A = BB T for some nonsingular B R n n Complexity of Gauss algorithm The number of ±,, / flop s (floating point operations) needed to solve Ax = b, where A R n n, is less than or equal to n 3. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 17/40

18 Table of Contents 1 Introduction 2 Gauss-elimination [MO, 2.1] 3 Orthogonal projection, Least Squares, [MO, 2.2] Projection and equivalent condition Constructing a solution Gram Matrix Gram-Schmidt Algorithm Eigenvalues 4 Linear Inequalities [MO, 2.4] 5 Integer Solutions of Linear Equations [MO, 2.3] Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 18/40

19 Projection and equivalent condition In this section: We see that the orthogonal projection problem is solved by solving a system of linear equations; and present some more results from matrix theory. Assumption: V is a linear vector space over R with inner product x, y and (induced) norm x = x, x. Minimization Problem: Given x V, subspace W V find x W such that: x x = min x y (1) y W The vector x is called the projection of x onto W. Lemma 2.8 ([MO, Lm. 2.1]) x W is the (unique) solution to (1) x x, w = 0 w W. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 19/40

20 Constructing a solution via Lm 2.8 Let a 1,..., a m be a basis of W, i.e., a 1,..., a m are linearly independent and W = span{a 1,..., a m }. Write x := m i=1 z ia i Then x x, w = 0, w W is equivalent with x m z ia i a j = 0, j = 1,..., m i=1 or m a i, a j z i = x, a j, j = 1,..., m i=1 Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 20/40

21 Gram Matrix Defining the Gram-matrix G = ( a i, a j ) i,j S m and considering b = ( x, a i ) i R m, this leads to the linear equation (for z): (2.16) Gz = b with solution ẑ = G 1 b Ex. 2.1 Prove that the Gram-matrix is positive definite, and thus non-singular (under our assumption). Summary Let W = span{a 1,..., a m }. Then the solution x of the minimization problem (1) is given by x := m i=1 ẑia i where ẑ is computed as the solution of G ẑ = b. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 21/40

22 Special case 1: V = R n, x, y = x T y and a 1,..., a m a basis of W. Then with A := [a 1,..., a m ] the projection of x onto W is given by ẑ = arg min{ x Az } = (A T A) 1 A T x, z x = arg min y { x y : y AR m } = Aẑ = A(A T A) 1 A T x. Special case 2: V = R n, x, y = x T y, a 1,..., a m R n lin. indep. and W = {w R n a T i w = 0, i = 1,..., m}. Then the projection of x onto W is given by x = arg min{ x y : a T y i y = 0 i} = x A(A T A) 1 A T x Special case 3: W = span{a 1,..., a m } with {a i }, an orthonormal basis, i.e., I = ( a i, a j ) i,j ). Then the projection of x onto W is given by ẑ = ( a j, x ) j R m, ( Fourier coefficients ) x = arg min{ x y : y W } = m ẑja j y j=1 Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 22/40

23 Gram-Schmidt Algorithm Given subspace W with basis a 1,..., a m, construct an orthonormal basis c 1,... c m for W, i.e. ( c i, c j ) i,j = I. Start with b 1 := a 1 and c 1 = b 1 / b 1. For k = 2,..., m let b k = a k k 1 i=1 c i, a k c i and c i = b i / b i. Gram-Schmidt in matrix form: With W V := R n. Put A = [a 1,..., a m ] T, B = [b 1,..., b m ] T, C = [c 1,..., c m ] T. Then the Gram-Schmidt-steps are equivalent with: add multiple of row j < k to row k (for B) multiply row k by scalar (for C) Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 23/40

24 Matrix form of Gram-Schmidt : Given A R m n with full row rank, there is a decomposition C = LA with lower triangular nonsingular matrix L (l ii = 1) and the rows c j of C are orthogonal, i.e. c i, c j = 0, i j. A corollary of this fact: Lemma 2.9 ([MO, Prop. 2.1]) Prop. 2.1 (Hadamard s inequality) Let A R m n with rows a T i. Then m 0 det (AA T ) a T i a i i=1 Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 24/40

25 Eigenvalues Definition. λ C is an eigenvalue of A R n n if there is an (eigenvector) 0 x C n with Ax = λx. The results above (together with the Theorem of Weierstrass) allow a proof of: Theorem 2.10 ([MO, Thm. 2.3], Spectral theorem for symmetric matrices) Let A S n. Then there exists an orthogonal matrix Q R n n (i.e. Q T Q = I) and eigenvalues λ 1,..., λ n R such that Q T AQ = D = diag (λ 1,..., λ n ) Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 25/40

26 Table of Contents 1 Introduction 2 Gauss-elimination [MO, 2.1] 3 Orthogonal projection, Least Squares, [MO, 2.2] 4 Linear Inequalities [MO, 2.4] Fourier-Motzkin Algorithm Solvability of linear systems Application: Markov chains 5 Integer Solutions of Linear Equations [MO, 2.3] Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 26/40

27 Linear Inequalities Ax b In this Section we learn: Finding a solution to Ax b. What the Gauss algorithm is for linear equations; is the Fourier-Motzkin algorithm for linear inequalities. The Fourier-Motzkin algorithm leads to a construcive proof of the Farkas Lemma (the basis for strong duality in linear programming). Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 27/40

28 Fourier-Motzkin algorithm for solving Ax b. Eliminate x 1 : a r1 x 1 + a s1 x 1 + n a rj x j b r j=2 n a sj x j b s j=2 n a tj x j b t j=2 r = 1,..., k s = k + 1,..., l t = l + 1,..., m with a r1 > 0, a s1 < 0. Divide by a r1, a s1, leading to (for r and s) x 1 + x 1 + n a rjx j b r j=2 n a sjx j b s j=2 r = 1,..., k s = k + 1,..., l Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 28/40

29 These two sets of inequalities have a solution x iff n a sjx j b s x 1 b r n j=2 j=2 a rjx j { r = 1,..., k s = k + 1,..., l or equivalently: n { (a sj + a rj)x j b s + b r = 1,..., k r s = k + 1,..., l j=2 Remark Explosion of number of inequalities. Before number = k + (l k) + (m l). Now the number = k (l k) + (m l) Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 29/40

30 So Ax b has a solution x = (x 1,.., x n ) if and only if there is a solution x = (x 2,.., x n ) of n (a sj + a rj)x j b r + b s r = 1,..., k; s = k , l j=2 n a tj x j b t t = l + 1,..., m. j=2 In matrix form: Ax b has a solution x = (x 1,.., x n ) if and only if there is a solution of the transformed system: A x b or ( 0 A )x b Remark: of (A b): Any row of (0 A b ) is a positive combination of rows any row is of the form y T (A b), y 0 Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 30/40

31 By eliminating x 1, x 2,..., x n, in this way we finally obtain an equivalent system à (n) x b where à (n) = 0 which is solvable iff 0 b i, i. Theorem 2.11 (Projection Theorem, [MO, Thm. 2.5]) Let P = {x R n Ax b}. Then all for k = 1,..., n, the projection P (k) = {(x k+1,.., x n ) (x 1,.., x k, x k+1,.., x n ) P is the solution set of a linear system in n k variables x (k) = (x k+1,..., x n ). for suitable x 1,..., x k R} A (k) x (k) b (k) In principle: Linear inequalities can be solved by FM. However this might be inefficient! (Why?) Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 31/40

32 Solvability of linear systems Theorem 2.12 (Farkas Lemma, [MO, 2.6]) Exactly one of the following statements is true: (I) Ax b has a solution x R n. (II) There exists y R m such that y T A = 0 T, y T b < 0 and y 0 Ex. 2.2 Let A R m n, C R k n, b R m, c R k. Then precisely one of the alternatives is valid. (I) There is a solution x of: Ax b, Cx = c (II) There is a solution µ R m, µ 0, λ R k of : ( A T b T ) µ + ( C T c T ) λ = ( ) 0 1 Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 32/40

33 Corollary 2.13 (Gordan, [MO, Cor. 2.5]) Given A R m n, exactly one of the following alternatives is true: (I) Ax = 0, x 0 has a solution x 0. (II) y T A < 0 T has a solution y. Remark: As we shall see in Chapter 3, the Farkas Lemma in the following form is the strong duality of LP in disguise. Corollary 2.14 (Farkas, implied inequalities, [MO, Cor. 2.6]) Let A R m n, b R m, c R n, z R. Assume that Ax b is feasible. Then the following are equivalent: (a) Ax b c T x z (b) y T A = c T, y T b z, y 0 has a solution y. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 33/40

34 Application: Markov chains (Existence of a steady state) Def. A vector π R n + with 1 T π := n i=1 π i = 1 is called a probability distribution on {1,.., n}. A matrix P = (p ij ) where each row P i is a probability distribution is called a stochastic matrix, i.e. P R+ n n and P1 = 1 In a stochastic process: (individuals in n possibly states) π i proportion of population is in state i p ij is probability of transition from state i j So the transition step k k + 1 is: π (k+1) = P T π (k) A probability distribution π is called steady state if As a corollary of Gordan s result: π = P T π Each stochastic matrix P has a steady state π. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 34/40

35 Table of Contents 1 Introduction 2 Gauss-elimination [MO, 2.1] 3 Orthogonal projection, Least Squares, [MO, 2.2] 4 Linear Inequalities [MO, 2.4] 5 Integer Solutions of Linear Equations [MO, 2.3] Two variables Lattices Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 35/40

36 Integer Solutions of Linear Equations Example Equation 3x 1 2x 2 = 1 has solution x = (1, 1) Z 2. But equation 6x 1 2x 2 = 1 does not allow an integer solution x. Key remark: Let a 1, a 2 Z and let a 1 x 1 + a 2 x 2 = b have a solution x 1, x 2 Z. Then b = λc with λ Z, c = gcd(a 1, a 2 ) Here: gcd(a 1, a 2 ) denotes the greatest common divisor of a 1, a 2. Lemma 2.15 (Euclid s Algorithm, [MO, Lm. 2.2]) Let c = gcd(a 1, a 2 ). Then L(a 1, a 2 ) := {a 1 λ 1 + a 2 λ 2 λ 1, λ 2 Z} = {cλ λ Z} =: L(c). (The proof of) this result allows to solve (if possible) a 1 x 1 + a 2 x 2 = b (in Z). Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 36/40

37 Algorithm to solve, a 1 x 1 + a 2 x 2 = b (in Z) Compute c = gcd(a 1, a 2 ). If λ := b/c / Z, no integer solution exists. If λ := b/c Z, compute solutions λ 1, λ 2 Z of λ 1 a 1 + λ 2 a 2 = c. Then (λ 1 λ)a 1 + (λ 2 λ)a 2 = b. General problem: Given a 1,..., a n, b Z m, find x = (x 1,, x n ) Z n such that ( ) a 1 x 1 + a 2 x a n x n = b or equivalently Ax = b where A := [a 1,..., a n ]. Def. We introduce the lattice generated by a 1,..., a n, { n } L = L(a 1,..., a n ) = a jλ j λ j Z R m. j=1 Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 37/40

38 Assumption 1: rank A = m (m n); w.l.o.g., a 1,..., a m are linearly independent. To solve the problem: Find C = [c 1... c m ] Z m m such that ( ) L(c 1,..., c m ) = L(a 1,..., a n ). Then ( ) has a solution x Z n iff λ := C 1 b Z m Bad news: As in the case of one equation: in general Lemma 2.16 ([MO, Lm. 2.3]) L(a 1,..., a m ) L(a 1,..., a n ). Let c 1,..., c m L(a 1,..., a n ). Then L(c 1,..., c m ) = L(a 1,..., a n ) if and only if for all j = 1,..., n, the system Cλ = a j has an integral solution. Last step: Find such c i s Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 38/40

39 Main Result: The algorithm Lattice Basis INIT: C = [c 1,..., c m ] = [a 1,..., a m ] ; ITER: Compute C 1 ; If C 1 a j Z m for j = 1,..., n, then stop; If λ = C 1 a j / Z m for some j, then Let a j = Cλ = m i=1 λ ic i and compute c = m i=1 (λ i [λ i ])c i = a j m i=1 [λ i]c i ; Let k be the largest index i such that λ i / Z ; Update C by replacing c k with c in column k; next iteration stops after at most K = log 2 (det[a 1,..., a m ]) steps with a matrix C satisfying ( ). Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 39/40

40 As an exercise: Explain how an integer solution x of ( ) can be constructed with the help of the results from the algorithm above. Complexity From the algorithm above we see: To solve integer systems of equations is polynomial. Under additional inequalities (such as x 0) the problem becomes NP-hard Theorem 2.17 ([MO, Thm. 2.4]) Let A Z m n and b Z m be given. Then exactly one of the following statements is true: (a) There exists some x Z n such that Ax = b. (b) There exists some y R m such that y T A Z n and y T b Z. Peter J.C. Dickinson MO18, Chpt 2: Linear Equations and inequalities 40/40

Mathematical Optimisation, Chpt 2: Linear Equations and inequalities

Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Introduction Gauss-elimination Orthogonal projection Linear Inequalities Integer Solutions Mathematical Optimisation, Chpt 2: Linear Equations and inequalities Peter J.C. Dickinson p.j.c.dickinson@utwente.nl

More information

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88 Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP) MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

Review problems for MA 54, Fall 2004.

Review problems for MA 54, Fall 2004. Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on

More information

Cheat Sheet for MATH461

Cheat Sheet for MATH461 Cheat Sheet for MATH46 Here is the stuff you really need to remember for the exams Linear systems Ax = b Problem: We consider a linear system of m equations for n unknowns x,,x n : For a given matrix A

More information

This can be accomplished by left matrix multiplication as follows: I

This can be accomplished by left matrix multiplication as follows: I 1 Numerical Linear Algebra 11 The LU Factorization Recall from linear algebra that Gaussian elimination is a method for solving linear systems of the form Ax = b, where A R m n and bran(a) In this method

More information

MAT Linear Algebra Collection of sample exams

MAT Linear Algebra Collection of sample exams MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

More information

Problem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show

Problem Set (T) If A is an m n matrix, B is an n p matrix and D is a p s matrix, then show MTH 0: Linear Algebra Department of Mathematics and Statistics Indian Institute of Technology - Kanpur Problem Set Problems marked (T) are for discussions in Tutorial sessions (T) If A is an m n matrix,

More information

Knowledge Discovery and Data Mining 1 (VO) ( )

Knowledge Discovery and Data Mining 1 (VO) ( ) Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory

More information

Algebra C Numerical Linear Algebra Sample Exam Problems

Algebra C Numerical Linear Algebra Sample Exam Problems Algebra C Numerical Linear Algebra Sample Exam Problems Notation. Denote by V a finite-dimensional Hilbert space with inner product (, ) and corresponding norm. The abbreviation SPD is used for symmetric

More information

Conceptual Questions for Review

Conceptual Questions for Review Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.

More information

MATH 240 Spring, Chapter 1: Linear Equations and Matrices

MATH 240 Spring, Chapter 1: Linear Equations and Matrices MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear

More information

Linear Algebra in Actuarial Science: Slides to the lecture

Linear Algebra in Actuarial Science: Slides to the lecture Linear Algebra in Actuarial Science: Slides to the lecture Fall Semester 2010/2011 Linear Algebra is a Tool-Box Linear Equation Systems Discretization of differential equations: solving linear equations

More information

Foundations of Matrix Analysis

Foundations of Matrix Analysis 1 Foundations of Matrix Analysis In this chapter we recall the basic elements of linear algebra which will be employed in the remainder of the text For most of the proofs as well as for the details, the

More information

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008

Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Math 520 Exam 2 Topic Outline Sections 1 3 (Xiao/Dumas/Liaw) Spring 2008 Exam 2 will be held on Tuesday, April 8, 7-8pm in 117 MacMillan What will be covered The exam will cover material from the lectures

More information

MATH 1120 (LINEAR ALGEBRA 1), FINAL EXAM FALL 2011 SOLUTIONS TO PRACTICE VERSION

MATH 1120 (LINEAR ALGEBRA 1), FINAL EXAM FALL 2011 SOLUTIONS TO PRACTICE VERSION MATH (LINEAR ALGEBRA ) FINAL EXAM FALL SOLUTIONS TO PRACTICE VERSION Problem (a) For each matrix below (i) find a basis for its column space (ii) find a basis for its row space (iii) determine whether

More information

LINEAR ALGEBRA KNOWLEDGE SURVEY

LINEAR ALGEBRA KNOWLEDGE SURVEY LINEAR ALGEBRA KNOWLEDGE SURVEY Instructions: This is a Knowledge Survey. For this assignment, I am only interested in your level of confidence about your ability to do the tasks on the following pages.

More information

A Review of Linear Algebra

A Review of Linear Algebra A Review of Linear Algebra Mohammad Emtiyaz Khan CS,UBC A Review of Linear Algebra p.1/13 Basics Column vector x R n, Row vector x T, Matrix A R m n. Matrix Multiplication, (m n)(n k) m k, AB BA. Transpose

More information

Chapter 3 Transformations

Chapter 3 Transformations Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases

More information

1 9/5 Matrices, vectors, and their applications

1 9/5 Matrices, vectors, and their applications 1 9/5 Matrices, vectors, and their applications Algebra: study of objects and operations on them. Linear algebra: object: matrices and vectors. operations: addition, multiplication etc. Algorithms/Geometric

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

Linear Analysis Lecture 16

Linear Analysis Lecture 16 Linear Analysis Lecture 16 The QR Factorization Recall the Gram-Schmidt orthogonalization process. Let V be an inner product space, and suppose a 1,..., a n V are linearly independent. Define q 1,...,

More information

MTH 464: Computational Linear Algebra

MTH 464: Computational Linear Algebra MTH 464: Computational Linear Algebra Lecture Outlines Exam 2 Material Prof. M. Beauregard Department of Mathematics & Statistics Stephen F. Austin State University March 2, 2018 Linear Algebra (MTH 464)

More information

A Brief Outline of Math 355

A Brief Outline of Math 355 A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting

More information

Recall the convention that, for us, all vectors are column vectors.

Recall the convention that, for us, all vectors are column vectors. Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

More information

Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same.

Equality: Two matrices A and B are equal, i.e., A = B if A and B have the same order and the entries of A and B are the same. Introduction Matrix Operations Matrix: An m n matrix A is an m-by-n array of scalars from a field (for example real numbers) of the form a a a n a a a n A a m a m a mn The order (or size) of A is m n (read

More information

2. Every linear system with the same number of equations as unknowns has a unique solution.

2. Every linear system with the same number of equations as unknowns has a unique solution. 1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations

More information

Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam

Math 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system

More information

Lecture 4 Orthonormal vectors and QR factorization

Lecture 4 Orthonormal vectors and QR factorization Orthonormal vectors and QR factorization 4 1 Lecture 4 Orthonormal vectors and QR factorization EE263 Autumn 2004 orthonormal vectors Gram-Schmidt procedure, QR factorization orthogonal decomposition induced

More information

ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3

ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3 ANALYTICAL MATHEMATICS FOR APPLICATIONS 2018 LECTURE NOTES 3 ISSUED 24 FEBRUARY 2018 1 Gaussian elimination Let A be an (m n)-matrix Consider the following row operations on A (1) Swap the positions any

More information

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v ) Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O

More information

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT

Math Camp II. Basic Linear Algebra. Yiqing Xu. Aug 26, 2014 MIT Math Camp II Basic Linear Algebra Yiqing Xu MIT Aug 26, 2014 1 Solving Systems of Linear Equations 2 Vectors and Vector Spaces 3 Matrices 4 Least Squares Systems of Linear Equations Definition A linear

More information

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB

Glossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the

More information

Total 170. Name. Final Examination M340L-CS

Total 170. Name. Final Examination M340L-CS 1 10 2 10 3 15 4 5 5 10 6 10 7 20 8 10 9 20 10 25 11 10 12 10 13 15 Total 170 Final Examination Name M340L-CS 1. Use extra paper to determine your solutions then neatly transcribe them onto these sheets.

More information

MATH 315 Linear Algebra Homework #1 Assigned: August 20, 2018

MATH 315 Linear Algebra Homework #1 Assigned: August 20, 2018 Homework #1 Assigned: August 20, 2018 Review the following subjects involving systems of equations and matrices from Calculus II. Linear systems of equations Converting systems to matrix form Pivot entry

More information

G1110 & 852G1 Numerical Linear Algebra

G1110 & 852G1 Numerical Linear Algebra The University of Sussex Department of Mathematics G & 85G Numerical Linear Algebra Lecture Notes Autumn Term Kerstin Hesse (w aw S w a w w (w aw H(wa = (w aw + w Figure : Geometric explanation of the

More information

MATH 369 Linear Algebra

MATH 369 Linear Algebra Assignment # Problem # A father and his two sons are together 00 years old. The father is twice as old as his older son and 30 years older than his younger son. How old is each person? Problem # 2 Determine

More information

1 Last time: least-squares problems

1 Last time: least-squares problems MATH Linear algebra (Fall 07) Lecture Last time: least-squares problems Definition. If A is an m n matrix and b R m, then a least-squares solution to the linear system Ax = b is a vector x R n such that

More information

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v

More information

Linear Algebra Review. Vectors

Linear Algebra Review. Vectors Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors

More information

Eigenvalues and Eigenvectors A =

Eigenvalues and Eigenvectors A = Eigenvalues and Eigenvectors Definition 0 Let A R n n be an n n real matrix A number λ R is a real eigenvalue of A if there exists a nonzero vector v R n such that A v = λ v The vector v is called an eigenvector

More information

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2016 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

More information

Math Linear Algebra Final Exam Review Sheet

Math Linear Algebra Final Exam Review Sheet Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

More information

Quantum Computing Lecture 2. Review of Linear Algebra

Quantum Computing Lecture 2. Review of Linear Algebra Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

LINEAR ALGEBRA 1, 2012-I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS

LINEAR ALGEBRA 1, 2012-I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS LINEAR ALGEBRA, -I PARTIAL EXAM SOLUTIONS TO PRACTICE PROBLEMS Problem (a) For each of the two matrices below, (i) determine whether it is diagonalizable, (ii) determine whether it is orthogonally diagonalizable,

More information

Linear Algebra March 16, 2019

Linear Algebra March 16, 2019 Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented

More information

This operation is - associative A + (B + C) = (A + B) + C; - commutative A + B = B + A; - has a neutral element O + A = A, here O is the null matrix

This operation is - associative A + (B + C) = (A + B) + C; - commutative A + B = B + A; - has a neutral element O + A = A, here O is the null matrix 1 Matrix Algebra Reading [SB] 81-85, pp 153-180 11 Matrix Operations 1 Addition a 11 a 12 a 1n a 21 a 22 a 2n a m1 a m2 a mn + b 11 b 12 b 1n b 21 b 22 b 2n b m1 b m2 b mn a 11 + b 11 a 12 + b 12 a 1n

More information

EE731 Lecture Notes: Matrix Computations for Signal Processing

EE731 Lecture Notes: Matrix Computations for Signal Processing EE731 Lecture Notes: Matrix Computations for Signal Processing James P. Reilly c Department of Electrical and Computer Engineering McMaster University September 22, 2005 0 Preface This collection of ten

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

LECTURE NOTES ELEMENTARY NUMERICAL METHODS. Eusebius Doedel

LECTURE NOTES ELEMENTARY NUMERICAL METHODS. Eusebius Doedel LECTURE NOTES on ELEMENTARY NUMERICAL METHODS Eusebius Doedel TABLE OF CONTENTS Vector and Matrix Norms 1 Banach Lemma 20 The Numerical Solution of Linear Systems 25 Gauss Elimination 25 Operation Count

More information

HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION)

HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) HOMEWORK PROBLEMS FROM STRANG S LINEAR ALGEBRA AND ITS APPLICATIONS (4TH EDITION) PROFESSOR STEVEN MILLER: BROWN UNIVERSITY: SPRING 2007 1. CHAPTER 1: MATRICES AND GAUSSIAN ELIMINATION Page 9, # 3: Describe

More information

Lecture Notes in Linear Algebra

Lecture Notes in Linear Algebra Lecture Notes in Linear Algebra Dr. Abdullah Al-Azemi Mathematics Department Kuwait University February 4, 2017 Contents 1 Linear Equations and Matrices 1 1.2 Matrices............................................

More information

Jim Lambers MAT 610 Summer Session Lecture 1 Notes

Jim Lambers MAT 610 Summer Session Lecture 1 Notes Jim Lambers MAT 60 Summer Session 2009-0 Lecture Notes Introduction This course is about numerical linear algebra, which is the study of the approximate solution of fundamental problems from linear algebra

More information

Study Guide for Linear Algebra Exam 2

Study Guide for Linear Algebra Exam 2 Study Guide for Linear Algebra Exam 2 Term Vector Space Definition A Vector Space is a nonempty set V of objects, on which are defined two operations, called addition and multiplication by scalars (real

More information

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition

Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Linear Algebra review Powers of a diagonalizable matrix Spectral decomposition Prof. Tesler Math 283 Fall 2018 Also see the separate version of this with Matlab and R commands. Prof. Tesler Diagonalizing

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

Math 407: Linear Optimization

Math 407: Linear Optimization Math 407: Linear Optimization Lecture 16: The Linear Least Squares Problem II Math Dept, University of Washington February 28, 2018 Lecture 16: The Linear Least Squares Problem II (Math Dept, University

More information

Solution of Linear Equations

Solution of Linear Equations Solution of Linear Equations (Com S 477/577 Notes) Yan-Bin Jia Sep 7, 07 We have discussed general methods for solving arbitrary equations, and looked at the special class of polynomial equations A subclass

More information

a s 1.3 Matrix Multiplication. Know how to multiply two matrices and be able to write down the formula

a s 1.3 Matrix Multiplication. Know how to multiply two matrices and be able to write down the formula Syllabus for Math 308, Paul Smith Book: Kolman-Hill Chapter 1. Linear Equations and Matrices 1.1 Systems of Linear Equations Definition of a linear equation and a solution to a linear equations. Meaning

More information

Linear Algebra: Matrix Eigenvalue Problems

Linear Algebra: Matrix Eigenvalue Problems CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given

More information

5.6. PSEUDOINVERSES 101. A H w.

5.6. PSEUDOINVERSES 101. A H w. 5.6. PSEUDOINVERSES 0 Corollary 5.6.4. If A is a matrix such that A H A is invertible, then the least-squares solution to Av = w is v = A H A ) A H w. The matrix A H A ) A H is the left inverse of A and

More information

Scientific Computing: Dense Linear Systems

Scientific Computing: Dense Linear Systems Scientific Computing: Dense Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 February 9th, 2012 A. Donev (Courant Institute)

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 3: Positive-Definite Systems; Cholesky Factorization Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical Analysis I 1 / 11 Symmetric

More information

Orthonormal Transformations and Least Squares

Orthonormal Transformations and Least Squares Orthonormal Transformations and Least Squares Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University of Oslo October 30, 2009 Applications of Qx with Q T Q = I 1. solving

More information

ELE/MCE 503 Linear Algebra Facts Fall 2018

ELE/MCE 503 Linear Algebra Facts Fall 2018 ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2

More information

ORIE 6300 Mathematical Programming I August 25, Recitation 1

ORIE 6300 Mathematical Programming I August 25, Recitation 1 ORIE 6300 Mathematical Programming I August 25, 2016 Lecturer: Calvin Wylie Recitation 1 Scribe: Mateo Díaz 1 Linear Algebra Review 1 1.1 Independence, Spanning, and Dimension Definition 1 A (usually infinite)

More information

Math 265 Linear Algebra Sample Spring 2002., rref (A) =

Math 265 Linear Algebra Sample Spring 2002., rref (A) = Math 265 Linear Algebra Sample Spring 22. It is given that A = rref (A T )= 2 3 5 3 2 6, rref (A) = 2 3 and (a) Find the rank of A. (b) Find the nullityof A. (c) Find a basis for the column space of A.

More information

LINEAR ALGEBRA SUMMARY SHEET.

LINEAR ALGEBRA SUMMARY SHEET. LINEAR ALGEBRA SUMMARY SHEET RADON ROSBOROUGH https://intuitiveexplanationscom/linear-algebra-summary-sheet/ This document is a concise collection of many of the important theorems of linear algebra, organized

More information

orthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis,

orthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis, 5 Orthogonality Goals: We use scalar products to find the length of a vector, the angle between 2 vectors, projections, orthogonal relations between vectors and subspaces Then we study some applications

More information

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0.

Linear Algebra. Matrices Operations. Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0. Matrices Operations Linear Algebra Consider, for example, a system of equations such as x + 2y z + 4w = 0, 3x 4y + 2z 6w = 0, x 3y 2z + w = 0 The rectangular array 1 2 1 4 3 4 2 6 1 3 2 1 in which the

More information

Lecture notes: Applied linear algebra Part 1. Version 2

Lecture notes: Applied linear algebra Part 1. Version 2 Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

Linear Algebra Primer

Linear Algebra Primer Linear Algebra Primer David Doria daviddoria@gmail.com Wednesday 3 rd December, 2008 Contents Why is it called Linear Algebra? 4 2 What is a Matrix? 4 2. Input and Output.....................................

More information

Theorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form. k k n-k. m-k k m-k n-k

Theorem A.1. If A is any nonzero m x n matrix, then A is equivalent to a partitioned matrix of the form. k k n-k. m-k k m-k n-k I. REVIEW OF LINEAR ALGEBRA A. Equivalence Definition A1. If A and B are two m x n matrices, then A is equivalent to B if we can obtain B from A by a finite sequence of elementary row or elementary column

More information

235 Final exam review questions

235 Final exam review questions 5 Final exam review questions Paul Hacking December 4, 0 () Let A be an n n matrix and T : R n R n, T (x) = Ax the linear transformation with matrix A. What does it mean to say that a vector v R n is an

More information

MTH 2032 SemesterII

MTH 2032 SemesterII MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents

More information

5. Orthogonal matrices

5. Orthogonal matrices L Vandenberghe EE133A (Spring 2017) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal

More information

Applied Linear Algebra

Applied Linear Algebra Applied Linear Algebra Gábor P. Nagy and Viktor Vígh University of Szeged Bolyai Institute Winter 2014 1 / 262 Table of contents I 1 Introduction, review Complex numbers Vectors and matrices Determinants

More information

Linear Algebra Highlights

Linear Algebra Highlights Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to

More information

MIT Final Exam Solutions, Spring 2017

MIT Final Exam Solutions, Spring 2017 MIT 8.6 Final Exam Solutions, Spring 7 Problem : For some real matrix A, the following vectors form a basis for its column space and null space: C(A) = span,, N(A) = span,,. (a) What is the size m n of

More information

Math 1553, Introduction to Linear Algebra

Math 1553, Introduction to Linear Algebra Learning goals articulate what students are expected to be able to do in a course that can be measured. This course has course-level learning goals that pertain to the entire course, and section-level

More information

Lecture 7: Positive Semidefinite Matrices

Lecture 7: Positive Semidefinite Matrices Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.

More information

SUMMARY OF MATH 1600

SUMMARY OF MATH 1600 SUMMARY OF MATH 1600 Note: The following list is intended as a study guide for the final exam. It is a continuation of the study guide for the midterm. It does not claim to be a comprehensive list. You

More information

MAT 610: Numerical Linear Algebra. James V. Lambers

MAT 610: Numerical Linear Algebra. James V. Lambers MAT 610: Numerical Linear Algebra James V Lambers January 16, 2017 2 Contents 1 Matrix Multiplication Problems 7 11 Introduction 7 111 Systems of Linear Equations 7 112 The Eigenvalue Problem 8 12 Basic

More information

Elementary maths for GMT

Elementary maths for GMT Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

More information

CS 246 Review of Linear Algebra 01/17/19

CS 246 Review of Linear Algebra 01/17/19 1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

More information

Math Linear Algebra II. 1. Inner Products and Norms

Math Linear Algebra II. 1. Inner Products and Norms Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,

More information

ANSWERS. E k E 2 E 1 A = B

ANSWERS. E k E 2 E 1 A = B MATH 7- Final Exam Spring ANSWERS Essay Questions points Define an Elementary Matrix Display the fundamental matrix multiply equation which summarizes a sequence of swap, combination and multiply operations,

More information

Review of Some Concepts from Linear Algebra: Part 2

Review of Some Concepts from Linear Algebra: Part 2 Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set

More information

I. Multiple Choice Questions (Answer any eight)

I. Multiple Choice Questions (Answer any eight) Name of the student : Roll No : CS65: Linear Algebra and Random Processes Exam - Course Instructor : Prashanth L.A. Date : Sep-24, 27 Duration : 5 minutes INSTRUCTIONS: The test will be evaluated ONLY

More information

Practical Linear Algebra: A Geometry Toolbox

Practical Linear Algebra: A Geometry Toolbox Practical Linear Algebra: A Geometry Toolbox Third edition Chapter 12: Gauss for Linear Systems Gerald Farin & Dianne Hansford CRC Press, Taylor & Francis Group, An A K Peters Book www.farinhansford.com/books/pla

More information

Math 240 Calculus III

Math 240 Calculus III The Calculus III Summer 2015, Session II Wednesday, July 8, 2015 Agenda 1. of the determinant 2. determinants 3. of determinants What is the determinant? Yesterday: Ax = b has a unique solution when A

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 1: Course Overview & Matrix-Vector Multiplication Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 20 Outline 1 Course

More information

Numerical Methods I Solving Square Linear Systems: GEM and LU factorization

Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Numerical Methods I Solving Square Linear Systems: GEM and LU factorization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 MATH-GA 2011.003 / CSCI-GA 2945.003, Fall 2014 September 18th,

More information

1. General Vector Spaces

1. General Vector Spaces 1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

More information

Mathematical foundations - linear algebra

Mathematical foundations - linear algebra Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar

More information