Linear Algebra Massoud Malek


 Jack Cobb
 2 years ago
 Views:
Transcription
1 CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product is a generalization of the dot product In a vector space, it is a way to multiply vectors together, with the result of this multiplication being a scalar More precisely, for a real vector space, an inner product, satisfies the following four properties Let u, v, and w be vectors and α be a scalar, then: Linearity: u + v, w = u, w + v, w α v, w = α v, w Symmetry: v, w = w, v Positivedefiniteness: v, v 0 and equal if and only if v = 0 A vector space together with an inner product on it is called an inner product space When given a complex vector space, the symmetry property above is usually replaced by Conjugate Symmetry: v, w = w, v, where w, v refers to complex conjugation With this property, the inner product is called a Hermitian inner product and a complex vector space with a Hermitian inner product is called a Hermitian inner product space An inner product space is a vector space with the additional inner product structure An inner product naturally induces an associated norm, u = u, u Thus an inner product space is also a normed vector space Examples of Inner Product Space (a) The Euclidean inner product on IF n, where IF is the set of real or complex number is defined by (u, u 2,, u n, v, v 2,, v n ) = u v + u 2 v u n v n (b) An inner product can be defined on the vector space of smooth realvalued functions on the interval [, ] by f)x), g(x) = f(x) g(x) d x
2 Linear Algebra Inner Product and Normed Space Page 2 (c) An inner product can be defined on the set of real polynomials by Examples of Vector Norms p(x), q(x) = Here are three mostly used vector norms: (a) norm: v = v i i (b) 2norm or Euclidean norm: v 2 = v i 2 (c) norm: v = max x i 0 p(x) q(x) e x d x i Also, note that if is a norm and M is any nonsingular square matrix, then v M v is also a norm The case where M is diagonal is particularly common in practice Note Any Hermitian positive definite matrix H induces an inner product u, v H = u H v; and thus a norm u H = u, u H In MatLab, the norm, 2norm, and norm are invoked by the statements norm (u, ), norm (u, 2), and norm (u, inf), respectively The 2norm is the default in MatLab The statement norm (u) is interpreted as the 2norm of u in MatLab Theorem If is a norm induce by an inner product, then u = 0, if and only if u = θ, otherwise u > 0 ; 2 λ u = λ u ; 3 (CauchySchwarz inequality) u, v u v ; 4 (Triangle inequality) u + v u + v, Orthogonality In an inner product vector space V, vectors u and v are orthogonal, if u, v = 0 A subset S of V is orthogonal, if any two distinct vectors of S are orthogonal A vector u is a unit vector, if u = Finally a subset S of V is orthonormal, if S is orthogonal and consists entirely of unit vectors If v θ, then the vector u = v is a unit vector; we say that v was normalized v Note The zero vector θ is orthogonal to every vector in V and θ is the only vector in V that is orthogonal to itself Pythagorean Theorem Suppose u and v are orthogonal vectors in V Then Proof we have u + v 2 = u 2 + v 2 u + v 2 = u + v, u + v = u, u + u, v + v, u + v, v = u, u 2 + v, v 2 as desired
3 Linear Algebra GramSchmidt Page 3 The CauchySchwarz inequality states that for all vectors u and v of an inner product space: u, v 2 u, u v, v, where, is the inner product Equivalently, by taking the square root of both sides, and referring to the norms of the vectors, the inequality is written as u, v u v Moreover, the two sides are equal if and only if u and v are linearly dependent Proof Let u, v be arbitrary vectors in a vector space V over IF with an inner product, where IF is the field of real or complex numbers If u, v = 0, the theorem holds trivially If not, then u 0, v 0 For any λ IF, we have 0 u λ v 2 = u λ v, u λ v = u, u λ v λ v, u λ v u, u λ u, v λ v, u + λ λ v, v Choose λ = u, v v, v, then 0 u, u λ u, v λ v, u + λ λ v, v = u, u u, v 2 v, v = u 2 u, v 2 v 2 It follows that u, v 2 u, u v, v or u, v u v Examples of the CauchySchwarz Inequality (a) If u, u 2,, u n, v, v 2,, v n are real numbers, then u v + u 2 v u n v n 2 ( u 2 + u u 2 n) ( v 2 + v v 2 n) (b) If f(x) and g(x) are smooth realvalued functions, then f(x) g(x) d x Triangle Inequality For any vectors u and v, Proof We have 2 ( ) ( f(x) 2 d x u + v u + v g(x) 2 d x u + v 2 = u + v, u + v = u, u + u, v + v, u + v, v = u Re u, v + v 2 u 2 + u, v + v 2 u u v + v 2 = u 2 + v 2 )
4 Linear Algebra GramSchmidt Page 4 GramSchmidt Process It is often more simpler to work in an orthogonal basis GramSchmidt process is a method for obtaining orthonormal vectors from a linearly independent set of vectors in an inner product space, most commonly the Euclidean space IR n S = {v, v 2, v 3,, v k } and generates an orthogonal set S G = {u, u 2, u 3,, u k } or orthonormal basis S N = {e, e 2, e 3,, e k } that spans the same subspace of IR n as the set S v, u We define the projection operator by proj u (v) = u, where v, u denotes the inner u, u product of the vectors v and u This operator projects the vector v orthogonally onto the line spanned by vector u The GramSchmidt process then works as follows: u = v, e = u u u 2 = v 2 proj u (v 2 ), e 2 = u 2 u 2 u 3 = v 3 proj u (v 3 ) proj u2 (v 3 ), e 3 = u 3 u 3 u 4 = v 4 proj u (v 4 ) proj u2 (v 4 ) proj u3 (v 4 ), e 4 = u 4 u 4 = = k u k = v k proj uj (v k ), e k = u k u k j= The sequence {u, u 2,, u k } is the required system of orthogonal vectors, and the normalized vectors {e, e 2,, e k } form an orthonormal set The calculation of the sequence u, u 2,, u k is known as GramSchmidt orthogonalization, while the calculation of the sequence e, e 2,, e k is known as GramSchmidt orthonormalization as the vectors are normalized Example Consider the basis B = v =, v 2 =, v 3 = Then use Gram Schmidt to find the orthogonal basis B G = { u, u 2, u 3 } and the orthonormal basis B N = { e, e 2, e 3 } associated with B u = v = ; e = u u = 3 u 2 = v 2 v 2, u u, u u = = 2 2 = u 2 = 2 ; e 2 = u u 2 = 2 6 u 3 = v 3 v 3, u u, u u v 3, u 2 u 2, u 2 u 2 = + 2 = 0 ; e 3 = u u 3 = 0 2 Hence B G = u =, u 2 = 2, u 3 = 0 and B N = e =, e 2 = 2, e 3 =
5 Linear Algebra Matrix Norms Page 5 Matrix Norms A matrix norm is an extension of the notion of a vector norm to matrices It is a vector norm on IF m n That is, if A denotes the norm of the matrix A, then, A 0; 2 A = 0 if and only if A = 0; 3 α A = α A for all α IF and A IF m n ; 4 A, B IF m n, A + B A + B Additionally, in the case of square matrices (thus, m = n), some (but not all) matrix norms satisfy the following condition, which is related to the fact that matrices are more than just vectors: 5 AB A B for all matrices A and B in IF m n A matrix norm that satisfies this additional property is called a submultiplicative norm Operator Norm If vector norms on IF m and IF n are given (IF is the field of real or complex numbers), then one defines the corresponding operator norm or induced norm or, on the space of mbyn matrices as the following maxima: A = sup{ Ax : x IF n, x = } { } Ax = sup x : x IF n, x θ The operator norm corresponding to the pnorm for vectors is: Ax p A p = sup x 0 x p Note All operator norms are submultiplicative Here are some example matrix norms: m Column norm or norm: A = max j n i= a ij, which is simply the maximum absolute column sum of the matrix n Row norm or infinitynorm: A = max i m j= a ij, which is simply the maximum absolute row sum of the matrix The 2norm induced by the euclidean vector norm is A = λ max, where λ max is the largest eigenvalue of the matrix A A Although the Frobenius norm, defined as: k n A F = a ij 2 = trace (A A) i= j=
6 Linear Algebra Matrix Norms Page 6 is not an induced norm but it is a submultiplicative norm In MatLab, the Frobenius nom is invoked by the statement norm (A, fro ) The max norm A max = max a ij is neither an induced norm nor a submultiplicative norm The largest eigenvalue of a square matrix A in modulus, denoted by ρ (A) and called the spectral radius of A, never exceeds its operator norm: ρ (A) A p Given two normed vector spaces V and W (over the same base field, either the real numbers or the complex numbers), a linear map A : V W is continuous if and only if there exists a real number λ such that for all v V, Condition Number A v λ v The condition number associated with the linear equation A x = b, gives a bound on how inaccurate the solution x will be after approximation Note that this is before the effects of roundoff error are taken into account; conditioning is a property of the matrix, not the algorithm or floating point accuracy of the computer used to solve the corresponding system Let x 0 be the solution of the linear equation A x = b and let x be the calculated solution of A x = b, so the error in the solution is e = x x 0, and the relative error in the solution is e = x x 0 x 0 Generally, we can t verify the error or relative error (we don t know x 0 ), so a possible way to check this is by testing the accuracy back in the system A x = b : and relative residual is = residual r = A x 0 A x = b b, A x 0 A x A x = b b b = r b A matrix A is said to be illconditioned, if relatively small changes in the input (in the matrix A) can cause large change in the output (the solution of A x = b), ie; the solution is not very accurate if input is rounded Otherwise it is wellconditioned The condition number of an invertible matrix A is defined to be κ (A) = A A This quantity is always greater than or equal to The condition number is defined more precisely to be the maximum ratio of the relative error in x divided by the relative error in b If A is nonsingular, then by knowing the eigenvalues of A A (λ λ 2 λ n > 0), we can compute the condition number using the 2norm: κ 2 (A) = A 2 A 2 = λ λ n
7 Linear Algebra Matrix Norms Page 7 The smallest nonzero eigenvalue λ n of A A is a measure of how close the matrix is to being singular If λ n is small, then κ 2 (A) is large Thus, the closer the matrix is to being singular, the more illconditioned it is Consistent Norms A matrix norm ab on IF m n is called consistent with a vector norm a on IF n and a vector norm b on IF m, if: Ax b A ab u a, for all A IF m n, u IF n All induced norms are consistent by definition Compatible Norms A matrix norm b on IF n n is called compatible with a vector norm a on IF n if: A x a A b x a, for all A IF n n, x IF n All induced norms are compatible by definition Equivalence of Norms Two matrix norms α and β are said to be equivalent, if for all matrices A IF m n, there exist some positive numbers r and s such that Examples of Norm Equivalence r A α A β s A α For matrix A R \ of rank r, the following inequalities hold: A 2 A F r A 2 A F A r A F A max A 2 mn A max A A 2 m A n A A 2 n A m Here, A p refers to the matrix norm induced by the vector pnorm Another useful inequality between matrix norms is: A 2 A A
Chapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 WeiTa Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationInner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:
Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v
More informationINNER PRODUCT SPACE. Definition 1
INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of
More informationDiagonalizing Matrices
Diagonalizing Matrices Massoud Malek A A Let A = A k be an n n nonsingular matrix and let B = A = [B, B,, B k,, B n ] Then A n A B = A A 0 0 A k [B, B,, B k,, B n ] = 0 0 = I n 0 A n Notice that A i B
More informationLinear Algebra Review. Vectors
Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data TwoDimensional Plots Programming in
More informationMATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)
MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m
More informationThere are two things that are particularly nice about the first basis
Orthogonality and the GramSchmidt Process In Chapter 4, we spent a great deal of time studying the problem of finding a basis for a vector space We know that a basis for a vector space can potentially
More informationDSGA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DSGA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationMath Linear Algebra II. 1. Inner Products and Norms
Math 342  Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,
More informationMath 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.
Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,
More informationMATH 235: Inner Product Spaces, Assignment 7
MATH 235: Inner Product Spaces, Assignment 7 Hand in questions 3,4,5,6,9, by 9:3 am on Wednesday March 26, 28. Contents Orthogonal Basis for Inner Product Space 2 2 InnerProduct Function Space 2 3 Weighted
More informationThe following definition is fundamental.
1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic
More information2. Review of Linear Algebra
2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear
More informationLecture Notes 1: Vector spaces
Optimizationbased data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector
More informationLinear Algebra. Session 12
Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)
More informationPart 1a: Inner product, Orthogonality, Vector/Matrix norm
Part 1a: Inner product, Orthogonality, Vector/Matrix norm September 19, 2018 Numerical Linear Algebra Part 1a September 19, 2018 1 / 16 1. Inner product on a linear space V over the number field F A map,
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationMathematical foundations  linear algebra
Mathematical foundations  linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More information1. What is the determinant of the following matrix? a 1 a 2 4a 3 2a 2 b 1 b 2 4b 3 2b c 1. = 4, then det
What is the determinant of the following matrix? 3 4 3 4 3 4 4 3 A 0 B 8 C 55 D 0 E 60 If det a a a 3 b b b 3 c c c 3 = 4, then det a a 4a 3 a b b 4b 3 b c c c 3 c = A 8 B 6 C 4 D E 3 Let A be an n n matrix
More information1. General Vector Spaces
1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More informationApplied Linear Algebra in Geoscience Using MATLAB
Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data TwoDimensional Plots Programming in
More informationVector spaces. DSGA 1013 / MATHGA 2824 Optimizationbased Data Analysis.
Vector spaces DSGA 1013 / MATHGA 2824 Optimizationbased Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos FernandezGranda Vector space Consists of: A set V A scalar
More informationMATH 240 Spring, Chapter 1: Linear Equations and Matrices
MATH 240 Spring, 2006 Chapter Summaries for Kolman / Hill, Elementary Linear Algebra, 8th Ed. Sections 1.1 1.6, 2.1 2.2, 3.2 3.8, 4.3 4.5, 5.1 5.3, 5.5, 6.1 6.5, 7.1 7.2, 7.4 DEFINITIONS Chapter 1: Linear
More information1. Foundations of Numerics from Advanced Mathematics. Linear Algebra
Foundations of Numerics from Advanced Mathematics Linear Algebra Linear Algebra, October 23, 22 Linear Algebra Mathematical Structures a mathematical structure consists of one or several sets and one or
More informationIr O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )
Section 3.2 Theorem 3.6. Let A be an m n matrix of rank r. Then r m, r n, and, by means of a finite number of elementary row and column operations, A can be transformed into the matrix ( ) Ir O D = 1 O
More informationLINEAR ALGEBRA W W L CHEN
LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,
More informationorthogonal relations between vectors and subspaces Then we study some applications in vector spaces and linear systems, including Orthonormal Basis,
5 Orthogonality Goals: We use scalar products to find the length of a vector, the angle between 2 vectors, projections, orthogonal relations between vectors and subspaces Then we study some applications
More informationAssignment 1 Math 5341 Linear Algebra Review. Give complete answers to each of the following questions. Show all of your work.
Assignment 1 Math 5341 Linear Algebra Review Give complete answers to each of the following questions Show all of your work Note: You might struggle with some of these questions, either because it has
More informationReview of Some Concepts from Linear Algebra: Part 2
Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set
More informationOrthonormal Bases; GramSchmidt Process; QRDecomposition
Orthonormal Bases; GramSchmidt Process; QRDecomposition MATH 322, Linear Algebra I J. Robert Buchanan Department of Mathematics Spring 205 Motivation When working with an inner product space, the most
More informationECS130 Scientific Computing. Lecture 1: Introduction. Monday, January 7, 10:00 10:50 am
ECS130 Scientific Computing Lecture 1: Introduction Monday, January 7, 10:00 10:50 am About Course: ECS130 Scientific Computing Professor: Zhaojun Bai Webpage: http://web.cs.ucdavis.edu/~bai/ecs130/ Today
More informationORTHOGONALITY AND LEASTSQUARES [CHAP. 6]
ORTHOGONALITY AND LEASTSQUARES [CHAP. 6] Inner products and Norms Inner product or dot product of 2 vectors u and v in R n : u.v = u 1 v 1 + u 2 v 2 + + u n v n Calculate u.v when u = 1 2 2 0 v = 1 0
More informationInner Product Spaces
Inner Product Spaces Introduction Recall in the lecture on vector spaces that geometric vectors (i.e. vectors in two and threedimensional Cartesian space have the properties of addition, subtraction,
More information6. Orthogonality and LeastSquares
Linear Algebra 6. Orthogonality and LeastSquares CSIE NCU 1 6. Orthogonality and LeastSquares 6.1 Inner product, length, and orthogonality. 2 6.2 Orthogonal sets... 8 6.3 Orthogonal projections... 13
More informationMTH 2032 SemesterII
MTH 202 SemesterII 201011 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents
More informationDot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem.
Dot Products K. Behrend April 3, 008 Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem. Contents The dot product 3. Length of a vector........................
More informationReview of Linear Algebra
Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=
More informationLecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.
Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an
More informationReview problems for MA 54, Fall 2004.
Review problems for MA 54, Fall 2004. Below are the review problems for the final. They are mostly homework problems, or very similar. If you are comfortable doing these problems, you should be fine on
More informationScientific Computing: Dense Linear Systems
Scientific Computing: Dense Linear Systems Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATHGA.2043 or CSCIGA.2112, Spring 2012 February 9th, 2012 A. Donev (Courant Institute)
More informationInner product spaces. Layers of structure:
Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty
More informationChapter 6: Orthogonality
Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products
More informationMAT Linear Algebra Collection of sample exams
MAT 342  Linear Algebra Collection of sample exams Ax. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,
More information2. Every linear system with the same number of equations as unknowns has a unique solution.
1. For matrices A, B, C, A + B = A + C if and only if A = B. 2. Every linear system with the same number of equations as unknowns has a unique solution. 3. Every linear system with the same number of equations
More informationDefinitions for Quizzes
Definitions for Quizzes Italicized text (or something close to it) will be given to you. Plain text is (an example of) what you should write as a definition. [Bracketed text will not be given, nor does
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,
More informationMath 24 Spring 2012 Sample Homework Solutions Week 8
Math 4 Spring Sample Homework Solutions Week 8 Section 5. (.) Test A M (R) for diagonalizability, and if possible find an invertible matrix Q and a diagonal matrix D such that Q AQ = D. ( ) 4 (c) A =.
More informationGlossary of Linear Algebra Terms. Prepared by Vince Zaccone For Campus Learning Assistance Services at UCSB
Glossary of Linear Algebra Terms Basis (for a subspace) A linearly independent set of vectors that spans the space Basic Variable A variable in a linear system that corresponds to a pivot column in the
More informationMATH 304 Linear Algebra Lecture 20: The GramSchmidt process (continued). Eigenvalues and eigenvectors.
MATH 304 Linear Algebra Lecture 20: The GramSchmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v
More informationMathematics Department Stanford University Math 61CM/DM Inner products
Mathematics Department Stanford University Math 61CM/DM Inner products Recall the definition of an inner product space; see Appendix A.8 of the textbook. Definition 1 An inner product space V is a vector
More informationMath 224, Fall 2007 Exam 3 Thursday, December 6, 2007
Math 224, Fall 2007 Exam 3 Thursday, December 6, 2007 You have 1 hour and 20 minutes. No notes, books, or other references. You are permitted to use Maple during this exam, but you must start with a blank
More informationMathematical Methods wk 1: Vectors
Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are workinprogress notes for the secondyear course on mathematical methods The most uptodate version is available from http://wwwthphysphysicsoxacuk/people/johnmagorrian/mm
More informationMathematical Methods wk 1: Vectors
Mathematical Methods wk : Vectors John Magorrian, magog@thphysoxacuk These are workinprogress notes for the secondyear course on mathematical methods The most uptodate version is available from http://wwwthphysphysicsoxacuk/people/johnmagorrian/mm
More informationSection 7.5 Inner Product Spaces
Section 7.5 Inner Product Spaces With the dot product defined in Chapter 6, we were able to study the following properties of vectors in R n. ) Length or norm of a vector u. ( u = p u u ) 2) Distance of
More informationAlgebra II. Paulius Drungilas and Jonas Jankauskas
Algebra II Paulius Drungilas and Jonas Jankauskas Contents 1. Quadratic forms 3 What is quadratic form? 3 Change of variables. 3 Equivalence of quadratic forms. 4 Canonical form. 4 Normal form. 7 Positive
More informationMATH 1120 (LINEAR ALGEBRA 1), FINAL EXAM FALL 2011 SOLUTIONS TO PRACTICE VERSION
MATH (LINEAR ALGEBRA ) FINAL EXAM FALL SOLUTIONS TO PRACTICE VERSION Problem (a) For each matrix below (i) find a basis for its column space (ii) find a basis for its row space (iii) determine whether
More informationLinear Algebra Practice Problems
Linear Algebra Practice Problems Page of 7 Linear Algebra Practice Problems These problems cover Chapters 4, 5, 6, and 7 of Elementary Linear Algebra, 6th ed, by Ron Larson and David Falvo (ISBN3 = 97868783762,
More informationDot product and linear least squares problems
Dot product and linear least squares problems Dot Product For vectors u,v R n we define the dot product Note that we can also write this as u v = u,,u n u v = u v + + u n v n v v n = u v + + u n v n The
More informationLINEAR ALGEBRA 1, 2012I PARTIAL EXAM 3 SOLUTIONS TO PRACTICE PROBLEMS
LINEAR ALGEBRA, I PARTIAL EXAM SOLUTIONS TO PRACTICE PROBLEMS Problem (a) For each of the two matrices below, (i) determine whether it is diagonalizable, (ii) determine whether it is orthogonally diagonalizable,
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tuberlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationMath 302 Outcome Statements Winter 2013
Math 302 Outcome Statements Winter 2013 1 Rectangular Space Coordinates; Vectors in the ThreeDimensional Space (a) Cartesian coordinates of a point (b) sphere (c) symmetry about a point, a line, and a
More information2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian
FE661  Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian
More informationConceptual Questions for Review
Conceptual Questions for Review Chapter 1 1.1 Which vectors are linear combinations of v = (3, 1) and w = (4, 3)? 1.2 Compare the dot product of v = (3, 1) and w = (4, 3) to the product of their lengths.
More informationELE/MCE 503 Linear Algebra Facts Fall 2018
ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2
More informationExercise Sheet 1.
Exercise Sheet 1 You can download my lecture and exercise sheets at the address http://sami.hust.edu.vn/giangvien/?name=huynt 1) Let A, B be sets. What does the statement "A is not a subset of B " mean?
More informationSingular Value Decompsition
Singular Value Decompsition Massoud Malek One of the most useful results from linear algebra, is a matrix decomposition known as the singular value decomposition It has many useful applications in almost
More informationwhich arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i
MODULE 6 Topics: GramSchmidt orthogonalization process We begin by observing that if the vectors {x j } N are mutually orthogonal in an inner product space V then they are necessarily linearly independent.
More informationLinear Algebra Final Exam Study Guide Solutions Fall 2012
. Let A = Given that v = 7 7 67 5 75 78 Linear Algebra Final Exam Study Guide Solutions Fall 5 explain why it is not possible to diagonalize A. is an eigenvector for A and λ = is an eigenvalue for A diagonalize
More informationChap 3. Linear Algebra
Chap 3. Linear Algebra Outlines 1. Introduction 2. Basis, Representation, and Orthonormalization 3. Linear Algebraic Equations 4. Similarity Transformation 5. Diagonal Form and Jordan Form 6. Functions
More informationJim Lambers MAT 610 Summer Session Lecture 2 Notes
Jim Lambers MAT 610 Summer Session 200910 Lecture 2 Notes These notes correspond to Sections 2.22.4 in the text. Vector Norms Given vectors x and y of length one, which are simply scalars x and y, the
More information1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)
Notes for 20170130 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty
More informationLinear Algebra problems
Linear Algebra problems 1. Show that the set F = ({1, 0}, +,.) is a field where + and. are defined as 1+1=0, 0+0=0, 0+1=1+0=1, 0.0=0.1=1.0=0, 1.1=1.. Let X be a nonempty set and F be any field. Let X
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Logistics Notes for 20160829 General announcement: we are switching from weekly to biweekly homeworks (mostly because the course is much bigger than planned). If you want to do HW but are not formally
More informationVectors in Function Spaces
Jim Lambers MAT 66 Spring Semester 1516 Lecture 18 Notes These notes correspond to Section 6.3 in the text. Vectors in Function Spaces We begin with some necessary terminology. A vector space V, also
More informationLinear Algebra Highlights
Linear Algebra Highlights Chapter 1 A linear equation in n variables is of the form a 1 x 1 + a 2 x 2 + + a n x n. We can have m equations in n variables, a system of linear equations, which we want to
More informationNumerical Linear Algebra
University of Alabama at Birmingham Department of Mathematics Numerical Linear Algebra Lecture Notes for MA 660 (1997 2014) Dr Nikolai Chernov April 2014 Chapter 0 Review of Linear Algebra 0.1 Matrices
More informationMathematical foundations  linear algebra
Mathematical foundations  linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationLecture 2: Linear Algebra Review
EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 26 of the web textbook 1 2.1 Vectors 2.1.1
More informationLecture 3: Linear Algebra Review, Part II
Lecture 3: Linear Algebra Review, Part II Brian Borchers January 4, Linear Independence Definition The vectors v, v,..., v n are linearly independent if the system of equations c v + c v +...+ c n v n
More information5.) For each of the given sets of vectors, determine whether or not the set spans R 3. Give reasons for your answers.
Linear Algebra  Test File  Spring Test # For problems  consider the following system of equations. x + y  z = x + y + 4z = x + y + 6z =.) Solve the system without using your calculator..) Find the
More informationChapter 4 Euclid Space
Chapter 4 Euclid Space Inner Product Spaces Definition.. Let V be a real vector space over IR. A real inner product on V is a real valued function on V V, denoted by (, ), which satisfies () (x, y) = (y,
More informationOctober 25, 2013 INNER PRODUCT SPACES
October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal
More informationMath Introduction to Numerical Analysis  Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620  Introduction to Numerical Analysis  Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More informationv = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :
Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =
More informationAN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES
AN ELEMENTARY PROOF OF THE SPECTRAL RADIUS FORMULA FOR MATRICES JOEL A. TROPP Abstract. We present an elementary proof that the spectral radius of a matrix A may be obtained using the formula ρ(a) lim
More information1 Linear Algebra Problems
Linear Algebra Problems. Let A be the conjugate transpose of the complex matrix A; i.e., A = A t : A is said to be Hermitian if A = A; real symmetric if A is real and A t = A; skewhermitian if A = A and
More informationy 2 . = x 1y 1 + x 2 y x + + x n y n 2 7 = 1(2) + 3(7) 5(4) = 3. x x = x x x2 n.
6.. Length, Angle, and Orthogonality In this section, we discuss the defintion of length and angle for vectors and define what it means for two vectors to be orthogonal. Then, we see that linear systems
More informationLecture 1: Review of linear algebra
Lecture 1: Review of linear algebra Linear functions and linearization Inverse matrix, leastsquares and leastnorm solutions Subspaces, basis, and dimension Change of basis and similarity transformations
More informationMath Linear Algebra Final Exam Review Sheet
Math 151 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a componentwise operation. Two vectors v and w may be added together as long as they contain the same number n of
More informationLecture Notes for InfMat 3350/4350, Tom Lyche
Lecture Notes for InfMat 3350/4350, 2007 Tom Lyche August 5, 2007 2 Contents Preface vii I A Review of Linear Algebra 1 1 Introduction 3 1.1 Notation............................... 3 2 Vectors 5 2.1 Vector
More informationInner products and Norms. Inner product of 2 vectors. Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y x n y n in R n
Inner products and Norms Inner product of 2 vectors Inner product of 2 vectors x and y in R n : x 1 y 1 + x 2 y 2 + + x n y n in R n Notation: (x, y) or y T x For complex vectors (x, y) = x 1 ȳ 1 + x 2
More informationSection 3.9. Matrix Norm
3.9. Matrix Norm 1 Section 3.9. Matrix Norm Note. We define several matrix norms, some similar to vector norms and some reflecting how multiplication by a matrix affects the norm of a vector. We use matrix
More informationQuantum Computing Lecture 2. Review of Linear Algebra
Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces
More informationSUMMARY OF MATH 1600
SUMMARY OF MATH 1600 Note: The following list is intended as a study guide for the final exam. It is a continuation of the study guide for the midterm. It does not claim to be a comprehensive list. You
More informationMath 18, Linear Algebra, Lecture C00, Spring 2017 Review and Practice Problems for Final Exam
Math 8, Linear Algebra, Lecture C, Spring 7 Review and Practice Problems for Final Exam. The augmentedmatrix of a linear system has been transformed by row operations into 5 4 8. Determine if the system
More informationLinear Algebra. Paul Yiu. Department of Mathematics Florida Atlantic University. Fall A: Inner products
Linear Algebra Paul Yiu Department of Mathematics Florida Atlantic University Fall 2011 6A: Inner products In this chapter, the field F = R or C. We regard F equipped with a conjugation χ : F F. If F =
More information