Lecture 20: 6.1 Inner Products

Size: px
Start display at page:

Download "Lecture 20: 6.1 Inner Products"

Transcription

1 Lecture 0: 6.1 Inner Products Wei-Ta Chu 011/1/5

2 Definition An inner product on a real vector space V is a function that associates a real number u, v with each pair of vectors u and v in V in such a way that the following axioms are satisfied for all vectors u, v, and w in V and all scalars k. u, v = v, u u + v, w = u, w + v, w ku, v = k u, v u, u 0 and u, u = 0 if and only if u = 0 A real vector space with an inner product is called a real inner product space.

3 Euclidean Inner Product on R n If u = (u 1, u,, u n ) and v = (v 1, v,, v n ) are vectors in R n, then the formula v, u = u v = u 1 v 1 + u v + + u n v n defines v, u to be the Euclidean inner product on R n. Inner products can be used to define norm and distance in a general inner product space. 3

4 Theorem If u and v are vectors in a real inner product space V, and if k is a scalar, then v 0 with equality if and only if v = 0 kv = k v d(u,v) = d(v,u) d(u,v) 0 with equality if and only if u = v 4

5 Euclidean Inner Product vs. Inner Product The Euclidean inner product is the most important inner product on R n. However, there are various applications in which it is desirable to modify the Euclidean inner product by weighting its terms differently. More precisely, if w 1,w,,w n are positive real numbers, and if u=(u 1,u,,u n ) and v=(v 1,v,,v n ) are vectors in R n, then it can be shown which is called weighted Euclidean inner product with weights w 1,w,,w n 5

6 Weighted Euclidean Inner Product Let u = (u 1, u ) and v = (v 1, v ) be vectors in R. Verify that the weighted Euclidean inner product u, v = 3u 1 v 1 + u v satisfies the four product axioms. Solution: Note first that if u and v are interchanged in this equation, the right side remains the same. Therefore, u, v = v, u. If w = (w 1, w ), then u + v, w = 3(u 1 +v 1 )w 1 + (u +v )w = (3u 1 w 1 + u w ) + (3v 1 w 1 + v w )= u, w + v, w which establishes the second axiom. 6

7 Weighted Euclidean Inner Product ku, v =3(ku 1 )v 1 + (ku )v = k(3u 1 v 1 + u v ) = k u, v which establishes the third axiom. v, v = 3v 1 v 1 +v v = 3v 1 + v.obviously, v, v = 3v 1 + v 0. Furthermore, v, v = 3v 1 + v = 0 if and only if v 1 = v = 0, That is, if and only if v = (v 1,v )=0. Thus, the fourth axiom is satisfied. 7

8 Applications of Weighted Euclidean Inner Products Suppose that some physical experiment can produce any of n possible numerical values x 1,x,,x n. We perform m repetitions of the experiment and yield these values with various frequencies; that is, x 1 occurs f 1 times, x occurs f times, and so forth. f 1 +f + +f n =m. Thus, the arithmetic average, or mean, is 8

9 Applications of Weighted Euclidean Inner Products If we let f=(f 1,f,,f n ), x=(x 1,x,,x n ), w 1 =w = =w n =1/m Then this equation can be expressed as the weighted inner product 9

10 Weighted Euclidean Inner Product The norm and distance depend on the inner product used. If the inner product is changed, then the norms and distances between vectors also change. For example, for the vectors u = (1,0) and v = (0,1) in R with the Euclidean inner product, we have u = 1 and d( u, v) = u v (1, 1) ( 1) However, if we change to the weighted Euclidean inner product u, v = 3u 1 v 1 + u v, then we obtain = = 1 = u = u, u 1/ = ( ) 1/ = 3 d( u, v) = u v = (1, 1),(1, 1) 1/ = [ ( 1) ( 1)] 1/ = 5 10

11 Definition If V is an inner product space, then the norm (or length) of a vector u in V is denoted by u and is defined by u = u, u ½ The distance between two points (vectors) u and v is denoted by d(u,v) and is defined by d(u, v) = u v If a vector has norm 1, then we say that it is a unit vector. 11

12 Unit Circles and Spheres in Inner Product Space If V is an inner product space, then the set of points in V that satisfy u = 1 is called the unite sphere or sometimes the unit circle in V. In R and R 3 these are the points that lie 1 unit away form the origin. 1

13 Unit Circles in R Sketch the unit circle in an xy-coordinate system in R using the Euclidean inner product u, v = u 1 v 1 + u v Sketch the unit circle in an xy-coordinate system in R using the Euclidean inner product u, v = 1/9u 1 v 1 + 1/4u v Solution If u = (x,y), then u = u, u ½ = (x + y ) ½, so the equation of the unit circle is x + y = 1. If u = (x,y), then u = u, u ½ = (1/9x + 1/4y ) ½, so the equation of the unit circle is x /9 + y /4 = 1. 13

14 Inner Products Generated by Matrices u1 v1 u v Let u = and v= be vectors in R n (expressed as n 1 : : un vn matrices), and let A be an invertible n n matrix. If u v is the Euclidean inner product on R n, then the formula u, v = Au Av defines an inner product; it is called the inner product on R n generated by A. Recalling that the Euclidean inner product u v can be written as the matrix product v T u, the above formula can be written in the alternative form u, v = (Av) T Au, or equivalently, u, v = v T A T Au 14

15 Inner Product Generated by the Identity Matrix The inner product on R n generated by the n n identity matrix is the Euclidean inner product: Let A = I, we have u, v = Iu Iv = u v The weighted Euclidean inner product u, v = 3u 1 v 1 + u v is the inner product on R generated by 15 the inner product on R generated by since A = [ ] [ ] , v u v u u u v v u u v v v u + = = =

16 Inner Product Generated by the Identity Matrix In general, the weighted Euclidean inner product u, v = w 1 u 1 v 1 + w u v + + w n u n v n is the inner product on R n generated by 16

17 An Inner Product on M If 1 1 U = u u and V v v u u = v v are any two matrices, then U, V = tr(u T V) = tr(v T U) = u 1 v 1 + u v + u 3 v 3 + u 4 v 4 defines an inner product on M For example, if U = and V 3 4 = 3 then U, V = 1(-1)+(0) + 3(3) + 4() = 16 The norm of a matrix U relative to this inner product is U =< U, U > = u + u + u + u 1/ and the unit sphere in this space consists of all matrices U whose entries satisfy the equation U = 1, which on squaring yields u 1 + u + u 3 + u 4 = 1 17

18 An Inner Product on P If p = a 0 + a 1 x + a x and q = b 0 + b 1 x + b x are any two vectors in P, then the following formula defines an inner product on P : p, q = a 0 b 0 + a 1 b 1 + a b The norm of the polynomial p relative to this inner product is 1/ p =< p, p > = a + a + a 0 1 and the unit sphere in this space consists of all polynomials p in P whose coefficients satisfy the equation p = 1, which on squaring yields a 0 + a 1 + a =1 18

19 Theorem 6.1. (Properties of Inner Products) If u, v, and w are vectors in a real inner product space, and k is any scalar, then: 0, v = v, 0 = 0 u, v + w = u, v + u, w u, kv =k u, v u v, w = u, w v, w u, v w = u, v u, w 19

20 Example u v, 3u + 4v = u, 3u + 4v v, 3u+4v = u, 3u + u, 4v v, 3u v, 4v = 3 u, u + 4 u, v 6 v, u 8 v, v = 3 u + 4 u, v 6 u, v 8 v = 3 u u, v 8 v 0

21 Example We are guaranteed without any further proof that the five properties given in Theorem 6.1. are true for the inner product on R n generated by any matrix A. u, v + w = (v+w) T A T Au =(v T +w T )A T Au =(v T A T Au) + (w T A T Au) = u, v + u, w [Property of transpose] [Property of matrix multiplication] 1

22 Lecture 0: 6. Angle and Orthogonality Wei-Ta Chu 011/1/5

23 Theorem 6..1 Theorem 6..1 (Cauchy-Schwarz Inequality) If u and v are vectors in a real inner product space, then u, v u v The inequality can be written in the following two forms The Cauchy-Schwarz inequality for R n follows as a special case of this theorem by taking u, v to be the Euclidean inner product u v. 3

24 Angle Between Vectors Cauchy-Schwarz inequality can be used to define angles in general inner product cases. We define to be the angle between u and v. 4

25 Example Let R 4 have the Euclidean inner product. Find the cosine of the angle θ between the vectors u = (4, 3, 1, -) and v = (-, 1,, 3). 5

26 Theorem 6.. Theorem 6.. (Properties of Length) If u, v and w are vectors in an inner product space V, and if k is any scalar, then : u + v u + v (Triangle inequality for vectors) d(u,v) d(u,w) + d(w,v) (Triangle inequality for distances) Proof of (a) (Property of absolute value) (Theorem 6..1) 6

27 Orthogonality Definition Two vectors u and v in an inner product space are called orthogonal if u, v = 0. Example ( U, V = tr(u T V) = tr(v T U) = u 1 v 1 + u v + u 3 v 3 + u 4 v 4 ) If M has the inner project defined previously, then the matrices U 1 = and V 0 = 0 are orthogonal, since U, V = 1(0) + 0() + 1(0) + 1(0) =

28 Orthogonal Vectors in P Let P have the inner product < p, q >= p( x) q( x) dx and let p = x and q = x. 1 Then p q 1 1/ 1 1/ 1/ =< p, p > = xxdx = x dx = / 1 1/ 1/ 4 =< q, q > = x x dx = x dx = < p, q >= xx dx = x dx = because p, q = 0, the vectors p = x and q = x are orthogonal relative to the given inner product

29 Theorem 6..3 (Generalized Theorem of Pythagoras) If u and v are orthogonal vectors in an inner product space, then Proof u + v = u + v 9

30 Orthogonality Definition Let W be a subspace of an inner product space V. A vector u in V is said to be orthogonal to W if it is orthogonal to every vector in W, and the set of all vectors in V that are orthogonal to W is called the orthogonal complement ( 正交 補餘 ) of W. If V is a plane through the origin of R 3 with Euclidean inner product, then the set of all vectors that are orthogonal to every vector in V forms the line L through the origin that is perpendicular to V. L V 30

31 Theorem 6..4 Theorem 6..4 (Properties of Orthogonal Complements) If W is a subspace of a finite-dimensional inner product space V, then: W is a subspace of V. (read W perp ) The only vector common to W and W is 0; that is,w W = 0. 31

32 Proof of Theorem 6..4(a) Note first that 0, w =0 for every vector w in W, so W contains at least the zero vector. We want to show that the sum of two vectors in W is orthogonal to every vector in W (closed under addition) and that any scalar multiple of a vector in W is orthogonal to every vector in W (closed under scalar multiplication). 3

33 Proof of Theorem 6..4(a) Let u and v be any vector in W, let k be any scalar, and let w be any vector in W. Then from the definition of W, we have u, w =0 and v, w =0. Using the basic properties of the inner product, we have u+v, w = u, w + v, w = = 0 ku, w = k u, w = k(0) = 0 Which proves that u+v and ku are in W. 33

34 Proof of Theorem 6..4(b) The only vector common to W and W is 0; that is,w W = 0. If v is common to W and W, then v, v =0, which implies that v=0 by Axiom 4 for inner products. 34

35 Theorem 6..5 Theorem 6..5 If W is a subspace of a finite-dimensional inner product space V, then the orthogonal complement of W is W; that is (W ) = W 35

36 Example (Basis for an Orthogonal Complement) Let W be the subspace of R 6 spanned by the vectors w 1 =(1, 3, -, 0,, 0), w =(, 6, -5, -, 4, -3), w 3 =(0, 0, 5, 10, 0, 15), w 4 =(, 6, 0, 8, 4, 18). Find a basis for the orthogonal complement of W. Solution The space W spanned by w 1, w, w 3, and w 4 is the same as the row space of the matrix Since the row space and null space of A are orthogonal complements, this problem reduces to finding a basis for the null space of A. form a basis for this null space. 36

Lecture 23: 6.1 Inner Products

Lecture 23: 6.1 Inner Products Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such

More information

LINEAR ALGEBRA W W L CHEN

LINEAR ALGEBRA W W L CHEN LINEAR ALGEBRA W W L CHEN c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied,

More information

Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning:

Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u u 2 2 = u 2. Geometric Meaning: Recall: Dot product on R 2 : u v = (u 1, u 2 ) (v 1, v 2 ) = u 1 v 1 + u 2 v 2, u u = u 2 1 + u 2 2 = u 2. Geometric Meaning: u v = u v cos θ. u θ v 1 Reason: The opposite side is given by u v. u v 2 =

More information

INNER PRODUCT SPACE. Definition 1

INNER PRODUCT SPACE. Definition 1 INNER PRODUCT SPACE Definition 1 Suppose u, v and w are all vectors in vector space V and c is any scalar. An inner product space on the vectors space V is a function that associates with each pair of

More information

ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6]

ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] ORTHOGONALITY AND LEAST-SQUARES [CHAP. 6] Inner products and Norms Inner product or dot product of 2 vectors u and v in R n : u.v = u 1 v 1 + u 2 v 2 + + u n v n Calculate u.v when u = 1 2 2 0 v = 1 0

More information

Inner Product Spaces 5.2 Inner product spaces

Inner Product Spaces 5.2 Inner product spaces Inner Product Spaces 5.2 Inner product spaces November 15 Goals Concept of length, distance, and angle in R 2 or R n is extended to abstract vector spaces V. Sucn a vector space will be called an Inner

More information

Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold:

Inner products. Theorem (basic properties): Given vectors u, v, w in an inner product space V, and a scalar k, the following properties hold: Inner products Definition: An inner product on a real vector space V is an operation (function) that assigns to each pair of vectors ( u, v) in V a scalar u, v satisfying the following axioms: 1. u, v

More information

6 Inner Product Spaces

6 Inner Product Spaces Lectures 16,17,18 6 Inner Product Spaces 6.1 Basic Definition Parallelogram law, the ability to measure angle between two vectors and in particular, the concept of perpendicularity make the euclidean space

More information

MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products.

MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products. MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products. Orthogonal projection Theorem 1 Let V be a subspace of R n. Then any vector x R n is uniquely represented

More information

Chapter 6. Orthogonality and Least Squares

Chapter 6. Orthogonality and Least Squares Chapter 6 Orthogonality and Least Squares Section 6.1 Inner Product, Length, and Orthogonality Orientation Recall: This course is about learning to: Solve the matrix equation Ax = b Solve the matrix equation

More information

MTH 2032 SemesterII

MTH 2032 SemesterII MTH 202 SemesterII 2010-11 Linear Algebra Worked Examples Dr. Tony Yee Department of Mathematics and Information Technology The Hong Kong Institute of Education December 28, 2011 ii Contents Table of Contents

More information

MATH 304 Linear Algebra Lecture 18: Orthogonal projection (continued). Least squares problems. Normed vector spaces.

MATH 304 Linear Algebra Lecture 18: Orthogonal projection (continued). Least squares problems. Normed vector spaces. MATH 304 Linear Algebra Lecture 18: Orthogonal projection (continued). Least squares problems. Normed vector spaces. Orthogonality Definition 1. Vectors x,y R n are said to be orthogonal (denoted x y)

More information

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n Lectures -2: Linear Algebra Background Almost all linear and nonlinear problems in scientific computation require the use of linear algebra These lectures review basic concepts in a way that has proven

More information

1. General Vector Spaces

1. General Vector Spaces 1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

More information

Inner Product Spaces 6.1 Length and Dot Product in R n

Inner Product Spaces 6.1 Length and Dot Product in R n Inner Product Spaces 6.1 Length and Dot Product in R n Summer 2017 Goals We imitate the concept of length and angle between two vectors in R 2, R 3 to define the same in the n space R n. Main topics are:

More information

Worksheet for Lecture 23 (due December 4) Section 6.1 Inner product, length, and orthogonality

Worksheet for Lecture 23 (due December 4) Section 6.1 Inner product, length, and orthogonality Worksheet for Lecture (due December 4) Name: Section 6 Inner product, length, and orthogonality u Definition Let u = u n product or dot product to be and v = v v n be vectors in R n We define their inner

More information

Math 290, Midterm II-key

Math 290, Midterm II-key Math 290, Midterm II-key Name (Print): (first) Signature: (last) The following rules apply: There are a total of 20 points on this 50 minutes exam. This contains 7 pages (including this cover page) and

More information

Section 6.1. Inner Product, Length, and Orthogonality

Section 6.1. Inner Product, Length, and Orthogonality Section 6. Inner Product, Length, and Orthogonality Orientation Almost solve the equation Ax = b Problem: In the real world, data is imperfect. x v u But due to measurement error, the measured x is not

More information

6.1. Inner Product, Length and Orthogonality

6.1. Inner Product, Length and Orthogonality These are brief notes for the lecture on Friday November 13, and Monday November 1, 2009: they are not complete, but they are a guide to what I want to say on those days. They are guaranteed to be incorrect..1.

More information

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 : Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =

More information

4.3 - Linear Combinations and Independence of Vectors

4.3 - Linear Combinations and Independence of Vectors - Linear Combinations and Independence of Vectors De nitions, Theorems, and Examples De nition 1 A vector v in a vector space V is called a linear combination of the vectors u 1, u,,u k in V if v can be

More information

Dot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem.

Dot Products. K. Behrend. April 3, Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem. Dot Products K. Behrend April 3, 008 Abstract A short review of some basic facts on the dot product. Projections. The spectral theorem. Contents The dot product 3. Length of a vector........................

More information

Applied Linear Algebra in Geoscience Using MATLAB

Applied Linear Algebra in Geoscience Using MATLAB Applied Linear Algebra in Geoscience Using MATLAB Contents Getting Started Creating Arrays Mathematical Operations with Arrays Using Script Files and Managing Data Two-Dimensional Plots Programming in

More information

Lecture 6: Geometry of OLS Estimation of Linear Regession

Lecture 6: Geometry of OLS Estimation of Linear Regession Lecture 6: Geometry of OLS Estimation of Linear Regession Xuexin Wang WISE Oct 2013 1 / 22 Matrix Algebra An n m matrix A is a rectangular array that consists of nm elements arranged in n rows and m columns

More information

x 1 x 2. x 1, x 2,..., x n R. x n

x 1 x 2. x 1, x 2,..., x n R. x n WEEK In general terms, our aim in this first part of the course is to use vector space theory to study the geometry of Euclidean space A good knowledge of the subject matter of the Matrix Applications

More information

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces. Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

More information

Definition 1. A set V is a vector space over the scalar field F {R, C} iff. there are two operations defined on V, called vector addition

Definition 1. A set V is a vector space over the scalar field F {R, C} iff. there are two operations defined on V, called vector addition 6 Vector Spaces with Inned Product Basis and Dimension Section Objective(s): Vector Spaces and Subspaces Linear (In)dependence Basis and Dimension Inner Product 6 Vector Spaces and Subspaces Definition

More information

Inner Product Spaces

Inner Product Spaces Inner Product Spaces Introduction Recall in the lecture on vector spaces that geometric vectors (i.e. vectors in two and three-dimensional Cartesian space have the properties of addition, subtraction,

More information

Chapter 3 Transformations

Chapter 3 Transformations Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases

More information

Math 3191 Applied Linear Algebra

Math 3191 Applied Linear Algebra Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have

More information

Unit 2. Projec.ons and Subspaces

Unit 2. Projec.ons and Subspaces Unit. Projec.ons and Subspaces Linear Algebra and Op.miza.on MSc Bioinforma.cs for Health Sciences Eduardo Eyras Pompeu Fabra University 8-9 hkp://comprna.upf.edu/courses/master_mat/ Inner product (scalar

More information

Typical Problem: Compute.

Typical Problem: Compute. Math 2040 Chapter 6 Orhtogonality and Least Squares 6.1 and some of 6.7: Inner Product, Length and Orthogonality. Definition: If x, y R n, then x y = x 1 y 1 +... + x n y n is the dot product of x and

More information

The Gram-Schmidt Process 1

The Gram-Schmidt Process 1 The Gram-Schmidt Process In this section all vector spaces will be subspaces of some R m. Definition.. Let S = {v...v n } R m. The set S is said to be orthogonal if v v j = whenever i j. If in addition

More information

Systems of Linear Equations

Systems of Linear Equations Systems of Linear Equations Math 108A: August 21, 2008 John Douglas Moore Our goal in these notes is to explain a few facts regarding linear systems of equations not included in the first few chapters

More information

Announcements Wednesday, November 15

Announcements Wednesday, November 15 Announcements Wednesday, November 15 The third midterm is on this Friday, November 17. The exam covers 3.1, 3.2, 5.1, 5.2, 5.3, and 5.5. About half the problems will be conceptual, and the other half computational.

More information

SECTION v 2 x + v 2 y, (5.1)

SECTION v 2 x + v 2 y, (5.1) CHAPTER 5 5.1 Normed Spaces SECTION 5.1 171 REAL AND COMPLEX NORMED, METRIC, AND INNER PRODUCT SPACES So far, our studies have concentrated only on properties of vector spaces that follow from Definition

More information

Linear Algebra. Session 12

Linear Algebra. Session 12 Linear Algebra. Session 12 Dr. Marco A Roque Sol 08/01/2017 Example 12.1 Find the constant function that is the least squares fit to the following data x 0 1 2 3 f(x) 1 0 1 2 Solution c = 1 c = 0 f (x)

More information

Linear Analysis Lecture 5

Linear Analysis Lecture 5 Linear Analysis Lecture 5 Inner Products and V Let dim V < with inner product,. Choose a basis B and let v, w V have coordinates in F n given by x 1. x n and y 1. y n, respectively. Let A F n n be the

More information

Lecture 16: 9.2 Geometry of Linear Operators

Lecture 16: 9.2 Geometry of Linear Operators Lecture 16: 9.2 Geometry of Linear Operators Wei-Ta Chu 2008/11/19 Theorem 9.2.1 If T: R 2 R 2 is multiplication by an invertible matrix A, then the geometric effect of T is the same as an appropriate

More information

Polar Form of a Complex Number (I)

Polar Form of a Complex Number (I) Polar Form of a Complex Number (I) The polar form of a complex number z = x + iy is given by z = r (cos θ + i sin θ) where θ is the angle from the real axes to the vector z and r is the modulus of z, i.e.

More information

Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.

Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set. Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an

More information

Exercise Sheet 1.

Exercise Sheet 1. Exercise Sheet 1 You can download my lecture and exercise sheets at the address http://sami.hust.edu.vn/giang-vien/?name=huynt 1) Let A, B be sets. What does the statement "A is not a subset of B " mean?

More information

Projection Theorem 1

Projection Theorem 1 Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

Lecture Note on Linear Algebra 17. Standard Inner Product

Lecture Note on Linear Algebra 17. Standard Inner Product Lecture Note on Linear Algebra 17. Standard Inner Product Wei-Shi Zheng, wszheng@ieee.org, 2011 November 21, 2011 1 What Do You Learn from This Note Up to this point, the theory we have established can

More information

The Transpose of a Vector

The Transpose of a Vector 8 CHAPTER Vectors The Transpose of a Vector We now consider the transpose of a vector in R n, which is a row vector. For a vector u 1 u. u n the transpose is denoted by u T = [ u 1 u u n ] EXAMPLE -5 Find

More information

(, ) : R n R n R. 1. It is bilinear, meaning it s linear in each argument: that is

(, ) : R n R n R. 1. It is bilinear, meaning it s linear in each argument: that is 17 Inner products Up until now, we have only examined the properties of vectors and matrices in R n. But normally, when we think of R n, we re really thinking of n-dimensional Euclidean space - that is,

More information

4.1 Distance and Length

4.1 Distance and Length Chapter Vector Geometry In this chapter we will look more closely at certain geometric aspects of vectors in R n. We will first develop an intuitive understanding of some basic concepts by looking at vectors

More information

Vector Geometry. Chapter 5

Vector Geometry. Chapter 5 Chapter 5 Vector Geometry In this chapter we will look more closely at certain geometric aspects of vectors in R n. We will first develop an intuitive understanding of some basic concepts by looking at

More information

Lecture 17: Section 4.2

Lecture 17: Section 4.2 Lecture 17: Section 4.2 Shuanglin Shao November 4, 2013 Subspaces We will discuss subspaces of vector spaces. Subspaces Definition. A subset W is a vector space V is called a subspace of V if W is itself

More information

7 Bilinear forms and inner products

7 Bilinear forms and inner products 7 Bilinear forms and inner products Definition 7.1 A bilinear form θ on a vector space V over a field F is a function θ : V V F such that θ(λu+µv,w) = λθ(u,w)+µθ(v,w) θ(u,λv +µw) = λθ(u,v)+µθ(u,w) for

More information

MATRICES ARE SIMILAR TO TRIANGULAR MATRICES

MATRICES ARE SIMILAR TO TRIANGULAR MATRICES MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

Last week we presented the following expression for the angles between two vectors u and v in R n ( ) θ = cos 1 u v

Last week we presented the following expression for the angles between two vectors u and v in R n ( ) θ = cos 1 u v Orthogonality (I) Last week we presented the following expression for the angles between two vectors u and v in R n ( ) θ = cos 1 u v u v which brings us to the fact that θ = π/2 u v = 0. Definition (Orthogonality).

More information

Assignment 1 Math 5341 Linear Algebra Review. Give complete answers to each of the following questions. Show all of your work.

Assignment 1 Math 5341 Linear Algebra Review. Give complete answers to each of the following questions. Show all of your work. Assignment 1 Math 5341 Linear Algebra Review Give complete answers to each of the following questions Show all of your work Note: You might struggle with some of these questions, either because it has

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

Chapter 6: Orthogonality

Chapter 6: Orthogonality Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products

More information

Linear Equations and Vectors

Linear Equations and Vectors Chapter Linear Equations and Vectors Linear Algebra, Fall 6 Matrices and Systems of Linear Equations Figure. Linear Algebra, Fall 6 Figure. Linear Algebra, Fall 6 Figure. Linear Algebra, Fall 6 Unique

More information

Math 3C Lecture 20. John Douglas Moore

Math 3C Lecture 20. John Douglas Moore Math 3C Lecture 20 John Douglas Moore May 18, 2009 TENTATIVE FORMULA I Midterm I: 20% Midterm II: 20% Homework: 10% Quizzes: 10% Final: 40% TENTATIVE FORMULA II Higher of two midterms: 30% Homework: 10%

More information

MATH 235: Inner Product Spaces, Assignment 7

MATH 235: Inner Product Spaces, Assignment 7 MATH 235: Inner Product Spaces, Assignment 7 Hand in questions 3,4,5,6,9, by 9:3 am on Wednesday March 26, 28. Contents Orthogonal Basis for Inner Product Space 2 2 Inner-Product Function Space 2 3 Weighted

More information

j=1 [We will show that the triangle inequality holds for each p-norm in Chapter 3 Section 6.] The 1-norm is A F = tr(a H A).

j=1 [We will show that the triangle inequality holds for each p-norm in Chapter 3 Section 6.] The 1-norm is A F = tr(a H A). Math 344 Lecture #19 3.5 Normed Linear Spaces Definition 3.5.1. A seminorm on a vector space V over F is a map : V R that for all x, y V and for all α F satisfies (i) x 0 (positivity), (ii) αx = α x (scale

More information

Best approximation in the 2-norm

Best approximation in the 2-norm Best approximation in the 2-norm Department of Mathematical Sciences, NTNU september 26th 2012 Vector space A real vector space V is a set with a 0 element and three operations: Addition: x, y V then x

More information

Extra Problems for Math 2050 Linear Algebra I

Extra Problems for Math 2050 Linear Algebra I Extra Problems for Math 5 Linear Algebra I Find the vector AB and illustrate with a picture if A = (,) and B = (,4) Find B, given A = (,4) and [ AB = A = (,4) and [ AB = 8 If possible, express x = 7 as

More information

Algebra II. Paulius Drungilas and Jonas Jankauskas

Algebra II. Paulius Drungilas and Jonas Jankauskas Algebra II Paulius Drungilas and Jonas Jankauskas Contents 1. Quadratic forms 3 What is quadratic form? 3 Change of variables. 3 Equivalence of quadratic forms. 4 Canonical form. 4 Normal form. 7 Positive

More information

Math 302 Outcome Statements Winter 2013

Math 302 Outcome Statements Winter 2013 Math 302 Outcome Statements Winter 2013 1 Rectangular Space Coordinates; Vectors in the Three-Dimensional Space (a) Cartesian coordinates of a point (b) sphere (c) symmetry about a point, a line, and a

More information

Solutions to Selected Questions from Denis Sevee s Vector Geometry. (Updated )

Solutions to Selected Questions from Denis Sevee s Vector Geometry. (Updated ) Solutions to Selected Questions from Denis Sevee s Vector Geometry. (Updated 24--27) Denis Sevee s Vector Geometry notes appear as Chapter 5 in the current custom textbook used at John Abbott College for

More information

Orthogonality and Least Squares

Orthogonality and Least Squares 6 Orthogonality and Least Squares 6.1 INNER PRODUCT, LENGTH, AND ORTHOGONALITY INNER PRODUCT If u and v are vectors in, then we regard u and v as matrices. n 1 n The transpose u T is a 1 n matrix, and

More information

Inner product spaces. Layers of structure:

Inner product spaces. Layers of structure: Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty

More information

Fourier and Wavelet Signal Processing

Fourier and Wavelet Signal Processing Ecole Polytechnique Federale de Lausanne (EPFL) Audio-Visual Communications Laboratory (LCAV) Fourier and Wavelet Signal Processing Martin Vetterli Amina Chebira, Ali Hormati Spring 2011 2/25/2011 1 Outline

More information

10 Orthogonality Orthogonal subspaces

10 Orthogonality Orthogonal subspaces 10 Orthogonality 10.1 Orthogonal subspaces In the plane R 2 we think of the coordinate axes as being orthogonal (perpendicular) to each other. We can express this in terms of vectors by saying that every

More information

Chapter 2. Vectors and Vector Spaces

Chapter 2. Vectors and Vector Spaces 2.1. Operations on Vectors 1 Chapter 2. Vectors and Vector Spaces Section 2.1. Operations on Vectors Note. In this section, we define several arithmetic operations on vectors (especially, vector addition

More information

Announcements Wednesday, November 15

Announcements Wednesday, November 15 3π 4 Announcements Wednesday, November 15 Reviews today: Recitation Style Solve and discuss Practice problems in groups Preparing for the exam tips and strategies It is not mandatory Eduardo at Culc 141,

More information

homogeneous 71 hyperplane 10 hyperplane 34 hyperplane 69 identity map 171 identity map 186 identity map 206 identity matrix 110 identity matrix 45

homogeneous 71 hyperplane 10 hyperplane 34 hyperplane 69 identity map 171 identity map 186 identity map 206 identity matrix 110 identity matrix 45 address 12 adjoint matrix 118 alternating 112 alternating 203 angle 159 angle 33 angle 60 area 120 associative 180 augmented matrix 11 axes 5 Axiom of Choice 153 basis 178 basis 210 basis 74 basis test

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP)

MATH 20F: LINEAR ALGEBRA LECTURE B00 (T. KEMP) MATH 20F: LINEAR ALGEBRA LECTURE B00 (T KEMP) Definition 01 If T (x) = Ax is a linear transformation from R n to R m then Nul (T ) = {x R n : T (x) = 0} = Nul (A) Ran (T ) = {Ax R m : x R n } = {b R m

More information

Review of Some Concepts from Linear Algebra: Part 2

Review of Some Concepts from Linear Algebra: Part 2 Review of Some Concepts from Linear Algebra: Part 2 Department of Mathematics Boise State University January 16, 2019 Math 566 Linear Algebra Review: Part 2 January 16, 2019 1 / 22 Vector spaces A set

More information

Representation of objects. Representation of objects. Transformations. Hierarchy of spaces. Dimensionality reduction

Representation of objects. Representation of objects. Transformations. Hierarchy of spaces. Dimensionality reduction Representation of objects Representation of objects For searching/mining, first need to represent the objects Images/videos: MPEG features Graphs: Matrix Text document: tf/idf vector, bag of words Kernels

More information

Chapter 2. General Vector Spaces. 2.1 Real Vector Spaces

Chapter 2. General Vector Spaces. 2.1 Real Vector Spaces Chapter 2 General Vector Spaces Outline : Real vector spaces Subspaces Linear independence Basis and dimension Row Space, Column Space, and Nullspace 2 Real Vector Spaces 2 Example () Let u and v be vectors

More information

Linear Algebra. 1.1 Introduction to vectors 1.2 Lengths and dot products. January 28th, 2013 Math 301. Monday, January 28, 13

Linear Algebra. 1.1 Introduction to vectors 1.2 Lengths and dot products. January 28th, 2013 Math 301. Monday, January 28, 13 Linear Algebra 1.1 Introduction to vectors 1.2 Lengths and dot products January 28th, 2013 Math 301 Notation for linear systems 12w +4x + 23y +9z =0 2u + v +5w 2x +2y +8z =1 5u + v 6w +2x +4y z =6 8u 4v

More information

Overview. Motivation for the inner product. Question. Definition

Overview. Motivation for the inner product. Question. Definition Overview Last time we studied the evolution of a discrete linear dynamical system, and today we begin the final topic of the course (loosely speaking) Today we ll recall the definition and properties of

More information

Vector Spaces. Commutativity of +: u + v = v + u, u, v, V ; Associativity of +: u + (v + w) = (u + v) + w, u, v, w V ;

Vector Spaces. Commutativity of +: u + v = v + u, u, v, V ; Associativity of +: u + (v + w) = (u + v) + w, u, v, w V ; Vector Spaces A vector space is defined as a set V over a (scalar) field F, together with two binary operations, i.e., vector addition (+) and scalar multiplication ( ), satisfying the following axioms:

More information

Definitions for Quizzes

Definitions for Quizzes Definitions for Quizzes Italicized text (or something close to it) will be given to you. Plain text is (an example of) what you should write as a definition. [Bracketed text will not be given, nor does

More information

Chapter 5 Eigenvalues and Eigenvectors

Chapter 5 Eigenvalues and Eigenvectors Chapter 5 Eigenvalues and Eigenvectors Outline 5.1 Eigenvalues and Eigenvectors 5.2 Diagonalization 5.3 Complex Vector Spaces 2 5.1 Eigenvalues and Eigenvectors Eigenvalue and Eigenvector If A is a n n

More information

Lecture Notes 1: Vector spaces

Lecture Notes 1: Vector spaces Optimization-based data analysis Fall 2017 Lecture Notes 1: Vector spaces In this chapter we review certain basic concepts of linear algebra, highlighting their application to signal processing. 1 Vector

More information

Vector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis.

Vector spaces. DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis. Vector spaces DS-GA 1013 / MATH-GA 2824 Optimization-based Data Analysis http://www.cims.nyu.edu/~cfgranda/pages/obda_fall17/index.html Carlos Fernandez-Granda Vector space Consists of: A set V A scalar

More information

Math 2331 Linear Algebra

Math 2331 Linear Algebra 6.1 Inner Product, Length & Orthogonality Math 2331 Linear Algebra 6.1 Inner Product, Length & Orthogonality Shang-Huan Chiu Department of Mathematics, University of Houston schiu@math.uh.edu math.uh.edu/

More information

LINEAR ALGEBRA - CHAPTER 1: VECTORS

LINEAR ALGEBRA - CHAPTER 1: VECTORS LINEAR ALGEBRA - CHAPTER 1: VECTORS A game to introduce Linear Algebra In measurement, there are many quantities whose description entirely rely on magnitude, i.e., length, area, volume, mass and temperature.

More information

Study Guide for Linear Algebra Exam 2

Study Guide for Linear Algebra Exam 2 Study Guide for Linear Algebra Exam 2 Term Vector Space Definition A Vector Space is a nonempty set V of objects, on which are defined two operations, called addition and multiplication by scalars (real

More information

7. Dimension and Structure.

7. Dimension and Structure. 7. Dimension and Structure 7.1. Basis and Dimension Bases for Subspaces Example 2 The standard unit vectors e 1, e 2,, e n are linearly independent, for if we write (2) in component form, then we obtain

More information

Lecture 11: Vector space and subspace

Lecture 11: Vector space and subspace Lecture : Vector space and subspace Vector space. R n space Definition.. The space R n consists of all column vector v with n real components, i.e. R n = { v : v = [v,v 2,...,v n ] T, v j R,j =,2,...,n

More information

Vectors. Vectors and the scalar multiplication and vector addition operations:

Vectors. Vectors and the scalar multiplication and vector addition operations: Vectors Vectors and the scalar multiplication and vector addition operations: x 1 x 1 y 1 2x 1 + 3y 1 x x n 1 = 2 x R n, 2 2 y + 3 2 2x = 2 + 3y 2............ x n x n y n 2x n + 3y n I ll use the two terms

More information

Inner Product and Orthogonality

Inner Product and Orthogonality Inner Product and Orthogonality P. Sam Johnson October 3, 2014 P. Sam Johnson (NITK) Inner Product and Orthogonality October 3, 2014 1 / 37 Overview In the Euclidean space R 2 and R 3 there are two concepts,

More information

Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence)

Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence) David Glickenstein December 7, 2015 1 Inner product spaces In this chapter, we will only consider the elds R and C. De nition 1 Let V be a vector

More information

Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014

Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Linear Algebra A Brief Reminder Purpose. The purpose of this document

More information

MAT 419 Lecture Notes Transcribed by Eowyn Cenek 6/1/2012

MAT 419 Lecture Notes Transcribed by Eowyn Cenek 6/1/2012 (Homework 1: Chapter 1: Exercises 1-7, 9, 11, 19, due Monday June 11th See also the course website for lectures, assignments, etc) Note: today s lecture is primarily about definitions Lots of definitions

More information

Inner Product, Length, and Orthogonality

Inner Product, Length, and Orthogonality Inner Product, Length, and Orthogonality Linear Algebra MATH 2076 Linear Algebra,, Chapter 6, Section 1 1 / 13 Algebraic Definition for Dot Product u 1 v 1 u 2 Let u =., v = v 2. be vectors in Rn. The

More information

MATH Linear Algebra

MATH Linear Algebra MATH 304 - Linear Algebra In the previous note we learned an important algorithm to produce orthogonal sequences of vectors called the Gramm-Schmidt orthogonalization process. Gramm-Schmidt orthogonalization

More information

Outline. Linear Algebra for Computer Vision

Outline. Linear Algebra for Computer Vision Outline Linear Algebra for Computer Vision Introduction CMSC 88 D Notation and Basics Motivation Linear systems of equations Gauss Elimination, LU decomposition Linear Spaces and Operators Addition, scalar

More information

March 27 Math 3260 sec. 56 Spring 2018

March 27 Math 3260 sec. 56 Spring 2018 March 27 Math 3260 sec. 56 Spring 2018 Section 4.6: Rank Definition: The row space, denoted Row A, of an m n matrix A is the subspace of R n spanned by the rows of A. We now have three vector spaces associated

More information

Chapter 4 Euclid Space

Chapter 4 Euclid Space Chapter 4 Euclid Space Inner Product Spaces Definition.. Let V be a real vector space over IR. A real inner product on V is a real valued function on V V, denoted by (, ), which satisfies () (x, y) = (y,

More information