Mathematical Preliminaries and Review

Size: px
Start display at page:

Download "Mathematical Preliminaries and Review"

Transcription

1 Mathematical Preliminaries and Review Ruye Wang for E59 1 Complex Variables and Sinusoids AcomplexnumbercanberepresentedineitherCartesianorpolarcoordinatesystem: z = x + jy= z z = re jθ = r(cos θ + j sin θ) where Re[z] =x = r cos θ Im[z] =y = r sin θ, he complex conjugate of z = x + jy is defined as: r = z = x2 + y 2 θ = z =tan 1 y/x z =(x + jy) = x jy =(re jθ ) = re jθ Arithmetic operations of complex variables (vectors in complex space): z + z =(x + jy)+(x jy)=2x =2Re [z] z z =(x + jy) (x jy)=2jy =2jIm[z] zz =(x + jy)(x jy)=x 2 + y 2 = z 2 z 1 ± z 2 =(x 1 + jy 1 ) ± (x 2 + jy 2 )=(x 1 ± x 2 )+j(y 1 ± y 2 ) z 1 z 2 = r 1 e jθ 1 r 2 e jθ 2 = r 1 r 2 e j(θ 1+θ 2 ) z 1 z 2 = r 1 e jθ1 r 2 e jθ 2 = r 1 r 2 e j(θ 1 θ 2 ) z 1/n =(re j(θ+2kπ) ) 1/n = r 1/n e j(θ/n+2kπ/n), (k =0, 1,,n 1) Some important values on the complex plane: e j2kπ =1, e j(2kπ+π/2) = j, e j(2kπ±π) = 1, e j(2kπ π/2) = j, (k =0, ±1, ±2, ) Euler s Formula: e jθ =cosθ + j sin θ e jθ =cosθ j sin θ, cos θ =(e jθ + e jθ )/2 sin θ =(e jθ e jθ )/2j 1

2 Asinusoidaltimefunctioncanberepresentedineitherofthetwoforms: he real and imaginary parts (horizontal and vertical projections)of a complex exponential(a vector rotating in CCW direction) represent a cosine and sine function, respectively: Re[Ae j(ωt+ϕ) ]=Re[A cos(ωt + ϕ)+jasin(ωt + ϕ)] = A cos(ωt + ϕ) Im[Ae j(ωt+ϕ) ]=Im[A cos(ωt + ϕ)+jasin(ωt + ϕ)] = A sin(ωt + ϕ) he rate of the rotation is determined by the angular frequency ω he vector sum of two complex exponentials for two vectors rotating in CW and CCW directions, corresponding to a positive frequency ω and a negative one ω, respectively: A cos(ωt + ϕ) = A 2 [ej(ωt+ϕ) + e j(ωt+ϕ) ] Similarly, a sine function can also be obtained as a vector sum: jacos(ωt + ϕ) = A 2 [ej(ωt+ϕ) e j(ωt+ϕ) ], ie A cos(ωt + ϕ) = A 2j [ej(ωt+ϕ) e j(ωt+ϕ) ] 2 Linear Algebra Vectors and matrices: An n-d vector is a column vector and its transpose is a row vector: x = x 1 x n, x =[x 1,,x n ] An m by n matrix is an array of m rows and n columns, which can also be represented in terms of its column (or row) vectors: a 11 a 1n A m n = =[a 1,, a n ] a m1 a mn 2

3 where a j is the jth column (j =1,,n): a j = a 1j a mj he transpose of A is: a 11 a 1n A = a m1 a mn a 11 a m1 = a 1n a nm =[a 1,, a n ] = a 1 a n he inverse A 1 of A satisfies: where I is the identity or unit matrix Inner (dot) product: A 1 A = AA 1 = I he inner (dot) product of two vectors x and y is a scaler defined as: < x, y >= x y =[x 1,,x n ] he following concepts are defined based on inner product: he norm (or length) of a vector is defined as Avectorx is normalized if x =1 y 1 y n n = x j y j x =< x, x > 1/2 n = x j x n j = x j 2 wo vectors x and y are orthogonal if < x, y >= 0 wo orthogonal vectors are orthonormal if they are both normalized he angle between two vectors is defined as: θ =cos 1 ( < x, y > ), ie, < x, y >= x y cos θ x y he scaler projection of x on y is defined as: P y (x) = < x, y > y = x cos θ, if y =1,then P y (x) =< x, y > (he projection of x on y could also be defined as a vector in the direction of y, whichisnot considered here) 3

4 Vector and matrix multiplication: An n-d column vector x n can be pre-multiplied by an matrix A m n to result an m-d column vector y m : y m = A m n x n which can be represented in element form as: y 1 y m and the ith element is (i =1,,m): = a 11 a 1n x 1 a m1 a mn x n n y i = a ij x j Given y and A, this linear system can be solved for x (by pre-multiplying both sides of the equation by A 1 ): A 1 y = A 1 Ax = x, ie x = A 1 y he product of two matrices A m k and B k n is also a matrix: C m n =[c 1,, c n ]=A m k B k n = A[b 1,, b n ] where the jth column is: c j = Ab j, ie, c 1j = a 11 a 1k b 1j c mj a m1 a mk b kj and its ith element is: k c ij = a il b lj, l=1 (i =1,,m) 4

5 Orthogonal and unitary matrices Some special matrices of interest are defined below: Ais symmetric if A = A Ais Hermitian if A = A, wherea is the conjugate transpose of A IfA = A is real, it is symmetric Ais orthogonal if A = A 1,ie,A A = AA = I Ais unitary if A = A 1 IfA = A is real, it is orthogonal All column (or row) vectors of a unitary matrix A =[a 1,, a n ]areorthonormal: 1 i = j < a i, a j >= a i a j = δ[i j] = 0 i j where δ[n] is the delta function defined as: δ[n] = 3 Inner Product (Hilbert) Space Euclidean space: 1 n =0 0 n 0 An n-d inner product vector space, called a Euclidean space, which is a set of all n-d vectors with inner product defined his space can be spanned by a set of n linearly independent basis vectors b 1,, b n } (none of them can be represented as a linear combination of the rest), so that any vector x in the space can be expressed as a linear combination of these basis vectors: c 1 b 11 b 1n c 1 n x = k=1 c k b k = c 1 b c n b n =[b 1,, b n ] c n = b n1 b nn c n = Bc where B = [b 1,, b n ]isannbynmatrixwiththenbasisvectorsasitscolumns,andc = [c 1,,c n ] is a column vector composed of n coefficients or weights for the basis vector hese coefficients can be obtained by solving this linear system (by pre-multiplying B 1 on both sides): c = B 1 x In particular, if the basis vectors are orthonormal: < b i, b j >= b i b j = δ[i j] then B becomes: and the kth element = B 1 is a unitary matrix (or orthogonal if B = B is real), and the equation above c = c 1 c n = B 1 x = B x = b 1 b n c k =< x, b k >= x b k = P bk (x) is the projection of vector x onto the kth basis vector b k 5 x

6 Any set of n linearly independent vectors can be used as a basis of an n-d vector space Any given basis of n vectors can be converted in to a set of orthogonal basis vectors by Gram- Schmidt process Any rotation of an orthogonal basis results another orthogonal basis, ie, any two different sets of orthogonal bases are related by a rotation corresponding to an orthogonal matrix All discussions above for n-d Euclidean space can be generalized to an infinite dimensional vector space by letting n Function space: he concept of n-d vector space can be generalized to n-d function space, which a set of all functions x(t) definedoveraparticularrange0 <t< with the inner product defined as: <x(t),y(t) >= x(t)y (t)dt, (compared to < x, y >= i x i yi ) he norm of a function is defined as: x(t) =< x(t),x(t) > 1/2 = x(t)x (t)dt = Afunctionx(t) isnormalizedifithasunitynorm x(t) =1 wo functions x(t) andy(t) areorthogonaliftheinnerproductiszero: <x(t),y(t) >= x(t)y (t)dt =0 x(t) 2 dt his function space can also be spanned by a set of basis functions b k (t) sothatanygivenfunction x(t) inthespacecanbeexpressedasalinearcombinationofthesebasisfunctions: x(t) = k c k b k (t) In particular, if the basis functions are orthogonal: <b k (t),b l (t) >= b k (t)b l (t)dt =0 (k l) then the coefficient c k can be obtained by taking an inner product with b l (t) onbothsidesofthe equation above: <x(t),b l (t) >=< k c k b k (t), b l (t) >= k c k <b k (t), b l (t) >= c l <b l (t),b l (t) >= c l b l (t) 2 he kth coefficient becomes: c k = <x(t),b k(t) > <b k (t),b k (t) > = x(t)b k (t)dt b k (t) 2 Moreover, if the basis functions are orthonormal (orthogonal and normalized) b k (t) =1,thenwe get c k =<x(t),b k (t) >= x(t)b k (t)dt ie, c k is the projection of x(t) ontothekthbasisfunctionb k (t) 6

7 4 Miscellaneous Integration of these expressions: sin 2 (at)dt, te t dt, t sin(at)dt Solution of 1st and 2nd-order ordinary differential equations: ẏ(t)+ 1 y(t) =af(t), τ ÿ(t)+2ζω nẏ(t)+ωn 2 y(t) =af(t) where τ, ζ, ω n and a are real constants, and the input f(t) canbe: f(t) =0[freeresponse]; f(t) =u(t) [unitstepresponse]; f(t) = sinωt [sinusoidal response]; f(t) =e jωt [harmonic (complex exponential) response]; Geometric series: Finite: Infinite ( x < 1): rigonometry identities: n 1 k=0 x k =1+x + x x n 1 = 1 xn 1 x k=0 x k =1+x + x 2 + = 1 1 x sin(α ± β) =sinα cos β ± cos α sin β cos(α ± β) =cosα cos β sin α sin β sin 2 α =(1 cos 2α)/2 cos 2 α =(1+cos2α)/2 sin 2α =2sinα cos α cos 2α =cos 2 α sin 2 α sin(α/2) = ± (1 cos α)/2 cos(α/2) = ± (1 + cos α)/2 7

CMPT 318: Lecture 5 Complex Exponentials, Spectrum Representation

CMPT 318: Lecture 5 Complex Exponentials, Spectrum Representation CMPT 318: Lecture 5 Complex Exponentials, Spectrum Representation Tamara Smyth, tamaras@cs.sfu.ca School of Computing Science, Simon Fraser University January 23, 2006 1 Exponentials The exponential is

More information

Music 270a: Complex Exponentials and Spectrum Representation

Music 270a: Complex Exponentials and Spectrum Representation Music 270a: Complex Exponentials and Spectrum Representation Tamara Smyth, trsmyth@ucsd.edu Department of Music, University of California, San Diego (UCSD) October 24, 2016 1 Exponentials The exponential

More information

CMPT 889: Lecture 2 Sinusoids, Complex Exponentials, Spectrum Representation

CMPT 889: Lecture 2 Sinusoids, Complex Exponentials, Spectrum Representation CMPT 889: Lecture 2 Sinusoids, Complex Exponentials, Spectrum Representation Tamara Smyth, tamaras@cs.sfu.ca School of Computing Science, Simon Fraser University September 26, 2005 1 Sinusoids Sinusoids

More information

Linear Algebra Review. Vectors

Linear Algebra Review. Vectors Linear Algebra Review 9/4/7 Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa (UCSD) Cogsci 8F Linear Algebra review Vectors

More information

Review of Linear Algebra

Review of Linear Algebra Review of Linear Algebra Definitions An m n (read "m by n") matrix, is a rectangular array of entries, where m is the number of rows and n the number of columns. 2 Definitions (Con t) A is square if m=

More information

5. Orthogonal matrices

5. Orthogonal matrices L Vandenberghe EE133A (Spring 2017) 5 Orthogonal matrices matrices with orthonormal columns orthogonal matrices tall matrices with orthonormal columns complex matrices with orthonormal columns 5-1 Orthonormal

More information

Sinusoids. Amplitude and Magnitude. Phase and Period. CMPT 889: Lecture 2 Sinusoids, Complex Exponentials, Spectrum Representation

Sinusoids. Amplitude and Magnitude. Phase and Period. CMPT 889: Lecture 2 Sinusoids, Complex Exponentials, Spectrum Representation Sinusoids CMPT 889: Lecture Sinusoids, Complex Exponentials, Spectrum Representation Tamara Smyth, tamaras@cs.sfu.ca School of Computing Science, Simon Fraser University September 6, 005 Sinusoids are

More information

Fourier Series. Spectral Analysis of Periodic Signals

Fourier Series. Spectral Analysis of Periodic Signals Fourier Series. Spectral Analysis of Periodic Signals he response of continuous-time linear invariant systems to the complex exponential with unitary magnitude response of a continuous-time LI system at

More information

Introduction to Signal Spaces

Introduction to Signal Spaces Introduction to Signal Spaces Selin Aviyente Department of Electrical and Computer Engineering Michigan State University January 12, 2010 Motivation Outline 1 Motivation 2 Vector Space 3 Inner Product

More information

Linear Algebra V = T = ( 4 3 ).

Linear Algebra V = T = ( 4 3 ). Linear Algebra Vectors A column vector is a list of numbers stored vertically The dimension of a column vector is the number of values in the vector W is a -dimensional column vector and V is a 5-dimensional

More information

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors.

MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. MATH 304 Linear Algebra Lecture 20: The Gram-Schmidt process (continued). Eigenvalues and eigenvectors. Orthogonal sets Let V be a vector space with an inner product. Definition. Nonzero vectors v 1,v

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

Introduction to Matrix Algebra

Introduction to Matrix Algebra Introduction to Matrix Algebra August 18, 2010 1 Vectors 1.1 Notations A p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the line. When p

More information

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p

Math Bootcamp An p-dimensional vector is p numbers put together. Written as. x 1 x =. x p Math Bootcamp 2012 1 Review of matrix algebra 1.1 Vectors and rules of operations An p-dimensional vector is p numbers put together. Written as x 1 x =. x p. When p = 1, this represents a point in the

More information

Linear Algebra II. 7 Inner product spaces. Notes 7 16th December Inner products and orthonormal bases

Linear Algebra II. 7 Inner product spaces. Notes 7 16th December Inner products and orthonormal bases MTH6140 Linear Algebra II Notes 7 16th December 2010 7 Inner product spaces Ordinary Euclidean space is a 3-dimensional vector space over R, but it is more than that: the extra geometric structure (lengths,

More information

We wish to solve a system of N simultaneous linear algebraic equations for the N unknowns x 1, x 2,...,x N, that are expressed in the general form

We wish to solve a system of N simultaneous linear algebraic equations for the N unknowns x 1, x 2,...,x N, that are expressed in the general form Linear algebra This chapter discusses the solution of sets of linear algebraic equations and defines basic vector/matrix operations The focus is upon elimination methods such as Gaussian elimination, and

More information

Linear algebra. 1.1 Numbers. d n x = 10 n (1.1) n=m x

Linear algebra. 1.1 Numbers. d n x = 10 n (1.1) n=m x 1 Linear algebra 1.1 Numbers The natural numbers are the positive integers and zero. Rational numbers are ratios of integers. Irrational numbers have decimal digits d n d n x = 10 n (1.1 n=m x that do

More information

Mathematical Review for Signal and Systems

Mathematical Review for Signal and Systems Mathematical Review for Signal and Systems 1 Trigonometric Identities It will be useful to memorize sin θ, cos θ, tan θ values for θ = 0, π/3, π/4, π/ and π ±θ, π θ for the above values of θ. The following

More information

2. Review of Linear Algebra

2. Review of Linear Algebra 2. Review of Linear Algebra ECE 83, Spring 217 In this course we will represent signals as vectors and operators (e.g., filters, transforms, etc) as matrices. This lecture reviews basic concepts from linear

More information

which arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i

which arises when we compute the orthogonal projection of a vector y in a subspace with an orthogonal basis. Hence assume that P y = A ij = x j, x i MODULE 6 Topics: Gram-Schmidt orthogonalization process We begin by observing that if the vectors {x j } N are mutually orthogonal in an inner product space V then they are necessarily linearly independent.

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,

More information

CS 246 Review of Linear Algebra 01/17/19

CS 246 Review of Linear Algebra 01/17/19 1 Linear algebra In this section we will discuss vectors and matrices. We denote the (i, j)th entry of a matrix A as A ij, and the ith entry of a vector as v i. 1.1 Vectors and vector operations A vector

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Matrix & Linear Algebra

Matrix & Linear Algebra Matrix & Linear Algebra Jamie Monogan University of Georgia For more information: http://monogan.myweb.uga.edu/teaching/mm/ Jamie Monogan (UGA) Matrix & Linear Algebra 1 / 84 Vectors Vectors Vector: A

More information

Linear Algebra: Matrix Eigenvalue Problems

Linear Algebra: Matrix Eigenvalue Problems CHAPTER8 Linear Algebra: Matrix Eigenvalue Problems Chapter 8 p1 A matrix eigenvalue problem considers the vector equation (1) Ax = λx. 8.0 Linear Algebra: Matrix Eigenvalue Problems Here A is a given

More information

3 What You Should Know About Complex Numbers

3 What You Should Know About Complex Numbers 3 What You Should Know About Complex Numbers Life is complex it has a real part, and an imaginary part Andrew Koenig. Complex numbers are an extension of the more familiar world of real numbers that make

More information

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections )

10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections ) c Dr. Igor Zelenko, Fall 2017 1 10. Linear Systems of ODEs, Matrix multiplication, superposition principle (parts of sections 7.2-7.4) 1. When each of the functions F 1, F 2,..., F n in right-hand side

More information

Lecture 2 INF-MAT : A boundary value problem and an eigenvalue problem; Block Multiplication; Tridiagonal Systems

Lecture 2 INF-MAT : A boundary value problem and an eigenvalue problem; Block Multiplication; Tridiagonal Systems Lecture 2 INF-MAT 4350 2008: A boundary value problem and an eigenvalue problem; Block Multiplication; Tridiagonal Systems Tom Lyche Centre of Mathematics for Applications, Department of Informatics, University

More information

Math Linear Algebra Final Exam Review Sheet

Math Linear Algebra Final Exam Review Sheet Math 15-1 Linear Algebra Final Exam Review Sheet Vector Operations Vector addition is a component-wise operation. Two vectors v and w may be added together as long as they contain the same number n of

More information

Supplementary information I Hilbert Space, Dirac Notation, and Matrix Mechanics. EE270 Fall 2017

Supplementary information I Hilbert Space, Dirac Notation, and Matrix Mechanics. EE270 Fall 2017 Supplementary information I Hilbert Space, Dirac Notation, and Matrix Mechanics Properties of Vector Spaces Unit vectors ~xi form a basis which spans the space and which are orthonormal ( if i = j ~xi

More information

Mathematical foundations - linear algebra

Mathematical foundations - linear algebra Mathematical foundations - linear algebra Andrea Passerini passerini@disi.unitn.it Machine Learning Vector space Definition (over reals) A set X is called a vector space over IR if addition and scalar

More information

Diagonalizing Matrices

Diagonalizing Matrices Diagonalizing Matrices Massoud Malek A A Let A = A k be an n n non-singular matrix and let B = A = [B, B,, B k,, B n ] Then A n A B = A A 0 0 A k [B, B,, B k,, B n ] = 0 0 = I n 0 A n Notice that A i B

More information

4 Linear Algebra Review

4 Linear Algebra Review 4 Linear Algebra Review For this topic we quickly review many key aspects of linear algebra that will be necessary for the remainder of the course 41 Vectors and Matrices For the context of data analysis,

More information

Elementary maths for GMT

Elementary maths for GMT Elementary maths for GMT Linear Algebra Part 2: Matrices, Elimination and Determinant m n matrices The system of m linear equations in n variables x 1, x 2,, x n a 11 x 1 + a 12 x 2 + + a 1n x n = b 1

More information

Matrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix

Matrix Operations. Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Linear Combination Vector Algebra Angle Between Vectors Projections and Reflections Equality of matrices, Augmented Matrix Matrix Operations Matrix Addition and Matrix Scalar Multiply Matrix Multiply Matrix

More information

Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in

Chapter 4 - MATRIX ALGEBRA. ... a 2j... a 2n. a i1 a i2... a ij... a in Chapter 4 - MATRIX ALGEBRA 4.1. Matrix Operations A a 11 a 12... a 1j... a 1n a 21. a 22.... a 2j... a 2n. a i1 a i2... a ij... a in... a m1 a m2... a mj... a mn The entry in the ith row and the jth column

More information

Phasor Young Won Lim 05/19/2015

Phasor Young Won Lim 05/19/2015 Phasor Copyright (c) 2009-2015 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version

More information

Matrix Algebra for Engineers Jeffrey R. Chasnov

Matrix Algebra for Engineers Jeffrey R. Chasnov Matrix Algebra for Engineers Jeffrey R. Chasnov The Hong Kong University of Science and Technology The Hong Kong University of Science and Technology Department of Mathematics Clear Water Bay, Kowloon

More information

Section 3: Complex numbers

Section 3: Complex numbers Essentially: Section 3: Complex numbers C (set of complex numbers) up to different notation: the same as R 2 (euclidean plane), (i) Write the real 1 instead of the first elementary unit vector e 1 = (1,

More information

Vectors. Vectors and the scalar multiplication and vector addition operations:

Vectors. Vectors and the scalar multiplication and vector addition operations: Vectors Vectors and the scalar multiplication and vector addition operations: x 1 x 1 y 1 2x 1 + 3y 1 x x n 1 = 2 x R n, 2 2 y + 3 2 2x = 2 + 3y 2............ x n x n y n 2x n + 3y n I ll use the two terms

More information

A Brief Outline of Math 355

A Brief Outline of Math 355 A Brief Outline of Math 355 Lecture 1 The geometry of linear equations; elimination with matrices A system of m linear equations with n unknowns can be thought of geometrically as m hyperplanes intersecting

More information

j=1 u 1jv 1j. 1/ 2 Lemma 1. An orthogonal set of vectors must be linearly independent.

j=1 u 1jv 1j. 1/ 2 Lemma 1. An orthogonal set of vectors must be linearly independent. Lecture Notes: Orthogonal and Symmetric Matrices Yufei Tao Department of Computer Science and Engineering Chinese University of Hong Kong taoyf@cse.cuhk.edu.hk Orthogonal Matrix Definition. Let u = [u

More information

Linear Algebra March 16, 2019

Linear Algebra March 16, 2019 Linear Algebra March 16, 2019 2 Contents 0.1 Notation................................ 4 1 Systems of linear equations, and matrices 5 1.1 Systems of linear equations..................... 5 1.2 Augmented

More information

Course Notes for Signals and Systems. Krishna R Narayanan

Course Notes for Signals and Systems. Krishna R Narayanan Course Notes for Signals and Systems Krishna R Narayanan May 7, 018 Contents 1 Math Review 5 1.1 Trigonometric Identities............................. 5 1. Complex Numbers................................

More information

Linear Algebra (Review) Volker Tresp 2017

Linear Algebra (Review) Volker Tresp 2017 Linear Algebra (Review) Volker Tresp 2017 1 Vectors k is a scalar (a number) c is a column vector. Thus in two dimensions, c = ( c1 c 2 ) (Advanced: More precisely, a vector is defined in a vector space.

More information

Inner product spaces. Layers of structure:

Inner product spaces. Layers of structure: Inner product spaces Layers of structure: vector space normed linear space inner product space The abstract definition of an inner product, which we will see very shortly, is simple (and by itself is pretty

More information

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian

2. Linear algebra. matrices and vectors. linear equations. range and nullspace of matrices. function of vectors, gradient and Hessian FE661 - Statistical Methods for Financial Engineering 2. Linear algebra Jitkomut Songsiri matrices and vectors linear equations range and nullspace of matrices function of vectors, gradient and Hessian

More information

Rank, Trace, Determinant, Transpose an Inverse of a Matrix Let A be an n n square matrix: A = a11 a1 a1n a1 a an a n1 a n a nn nn where is the jth col

Rank, Trace, Determinant, Transpose an Inverse of a Matrix Let A be an n n square matrix: A = a11 a1 a1n a1 a an a n1 a n a nn nn where is the jth col Review of Linear Algebra { E18 Hanout Vectors an Their Inner Proucts Let X an Y be two vectors: an Their inner prouct is ene as X =[x1; ;x n ] T Y =[y1; ;y n ] T (X; Y ) = X T Y = x k y k k=1 where T an

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 2: Orthogonal Vectors and Matrices; Vector Norms Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 11 Outline 1 Orthogonal

More information

Hilbert Inner Product Space (2B) Young Won Lim 2/7/12

Hilbert Inner Product Space (2B) Young Won Lim 2/7/12 Hilbert nner Product Space (2B) Copyright (c) 2009-2011 Young W. Lim. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version

More information

Matrix Representation

Matrix Representation Matrix Representation Matrix Rep. Same basics as introduced already. Convenient method of working with vectors. Superposition Complete set of vectors can be used to express any other vector. Complete set

More information

Linear Algebra Review (Course Notes for Math 308H - Spring 2016)

Linear Algebra Review (Course Notes for Math 308H - Spring 2016) Linear Algebra Review (Course Notes for Math 308H - Spring 2016) Dr. Michael S. Pilant February 12, 2016 1 Background: We begin with one of the most fundamental notions in R 2, distance. Letting (x 1,

More information

Quantum Computing Lecture 2. Review of Linear Algebra

Quantum Computing Lecture 2. Review of Linear Algebra Quantum Computing Lecture 2 Review of Linear Algebra Maris Ozols Linear algebra States of a quantum system form a vector space and their transformations are described by linear operators Vector spaces

More information

MATH 22A: LINEAR ALGEBRA Chapter 4

MATH 22A: LINEAR ALGEBRA Chapter 4 MATH 22A: LINEAR ALGEBRA Chapter 4 Jesús De Loera, UC Davis November 30, 2012 Orthogonality and Least Squares Approximation QUESTION: Suppose Ax = b has no solution!! Then what to do? Can we find an Approximate

More information

Electric Circuit Theory

Electric Circuit Theory Electric Circuit Theory Nam Ki Min nkmin@korea.ac.kr 010-9419-2320 Chapter 11 Sinusoidal Steady-State Analysis Nam Ki Min nkmin@korea.ac.kr 010-9419-2320 Contents and Objectives 3 Chapter Contents 11.1

More information

Linear Algebra (Review) Volker Tresp 2018

Linear Algebra (Review) Volker Tresp 2018 Linear Algebra (Review) Volker Tresp 2018 1 Vectors k, M, N are scalars A one-dimensional array c is a column vector. Thus in two dimensions, ( ) c1 c = c 2 c i is the i-th component of c c T = (c 1, c

More information

ELE/MCE 503 Linear Algebra Facts Fall 2018

ELE/MCE 503 Linear Algebra Facts Fall 2018 ELE/MCE 503 Linear Algebra Facts Fall 2018 Fact N.1 A set of vectors is linearly independent if and only if none of the vectors in the set can be written as a linear combination of the others. Fact N.2

More information

Lecture 7. Econ August 18

Lecture 7. Econ August 18 Lecture 7 Econ 2001 2015 August 18 Lecture 7 Outline First, the theorem of the maximum, an amazing result about continuity in optimization problems. Then, we start linear algebra, mostly looking at familiar

More information

Knowledge Discovery and Data Mining 1 (VO) ( )

Knowledge Discovery and Data Mining 1 (VO) ( ) Knowledge Discovery and Data Mining 1 (VO) (707.003) Review of Linear Algebra Denis Helic KTI, TU Graz Oct 9, 2014 Denis Helic (KTI, TU Graz) KDDM1 Oct 9, 2014 1 / 74 Big picture: KDDM Probability Theory

More information

3 a 21 a a 2N. 3 a 21 a a 2M

3 a 21 a a 2N. 3 a 21 a a 2M APPENDIX: MATHEMATICS REVIEW G 12.1.1 Matrices and Vectors Definition of Matrix. An MxN matrix A is a two-dimensional array of numbers 2 A = 6 4 a 11 a 12... a 1N a 21 a 22... a 2N. 7..... 5 a M1 a M2...

More information

1. General Vector Spaces

1. General Vector Spaces 1.1. Vector space axioms. 1. General Vector Spaces Definition 1.1. Let V be a nonempty set of objects on which the operations of addition and scalar multiplication are defined. By addition we mean a rule

More information

Tensors, and differential forms - Lecture 2

Tensors, and differential forms - Lecture 2 Tensors, and differential forms - Lecture 2 1 Introduction The concept of a tensor is derived from considering the properties of a function under a transformation of the coordinate system. A description

More information

EDISP (NWL2) (English) Digital Signal Processing Transform, FT, DFT. March 11, 2015

EDISP (NWL2) (English) Digital Signal Processing Transform, FT, DFT. March 11, 2015 EDISP (NWL2) (English) Digital Signal Processing Transform, FT, DFT March 11, 2015 Transform concept We want to analyze the signal represent it as built of some building blocks (well known signals), possibly

More information

Linear Algebra. Min Yan

Linear Algebra. Min Yan Linear Algebra Min Yan January 2, 2018 2 Contents 1 Vector Space 7 1.1 Definition................................. 7 1.1.1 Axioms of Vector Space..................... 7 1.1.2 Consequence of Axiom......................

More information

AMS526: Numerical Analysis I (Numerical Linear Algebra)

AMS526: Numerical Analysis I (Numerical Linear Algebra) AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 7: More on Householder Reflectors; Least Squares Problems Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 15 Outline

More information

Examples and MatLab. Vector and Matrix Material. Matrix Addition R = A + B. Matrix Equality A = B. Matrix Multiplication R = A * B.

Examples and MatLab. Vector and Matrix Material. Matrix Addition R = A + B. Matrix Equality A = B. Matrix Multiplication R = A * B. Vector and Matrix Material Examples and MatLab Matrix = Rectangular array of numbers, complex numbers or functions If r rows & c columns, r*c elements, r*c is order of matrix. A is n * m matrix Square

More information

MATH 135: COMPLEX NUMBERS

MATH 135: COMPLEX NUMBERS MATH 135: COMPLEX NUMBERS (WINTER, 010) The complex numbers C are important in just about every branch of mathematics. These notes 1 present some basic facts about them. 1. The Complex Plane A complex

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

Math Linear Algebra II. 1. Inner Products and Norms

Math Linear Algebra II. 1. Inner Products and Norms Math 342 - Linear Algebra II Notes 1. Inner Products and Norms One knows from a basic introduction to vectors in R n Math 254 at OSU) that the length of a vector x = x 1 x 2... x n ) T R n, denoted x,

More information

Math 4377/6308 Advanced Linear Algebra

Math 4377/6308 Advanced Linear Algebra 2.3 Composition Math 4377/6308 Advanced Linear Algebra 2.3 Composition of Linear Transformations Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/ jiwenhe/math4377

More information

Differential equations

Differential equations Differential equations Math 7 Spring Practice problems for April Exam Problem Use the method of elimination to find the x-component of the general solution of x y = 6x 9x + y = x 6y 9y Soln: The system

More information

Review of similarity transformation and Singular Value Decomposition

Review of similarity transformation and Singular Value Decomposition Review of similarity transformation and Singular Value Decomposition Nasser M Abbasi Applied Mathematics Department, California State University, Fullerton July 8 7 page compiled on June 9, 5 at 9:5pm

More information

MATH 167: APPLIED LINEAR ALGEBRA Least-Squares

MATH 167: APPLIED LINEAR ALGEBRA Least-Squares MATH 167: APPLIED LINEAR ALGEBRA Least-Squares October 30, 2014 Least Squares We do a series of experiments, collecting data. We wish to see patterns!! We expect the output b to be a linear function of

More information

Lecture 4 Orthonormal vectors and QR factorization

Lecture 4 Orthonormal vectors and QR factorization Orthonormal vectors and QR factorization 4 1 Lecture 4 Orthonormal vectors and QR factorization EE263 Autumn 2004 orthonormal vectors Gram-Schmidt procedure, QR factorization orthogonal decomposition induced

More information

Chapter 8: Frequency Domain Analysis

Chapter 8: Frequency Domain Analysis Chapter 8: Frequency Domain Analysis Samantha Ramirez Preview Questions 1. What is the steady-state response of a linear system excited by a cyclic or oscillatory input? 2. How does one characterize the

More information

Review of Linear Time-Invariant Network Analysis

Review of Linear Time-Invariant Network Analysis D1 APPENDIX D Review of Linear Time-Invariant Network Analysis Consider a network with input x(t) and output y(t) as shown in Figure D-1. If an input x 1 (t) produces an output y 1 (t), and an input x

More information

1 Matrices and vector spaces

1 Matrices and vector spaces Matrices and vector spaces. Which of the following statements about linear vector spaces are true? Where a statement is false, give a counter-example to demonstrate this. (a) Non-singular N N matrices

More information

Linear Algebra. Linear Equations and Matrices. Copyright 2005, W.R. Winfrey

Linear Algebra. Linear Equations and Matrices. Copyright 2005, W.R. Winfrey Copyright 2005, W.R. Winfrey Topics Preliminaries Systems of Linear Equations Matrices Algebraic Properties of Matrix Operations Special Types of Matrices and Partitioned Matrices Matrix Transformations

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

EE292: Fundamentals of ECE

EE292: Fundamentals of ECE EE292: Fundamentals of ECE Fall 2012 TTh 10:00-11:15 SEB 1242 Lecture 18 121025 http://www.ee.unlv.edu/~b1morris/ee292/ 2 Outline Review RMS Values Complex Numbers Phasors Complex Impedance Circuit Analysis

More information

Signals and Systems Lecture (S2) Orthogonal Functions and Fourier Series March 17, 2008

Signals and Systems Lecture (S2) Orthogonal Functions and Fourier Series March 17, 2008 Signals and Systems Lecture (S) Orthogonal Functions and Fourier Series March 17, 008 Today s Topics 1. Analogy between functions of time and vectors. Fourier series Take Away Periodic complex exponentials

More information

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 :

v = v 1 2 +v 2 2. Two successive applications of this idea give the length of the vector v R 3 : Length, Angle and the Inner Product The length (or norm) of a vector v R 2 (viewed as connecting the origin to a point (v 1,v 2 )) is easily determined by the Pythagorean Theorem and is denoted v : v =

More information

CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33

CS123 INTRODUCTION TO COMPUTER GRAPHICS. Linear Algebra 1/33 Linear Algebra 1/33 Vectors A vector is a magnitude and a direction Magnitude = v Direction Also known as norm, length Represented by unit vectors (vectors with a length of 1 that point along distinct

More information

Review of linear algebra

Review of linear algebra Review of linear algebra 1 Vectors and matrices We will just touch very briefly on certain aspects of linear algebra, most of which should be familiar. Recall that we deal with vectors, i.e. elements of

More information

MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix

MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix MODULE 8 Topics: Null space, range, column space, row space and rank of a matrix Definition: Let L : V 1 V 2 be a linear operator. The null space N (L) of L is the subspace of V 1 defined by N (L) = {x

More information

CITY UNIVERSITY LONDON. BEng (Hons) in Electrical and Electronic Engineering PART 2 EXAMINATION. ENGINEERING MATHEMATICS 2 (resit) EX2003

CITY UNIVERSITY LONDON. BEng (Hons) in Electrical and Electronic Engineering PART 2 EXAMINATION. ENGINEERING MATHEMATICS 2 (resit) EX2003 No: CITY UNIVERSITY LONDON BEng (Hons) in Electrical and Electronic Engineering PART 2 EXAMINATION ENGINEERING MATHEMATICS 2 (resit) EX2003 Date: August 2004 Time: 3 hours Attempt Five out of EIGHT questions

More information

Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.

Lecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set. Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an

More information

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n

b 1 b 2.. b = b m A = [a 1,a 2,...,a n ] where a 1,j a 2,j a j = a m,j Let A R m n and x 1 x 2 x = x n Lectures -2: Linear Algebra Background Almost all linear and nonlinear problems in scientific computation require the use of linear algebra These lectures review basic concepts in a way that has proven

More information

1 (2n)! (-1)n (θ) 2n

1 (2n)! (-1)n (θ) 2n Complex Numbers and Algebra The real numbers are complete for the operations addition, subtraction, multiplication, and division, or more suggestively, for the operations of addition and multiplication

More information

2. Matrix Algebra and Random Vectors

2. Matrix Algebra and Random Vectors 2. Matrix Algebra and Random Vectors 2.1 Introduction Multivariate data can be conveniently display as array of numbers. In general, a rectangular array of numbers with, for instance, n rows and p columns

More information

ICS 6N Computational Linear Algebra Matrix Algebra

ICS 6N Computational Linear Algebra Matrix Algebra ICS 6N Computational Linear Algebra Matrix Algebra Xiaohui Xie University of California, Irvine xhx@uci.edu February 2, 2017 Xiaohui Xie (UCI) ICS 6N February 2, 2017 1 / 24 Matrix Consider an m n matrix

More information

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET This is a (not quite comprehensive) list of definitions and theorems given in Math 1553. Pay particular attention to the ones in red. Study Tip For each

More information

Linear Algebra Review

Linear Algebra Review Chapter 1 Linear Algebra Review It is assumed that you have had a course in linear algebra, and are familiar with matrix multiplication, eigenvectors, etc. I will review some of these terms here, but quite

More information

Lecture 3: Linear Algebra Review, Part II

Lecture 3: Linear Algebra Review, Part II Lecture 3: Linear Algebra Review, Part II Brian Borchers January 4, Linear Independence Definition The vectors v, v,..., v n are linearly independent if the system of equations c v + c v +...+ c n v n

More information

Example. We can represent the information on July sales more simply as

Example. We can represent the information on July sales more simply as CHAPTER 1 MATRICES, VECTORS, AND SYSTEMS OF LINEAR EQUATIONS 11 Matrices and Vectors In many occasions, we can arrange a number of values of interest into an rectangular array For example: Example We can

More information

Chapter 6. Orthogonality

Chapter 6. Orthogonality 6.4 The Projection Matrix 1 Chapter 6. Orthogonality 6.4 The Projection Matrix Note. In Section 6.1 (Projections), we projected a vector b R n onto a subspace W of R n. We did so by finding a basis for

More information

Algebraic Properties of Solutions of Linear Systems

Algebraic Properties of Solutions of Linear Systems Algebraic Properties of Solutions of Linear Systems In this chapter we will consider simultaneous first-order differential equations in several variables, that is, equations of the form f 1t,,,x n d f

More information

A Second Course in Elementary Differential Equations

A Second Course in Elementary Differential Equations A Second Course in Elementary Differential Equations Marcel B Finan Arkansas Tech University c All Rights Reserved August 3, 23 Contents 28 Calculus of Matrix-Valued Functions of a Real Variable 4 29 nth

More information

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces. Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

More information