Singular value decomposition. Mathématiques appliquées (MATH0504-1) B. Dewals, Ch. Geuzaine
|
|
- Dora Rich
- 5 years ago
- Views:
Transcription
1 Lecture 11 Sigular value decompositio Mathématiques appliquées (MATH0504-1) B. Dewals, Ch. Geuzaie V1.2 07/12/2018 1
2 Sigular value decompositio (SVD) at a glace Motivatio: the image of the uit sphere S uder ay m matrix trasformatio is a hyperellipse. x m A A x = Ax m x Ax S Through the SVD, we will ifer importat properties of matrix A from the shape of AS! 2
3 Sigular value decompositio (SVD) at a glace The sigular value decompositio (SVD) is a particular matrix factorizatio. x m A A x = Ax m x Ax S Through the SVD, we will ifer importat properties of matrix A from the shape of AS! 3
4 Why is the sigular value decompositio of particular importace? The reasos for lookig at SVD are twofold: 1. The computatio of SVD the SVD is used is used as a as a itermediate step i may algorithms of of practical iterest. 2. From a coceptual poit of view, SVD also eables the SVD a eables deeper a uderstadig deeper uderstadig of may problems of may aspects i liear of algebra. liear algebra. 4
5 Learig objectives & outlie Become familiar with the SVD ad its geometric iterpretatio, ad get aware of its sigificace 1. Geometric Remider of observatios some fudametals i liear algebra 2. Reduced Geometric SVD iterpretatio 3. From reduced SVD to full SVD, ull ad SVD formal defiitio 4. Formal Existece defiitio ad uiqueess 5
6 1 - Remider: fudametals i liear algebra I this sectio, we briefly review the cocepts of adjoit matrix, matrix rak, uitary matrix as well as matrix orms (Chapters 2 ad 3 i Trefethe & Bau, 1997).
7 Adjoit of a matrix The adjoit (or Hermitia cojugate) of a m matrix A, writte A *, is the m matrix whose i, j etry is the complex cojugate of the j, i etry of A. If A = A *, A is Hermitia (or self-adjoit). For a real matrix A, the adjoit is the traspose: A * = A T, if the matrix is Hermitia, that is A = A T, the it is symmetric. 7
8 Matrix rak The rak of a matrix is the umber of liearly idepedet colums (or rows) of a matrix. The umbers of liearly idepedet colums ad rows of a matrix are equal. A m matrix of full rak is oe that has the maximal possible rak (the lesser of m ad ). If m, such a matrix is characterized by the property that it maps o two distict vectors to the same vector. 8
9 Uitary matrix A square matrix Q C mm, is uitary (or orthogoal, i the real case), if i.e. Q * = Q 1, Q * Q = I. The colums q i of a uitary matrix form a othoormal basis of C m : (q i ) * q j = δ ij, with δ ij the Kroecker delta. 9
10 A rotatio matrix is a typical example of a uitary matrix A rotatio matrix R may write: R cosq siq siq cosq The image of a vector is the same vector, rotated couter clockwise by a agle q. Matrix R is orthogoal ad R * R = R T R = I. 10
11 (Iduced) matrix orms are defied from the actio of the matrix o vectors For a matrix A C m, ad give vector orms () o the domai of A (m) o the rage of A the iduced matrix orm () is the smallest umber C for which the followig iequality holds for all x C : Ax C x m It is the maximum factor by which A ca stretch a vector x. 11
12 (Iduced) matrix orms are defied from the actio of the matrix o vectors The matrix orm ca be defied equivaletly i terms of the images of the uit vectors uder A: A m, max x x x 0 x 1 This form is coveiet for visualizig iduced matrix orms, as i this example. A x Ax x 1 2 m max Ax A Ax m 12
13 2 Geometric iterpretatio I this sectio, we itroduce coceptually the SVD, by meas of a simple geometric iterpretatio (Chapter 4 i Trefethe & Bau, 1997).
14 Geometric iterpretatio Let S be the uit sphere i R. Cosider ay matrix A R m, with m. Assume for the momet that A has full rak. S x x A : x 1 x 2 i1 x i
15 Geometric iterpretatio The imaghe AS is a hyperellipse i R m. This fact is ot obvious; but let us assume for ow that it is true. It will be proved later. x A Ax S AS 15
16 A hyperellipse is the m-dimesioal geeralizatio of a ellipse i 2D I R m, a hyperellipse is a surface obtaied by stretchig the uit sphere i R m by some factors s 1,, s m (possibly zero) i some orthogoal directios u 1,, u m R m For coveiece, let us take the u i to be uit vectors, i.e. u i 2 = 1. The vectors {s i u i } are the pricipal semiaxes of the hyperellipse. s 1 u 1 s 2 u 2 AS 16
17 A hyperellipse is the m-dimesioal geeralizatio of a ellipse i 2D If A has rak r, exactly r of the legths s i will be ozero. I particular, if m, at most of them will be ozero. x A s 2 u 2 AS S s 1 u 1 17
18 Sigular values We stated at the begiig that the SVD eables characterizig properties of matrix A from the shape of AS. Here we go for three defiitios We defie the sigular values of matrix A as the legths of the pricipal semiaxes of AS, oted s 1,, s. It is covetioal to umber the sigular values i descedig order: s 1 s 2 s. s 1 u 1 s 2 u 2 AS 18
19 Left sigular vectors We also defie the left sigular vectors of matrix A as the uit vectors {u 1,, u } orieted i the directios of the pricipal semiaxes of AS, umbered to correspod with the sigular values. s 2 u 2 Thus, the vector s i u i is the i th largest pricipal semiaxis. s 1 u 1 AS 19
20 Right sigular vectors We also defie the right sigular vectors of matrix A as the uit vectors {v 1,, v } S that are the preimages of the pricipal semiaxes of AS, umbered so that A v j = s j u j. v 1 v 2 A s 2 u 2 S s 1 u 1 AS 20
21 Importat remarks The terms left ad right sigular vectors will be uderstood later as we move forward with a more formal descriptio of the SVD. I the geometric iterpretatio preseted so far, we assumed that matrix A is real ad m = = 2. v 1 v 2 Actually, the SVD applies A s 2 u 2 to both real ad complex matrices, S whatever the umber of dimesios. s 1 u AS 1 21
22 3 From reduced to full SVD, ad formal defiitio I this sectio, we distiguish betwee the so-called reduced SVD, ofte used i practice, ad the full SVD. We also itroduce the formal defiitio of SVD (Chapter 4 i Trefethe & Bau, 1997).
23 The equatios relatig right ad left sigular vectors ca be expressed i matrix form We just metioed that the equatios relatig right sigular vectors {v j } ad left sigular vectors {u j } ca be writte A v j = s j u j 1 j This collectio of vector equatios ca be expressed as a matrix equatio. s 1 A s v 1 v 2 v = u 1 u 2 u 2 s 23
24 The equatios relatig right ad left sigular vectors ca be expressed i matrix form This matrix equatio ca be writte i a more compact form: AV Uˆ Sˆ with SŜ a diagoal matrix with real etries (as A was assumed to have full rak ) Uˆ a m matrix with orthoormal colums V a matrix with orthoormal colums A s v 1 v 2 v = u 1 u 2 u 2 Thus, V is uitary (i.e. V * = V 1 ), ad we obtai: A Uˆ Sˆ V * to distiguish from U, S i the full SVD s 1 s 24
25 Reduced SVD The factorizatio of matrix A i the form ˆ ˆ * A USV is called a reduced sigular values decompositio, or reduced SVD, of matrix A. Schematically, it looks like this (m ): m = A ˆ U Ŝ V * 25
26 From reduced SVD to full SVD The colums of Û are orthoormal vectors i the m-dimesioal space C m. Uless m =, they do ot form a basis of C m, or is Û a uitary matrix. However, we may upgrade Û to a uitary matrix! m = A ˆ U Ŝ V * 26
27 From reduced SVD to full SVD Let us adjoi a additioal m orthoormal colums to matrix Û, so that it becomes uitary. The m additioal orthoormal colums are chose arbitrarily ad the result is oted U. However, S must chage too m = A U Ŝ V * 27
28 silet colums From reduced SVD to full SVD For the product to remai ualtered, the last m colums of U should be multiplied by zero. Accordigly, let S be the m matrix cosistig of S i the upper block together with m rows of zeros below. m = * A U S V 28
29 From reduced SVD to full SVD We get a ew factorizatio of A, called full SVD: A = U S V * U is a m m uitary matrix, V is a uitary matrix, S is a m diagoal matrix with real etries m = * A U S V 29
30 Geeralizatio to the case of a matrix A which does ot have full rak m If matrix A is rak-deficiet (i.e. of rak r < ), oly r (istead of ) of the left sigular vectors are deduced from the size of the hyperellipse BUT the full SVD still applies, by itroducig m r (istead of m ) additioal arbitrary orthoormal colums to costruct the uitary matrix U; the matrix V also eeds r arbitrary orthoormal = colums to exted the r colums determied from the hyperellipse geometry * Amatrix S has oly Ur o-zero diagoal S etries. V 30
31 Formal defiitio of the SVD m Let m ad be arbitrary (we do ot require m ). Give A C m, ot ecessarily of full rak, a sigular value decompositio of A is a factorizatio where = A = U S V * s 1 s p oegative, i oicreasig order U C mm is square, uitary S R m is real diagoal V * C is square, uitary * A U S V 31
32 Cosequetly, the image of the uit sphere i R uder a map A = U S V * is a hyperellipse i R m m Thus, 1. The uitary map V * preserves the sphere 2. The diagoal matrix S stretches the sphere ito a hyperellipse 3. The fial uitary map U rotates, or reflects, the hyperellipse without chagig its shape. if we ca prove that every U C mm S matrix C m has a SVD, V * C we will have is = is proved square, that the image of the uit is square, sphere uder ay real uitary liear map uitary is ideed a hyperellipse. diagoal * A U S V 32
33 4 Existece ad uiqueess I this sectio, we demostrate the existece of the SVD, the uiqueess of the sigular values, as well as uder some specific coditios, the uiqueess of the sigular vectors (Chapter 4 i Trefethe & Bau, 1997).
34 Every matrix A C m has a sigular value decompositio A = U S V * To prove the existece of the SVD, we first isolate the directio of the largest actio of A, the we proceed by iductio o the dimesio of A. The proof takes 5 steps. 34
35 Every matrix A C m has a sigular value decompositio A = U S V * Set s 1 = A 2. From the defiitio of the matrix orm, there must be a vector v 1 C m A max x x 1 Ax with v 1 2 = 1 ad Av 1 2 = s 1 v 1 2 = 1 m A 1 v = Av1 Av 1 2 = s 1 We ote: Av1 u1 s 1 35
36 Every matrix A C m has a sigular value decompositio A = U S V * Cosider ay extesios of v 1 to a orthoormal basis {v j } of C ad of u 1 to a orthoormal basis {u j } of C m Let U 1 ad V 1 deote the uitary matrices with colums u j ad v j, respectively. m A v1 v j 1 ad u1 u j 1 V 1 U 1 36
37 Every matrix A C m has a sigular value decompositio A = U S V * The we have * * s 1 w U1 AV1 S 0 B where 0 is a colum vector of dimesio m 1, w * is a row vector of dimesio 1, ad B has dimesios (m 1) ( 1). * * u 1 u s u * w s m A v1 = 0 B U 1 * V 1 u * j1s 1u1 0 37
38 Every matrix A C m has a sigular value decompositio A = U S V * Furthermore, A m, max x x 0 Ax x m s s 0 B w w * s 1 2 * 2 * 12 1 w 1 s1 w w s1 w w Implyig (from the defiitio of matrix orms) BUT, sice U 1 ad V 1 are uitary, we kow that S This implies w = 0. 2 S 2 s 2 * 12 1 w w U AV A s *
39 Every matrix A C m has a sigular value decompositio A = U S V * If = 1 or m = 1, we are doe! Otherwise, the submatrix B describes the actio of A o the subspace orthogoal to v 1. By the iductio hypothesis, B has a SVD B = U 2 S 2 V 2*. Now it is easily verified that 1 0 s * A U1 V1 0 U V S 2 is a SVD of A, completig the proof of existece. * 39
40 Uiqueess The sigular values {s j } are uiquely determied. If A is square ad the s j are distict, the left ad right sigular vectors {u j } ad {v j } are uiquely determied up to complex sigs. v 1 v 2 A s 2 u 2 S s 1 u 1 AS 40
41 Uiqueess Geometrically, the proof is straightforward: if the semiaxis legths of a hyperellipse are distict, the the semiaxes themselves are determied by the geometry, up to sigs. v 1 v 2 A s 2 u 2 S s 1 u 1 AS 41
42 Take-home messages SVD is a importat factorizatio method, which applies for all rectagular, real or complex matrices It decomposes the matrix ito three factors a uitary matrix a real diagoal matrix, with oegative etries aother uitary matrix It has a broad rage of implicatios ad applicatios! 42
43 What s ext? Every matrix is diagoal if oly oe uses the proper bases for the domai ad rage spaces. SVD vs. eigevalue decompositio existece rectagular vs. square matrices orthoormal bases i the SVD, ot eigevectors Lik with matrix rak, rage, ull space, orm Low-rak approximatios 43
Lecture 8: October 20, Applications of SVD: least squares approximation
Mathematical Toolkit Autum 2016 Lecturer: Madhur Tulsiai Lecture 8: October 20, 2016 1 Applicatios of SVD: least squares approximatio We discuss aother applicatio of sigular value decompositio (SVD) of
More informationInverse Matrix. A meaning that matrix B is an inverse of matrix A.
Iverse Matrix Two square matrices A ad B of dimesios are called iverses to oe aother if the followig holds, AB BA I (11) The otio is dual but we ofte write 1 B A meaig that matrix B is a iverse of matrix
More informationSymmetric Matrices and Quadratic Forms
7 Symmetric Matrices ad Quadratic Forms 7.1 DIAGONALIZAION OF SYMMERIC MARICES SYMMERIC MARIX A symmetric matrix is a matrix A such that. A = A Such a matrix is ecessarily square. Its mai diagoal etries
More information5.1 Review of Singular Value Decomposition (SVD)
MGMT 69000: Topics i High-dimesioal Data Aalysis Falll 06 Lecture 5: Spectral Clusterig: Overview (cotd) ad Aalysis Lecturer: Jiamig Xu Scribe: Adarsh Barik, Taotao He, September 3, 06 Outlie Review of
More informationLECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK)
LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK) Everythig marked by is ot required by the course syllabus I this lecture, all vector spaces is over the real umber R. All vectors i R is viewed as a colum
More information(VII.A) Review of Orthogonality
VII.A Review of Orthogoality At the begiig of our study of liear trasformatios i we briefly discussed projectios, rotatios ad projectios. I III.A, projectios were treated i the abstract ad without regard
More informationTopics in Eigen-analysis
Topics i Eige-aalysis Li Zajiag 28 July 2014 Cotets 1 Termiology... 2 2 Some Basic Properties ad Results... 2 3 Eige-properties of Hermitia Matrices... 5 3.1 Basic Theorems... 5 3.2 Quadratic Forms & Noegative
More information6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition
6. Kalma filter implemetatio for liear algebraic equatios. Karhue-Loeve decompositio 6.1. Solvable liear algebraic systems. Probabilistic iterpretatio. Let A be a quadratic matrix (ot obligatory osigular.
More informationApply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.
Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α
More informationCHAPTER 3. GOE and GUE
CHAPTER 3 GOE ad GUE We quicly recall that a GUE matrix ca be defied i the followig three equivalet ways. We leave it to the reader to mae the three aalogous statemets for GOE. I the previous chapters,
More informationMATH10212 Linear Algebra B Proof Problems
MATH22 Liear Algebra Proof Problems 5 Jue 26 Each problem requests a proof of a simple statemet Problems placed lower i the list may use the results of previous oes Matrices ermiats If a b R the matrix
More informationa for a 1 1 matrix. a b a b 2 2 matrix: We define det ad bc 3 3 matrix: We define a a a a a a a a a a a a a a a a a a
Math E-2b Lecture #8 Notes This week is all about determiats. We ll discuss how to defie them, how to calculate them, lear the allimportat property kow as multiliearity, ad show that a square matrix A
More informationZeros of Polynomials
Math 160 www.timetodare.com 4.5 4.6 Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered with fidig the solutios of polyomial equatios of ay degree
More informationBrief Review of Functions of Several Variables
Brief Review of Fuctios of Several Variables Differetiatio Differetiatio Recall, a fuctio f : R R is differetiable at x R if ( ) ( ) lim f x f x 0 exists df ( x) Whe this limit exists we call it or f(
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture 9: Pricipal Compoet Aalysis The text i black outlies mai ideas to retai from the lecture. The text i blue give a deeper uderstadig of how we derive or get
More informationChimica Inorganica 3
himica Iorgaica Irreducible Represetatios ad haracter Tables Rather tha usig geometrical operatios, it is ofte much more coveiet to employ a ew set of group elemets which are matrices ad to make the rule
More informationPhysics 324, Fall Dirac Notation. These notes were produced by David Kaplan for Phys. 324 in Autumn 2001.
Physics 324, Fall 2002 Dirac Notatio These otes were produced by David Kapla for Phys. 324 i Autum 2001. 1 Vectors 1.1 Ier product Recall from liear algebra: we ca represet a vector V as a colum vector;
More information1 Last time: similar and diagonalizable matrices
Last time: similar ad diagoalizable matrices Let be a positive iteger Suppose A is a matrix, v R, ad λ R Recall that v a eigevector for A with eigevalue λ if v ad Av λv, or equivaletly if v is a ozero
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm
More informationLinear Regression Demystified
Liear Regressio Demystified Liear regressio is a importat subject i statistics. I elemetary statistics courses, formulae related to liear regressio are ofte stated without derivatio. This ote iteds to
More informationa for a 1 1 matrix. a b a b 2 2 matrix: We define det ad bc 3 3 matrix: We define a a a a a a a a a a a a a a a a a a
Math S-b Lecture # Notes This wee is all about determiats We ll discuss how to defie them, how to calculate them, lear the allimportat property ow as multiliearity, ad show that a square matrix A is ivertible
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More information2 Geometric interpretation of complex numbers
2 Geometric iterpretatio of complex umbers 2.1 Defiitio I will start fially with a precise defiitio, assumig that such mathematical object as vector space R 2 is well familiar to the studets. Recall that
More information5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0.
40 RODICA D. COSTIN 5. The Rayleigh s priciple ad the i priciple for the eigevalues of a self-adjoit matrix Eigevalues of self-adjoit matrices are easy to calculate. This sectio shows how this is doe usig
More informationAN INTRODUCTION TO SPECTRAL GRAPH THEORY
AN INTRODUCTION TO SPECTRAL GRAPH THEORY JIAQI JIANG Abstract. Spectral graph theory is the study of properties of the Laplacia matrix or adjacecy matrix associated with a graph. I this paper, we focus
More information4 The Sperner property.
4 The Sperer property. I this sectio we cosider a surprisig applicatio of certai adjacecy matrices to some problems i extremal set theory. A importat role will also be played by fiite groups. I geeral,
More informationMatrix Algebra from a Statistician s Perspective BIOS 524/ Scalar multiple: ka
Matrix Algebra from a Statisticia s Perspective BIOS 524/546. Matrices... Basic Termiology a a A = ( aij ) deotes a m matrix of values. Whe =, this is a am a m colum vector. Whe m= this is a row vector..2.
More informationPrinciple Of Superposition
ecture 5: PREIMINRY CONCEP O RUCUR NYI Priciple Of uperpositio Mathematically, the priciple of superpositio is stated as ( a ) G( a ) G( ) G a a or for a liear structural system, the respose at a give
More informationMatrix Algebra 2.2 THE INVERSE OF A MATRIX Pearson Education, Inc.
2 Matrix Algebra 2.2 THE INVERSE OF A MATRIX MATRIX OPERATIONS A matrix A is said to be ivertible if there is a matrix C such that CA = I ad AC = I where, the idetity matrix. I = I I this case, C is a
More informationNotes for Lecture 11
U.C. Berkeley CS78: Computatioal Complexity Hadout N Professor Luca Trevisa 3/4/008 Notes for Lecture Eigevalues, Expasio, ad Radom Walks As usual by ow, let G = (V, E) be a udirected d-regular graph with
More informationAbstract Vector Spaces. Abstract Vector Spaces
Astract Vector Spaces The process of astractio is critical i egieerig! Physical Device Data Storage Vector Space MRI machie Optical receiver 0 0 1 0 1 0 0 1 Icreasig astractio 6.1 Astract Vector Spaces
More informationFor a 3 3 diagonal matrix we find. Thus e 1 is a eigenvector corresponding to eigenvalue λ = a 11. Thus matrix A has eigenvalues 2 and 3.
Closed Leotief Model Chapter 6 Eigevalues I a closed Leotief iput-output-model cosumptio ad productio coicide, i.e. V x = x = x Is this possible for the give techology matrix V? This is a special case
More informationIntroduction to Optimization Techniques
Itroductio to Optimizatio Techiques Basic Cocepts of Aalysis - Real Aalysis, Fuctioal Aalysis 1 Basic Cocepts of Aalysis Liear Vector Spaces Defiitio: A vector space X is a set of elemets called vectors
More informationSession 5. (1) Principal component analysis and Karhunen-Loève transformation
200 Autum semester Patter Iformatio Processig Topic 2 Image compressio by orthogoal trasformatio Sessio 5 () Pricipal compoet aalysis ad Karhue-Loève trasformatio Topic 2 of this course explais the image
More informationDeterminants of order 2 and 3 were defined in Chapter 2 by the formulae (5.1)
5. Determiats 5.. Itroductio 5.2. Motivatio for the Choice of Axioms for a Determiat Fuctios 5.3. A Set of Axioms for a Determiat Fuctio 5.4. The Determiat of a Diagoal Matrix 5.5. The Determiat of a Upper
More information3.2 Properties of Division 3.3 Zeros of Polynomials 3.4 Complex and Rational Zeros of Polynomials
Math 60 www.timetodare.com 3. Properties of Divisio 3.3 Zeros of Polyomials 3.4 Complex ad Ratioal Zeros of Polyomials I these sectios we will study polyomials algebraically. Most of our work will be cocered
More information, then cv V. Differential Equations Elements of Lineaer Algebra Name: Consider the differential equation. and y2 cos( kx)
Cosider the differetial equatio y '' k y 0 has particular solutios y1 si( kx) ad y cos( kx) I geeral, ay liear combiatio of y1 ad y, cy 1 1 cy where c1, c is also a solutio to the equatio above The reaso
More informationMatrix Theory, Math6304 Lecture Notes from October 23, 2012 taken by Satish Pandey
Matrix Theory, Math6304 Lecture Notes from October 3, 01 take by Satish Padey Warm up from last time Example of low rak perturbatio; re-examied We had this operator S = 0 1 0......... 1 1 0 M (C) S is
More information24 MATH 101B: ALGEBRA II, PART D: REPRESENTATIONS OF GROUPS
24 MATH 101B: ALGEBRA II, PART D: REPRESENTATIONS OF GROUPS Corollary 2.30. Suppose that the semisimple decompositio of the G- module V is V = i S i. The i = χ V,χ i Proof. Sice χ V W = χ V + χ W, we have:
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 9 Multicolliearity Dr Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Multicolliearity diagostics A importat questio that
More informationR is a scalar defined as follows:
Math 8. Notes o Dot Product, Cross Product, Plaes, Area, ad Volumes This lecture focuses primarily o the dot product ad its may applicatios, especially i the measuremet of agles ad scalar projectio ad
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationGeometry of LS. LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT
OCTOBER 7, 2016 LECTURE 3 GEOMETRY OF LS, PROPERTIES OF σ 2, PARTITIONED REGRESSION, GOODNESS OF FIT Geometry of LS We ca thik of y ad the colums of X as members of the -dimesioal Euclidea space R Oe ca
More informationCOLLIN COUNTY COMMUNITY COLLEGE COURSE SYLLABUS CREDIT HOURS: 3 LECTURE HOURS: 3 LAB HOURS: 0
COLLIN COUNTY COMMUNITY COLLEGE COURSE SYLLABUS Revised Fall 2017 COURSE NUMBER: MATH 2318 COURSE TITLE: Liear Algebra CREDIT HOURS: 3 LECTURE HOURS: 3 LAB HOURS: 0 ASSESSMENTS: Noe PREREQUISITE: MATH
More informationCHAPTER 5. Theory and Solution Using Matrix Techniques
A SERIES OF CLASS NOTES FOR 2005-2006 TO INTRODUCE LINEAR AND NONLINEAR PROBLEMS TO ENGINEERS, SCIENTISTS, AND APPLIED MATHEMATICIANS DE CLASS NOTES 3 A COLLECTION OF HANDOUTS ON SYSTEMS OF ORDINARY DIFFERENTIAL
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationEigenvalues and Eigenvectors
5 Eigevalues ad Eigevectors 5.3 DIAGONALIZATION DIAGONALIZATION Example 1: Let. Fid a formula for A k, give that P 1 1 = 1 2 ad, where Solutio: The stadard formula for the iverse of a 2 2 matrix yields
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationOn Involutions which Preserve Natural Filtration
Proceedigs of Istitute of Mathematics of NAS of Ukraie 00, Vol. 43, Part, 490 494 O Ivolutios which Preserve Natural Filtratio Alexader V. STRELETS Istitute of Mathematics of the NAS of Ukraie, 3 Tereshchekivska
More informationRepresentations of State Vectors and Operators
Chapter 10 Represetatios of State Vectors ad Operators I the precedig Chapters, the mathematical ideas uderpiig the quatum theory have bee developed i a fairly geeral (though, admittedly, ot a mathematically
More informationIn the preceding Chapters, the mathematical ideas underpinning the quantum theory have been
Chapter Matrix Represetatios of State Vectors ad Operators I the precedig Chapters, the mathematical ideas uderpiig the quatum theory have bee developed i a fairly geeral (though, admittedly, ot a mathematically
More informationAlgebra of Least Squares
October 19, 2018 Algebra of Least Squares Geometry of Least Squares Recall that out data is like a table [Y X] where Y collects observatios o the depedet variable Y ad X collects observatios o the k-dimesioal
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationsubcaptionfont+=small,labelformat=parens,labelsep=space,skip=6pt,list=0,hypcap=0 subcaption ALGEBRAIC COMBINATORICS LECTURE 8 TUESDAY, 2/16/2016
subcaptiofot+=small,labelformat=pares,labelsep=space,skip=6pt,list=0,hypcap=0 subcaptio ALGEBRAIC COMBINATORICS LECTURE 8 TUESDAY, /6/06. Self-cojugate Partitios Recall that, give a partitio λ, we may
More information(3) If you replace row i of A by its sum with a multiple of another row, then the determinant is unchanged! Expand across the i th row:
Math 50-004 Tue Feb 4 Cotiue with sectio 36 Determiats The effective way to compute determiats for larger-sized matrices without lots of zeroes is to ot use the defiitio, but rather to use the followig
More informationNICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) =
AN INTRODUCTION TO SCHRÖDER AND UNKNOWN NUMBERS NICK DUFRESNE Abstract. I this article we will itroduce two types of lattice paths, Schröder paths ad Ukow paths. We will examie differet properties of each,
More informationChapter Vectors
Chapter 4. Vectors fter readig this chapter you should be able to:. defie a vector. add ad subtract vectors. fid liear combiatios of vectors ad their relatioship to a set of equatios 4. explai what it
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationMatrix Theory, Math6304 Lecture Notes from October 25, 2012 taken by Manisha Bhardwaj
Matrix Theory, Math6304 Lecture Notes from October 25, 2012 take by Maisha Bhardwaj Last Time (10/23/12) Example for low-rak perturbatio, re-examied Relatig eigevalues of matrices ad pricipal submatrices
More informationMath 155 (Lecture 3)
Math 55 (Lecture 3) September 8, I this lecture, we ll cosider the aswer to oe of the most basic coutig problems i combiatorics Questio How may ways are there to choose a -elemet subset of the set {,,,
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationMath 778S Spectral Graph Theory Handout #3: Eigenvalues of Adjacency Matrix
Math 778S Spectral Graph Theory Hadout #3: Eigevalues of Adjacecy Matrix The Cartesia product (deoted by G H) of two simple graphs G ad H has the vertex-set V (G) V (H). For ay u, v V (G) ad x, y V (H),
More informationA Hadamard-type lower bound for symmetric diagonally dominant positive matrices
A Hadamard-type lower boud for symmetric diagoally domiat positive matrices Christopher J. Hillar, Adre Wibisoo Uiversity of Califoria, Berkeley Jauary 7, 205 Abstract We prove a ew lower-boud form of
More informationThe multiplicative structure of finite field and a construction of LRC
IERG6120 Codig for Distributed Storage Systems Lecture 8-06/10/2016 The multiplicative structure of fiite field ad a costructio of LRC Lecturer: Keeth Shum Scribe: Zhouyi Hu Notatios: We use the otatio
More information(3) If you replace row i of A by its sum with a multiple of another row, then the determinant is unchanged! Expand across the i th row:
Math 5-4 Tue Feb 4 Cotiue with sectio 36 Determiats The effective way to compute determiats for larger-sized matrices without lots of zeroes is to ot use the defiitio, but rather to use the followig facts,
More informationBeurling Integers: Part 2
Beurlig Itegers: Part 2 Isomorphisms Devi Platt July 11, 2015 1 Prime Factorizatio Sequeces I the last article we itroduced the Beurlig geeralized itegers, which ca be represeted as a sequece of real umbers
More information8. Applications To Linear Differential Equations
8. Applicatios To Liear Differetial Equatios 8.. Itroductio 8.. Review Of Results Cocerig Liear Differetial Equatios Of First Ad Secod Orders 8.3. Eercises 8.4. Liear Differetial Equatios Of Order N 8.5.
More informationSome examples of vector spaces
Roberto s Notes o Liear Algebra Chapter 11: Vector spaces Sectio 2 Some examples of vector spaces What you eed to kow already: The te axioms eeded to idetify a vector space. What you ca lear here: Some
More informationMatrices and vectors
Oe Matrices ad vectors This book takes for grated that readers have some previous kowledge of the calculus of real fuctios of oe real variable It would be helpful to also have some kowledge of liear algebra
More informationState Space Representation
Optimal Cotrol, Guidace ad Estimatio Lecture 2 Overview of SS Approach ad Matrix heory Prof. Radhakat Padhi Dept. of Aerospace Egieerig Idia Istitute of Sciece - Bagalore State Space Represetatio Prof.
More informationMon Feb matrix inverses. Announcements: Warm-up Exercise:
Math 225-4 Week 6 otes We will ot ecessarily fiish the material from a give day's otes o that day We may also add or subtract some material as the week progresses, but these otes represet a i-depth outlie
More informationTENSOR PRODUCTS AND PARTIAL TRACES
Lecture 2 TENSOR PRODUCTS AND PARTIAL TRACES Stéphae ATTAL Abstract This lecture cocers special aspects of Operator Theory which are of much use i Quatum Mechaics, i particular i the theory of Quatum Ope
More informationMatrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Pearson Education, Inc.
2 Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES 2012 Pearso Educatio, Ic. Theorem 8: Let A be a square matrix. The the followig statemets are equivalet. That is, for a give A, the statemets
More informationLinear Transformations
Liear rasformatios 6. Itroductio to Liear rasformatios 6. he Kerel ad Rage of a Liear rasformatio 6. Matrices for Liear rasformatios 6.4 rasitio Matrices ad Similarity 6.5 Applicatios of Liear rasformatios
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationMath E-21b Spring 2018 Homework #2
Math E- Sprig 08 Homework # Prolems due Thursday, Feruary 8: Sectio : y = + 7 8 Fid the iverse of the liear trasformatio [That is, solve for, i terms of y, y ] y = + 0 Cosider the circular face i the accompayig
More informationStochastic Matrices in a Finite Field
Stochastic Matrices i a Fiite Field Abstract: I this project we will explore the properties of stochastic matrices i both the real ad the fiite fields. We first explore what properties 2 2 stochastic matrices
More informationc 2006 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 7, No. 3, pp. 851 860 c 006 Society for Idustrial ad Applied Mathematics EXTREMAL EIGENVALUES OF REAL SYMMETRIC MATRICES WITH ENTRIES IN AN INTERVAL XINGZHI ZHAN Abstract.
More informationBertrand s Postulate
Bertrad s Postulate Lola Thompso Ross Program July 3, 2009 Lola Thompso (Ross Program Bertrad s Postulate July 3, 2009 1 / 33 Bertrad s Postulate I ve said it oce ad I ll say it agai: There s always a
More informationSOME TRIBONACCI IDENTITIES
Mathematics Today Vol.7(Dec-011) 1-9 ISSN 0976-38 Abstract: SOME TRIBONACCI IDENTITIES Shah Devbhadra V. Sir P.T.Sarvajaik College of Sciece, Athwalies, Surat 395001. e-mail : drdvshah@yahoo.com The sequece
More informationMachine Learning Theory Tübingen University, WS 2016/2017 Lecture 11
Machie Learig Theory Tübige Uiversity, WS 06/07 Lecture Tolstikhi Ilya Abstract We will itroduce the otio of reproducig kerels ad associated Reproducig Kerel Hilbert Spaces (RKHS). We will cosider couple
More informationb i u x i U a i j u x i u x j
M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here
More informationCMSE 820: Math. Foundations of Data Sci.
Lecture 17 8.4 Weighted path graphs Take from [10, Lecture 3] As alluded to at the ed of the previous sectio, we ow aalyze weighted path graphs. To that ed, we prove the followig: Theorem 6 (Fiedler).
More informationTMA4205 Numerical Linear Algebra. The Poisson problem in R 2 : diagonalization methods
TMA4205 Numerical Liear Algebra The Poisso problem i R 2 : diagoalizatio methods September 3, 2007 c Eiar M Røquist Departmet of Mathematical Scieces NTNU, N-749 Trodheim, Norway All rights reserved A
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationRandom Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices
Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio
More informationEE / EEE SAMPLE STUDY MATERIAL. GATE, IES & PSUs Signal System. Electrical Engineering. Postal Correspondence Course
Sigal-EE Postal Correspodece Course 1 SAMPLE STUDY MATERIAL Electrical Egieerig EE / EEE Postal Correspodece Course GATE, IES & PSUs Sigal System Sigal-EE Postal Correspodece Course CONTENTS 1. SIGNAL
More informationLINEAR ALGEBRA. Paul Dawkins
LINEAR ALGEBRA Paul Dawkis Table of Cotets Preface... ii Outlie... iii Systems of Equatios ad Matrices... Itroductio... Systems of Equatios... Solvig Systems of Equatios... 5 Matrices... 7 Matrix Arithmetic
More informationIterative method for computing a Schur form of symplectic matrix
Aals of the Uiversity of Craiova, Mathematics ad Computer Sciece Series Volume 421, 2015, Pages 158 166 ISSN: 1223-6934 Iterative method for computig a Schur form of symplectic matrix A Mesbahi, AH Betbib,
More informationThe inverse eigenvalue problem for symmetric doubly stochastic matrices
Liear Algebra ad its Applicatios 379 (004) 77 83 www.elsevier.com/locate/laa The iverse eigevalue problem for symmetric doubly stochastic matrices Suk-Geu Hwag a,,, Sug-Soo Pyo b, a Departmet of Mathematics
More informationSequences, Mathematical Induction, and Recursion. CSE 2353 Discrete Computational Structures Spring 2018
CSE 353 Discrete Computatioal Structures Sprig 08 Sequeces, Mathematical Iductio, ad Recursio (Chapter 5, Epp) Note: some course slides adopted from publisher-provided material Overview May mathematical
More informationLECTURE 11: POSTNIKOV AND WHITEHEAD TOWERS
LECTURE 11: POSTNIKOV AND WHITEHEAD TOWERS I the previous sectio we used the techique of adjoiig cells i order to costruct CW approximatios for arbitrary spaces Here we will see that the same techique
More informationComplex Analysis Spring 2001 Homework I Solution
Complex Aalysis Sprig 2001 Homework I Solutio 1. Coway, Chapter 1, sectio 3, problem 3. Describe the set of poits satisfyig the equatio z a z + a = 2c, where c > 0 ad a R. To begi, we see from the triagle
More informationDiscrete-Time Systems, LTI Systems, and Discrete-Time Convolution
EEL5: Discrete-Time Sigals ad Systems. Itroductio I this set of otes, we begi our mathematical treatmet of discrete-time s. As show i Figure, a discrete-time operates or trasforms some iput sequece x [
More informationAssignment 2 Solutions SOLUTION. ϕ 1 Â = 3 ϕ 1 4i ϕ 2. The other case can be dealt with in a similar way. { ϕ 2 Â} χ = { 4i ϕ 1 3 ϕ 2 } χ.
PHYSICS 34 QUANTUM PHYSICS II (25) Assigmet 2 Solutios 1. With respect to a pair of orthoormal vectors ϕ 1 ad ϕ 2 that spa the Hilbert space H of a certai system, the operator  is defied by its actio
More informationOn Nonsingularity of Saddle Point Matrices. with Vectors of Ones
Iteratioal Joural of Algebra, Vol. 2, 2008, o. 4, 197-204 O Nosigularity of Saddle Poit Matrices with Vectors of Oes Tadeusz Ostrowski Istitute of Maagemet The State Vocatioal Uiversity -400 Gorzów, Polad
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More information