Matrix Theory, Math6304 Lecture Notes from March 22, 2016 taken by Kazem Safari

Size: px
Start display at page:

Download "Matrix Theory, Math6304 Lecture Notes from March 22, 2016 taken by Kazem Safari"

Transcription

1 Matrix Theory, Math6304 Lecture Notes from March 22, 2016 taken by Kazem Safari 1.1 Applications of Courant-Fisher and min- or -min Last time: Courant -Fishert -min or min- for eigenvalues Warm-up: Sums of eigenvalues from optimization problems Proposition. Let A M n be Hermitian, i.e. A A, and with eigenvalues λ 1 λ 2... λ n. Then λ j min tr[ap ]. In order to prove this remarkable result, we need to recall some Orthogonal Projection theory from Real Analysis 1 : If A is a closed subspace 2 of a Hilbert space H 3, for every x H we can define: δ inf x y. y A It then follows that δ < and there exist a unique z A that achieves this infimum, i.e. δ inf x y x z. y A 1 For the proof of these results cf Folland, Real Analysis Modern Techniques and Their Application, for the purpose of defining the orthogonal projection, it suffices for our subset A to be convex. 3 In this course H R n or C n. Therefore we can identify each linear maps with a matrix and vice-versa. 1

2 P (x) : z is called the orthogonal projection onto closed subspace A. Then, we have: x P (x) A 4 And every element x H can be uniquely written as: x P x + (x P x) where P x A and x P x A. In other words: H A A Proposition. The orthogonal projection operator P : H A has the following properties: 1) P is a linear continuous map, and P 1. 2) P 2 P, i.e. P A id. 3) R(P ) A and null(p ) A. 4) P P. 5) rank(p ) tr(p ). 6) Any eigenvalue of P is either 0 or 1. Conversely: Proposition. Suppose that P L(H, H) satisfies P 2 P P. Then R(P ) is closed and P is the orthogonal projection onto R(P ) Definition. we define P k as the set of orthogonal projections of rank k. 5 4 A {x H x, a 0 x A} 5 Unfortunately, the space of orthogonal projections of rank k is not a linear space but it is what we call a Variety. 2

3 Proof of the warm-up. Consider n orthonormal eigenvectors {u j } n corresponding to eigenvalues {λ j } n of our Hermitian matrix A. If we define U [u 1... u n ], then one eigendecomposition of A is A UΛU UU U U I and Λ diag[λ 1,..., λ n ]. Therefore where A n λ ju j u j. Next, if P P k, then since P 1. Now, P u j 2 u j 2 1. Since P 2 P, and since P P, k rank(p ) tr(p ) tr(u P U) u 1 u 2 tr. P [u 1 u 2... u n ] u n u 1 u 2 u n tr. [P u 1 P u 2... P u n ] u 1P u 1 u 1P u 2 u 1P u n u 2P u 1 u 2P u 2 u 2P u n tr u np u 1 u np u 2 u np u n tr[p u j u j] u jp u j P u j, u j P 2 u j, u j 3

4 So by the definition of the adjoint operator, P P u j, u j P u j, P u j P u j 2. Thus, if we define x j : tr(p u j u j) then by the same procedure as above: x j P u j u ju l, u l P u j δ j,l, u l P u j, u j P u j 2. Then by combining the two previous results, for each j we have: 0 x j 1 and x j k. Now let X k : {x [0, 1] n s.t. n x j k}. Then 6 : min tr[ap ] min tr[p A] min tr[ P λ j u j u j] min min λ j tr[p u j u j] λ j x j 6 since tr is linear and tr(ab) tr(ba) for all A, B M n n (C). 4

5 Now we are going to invoke the variational principal in optimization, which in essence says that by properly relaxing the conditions of a structurally highly complicated problem, the min only goes lower and goes higher. min x X k λ j x j Which turns the problem into minimizing over a linear k-polytope. Claim 7 : min x X k λ j x j λ j proof of the claim. if x l 0 for any l > k then there exist a ɛ > 0 such that λ l λ k + ɛ. Then: λ j x j λ j x j + λ l x l λ j x j + (λ k + ɛ)x l k 1 λ j x j + λ k (x k + x l ) + ɛx l k 1 λ j x j + λ k (x k + x l ) Meaning whenever any eigenvalue greater than λ k has a positive weight we can redistribute that weight among the eigenvalues less than or equal to λ k and achieve a lower overall value. Therefore we must have x k+1 x k+2... x n 0 7 This problem is very similar to water-filling algorithm in Singal Processing. 5

6 On the other hand, since x X k, we must fully scale the first k eigenvalues: Conversely, Choosing P k u ju j x 1 x 2... x k 1. It is straightforward to check that P L(H, H) and P 2 P P. Therefore by the latter Prop, P is the orthogonal projection into R(P ), which is a closed linear subspace of H. Moreover we have: P u j u j u 1 j u 2 j u n j. [ ] u 1 j u 2 j u n j u 1 ju 1 j u 1 ju 2 j u 1 ju n j u 2 ju 1 1 u 2 ju 2 j u 2 ju n j u n j u 1 1 u n j u 2 j u n j u n j Therefore we can easily see that: tr(p ) u j 2 k. But rank(p ) tr(p ) k by the former Prop, therefore we see that P P k as well. Thus: 6

7 min P P tr[ap ] tr[ap ] tr[a u j u j] tr[au j u j] tr[λ j u j u j] λ j tr(u j u j) λ j u j 2 λ j. And now we conclude that min tr[ap ] λ j. Moral: We can replace /min or min/ by a sequence of minimizations over P k, and therefore, we can relate the spectrum of submatrices to the whole matrix. Recall: Theorem (Courant-Fischer). Suppose A M n is Hermitian, i.e. A A. Now, for each 1 k n, let {S α k } α I k, where α I k denote the set of all k dimensional linear subspaces of H, and enumerate the n eigenvalues λ 1,..., λ n (counting multiplicity) in increasing order, i.e. λ 1 λ 2,..., λ n. Then, we have 7

8 (i) min α I k x Sk α\{0} Ax, x / x 2 λ k. (ii) α J n k+1 min x Sn k+1 α \{0} Ax x / x 2 λ k. Proof of part (ii). Let W Span{u 1, u 2,..., u k }, dim W k. Then if dim S n k+1 n k+1 then by dimension counting, i.e. 1 dim(w S n k+1 ) dim W + dim S n k+1 dim(w S n k+1 ), S n k+1 W 0, and therefore, there exists an x (S n k+1 W ) {0}, with x k x, u j u j. Therefore R A (x) Ax, x k λ j x, u j u j, x k λ j x, u j u j, x k λ j u j, x 2 λ k Since λ k λ k 1... λ 1, and we have x, u j 2 since {u j } k is an orthonormal basis for W k. Therefore we have: Ax, x min x S n k+1 x 0 8 λ k

9 So since the choice of S n k+1 was arbitrary: S n k+1 min x S n k+1 x 0 Ax, x λ k But, on the other hand, there is a Special Choice of S n k+1, namely S n k+1 Span{u k, u k+1,..., u n }, and then we have S n k+1 W Span{u k }. Finally, using Rayleigh-Ritz for A Sn k+1 : Ax, x min smallest eiqenvalue of A x S n k+1 Sn k+1 λ k. x 0 Note: If k 1 in (1) or (2), we recover Rayleigh-Ritz as a special case. Counterexample in the non-hermitian case [ ] 0 1 Let N be the nilpotent matrix. 0 0 Define the Rayleigh quotient R N (x) exactly as above in the Hermitian case. Then it is easy to see that the only eigenvalue of N is zero, while the imum value of the Rayleigh ratio is 1/2. That is, the imum value of the Rayleigh quotient is larger than the imum eigenvalue. Applications of Courant-Fisher Theorem (Weyl). Let A, B M n be Hermitian with eigenvalues {λ j (A)} n and {λ j (B)} n, and {λ j (A + B)} n, all arranged in non-decreasing order. 8 We then have: λ k (A) + λ 1 (B) λ k (A + B) λ k (A) + λ n (B) Proof. We know from Rayleigh-Ritz that for any nonzero vector x C n : 8 A and B could be considered as kenetic and potential energy matrices of the Schrdinger Hamiltonian operator in Quantum Mechanics. 9

10 λ 1 (B) Bx, x λ n (B), So in order to prove the first inequality, considering A + B by Courant-Fisher we have: λ k (A + B) min S k min S k min S k min S k x S k,x 0 x S k,x 0 x S k,x 0 x S k,x 0 λ k (A) + λ k (B), ( ) (A + B)x, x x ( 2 ) Ax, x Bx, x + ( ) x ( 2 ) Ax, x + λ 1 (B) ( ) Ax, x + λ 1 (B) where the last equality follows from Courant-Fisher for A. Now to prove the second inequality we instead, estimate the second term in ( ) by Bx, x λ n (B) which similarly gives λ k (A + B) λ k (A) + λ n (B). In special cases, we can deduce simpler inequalities Definition. A matrix B M n is called positive semidefinite if, it is Hermitian, and for each x C, Bx, x 0. 10

11 1.1.8 Corollary. Let A, B M n be Hermitian and B positive semidefinite, then λ k (A) λ k (A + B). Proof. Follows from Weyl s Theorem and the fact that λ 1 (B) Remark. A positive semidefinite rank-one matrix B is of the form B zz. Since B is Hermitian, we can write it as B λuu, where λ 0. So we can choose z λu Theorem (Interlacing Theorem). 9 Let A M n be Hermitian and z C n. If {λ j (A)} n and {λ j (A) ± zz } n are in non-decreasing order, then the eigenvalues interlace, that is: λ k (A ± zz ) λ k+1(a) λ k+2 (A ± zz ) Application of min- -min Theorem in Game Theory: 10 Finding the Nash Equilibrium Game theory attempts to mathematically explain behavior in situations in which an individual s outcome depends on the actions of others Definition. An n-person game is one in which there are n players, and a payoff function, which assigns an n-vector to each terminal vertex of the game, indicating each players earnings Definition. A strategy refers to a players plan specifying which choices it will make in every possible situation, leading to an eventual outcome. Let Σ i denote the set of all strategies for player i. In order to decide which strategy is best, player i will have to choose the strategy which imizes its payoff (i.e., the i -th component of the payoff function). Letting π denote the probability of a certain combination of strategies occuring, we can derive a mathematical expression for the payoff function, given player i uses strategy σ i Σ i : 9 If you imagine the eigenvalues of A ± zz and A are arranged in ascending order on two vertical lines parallel to each other, then the comparative order of them somehow resembles how you use your shoe-laces to tie your shoes. 10 cf John Von Neumann and Oskar Morgenstern. Theory of Games and Economic Behavior. Princeton University Press

12 π(σ 1, σ 2,..., σ n ) (π 1 (σ 1...σ n ), π 2 (σ 1...σ n ),..., π n (σ 1...σ n )) where σ 1 represents player 1 s strategy, σ 2 represents player 2 s strategy, and so on, while π 1 represents the probability of player 1 choosing strategy σ 1, π 2 represents the probability of player 2 choosing strategy σ 2, and so on. It is possible to express this function through an n-dimensional array of n-vectors, called the normal form of the game Definition. A strategy n-tuple (σ 1, σ 2,..., σ n ) is said to be a Nash equilibrium if and only if no player has any reason to change its strategy, assuming the other players do not change theirs. That is, the strategy n-tuple (σ 1, σ 2,..., σ n ) is in equilibrium, for any i 1,...n, and any σ i Σ i : π i (σ 1,..., σ i 1, σ i, σ i+1,..., σ n ) π i (σ 1, σ 2,..., σ n ) Definition. A mixed strategy is a probability distribution on the set of a players pure strategies. When a player has a finite number of m strategies, its mixed strategy can be expressed as an m-vector, x (x 1,..., x m ) such that x i 0 and n i1 x i 1 Suppose players (1,2) have pay-off matrices (A n, B n ). Let X denote the set of all mixed strategies for player 1, and Y represent the set of all mixed strategies for player 2. If player 1 chooses mixed strategy x while player 2 chooses mixed strategy y, then the expected pay-off matrices can be written as P A x Ay and P B y Bx. So the Nash Equilibrium would be: P A min y x y Ax P B min x y x By It is straightforward to check that in the case of the pay-off matrix of the famous prisoner s dilemma, the NE is in fact the pair of smallest eigenvalues. 12

13 Prisoner s dilemma Example of PD payoff matrix M Cooperate (with other) Defect (betray other) ( ) Cooperate (with other) 2, 2 0, 3 Defect (betray other) 3, 0 1, 1 Since A ( ) 2 0, B 3 1 ( ) 2 3 and the NE (1, 1) (λ 0 1 min (A), λ min (B)). 13

Matrix Theory, Math6304 Lecture Notes from October 25, 2012

Matrix Theory, Math6304 Lecture Notes from October 25, 2012 Matrix Theory, Math6304 Lecture Notes from October 25, 2012 taken by John Haas Last Time (10/23/12) Example of Low Rank Perturbation Relationship Between Eigenvalues and Principal Submatrices: We started

More information

MATRICES ARE SIMILAR TO TRIANGULAR MATRICES

MATRICES ARE SIMILAR TO TRIANGULAR MATRICES MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,

More information

KAKUTANI S FIXED POINT THEOREM AND THE MINIMAX THEOREM IN GAME THEORY

KAKUTANI S FIXED POINT THEOREM AND THE MINIMAX THEOREM IN GAME THEORY KAKUTANI S FIXED POINT THEOREM AND THE MINIMAX THEOREM IN GAME THEORY YOUNGGEUN YOO Abstract. The imax theorem is one of the most important results in game theory. It was first introduced by John von Neumann

More information

Lecture 9: Low Rank Approximation

Lecture 9: Low Rank Approximation CSE 521: Design and Analysis of Algorithms I Fall 2018 Lecture 9: Low Rank Approximation Lecturer: Shayan Oveis Gharan February 8th Scribe: Jun Qi Disclaimer: These notes have not been subjected to the

More information

Math 408 Advanced Linear Algebra

Math 408 Advanced Linear Algebra Math 408 Advanced Linear Algebra Chi-Kwong Li Chapter 4 Hermitian and symmetric matrices Basic properties Theorem Let A M n. The following are equivalent. Remark (a) A is Hermitian, i.e., A = A. (b) x

More information

6.891 Games, Decision, and Computation February 5, Lecture 2

6.891 Games, Decision, and Computation February 5, Lecture 2 6.891 Games, Decision, and Computation February 5, 2015 Lecture 2 Lecturer: Constantinos Daskalakis Scribe: Constantinos Daskalakis We formally define games and the solution concepts overviewed in Lecture

More information

SPECTRAL PROPERTIES OF THE LAPLACIAN ON BOUNDED DOMAINS

SPECTRAL PROPERTIES OF THE LAPLACIAN ON BOUNDED DOMAINS SPECTRAL PROPERTIES OF THE LAPLACIAN ON BOUNDED DOMAINS TSOGTGEREL GANTUMUR Abstract. After establishing discrete spectra for a large class of elliptic operators, we present some fundamental spectral properties

More information

SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS

SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS G. RAMESH Contents Introduction 1 1. Bounded Operators 1 1.3. Examples 3 2. Compact Operators 5 2.1. Properties 6 3. The Spectral Theorem 9 3.3. Self-adjoint

More information

Exercise Sheet 1.

Exercise Sheet 1. Exercise Sheet 1 You can download my lecture and exercise sheets at the address http://sami.hust.edu.vn/giang-vien/?name=huynt 1) Let A, B be sets. What does the statement "A is not a subset of B " mean?

More information

Spectral Clustering on Handwritten Digits Database

Spectral Clustering on Handwritten Digits Database University of Maryland-College Park Advance Scientific Computing I,II Spectral Clustering on Handwritten Digits Database Author: Danielle Middlebrooks Dmiddle1@math.umd.edu Second year AMSC Student Advisor:

More information

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces. Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

More information

The following definition is fundamental.

The following definition is fundamental. 1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic

More information

Throughout these notes we assume V, W are finite dimensional inner product spaces over C.

Throughout these notes we assume V, W are finite dimensional inner product spaces over C. Math 342 - Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal

More information

STAT200C: Review of Linear Algebra

STAT200C: Review of Linear Algebra Stat200C Instructor: Zhaoxia Yu STAT200C: Review of Linear Algebra 1 Review of Linear Algebra 1.1 Vector Spaces, Rank, Trace, and Linear Equations 1.1.1 Rank and Vector Spaces Definition A vector whose

More information

1 Review: symmetric matrices, their eigenvalues and eigenvectors

1 Review: symmetric matrices, their eigenvalues and eigenvectors Cornell University, Fall 2012 Lecture notes on spectral methods in algorithm design CS 6820: Algorithms Studying the eigenvalues and eigenvectors of matrices has powerful consequences for at least three

More information

THE SINGULAR VALUE DECOMPOSITION AND LOW RANK APPROXIMATION

THE SINGULAR VALUE DECOMPOSITION AND LOW RANK APPROXIMATION THE SINGULAR VALUE DECOMPOSITION AND LOW RANK APPROXIMATION MANTAS MAŽEIKA Abstract. The purpose of this paper is to present a largely self-contained proof of the singular value decomposition (SVD), and

More information

Recall the convention that, for us, all vectors are column vectors.

Recall the convention that, for us, all vectors are column vectors. Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists

More information

Prisoner s Dilemma. Veronica Ciocanel. February 25, 2013

Prisoner s Dilemma. Veronica Ciocanel. February 25, 2013 n-person February 25, 2013 n-person Table of contents 1 Equations 5.4, 5.6 2 3 Types of dilemmas 4 n-person n-person GRIM, GRIM, ALLD Useful to think of equations 5.4 and 5.6 in terms of cooperation and

More information

Lecture 2: Linear Algebra Review

Lecture 2: Linear Algebra Review EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1

More information

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination

Math 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column

More information

Lecture notes: Applied linear algebra Part 1. Version 2

Lecture notes: Applied linear algebra Part 1. Version 2 Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and

More information

Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5

Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize

More information

Problem # Max points possible Actual score Total 120

Problem # Max points possible Actual score Total 120 FINAL EXAMINATION - MATH 2121, FALL 2017. Name: ID#: Email: Lecture & Tutorial: Problem # Max points possible Actual score 1 15 2 15 3 10 4 15 5 15 6 15 7 10 8 10 9 15 Total 120 You have 180 minutes to

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,

More information

Chapter 0 Miscellaneous Preliminaries

Chapter 0 Miscellaneous Preliminaries EE 520: Topics Compressed Sensing Linear Algebra Review Notes scribed by Kevin Palmowski, Spring 2013, for Namrata Vaswani s course Notes on matrix spark courtesy of Brian Lois More notes added by Namrata

More information

Lecture 3: Review of Linear Algebra

Lecture 3: Review of Linear Algebra ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,

More information

Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014

Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Linear Algebra A Brief Reminder Purpose. The purpose of this document

More information

LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT FALL 2006 PRINCETON UNIVERSITY

LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT FALL 2006 PRINCETON UNIVERSITY LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT 204 - FALL 2006 PRINCETON UNIVERSITY ALFONSO SORRENTINO 1 Adjoint of a linear operator Note: In these notes, V will denote a n-dimensional euclidean vector

More information

Trace Class Operators and Lidskii s Theorem

Trace Class Operators and Lidskii s Theorem Trace Class Operators and Lidskii s Theorem Tom Phelan Semester 2 2009 1 Introduction The purpose of this paper is to provide the reader with a self-contained derivation of the celebrated Lidskii Trace

More information

Lecture 4: Purifications and fidelity

Lecture 4: Purifications and fidelity CS 766/QIC 820 Theory of Quantum Information (Fall 2011) Lecture 4: Purifications and fidelity Throughout this lecture we will be discussing pairs of registers of the form (X, Y), and the relationships

More information

08a. Operators on Hilbert spaces. 1. Boundedness, continuity, operator norms

08a. Operators on Hilbert spaces. 1. Boundedness, continuity, operator norms (February 24, 2017) 08a. Operators on Hilbert spaces Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ [This document is http://www.math.umn.edu/ garrett/m/real/notes 2016-17/08a-ops

More information

Functional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...

Functional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability... Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................

More information

Lectures 6, 7 and part of 8

Lectures 6, 7 and part of 8 Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,

More information

MATH 829: Introduction to Data Mining and Analysis Principal component analysis

MATH 829: Introduction to Data Mining and Analysis Principal component analysis 1/11 MATH 829: Introduction to Data Mining and Analysis Principal component analysis Dominique Guillot Departments of Mathematical Sciences University of Delaware April 4, 2016 Motivation 2/11 High-dimensional

More information

Lecture 12 : Graph Laplacians and Cheeger s Inequality

Lecture 12 : Graph Laplacians and Cheeger s Inequality CPS290: Algorithmic Foundations of Data Science March 7, 2017 Lecture 12 : Graph Laplacians and Cheeger s Inequality Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Graph Laplacian Maybe the most beautiful

More information

Introduction to quantum information processing

Introduction to quantum information processing Introduction to quantum information processing Measurements and quantum probability Brad Lackey 25 October 2016 MEASUREMENTS AND QUANTUM PROBABILITY 1 of 22 OUTLINE 1 Probability 2 Density Operators 3

More information

Lecture 1 and 2: Random Spanning Trees

Lecture 1 and 2: Random Spanning Trees Recent Advances in Approximation Algorithms Spring 2015 Lecture 1 and 2: Random Spanning Trees Lecturer: Shayan Oveis Gharan March 31st Disclaimer: These notes have not been subjected to the usual scrutiny

More information

Lecture 7: Positive Semidefinite Matrices

Lecture 7: Positive Semidefinite Matrices Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.

More information

Linear Algebra Formulas. Ben Lee

Linear Algebra Formulas. Ben Lee Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries

More information

Linear Algebra Review

Linear Algebra Review January 29, 2013 Table of contents Metrics Metric Given a space X, then d : X X R + 0 and z in X if: d(x, y) = 0 is equivalent to x = y d(x, y) = d(y, x) d(x, y) d(x, z) + d(z, y) is a metric is for all

More information

MATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS

MATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS MATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS There will be eight problems on the final. The following are sample problems. Problem 1. Let F be the vector space of all real valued functions on

More information

October 25, 2013 INNER PRODUCT SPACES

October 25, 2013 INNER PRODUCT SPACES October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal

More information

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra. DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

More information

SPECTRAL THEORY EVAN JENKINS

SPECTRAL THEORY EVAN JENKINS SPECTRAL THEORY EVAN JENKINS Abstract. These are notes from two lectures given in MATH 27200, Basic Functional Analysis, at the University of Chicago in March 2010. The proof of the spectral theorem for

More information

Elementary linear algebra

Elementary linear algebra Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The

More information

Evolution & Learning in Games

Evolution & Learning in Games 1 / 27 Evolution & Learning in Games Econ 243B Jean-Paul Carvalho Lecture 2. Foundations of Evolution & Learning in Games II 2 / 27 Outline In this lecture, we shall: Take a first look at local stability.

More information

LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM

LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM Unless otherwise stated, all vector spaces in this worksheet are finite dimensional and the scalar field F is R or C. Definition 1. A linear operator

More information

A Linear Algebra Primer

A Linear Algebra Primer CSE 594: Combinatorial and Graph Algorithms SUNY at Buffalo, Fall 2006 Lecturer: Hung Q. Ngo Scribe: Hung Q. Ngo A Linear Algebra Primer Standard texts on Linear Algebra and Algebra are [1, 8]. 1 Preliminaries

More information

Lecture 18. Ramanujan Graphs continued

Lecture 18. Ramanujan Graphs continued Stanford University Winter 218 Math 233A: Non-constructive methods in combinatorics Instructor: Jan Vondrák Lecture date: March 8, 218 Original scribe: László Miklós Lovász Lecture 18 Ramanujan Graphs

More information

NORMS ON SPACE OF MATRICES

NORMS ON SPACE OF MATRICES NORMS ON SPACE OF MATRICES. Operator Norms on Space of linear maps Let A be an n n real matrix and x 0 be a vector in R n. We would like to use the Picard iteration method to solve for the following system

More information

1 Quantum states and von Neumann entropy

1 Quantum states and von Neumann entropy Lecture 9: Quantum entropy maximization CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: February 15, 2016 1 Quantum states and von Neumann entropy Recall that S sym n n

More information

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88

Math Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88 Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant

More information

OPTIMALITY AND STABILITY OF SYMMETRIC EVOLUTIONARY GAMES WITH APPLICATIONS IN GENETIC SELECTION. (Communicated by Yang Kuang)

OPTIMALITY AND STABILITY OF SYMMETRIC EVOLUTIONARY GAMES WITH APPLICATIONS IN GENETIC SELECTION. (Communicated by Yang Kuang) MATHEMATICAL BIOSCIENCES doi:10.3934/mbe.2015.12.503 AND ENGINEERING Volume 12, Number 3, June 2015 pp. 503 523 OPTIMALITY AND STABILITY OF SYMMETRIC EVOLUTIONARY GAMES WITH APPLICATIONS IN GENETIC SELECTION

More information

Math 3191 Applied Linear Algebra

Math 3191 Applied Linear Algebra Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have

More information

Chapter 4 Euclid Space

Chapter 4 Euclid Space Chapter 4 Euclid Space Inner Product Spaces Definition.. Let V be a real vector space over IR. A real inner product on V is a real valued function on V V, denoted by (, ), which satisfies () (x, y) = (y,

More information

Linear algebra and applications to graphs Part 1

Linear algebra and applications to graphs Part 1 Linear algebra and applications to graphs Part 1 Written up by Mikhail Belkin and Moon Duchin Instructor: Laszlo Babai June 17, 2001 1 Basic Linear Algebra Exercise 1.1 Let V and W be linear subspaces

More information

Coalitional Structure of the Muller-Satterthwaite Theorem

Coalitional Structure of the Muller-Satterthwaite Theorem Coalitional Structure of the Muller-Satterthwaite Theorem Pingzhong Tang and Tuomas Sandholm Computer Science Department Carnegie Mellon University {kenshin,sandholm}@cscmuedu Abstract The Muller-Satterthwaite

More information

5 Compact linear operators

5 Compact linear operators 5 Compact linear operators One of the most important results of Linear Algebra is that for every selfadjoint linear map A on a finite-dimensional space, there exists a basis consisting of eigenvectors.

More information

Math 113 Winter 2013 Prof. Church Midterm Solutions

Math 113 Winter 2013 Prof. Church Midterm Solutions Math 113 Winter 2013 Prof. Church Midterm Solutions Name: Student ID: Signature: Question 1 (20 points). Let V be a finite-dimensional vector space, and let T L(V, W ). Assume that v 1,..., v n is a basis

More information

Principal Component Analysis

Principal Component Analysis Principal Component Analysis Laurenz Wiskott Institute for Theoretical Biology Humboldt-University Berlin Invalidenstraße 43 D-10115 Berlin, Germany 11 March 2004 1 Intuition Problem Statement Experimental

More information

Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,

Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, 2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for

More information

Computing Minmax; Dominance

Computing Minmax; Dominance Computing Minmax; Dominance CPSC 532A Lecture 5 Computing Minmax; Dominance CPSC 532A Lecture 5, Slide 1 Lecture Overview 1 Recap 2 Linear Programming 3 Computational Problems Involving Maxmin 4 Domination

More information

Linear Algebra and Dirac Notation, Pt. 2

Linear Algebra and Dirac Notation, Pt. 2 Linear Algebra and Dirac Notation, Pt. 2 PHYS 500 - Southern Illinois University February 1, 2017 PHYS 500 - Southern Illinois University Linear Algebra and Dirac Notation, Pt. 2 February 1, 2017 1 / 14

More information

Nullity of Measurement-induced Nonlocality. Yu Guo

Nullity of Measurement-induced Nonlocality. Yu Guo Jul. 18-22, 2011, at Taiyuan. Nullity of Measurement-induced Nonlocality Yu Guo (Joint work with Pro. Jinchuan Hou) 1 1 27 Department of Mathematics Shanxi Datong University Datong, China guoyu3@yahoo.com.cn

More information

UCSD ECE269 Handout #18 Prof. Young-Han Kim Monday, March 19, Final Examination (Total: 130 points)

UCSD ECE269 Handout #18 Prof. Young-Han Kim Monday, March 19, Final Examination (Total: 130 points) UCSD ECE269 Handout #8 Prof Young-Han Kim Monday, March 9, 208 Final Examination (Total: 30 points) There are 5 problems, each with multiple parts Your answer should be as clear and succinct as possible

More information

7. Symmetric Matrices and Quadratic Forms

7. Symmetric Matrices and Quadratic Forms Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value

More information

5 Quiver Representations

5 Quiver Representations 5 Quiver Representations 5. Problems Problem 5.. Field embeddings. Recall that k(y,..., y m ) denotes the field of rational functions of y,..., y m over a field k. Let f : k[x,..., x n ] k(y,..., y m )

More information

Fiedler s Theorems on Nodal Domains

Fiedler s Theorems on Nodal Domains Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A. Spielman September 19, 2018 7.1 Overview In today s lecture we will justify some of the behavior we observed when using eigenvectors

More information

Math Matrix Algebra

Math Matrix Algebra Math 44 - Matrix Algebra Review notes - (Alberto Bressan, Spring 7) sec: Orthogonal diagonalization of symmetric matrices When we seek to diagonalize a general n n matrix A, two difficulties may arise:

More information

Homework 2. Solutions T =

Homework 2. Solutions T = Homework. s Let {e x, e y, e z } be an orthonormal basis in E. Consider the following ordered triples: a) {e x, e x + e y, 5e z }, b) {e y, e x, 5e z }, c) {e y, e x, e z }, d) {e y, e x, 5e z }, e) {

More information

Eigenvalues and diagonalization

Eigenvalues and diagonalization Eigenvalues and diagonalization Patrick Breheny November 15 Patrick Breheny BST 764: Applied Statistical Modeling 1/20 Introduction The next topic in our course, principal components analysis, revolves

More information

Math 2331 Linear Algebra

Math 2331 Linear Algebra 6.2 Orthogonal Sets Math 233 Linear Algebra 6.2 Orthogonal Sets Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/ jiwenhe/math233 Jiwen He, University of Houston

More information

Asymptotic distribution of eigenvalues of Laplace operator

Asymptotic distribution of eigenvalues of Laplace operator Asymptotic distribution of eigenvalues of Laplace operator 23.8.2013 Topics We will talk about: the number of eigenvalues of Laplace operator smaller than some λ as a function of λ asymptotic behaviour

More information

Lecture December 2009 Fall 2009 Scribe: R. Ring In this lecture we will talk about

Lecture December 2009 Fall 2009 Scribe: R. Ring In this lecture we will talk about 0368.4170: Cryptography and Game Theory Ran Canetti and Alon Rosen Lecture 7 02 December 2009 Fall 2009 Scribe: R. Ring In this lecture we will talk about Two-Player zero-sum games (min-max theorem) Mixed

More information

ECE 275A Homework #3 Solutions

ECE 275A Homework #3 Solutions ECE 75A Homework #3 Solutions. Proof of (a). Obviously Ax = 0 y, Ax = 0 for all y. To show sufficiency, note that if y, Ax = 0 for all y, then it must certainly be true for the particular value of y =

More information

Finite and infinite dimensional generalizations of Klyachko theorem. Shmuel Friedland. August 15, 1999

Finite and infinite dimensional generalizations of Klyachko theorem. Shmuel Friedland. August 15, 1999 Finite and infinite dimensional generalizations of Klyachko theorem Shmuel Friedland Department of Mathematics, Statistics, and Computer Science University of Illinois Chicago 322 SEO, 851 S. Morgan, Chicago,

More information

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.

MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation

More information

CS286.2 Lecture 15: Tsirelson s characterization of XOR games

CS286.2 Lecture 15: Tsirelson s characterization of XOR games CS86. Lecture 5: Tsirelson s characterization of XOR games Scribe: Zeyu Guo We first recall the notion of quantum multi-player games: a quantum k-player game involves a verifier V and k players P,...,

More information

arxiv: v1 [math.na] 5 May 2011

arxiv: v1 [math.na] 5 May 2011 ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU arxiv:1105.1185v1 [math.na] 5 May 2011 Abstract. We examine some numerical iterative methods for computing the eigenvalues and

More information

The Principles of Quantum Mechanics: Pt. 1

The Principles of Quantum Mechanics: Pt. 1 The Principles of Quantum Mechanics: Pt. 1 PHYS 476Q - Southern Illinois University February 15, 2018 PHYS 476Q - Southern Illinois University The Principles of Quantum Mechanics: Pt. 1 February 15, 2018

More information

MATH 426, TOPOLOGY. p 1.

MATH 426, TOPOLOGY. p 1. MATH 426, TOPOLOGY THE p-norms In this document we assume an extended real line, where is an element greater than all real numbers; the interval notation [1, ] will be used to mean [1, ) { }. 1. THE p

More information

Linear Algebra Massoud Malek

Linear Algebra Massoud Malek CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product

More information

Homework 11 Solutions. Math 110, Fall 2013.

Homework 11 Solutions. Math 110, Fall 2013. Homework 11 Solutions Math 110, Fall 2013 1 a) Suppose that T were self-adjoint Then, the Spectral Theorem tells us that there would exist an orthonormal basis of P 2 (R), (p 1, p 2, p 3 ), consisting

More information

Linear Algebra 1. M.T.Nair Department of Mathematics, IIT Madras. and in that case x is called an eigenvector of T corresponding to the eigenvalue λ.

Linear Algebra 1. M.T.Nair Department of Mathematics, IIT Madras. and in that case x is called an eigenvector of T corresponding to the eigenvalue λ. Linear Algebra 1 M.T.Nair Department of Mathematics, IIT Madras 1 Eigenvalues and Eigenvectors 1.1 Definition and Examples Definition 1.1. Let V be a vector space (over a field F) and T : V V be a linear

More information

Variational Principles for Nonlinear Eigenvalue Problems

Variational Principles for Nonlinear Eigenvalue Problems Variational Principles for Nonlinear Eigenvalue Problems Heinrich Voss voss@tuhh.de Hamburg University of Technology Institute of Mathematics TUHH Heinrich Voss Variational Principles for Nonlinear EVPs

More information

Belief-based Learning

Belief-based Learning Belief-based Learning Algorithmic Game Theory Marcello Restelli Lecture Outline Introdutcion to multi-agent learning Belief-based learning Cournot adjustment Fictitious play Bayesian learning Equilibrium

More information

Lecture 21: HSP via the Pretty Good Measurement

Lecture 21: HSP via the Pretty Good Measurement Quantum Computation (CMU 5-859BB, Fall 205) Lecture 2: HSP via the Pretty Good Measurement November 8, 205 Lecturer: John Wright Scribe: Joshua Brakensiek The Average-Case Model Recall from Lecture 20

More information

Near-Potential Games: Geometry and Dynamics

Near-Potential Games: Geometry and Dynamics Near-Potential Games: Geometry and Dynamics Ozan Candogan, Asuman Ozdaglar and Pablo A. Parrilo January 29, 2012 Abstract Potential games are a special class of games for which many adaptive user dynamics

More information

Fiedler s Theorems on Nodal Domains

Fiedler s Theorems on Nodal Domains Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A Spielman September 9, 202 7 About these notes These notes are not necessarily an accurate representation of what happened in

More information

Spectra of Adjacency and Laplacian Matrices

Spectra of Adjacency and Laplacian Matrices Spectra of Adjacency and Laplacian Matrices Definition: University of Alicante (Spain) Matrix Computing (subject 3168 Degree in Maths) 30 hours (theory)) + 15 hours (practical assignment) Contents 1. Spectra

More information

Linear Algebra Lecture Notes-II

Linear Algebra Lecture Notes-II Linear Algebra Lecture Notes-II Vikas Bist Department of Mathematics Panjab University, Chandigarh-64 email: bistvikas@gmail.com Last revised on March 5, 8 This text is based on the lectures delivered

More information

Chapter 3 Transformations

Chapter 3 Transformations Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases

More information

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012

Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.

More information

REPRESENTATION THEORY WEEK 7

REPRESENTATION THEORY WEEK 7 REPRESENTATION THEORY WEEK 7 1. Characters of L k and S n A character of an irreducible representation of L k is a polynomial function constant on every conjugacy class. Since the set of diagonalizable

More information

Chapter 6: Orthogonality

Chapter 6: Orthogonality Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products

More information

D-bounded Distance-Regular Graphs

D-bounded Distance-Regular Graphs D-bounded Distance-Regular Graphs CHIH-WEN WENG 53706 Abstract Let Γ = (X, R) denote a distance-regular graph with diameter D 3 and distance function δ. A (vertex) subgraph X is said to be weak-geodetically

More information

Lecture 8 : Eigenvalues and Eigenvectors

Lecture 8 : Eigenvalues and Eigenvectors CPS290: Algorithmic Foundations of Data Science February 24, 2017 Lecture 8 : Eigenvalues and Eigenvectors Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Hermitian Matrices It is simpler to begin with

More information

Section 6.2, 6.3 Orthogonal Sets, Orthogonal Projections

Section 6.2, 6.3 Orthogonal Sets, Orthogonal Projections Section 6. 6. Orthogonal Sets Orthogonal Projections Main Ideas in these sections: Orthogonal set = A set of mutually orthogonal vectors. OG LI. Orthogonal Projection of y onto u or onto an OG set {u u

More information

Chapter 1 Vector Spaces

Chapter 1 Vector Spaces Chapter 1 Vector Spaces Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 110 Linear Algebra Vector Spaces Definition A vector space V over a field

More information

Lecture 4 Eigenvalue problems

Lecture 4 Eigenvalue problems Lecture 4 Eigenvalue problems Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn

More information