Matrix Theory, Math6304 Lecture Notes from March 22, 2016 taken by Kazem Safari
|
|
- Edwin Booker
- 6 years ago
- Views:
Transcription
1 Matrix Theory, Math6304 Lecture Notes from March 22, 2016 taken by Kazem Safari 1.1 Applications of Courant-Fisher and min- or -min Last time: Courant -Fishert -min or min- for eigenvalues Warm-up: Sums of eigenvalues from optimization problems Proposition. Let A M n be Hermitian, i.e. A A, and with eigenvalues λ 1 λ 2... λ n. Then λ j min tr[ap ]. In order to prove this remarkable result, we need to recall some Orthogonal Projection theory from Real Analysis 1 : If A is a closed subspace 2 of a Hilbert space H 3, for every x H we can define: δ inf x y. y A It then follows that δ < and there exist a unique z A that achieves this infimum, i.e. δ inf x y x z. y A 1 For the proof of these results cf Folland, Real Analysis Modern Techniques and Their Application, for the purpose of defining the orthogonal projection, it suffices for our subset A to be convex. 3 In this course H R n or C n. Therefore we can identify each linear maps with a matrix and vice-versa. 1
2 P (x) : z is called the orthogonal projection onto closed subspace A. Then, we have: x P (x) A 4 And every element x H can be uniquely written as: x P x + (x P x) where P x A and x P x A. In other words: H A A Proposition. The orthogonal projection operator P : H A has the following properties: 1) P is a linear continuous map, and P 1. 2) P 2 P, i.e. P A id. 3) R(P ) A and null(p ) A. 4) P P. 5) rank(p ) tr(p ). 6) Any eigenvalue of P is either 0 or 1. Conversely: Proposition. Suppose that P L(H, H) satisfies P 2 P P. Then R(P ) is closed and P is the orthogonal projection onto R(P ) Definition. we define P k as the set of orthogonal projections of rank k. 5 4 A {x H x, a 0 x A} 5 Unfortunately, the space of orthogonal projections of rank k is not a linear space but it is what we call a Variety. 2
3 Proof of the warm-up. Consider n orthonormal eigenvectors {u j } n corresponding to eigenvalues {λ j } n of our Hermitian matrix A. If we define U [u 1... u n ], then one eigendecomposition of A is A UΛU UU U U I and Λ diag[λ 1,..., λ n ]. Therefore where A n λ ju j u j. Next, if P P k, then since P 1. Now, P u j 2 u j 2 1. Since P 2 P, and since P P, k rank(p ) tr(p ) tr(u P U) u 1 u 2 tr. P [u 1 u 2... u n ] u n u 1 u 2 u n tr. [P u 1 P u 2... P u n ] u 1P u 1 u 1P u 2 u 1P u n u 2P u 1 u 2P u 2 u 2P u n tr u np u 1 u np u 2 u np u n tr[p u j u j] u jp u j P u j, u j P 2 u j, u j 3
4 So by the definition of the adjoint operator, P P u j, u j P u j, P u j P u j 2. Thus, if we define x j : tr(p u j u j) then by the same procedure as above: x j P u j u ju l, u l P u j δ j,l, u l P u j, u j P u j 2. Then by combining the two previous results, for each j we have: 0 x j 1 and x j k. Now let X k : {x [0, 1] n s.t. n x j k}. Then 6 : min tr[ap ] min tr[p A] min tr[ P λ j u j u j] min min λ j tr[p u j u j] λ j x j 6 since tr is linear and tr(ab) tr(ba) for all A, B M n n (C). 4
5 Now we are going to invoke the variational principal in optimization, which in essence says that by properly relaxing the conditions of a structurally highly complicated problem, the min only goes lower and goes higher. min x X k λ j x j Which turns the problem into minimizing over a linear k-polytope. Claim 7 : min x X k λ j x j λ j proof of the claim. if x l 0 for any l > k then there exist a ɛ > 0 such that λ l λ k + ɛ. Then: λ j x j λ j x j + λ l x l λ j x j + (λ k + ɛ)x l k 1 λ j x j + λ k (x k + x l ) + ɛx l k 1 λ j x j + λ k (x k + x l ) Meaning whenever any eigenvalue greater than λ k has a positive weight we can redistribute that weight among the eigenvalues less than or equal to λ k and achieve a lower overall value. Therefore we must have x k+1 x k+2... x n 0 7 This problem is very similar to water-filling algorithm in Singal Processing. 5
6 On the other hand, since x X k, we must fully scale the first k eigenvalues: Conversely, Choosing P k u ju j x 1 x 2... x k 1. It is straightforward to check that P L(H, H) and P 2 P P. Therefore by the latter Prop, P is the orthogonal projection into R(P ), which is a closed linear subspace of H. Moreover we have: P u j u j u 1 j u 2 j u n j. [ ] u 1 j u 2 j u n j u 1 ju 1 j u 1 ju 2 j u 1 ju n j u 2 ju 1 1 u 2 ju 2 j u 2 ju n j u n j u 1 1 u n j u 2 j u n j u n j Therefore we can easily see that: tr(p ) u j 2 k. But rank(p ) tr(p ) k by the former Prop, therefore we see that P P k as well. Thus: 6
7 min P P tr[ap ] tr[ap ] tr[a u j u j] tr[au j u j] tr[λ j u j u j] λ j tr(u j u j) λ j u j 2 λ j. And now we conclude that min tr[ap ] λ j. Moral: We can replace /min or min/ by a sequence of minimizations over P k, and therefore, we can relate the spectrum of submatrices to the whole matrix. Recall: Theorem (Courant-Fischer). Suppose A M n is Hermitian, i.e. A A. Now, for each 1 k n, let {S α k } α I k, where α I k denote the set of all k dimensional linear subspaces of H, and enumerate the n eigenvalues λ 1,..., λ n (counting multiplicity) in increasing order, i.e. λ 1 λ 2,..., λ n. Then, we have 7
8 (i) min α I k x Sk α\{0} Ax, x / x 2 λ k. (ii) α J n k+1 min x Sn k+1 α \{0} Ax x / x 2 λ k. Proof of part (ii). Let W Span{u 1, u 2,..., u k }, dim W k. Then if dim S n k+1 n k+1 then by dimension counting, i.e. 1 dim(w S n k+1 ) dim W + dim S n k+1 dim(w S n k+1 ), S n k+1 W 0, and therefore, there exists an x (S n k+1 W ) {0}, with x k x, u j u j. Therefore R A (x) Ax, x k λ j x, u j u j, x k λ j x, u j u j, x k λ j u j, x 2 λ k Since λ k λ k 1... λ 1, and we have x, u j 2 since {u j } k is an orthonormal basis for W k. Therefore we have: Ax, x min x S n k+1 x 0 8 λ k
9 So since the choice of S n k+1 was arbitrary: S n k+1 min x S n k+1 x 0 Ax, x λ k But, on the other hand, there is a Special Choice of S n k+1, namely S n k+1 Span{u k, u k+1,..., u n }, and then we have S n k+1 W Span{u k }. Finally, using Rayleigh-Ritz for A Sn k+1 : Ax, x min smallest eiqenvalue of A x S n k+1 Sn k+1 λ k. x 0 Note: If k 1 in (1) or (2), we recover Rayleigh-Ritz as a special case. Counterexample in the non-hermitian case [ ] 0 1 Let N be the nilpotent matrix. 0 0 Define the Rayleigh quotient R N (x) exactly as above in the Hermitian case. Then it is easy to see that the only eigenvalue of N is zero, while the imum value of the Rayleigh ratio is 1/2. That is, the imum value of the Rayleigh quotient is larger than the imum eigenvalue. Applications of Courant-Fisher Theorem (Weyl). Let A, B M n be Hermitian with eigenvalues {λ j (A)} n and {λ j (B)} n, and {λ j (A + B)} n, all arranged in non-decreasing order. 8 We then have: λ k (A) + λ 1 (B) λ k (A + B) λ k (A) + λ n (B) Proof. We know from Rayleigh-Ritz that for any nonzero vector x C n : 8 A and B could be considered as kenetic and potential energy matrices of the Schrdinger Hamiltonian operator in Quantum Mechanics. 9
10 λ 1 (B) Bx, x λ n (B), So in order to prove the first inequality, considering A + B by Courant-Fisher we have: λ k (A + B) min S k min S k min S k min S k x S k,x 0 x S k,x 0 x S k,x 0 x S k,x 0 λ k (A) + λ k (B), ( ) (A + B)x, x x ( 2 ) Ax, x Bx, x + ( ) x ( 2 ) Ax, x + λ 1 (B) ( ) Ax, x + λ 1 (B) where the last equality follows from Courant-Fisher for A. Now to prove the second inequality we instead, estimate the second term in ( ) by Bx, x λ n (B) which similarly gives λ k (A + B) λ k (A) + λ n (B). In special cases, we can deduce simpler inequalities Definition. A matrix B M n is called positive semidefinite if, it is Hermitian, and for each x C, Bx, x 0. 10
11 1.1.8 Corollary. Let A, B M n be Hermitian and B positive semidefinite, then λ k (A) λ k (A + B). Proof. Follows from Weyl s Theorem and the fact that λ 1 (B) Remark. A positive semidefinite rank-one matrix B is of the form B zz. Since B is Hermitian, we can write it as B λuu, where λ 0. So we can choose z λu Theorem (Interlacing Theorem). 9 Let A M n be Hermitian and z C n. If {λ j (A)} n and {λ j (A) ± zz } n are in non-decreasing order, then the eigenvalues interlace, that is: λ k (A ± zz ) λ k+1(a) λ k+2 (A ± zz ) Application of min- -min Theorem in Game Theory: 10 Finding the Nash Equilibrium Game theory attempts to mathematically explain behavior in situations in which an individual s outcome depends on the actions of others Definition. An n-person game is one in which there are n players, and a payoff function, which assigns an n-vector to each terminal vertex of the game, indicating each players earnings Definition. A strategy refers to a players plan specifying which choices it will make in every possible situation, leading to an eventual outcome. Let Σ i denote the set of all strategies for player i. In order to decide which strategy is best, player i will have to choose the strategy which imizes its payoff (i.e., the i -th component of the payoff function). Letting π denote the probability of a certain combination of strategies occuring, we can derive a mathematical expression for the payoff function, given player i uses strategy σ i Σ i : 9 If you imagine the eigenvalues of A ± zz and A are arranged in ascending order on two vertical lines parallel to each other, then the comparative order of them somehow resembles how you use your shoe-laces to tie your shoes. 10 cf John Von Neumann and Oskar Morgenstern. Theory of Games and Economic Behavior. Princeton University Press
12 π(σ 1, σ 2,..., σ n ) (π 1 (σ 1...σ n ), π 2 (σ 1...σ n ),..., π n (σ 1...σ n )) where σ 1 represents player 1 s strategy, σ 2 represents player 2 s strategy, and so on, while π 1 represents the probability of player 1 choosing strategy σ 1, π 2 represents the probability of player 2 choosing strategy σ 2, and so on. It is possible to express this function through an n-dimensional array of n-vectors, called the normal form of the game Definition. A strategy n-tuple (σ 1, σ 2,..., σ n ) is said to be a Nash equilibrium if and only if no player has any reason to change its strategy, assuming the other players do not change theirs. That is, the strategy n-tuple (σ 1, σ 2,..., σ n ) is in equilibrium, for any i 1,...n, and any σ i Σ i : π i (σ 1,..., σ i 1, σ i, σ i+1,..., σ n ) π i (σ 1, σ 2,..., σ n ) Definition. A mixed strategy is a probability distribution on the set of a players pure strategies. When a player has a finite number of m strategies, its mixed strategy can be expressed as an m-vector, x (x 1,..., x m ) such that x i 0 and n i1 x i 1 Suppose players (1,2) have pay-off matrices (A n, B n ). Let X denote the set of all mixed strategies for player 1, and Y represent the set of all mixed strategies for player 2. If player 1 chooses mixed strategy x while player 2 chooses mixed strategy y, then the expected pay-off matrices can be written as P A x Ay and P B y Bx. So the Nash Equilibrium would be: P A min y x y Ax P B min x y x By It is straightforward to check that in the case of the pay-off matrix of the famous prisoner s dilemma, the NE is in fact the pair of smallest eigenvalues. 12
13 Prisoner s dilemma Example of PD payoff matrix M Cooperate (with other) Defect (betray other) ( ) Cooperate (with other) 2, 2 0, 3 Defect (betray other) 3, 0 1, 1 Since A ( ) 2 0, B 3 1 ( ) 2 3 and the NE (1, 1) (λ 0 1 min (A), λ min (B)). 13
Matrix Theory, Math6304 Lecture Notes from October 25, 2012
Matrix Theory, Math6304 Lecture Notes from October 25, 2012 taken by John Haas Last Time (10/23/12) Example of Low Rank Perturbation Relationship Between Eigenvalues and Principal Submatrices: We started
More informationMATRICES ARE SIMILAR TO TRIANGULAR MATRICES
MATRICES ARE SIMILAR TO TRIANGULAR MATRICES 1 Complex matrices Recall that the complex numbers are given by a + ib where a and b are real and i is the imaginary unity, ie, i 2 = 1 In what we describe below,
More informationKAKUTANI S FIXED POINT THEOREM AND THE MINIMAX THEOREM IN GAME THEORY
KAKUTANI S FIXED POINT THEOREM AND THE MINIMAX THEOREM IN GAME THEORY YOUNGGEUN YOO Abstract. The imax theorem is one of the most important results in game theory. It was first introduced by John von Neumann
More informationLecture 9: Low Rank Approximation
CSE 521: Design and Analysis of Algorithms I Fall 2018 Lecture 9: Low Rank Approximation Lecturer: Shayan Oveis Gharan February 8th Scribe: Jun Qi Disclaimer: These notes have not been subjected to the
More informationMath 408 Advanced Linear Algebra
Math 408 Advanced Linear Algebra Chi-Kwong Li Chapter 4 Hermitian and symmetric matrices Basic properties Theorem Let A M n. The following are equivalent. Remark (a) A is Hermitian, i.e., A = A. (b) x
More information6.891 Games, Decision, and Computation February 5, Lecture 2
6.891 Games, Decision, and Computation February 5, 2015 Lecture 2 Lecturer: Constantinos Daskalakis Scribe: Constantinos Daskalakis We formally define games and the solution concepts overviewed in Lecture
More informationSPECTRAL PROPERTIES OF THE LAPLACIAN ON BOUNDED DOMAINS
SPECTRAL PROPERTIES OF THE LAPLACIAN ON BOUNDED DOMAINS TSOGTGEREL GANTUMUR Abstract. After establishing discrete spectra for a large class of elliptic operators, we present some fundamental spectral properties
More informationSPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS
SPECTRAL THEOREM FOR COMPACT SELF-ADJOINT OPERATORS G. RAMESH Contents Introduction 1 1. Bounded Operators 1 1.3. Examples 3 2. Compact Operators 5 2.1. Properties 6 3. The Spectral Theorem 9 3.3. Self-adjoint
More informationExercise Sheet 1.
Exercise Sheet 1 You can download my lecture and exercise sheets at the address http://sami.hust.edu.vn/giang-vien/?name=huynt 1) Let A, B be sets. What does the statement "A is not a subset of B " mean?
More informationSpectral Clustering on Handwritten Digits Database
University of Maryland-College Park Advance Scientific Computing I,II Spectral Clustering on Handwritten Digits Database Author: Danielle Middlebrooks Dmiddle1@math.umd.edu Second year AMSC Student Advisor:
More informationMath 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.
Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,
More informationThe following definition is fundamental.
1. Some Basics from Linear Algebra With these notes, I will try and clarify certain topics that I only quickly mention in class. First and foremost, I will assume that you are familiar with many basic
More informationThroughout these notes we assume V, W are finite dimensional inner product spaces over C.
Math 342 - Linear Algebra II Notes Throughout these notes we assume V, W are finite dimensional inner product spaces over C 1 Upper Triangular Representation Proposition: Let T L(V ) There exists an orthonormal
More informationSTAT200C: Review of Linear Algebra
Stat200C Instructor: Zhaoxia Yu STAT200C: Review of Linear Algebra 1 Review of Linear Algebra 1.1 Vector Spaces, Rank, Trace, and Linear Equations 1.1.1 Rank and Vector Spaces Definition A vector whose
More information1 Review: symmetric matrices, their eigenvalues and eigenvectors
Cornell University, Fall 2012 Lecture notes on spectral methods in algorithm design CS 6820: Algorithms Studying the eigenvalues and eigenvectors of matrices has powerful consequences for at least three
More informationTHE SINGULAR VALUE DECOMPOSITION AND LOW RANK APPROXIMATION
THE SINGULAR VALUE DECOMPOSITION AND LOW RANK APPROXIMATION MANTAS MAŽEIKA Abstract. The purpose of this paper is to present a largely self-contained proof of the singular value decomposition (SVD), and
More informationRecall the convention that, for us, all vectors are column vectors.
Some linear algebra Recall the convention that, for us, all vectors are column vectors. 1. Symmetric matrices Let A be a real matrix. Recall that a complex number λ is an eigenvalue of A if there exists
More informationPrisoner s Dilemma. Veronica Ciocanel. February 25, 2013
n-person February 25, 2013 n-person Table of contents 1 Equations 5.4, 5.6 2 3 Types of dilemmas 4 n-person n-person GRIM, GRIM, ALLD Useful to think of equations 5.4 and 5.6 in terms of cooperation and
More informationLecture 2: Linear Algebra Review
EE 227A: Convex Optimization and Applications January 19 Lecture 2: Linear Algebra Review Lecturer: Mert Pilanci Reading assignment: Appendix C of BV. Sections 2-6 of the web textbook 1 2.1 Vectors 2.1.1
More informationMath 102, Winter Final Exam Review. Chapter 1. Matrices and Gaussian Elimination
Math 0, Winter 07 Final Exam Review Chapter. Matrices and Gaussian Elimination { x + x =,. Different forms of a system of linear equations. Example: The x + 4x = 4. [ ] [ ] [ ] vector form (or the column
More informationLecture notes: Applied linear algebra Part 1. Version 2
Lecture notes: Applied linear algebra Part 1. Version 2 Michael Karow Berlin University of Technology karow@math.tu-berlin.de October 2, 2008 1 Notation, basic notions and facts 1.1 Subspaces, range and
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5
Semidefinite and Second Order Cone Programming Seminar Fall 2001 Lecture 5 Instructor: Farid Alizadeh Scribe: Anton Riabov 10/08/2001 1 Overview We continue studying the maximum eigenvalue SDP, and generalize
More informationProblem # Max points possible Actual score Total 120
FINAL EXAMINATION - MATH 2121, FALL 2017. Name: ID#: Email: Lecture & Tutorial: Problem # Max points possible Actual score 1 15 2 15 3 10 4 15 5 15 6 15 7 10 8 10 9 15 Total 120 You have 180 minutes to
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters, transforms,
More informationChapter 0 Miscellaneous Preliminaries
EE 520: Topics Compressed Sensing Linear Algebra Review Notes scribed by Kevin Palmowski, Spring 2013, for Namrata Vaswani s course Notes on matrix spark courtesy of Brian Lois More notes added by Namrata
More informationLecture 3: Review of Linear Algebra
ECE 83 Fall 2 Statistical Signal Processing instructor: R Nowak, scribe: R Nowak Lecture 3: Review of Linear Algebra Very often in this course we will represent signals as vectors and operators (eg, filters,
More informationDuke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014
Duke University, Department of Electrical and Computer Engineering Optimization for Scientists and Engineers c Alex Bronstein, 2014 Linear Algebra A Brief Reminder Purpose. The purpose of this document
More informationLECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT FALL 2006 PRINCETON UNIVERSITY
LECTURE VI: SELF-ADJOINT AND UNITARY OPERATORS MAT 204 - FALL 2006 PRINCETON UNIVERSITY ALFONSO SORRENTINO 1 Adjoint of a linear operator Note: In these notes, V will denote a n-dimensional euclidean vector
More informationTrace Class Operators and Lidskii s Theorem
Trace Class Operators and Lidskii s Theorem Tom Phelan Semester 2 2009 1 Introduction The purpose of this paper is to provide the reader with a self-contained derivation of the celebrated Lidskii Trace
More informationLecture 4: Purifications and fidelity
CS 766/QIC 820 Theory of Quantum Information (Fall 2011) Lecture 4: Purifications and fidelity Throughout this lecture we will be discussing pairs of registers of the form (X, Y), and the relationships
More information08a. Operators on Hilbert spaces. 1. Boundedness, continuity, operator norms
(February 24, 2017) 08a. Operators on Hilbert spaces Paul Garrett garrett@math.umn.edu http://www.math.umn.edu/ garrett/ [This document is http://www.math.umn.edu/ garrett/m/real/notes 2016-17/08a-ops
More informationFunctional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...
Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................
More informationLectures 6, 7 and part of 8
Lectures 6, 7 and part of 8 Uriel Feige April 26, May 3, May 10, 2015 1 Linear programming duality 1.1 The diet problem revisited Recall the diet problem from Lecture 1. There are n foods, m nutrients,
More informationMATH 829: Introduction to Data Mining and Analysis Principal component analysis
1/11 MATH 829: Introduction to Data Mining and Analysis Principal component analysis Dominique Guillot Departments of Mathematical Sciences University of Delaware April 4, 2016 Motivation 2/11 High-dimensional
More informationLecture 12 : Graph Laplacians and Cheeger s Inequality
CPS290: Algorithmic Foundations of Data Science March 7, 2017 Lecture 12 : Graph Laplacians and Cheeger s Inequality Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Graph Laplacian Maybe the most beautiful
More informationIntroduction to quantum information processing
Introduction to quantum information processing Measurements and quantum probability Brad Lackey 25 October 2016 MEASUREMENTS AND QUANTUM PROBABILITY 1 of 22 OUTLINE 1 Probability 2 Density Operators 3
More informationLecture 1 and 2: Random Spanning Trees
Recent Advances in Approximation Algorithms Spring 2015 Lecture 1 and 2: Random Spanning Trees Lecturer: Shayan Oveis Gharan March 31st Disclaimer: These notes have not been subjected to the usual scrutiny
More informationLecture 7: Positive Semidefinite Matrices
Lecture 7: Positive Semidefinite Matrices Rajat Mittal IIT Kanpur The main aim of this lecture note is to prepare your background for semidefinite programming. We have already seen some linear algebra.
More informationLinear Algebra Formulas. Ben Lee
Linear Algebra Formulas Ben Lee January 27, 2016 Definitions and Terms Diagonal: Diagonal of matrix A is a collection of entries A ij where i = j. Diagonal Matrix: A matrix (usually square), where entries
More informationLinear Algebra Review
January 29, 2013 Table of contents Metrics Metric Given a space X, then d : X X R + 0 and z in X if: d(x, y) = 0 is equivalent to x = y d(x, y) = d(y, x) d(x, y) d(x, z) + d(z, y) is a metric is for all
More informationMATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS
MATH 320: PRACTICE PROBLEMS FOR THE FINAL AND SOLUTIONS There will be eight problems on the final. The following are sample problems. Problem 1. Let F be the vector space of all real valued functions on
More informationOctober 25, 2013 INNER PRODUCT SPACES
October 25, 2013 INNER PRODUCT SPACES RODICA D. COSTIN Contents 1. Inner product 2 1.1. Inner product 2 1.2. Inner product spaces 4 2. Orthogonal bases 5 2.1. Existence of an orthogonal basis 7 2.2. Orthogonal
More informationDS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationSPECTRAL THEORY EVAN JENKINS
SPECTRAL THEORY EVAN JENKINS Abstract. These are notes from two lectures given in MATH 27200, Basic Functional Analysis, at the University of Chicago in March 2010. The proof of the spectral theorem for
More informationElementary linear algebra
Chapter 1 Elementary linear algebra 1.1 Vector spaces Vector spaces owe their importance to the fact that so many models arising in the solutions of specific problems turn out to be vector spaces. The
More informationEvolution & Learning in Games
1 / 27 Evolution & Learning in Games Econ 243B Jean-Paul Carvalho Lecture 2. Foundations of Evolution & Learning in Games II 2 / 27 Outline In this lecture, we shall: Take a first look at local stability.
More informationLINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM
LINEAR ALGEBRA BOOT CAMP WEEK 4: THE SPECTRAL THEOREM Unless otherwise stated, all vector spaces in this worksheet are finite dimensional and the scalar field F is R or C. Definition 1. A linear operator
More informationA Linear Algebra Primer
CSE 594: Combinatorial and Graph Algorithms SUNY at Buffalo, Fall 2006 Lecturer: Hung Q. Ngo Scribe: Hung Q. Ngo A Linear Algebra Primer Standard texts on Linear Algebra and Algebra are [1, 8]. 1 Preliminaries
More informationLecture 18. Ramanujan Graphs continued
Stanford University Winter 218 Math 233A: Non-constructive methods in combinatorics Instructor: Jan Vondrák Lecture date: March 8, 218 Original scribe: László Miklós Lovász Lecture 18 Ramanujan Graphs
More informationNORMS ON SPACE OF MATRICES
NORMS ON SPACE OF MATRICES. Operator Norms on Space of linear maps Let A be an n n real matrix and x 0 be a vector in R n. We would like to use the Picard iteration method to solve for the following system
More information1 Quantum states and von Neumann entropy
Lecture 9: Quantum entropy maximization CSE 599S: Entropy optimality, Winter 2016 Instructor: James R. Lee Last updated: February 15, 2016 1 Quantum states and von Neumann entropy Recall that S sym n n
More informationMath Camp Lecture 4: Linear Algebra. Xiao Yu Wang. Aug 2010 MIT. Xiao Yu Wang (MIT) Math Camp /10 1 / 88
Math Camp 2010 Lecture 4: Linear Algebra Xiao Yu Wang MIT Aug 2010 Xiao Yu Wang (MIT) Math Camp 2010 08/10 1 / 88 Linear Algebra Game Plan Vector Spaces Linear Transformations and Matrices Determinant
More informationOPTIMALITY AND STABILITY OF SYMMETRIC EVOLUTIONARY GAMES WITH APPLICATIONS IN GENETIC SELECTION. (Communicated by Yang Kuang)
MATHEMATICAL BIOSCIENCES doi:10.3934/mbe.2015.12.503 AND ENGINEERING Volume 12, Number 3, June 2015 pp. 503 523 OPTIMALITY AND STABILITY OF SYMMETRIC EVOLUTIONARY GAMES WITH APPLICATIONS IN GENETIC SELECTION
More informationMath 3191 Applied Linear Algebra
Math 191 Applied Linear Algebra Lecture 1: Inner Products, Length, Orthogonality Stephen Billups University of Colorado at Denver Math 191Applied Linear Algebra p.1/ Motivation Not all linear systems have
More informationChapter 4 Euclid Space
Chapter 4 Euclid Space Inner Product Spaces Definition.. Let V be a real vector space over IR. A real inner product on V is a real valued function on V V, denoted by (, ), which satisfies () (x, y) = (y,
More informationLinear algebra and applications to graphs Part 1
Linear algebra and applications to graphs Part 1 Written up by Mikhail Belkin and Moon Duchin Instructor: Laszlo Babai June 17, 2001 1 Basic Linear Algebra Exercise 1.1 Let V and W be linear subspaces
More informationCoalitional Structure of the Muller-Satterthwaite Theorem
Coalitional Structure of the Muller-Satterthwaite Theorem Pingzhong Tang and Tuomas Sandholm Computer Science Department Carnegie Mellon University {kenshin,sandholm}@cscmuedu Abstract The Muller-Satterthwaite
More information5 Compact linear operators
5 Compact linear operators One of the most important results of Linear Algebra is that for every selfadjoint linear map A on a finite-dimensional space, there exists a basis consisting of eigenvectors.
More informationMath 113 Winter 2013 Prof. Church Midterm Solutions
Math 113 Winter 2013 Prof. Church Midterm Solutions Name: Student ID: Signature: Question 1 (20 points). Let V be a finite-dimensional vector space, and let T L(V, W ). Assume that v 1,..., v n is a basis
More informationPrincipal Component Analysis
Principal Component Analysis Laurenz Wiskott Institute for Theoretical Biology Humboldt-University Berlin Invalidenstraße 43 D-10115 Berlin, Germany 11 March 2004 1 Intuition Problem Statement Experimental
More informationLecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε,
2. REVIEW OF LINEAR ALGEBRA 1 Lecture 1 Review: Linear models have the form (in matrix notation) Y = Xβ + ε, where Y n 1 response vector and X n p is the model matrix (or design matrix ) with one row for
More informationComputing Minmax; Dominance
Computing Minmax; Dominance CPSC 532A Lecture 5 Computing Minmax; Dominance CPSC 532A Lecture 5, Slide 1 Lecture Overview 1 Recap 2 Linear Programming 3 Computational Problems Involving Maxmin 4 Domination
More informationLinear Algebra and Dirac Notation, Pt. 2
Linear Algebra and Dirac Notation, Pt. 2 PHYS 500 - Southern Illinois University February 1, 2017 PHYS 500 - Southern Illinois University Linear Algebra and Dirac Notation, Pt. 2 February 1, 2017 1 / 14
More informationNullity of Measurement-induced Nonlocality. Yu Guo
Jul. 18-22, 2011, at Taiyuan. Nullity of Measurement-induced Nonlocality Yu Guo (Joint work with Pro. Jinchuan Hou) 1 1 27 Department of Mathematics Shanxi Datong University Datong, China guoyu3@yahoo.com.cn
More informationUCSD ECE269 Handout #18 Prof. Young-Han Kim Monday, March 19, Final Examination (Total: 130 points)
UCSD ECE269 Handout #8 Prof Young-Han Kim Monday, March 9, 208 Final Examination (Total: 30 points) There are 5 problems, each with multiple parts Your answer should be as clear and succinct as possible
More information7. Symmetric Matrices and Quadratic Forms
Linear Algebra 7. Symmetric Matrices and Quadratic Forms CSIE NCU 1 7. Symmetric Matrices and Quadratic Forms 7.1 Diagonalization of symmetric matrices 2 7.2 Quadratic forms.. 9 7.4 The singular value
More information5 Quiver Representations
5 Quiver Representations 5. Problems Problem 5.. Field embeddings. Recall that k(y,..., y m ) denotes the field of rational functions of y,..., y m over a field k. Let f : k[x,..., x n ] k(y,..., y m )
More informationFiedler s Theorems on Nodal Domains
Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A. Spielman September 19, 2018 7.1 Overview In today s lecture we will justify some of the behavior we observed when using eigenvectors
More informationMath Matrix Algebra
Math 44 - Matrix Algebra Review notes - (Alberto Bressan, Spring 7) sec: Orthogonal diagonalization of symmetric matrices When we seek to diagonalize a general n n matrix A, two difficulties may arise:
More informationHomework 2. Solutions T =
Homework. s Let {e x, e y, e z } be an orthonormal basis in E. Consider the following ordered triples: a) {e x, e x + e y, 5e z }, b) {e y, e x, 5e z }, c) {e y, e x, e z }, d) {e y, e x, 5e z }, e) {
More informationEigenvalues and diagonalization
Eigenvalues and diagonalization Patrick Breheny November 15 Patrick Breheny BST 764: Applied Statistical Modeling 1/20 Introduction The next topic in our course, principal components analysis, revolves
More informationMath 2331 Linear Algebra
6.2 Orthogonal Sets Math 233 Linear Algebra 6.2 Orthogonal Sets Jiwen He Department of Mathematics, University of Houston jiwenhe@math.uh.edu math.uh.edu/ jiwenhe/math233 Jiwen He, University of Houston
More informationAsymptotic distribution of eigenvalues of Laplace operator
Asymptotic distribution of eigenvalues of Laplace operator 23.8.2013 Topics We will talk about: the number of eigenvalues of Laplace operator smaller than some λ as a function of λ asymptotic behaviour
More informationLecture December 2009 Fall 2009 Scribe: R. Ring In this lecture we will talk about
0368.4170: Cryptography and Game Theory Ran Canetti and Alon Rosen Lecture 7 02 December 2009 Fall 2009 Scribe: R. Ring In this lecture we will talk about Two-Player zero-sum games (min-max theorem) Mixed
More informationECE 275A Homework #3 Solutions
ECE 75A Homework #3 Solutions. Proof of (a). Obviously Ax = 0 y, Ax = 0 for all y. To show sufficiency, note that if y, Ax = 0 for all y, then it must certainly be true for the particular value of y =
More informationFinite and infinite dimensional generalizations of Klyachko theorem. Shmuel Friedland. August 15, 1999
Finite and infinite dimensional generalizations of Klyachko theorem Shmuel Friedland Department of Mathematics, Statistics, and Computer Science University of Illinois Chicago 322 SEO, 851 S. Morgan, Chicago,
More informationMATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators.
MATH 423 Linear Algebra II Lecture 33: Diagonalization of normal operators. Adjoint operator and adjoint matrix Given a linear operator L on an inner product space V, the adjoint of L is a transformation
More informationCS286.2 Lecture 15: Tsirelson s characterization of XOR games
CS86. Lecture 5: Tsirelson s characterization of XOR games Scribe: Zeyu Guo We first recall the notion of quantum multi-player games: a quantum k-player game involves a verifier V and k players P,...,
More informationarxiv: v1 [math.na] 5 May 2011
ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU arxiv:1105.1185v1 [math.na] 5 May 2011 Abstract. We examine some numerical iterative methods for computing the eigenvalues and
More informationThe Principles of Quantum Mechanics: Pt. 1
The Principles of Quantum Mechanics: Pt. 1 PHYS 476Q - Southern Illinois University February 15, 2018 PHYS 476Q - Southern Illinois University The Principles of Quantum Mechanics: Pt. 1 February 15, 2018
More informationMATH 426, TOPOLOGY. p 1.
MATH 426, TOPOLOGY THE p-norms In this document we assume an extended real line, where is an element greater than all real numbers; the interval notation [1, ] will be used to mean [1, ) { }. 1. THE p
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More informationHomework 11 Solutions. Math 110, Fall 2013.
Homework 11 Solutions Math 110, Fall 2013 1 a) Suppose that T were self-adjoint Then, the Spectral Theorem tells us that there would exist an orthonormal basis of P 2 (R), (p 1, p 2, p 3 ), consisting
More informationLinear Algebra 1. M.T.Nair Department of Mathematics, IIT Madras. and in that case x is called an eigenvector of T corresponding to the eigenvalue λ.
Linear Algebra 1 M.T.Nair Department of Mathematics, IIT Madras 1 Eigenvalues and Eigenvectors 1.1 Definition and Examples Definition 1.1. Let V be a vector space (over a field F) and T : V V be a linear
More informationVariational Principles for Nonlinear Eigenvalue Problems
Variational Principles for Nonlinear Eigenvalue Problems Heinrich Voss voss@tuhh.de Hamburg University of Technology Institute of Mathematics TUHH Heinrich Voss Variational Principles for Nonlinear EVPs
More informationBelief-based Learning
Belief-based Learning Algorithmic Game Theory Marcello Restelli Lecture Outline Introdutcion to multi-agent learning Belief-based learning Cournot adjustment Fictitious play Bayesian learning Equilibrium
More informationLecture 21: HSP via the Pretty Good Measurement
Quantum Computation (CMU 5-859BB, Fall 205) Lecture 2: HSP via the Pretty Good Measurement November 8, 205 Lecturer: John Wright Scribe: Joshua Brakensiek The Average-Case Model Recall from Lecture 20
More informationNear-Potential Games: Geometry and Dynamics
Near-Potential Games: Geometry and Dynamics Ozan Candogan, Asuman Ozdaglar and Pablo A. Parrilo January 29, 2012 Abstract Potential games are a special class of games for which many adaptive user dynamics
More informationFiedler s Theorems on Nodal Domains
Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A Spielman September 9, 202 7 About these notes These notes are not necessarily an accurate representation of what happened in
More informationSpectra of Adjacency and Laplacian Matrices
Spectra of Adjacency and Laplacian Matrices Definition: University of Alicante (Spain) Matrix Computing (subject 3168 Degree in Maths) 30 hours (theory)) + 15 hours (practical assignment) Contents 1. Spectra
More informationLinear Algebra Lecture Notes-II
Linear Algebra Lecture Notes-II Vikas Bist Department of Mathematics Panjab University, Chandigarh-64 email: bistvikas@gmail.com Last revised on March 5, 8 This text is based on the lectures delivered
More informationChapter 3 Transformations
Chapter 3 Transformations An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Linear Transformations A function is called a linear transformation if 1. for every and 2. for every If we fix the bases
More informationPreliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012
Instructions Preliminary/Qualifying Exam in Numerical Analysis (Math 502a) Spring 2012 The exam consists of four problems, each having multiple parts. You should attempt to solve all four problems. 1.
More informationREPRESENTATION THEORY WEEK 7
REPRESENTATION THEORY WEEK 7 1. Characters of L k and S n A character of an irreducible representation of L k is a polynomial function constant on every conjugacy class. Since the set of diagonalizable
More informationChapter 6: Orthogonality
Chapter 6: Orthogonality (Last Updated: November 7, 7) These notes are derived primarily from Linear Algebra and its applications by David Lay (4ed). A few theorems have been moved around.. Inner products
More informationD-bounded Distance-Regular Graphs
D-bounded Distance-Regular Graphs CHIH-WEN WENG 53706 Abstract Let Γ = (X, R) denote a distance-regular graph with diameter D 3 and distance function δ. A (vertex) subgraph X is said to be weak-geodetically
More informationLecture 8 : Eigenvalues and Eigenvectors
CPS290: Algorithmic Foundations of Data Science February 24, 2017 Lecture 8 : Eigenvalues and Eigenvectors Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Hermitian Matrices It is simpler to begin with
More informationSection 6.2, 6.3 Orthogonal Sets, Orthogonal Projections
Section 6. 6. Orthogonal Sets Orthogonal Projections Main Ideas in these sections: Orthogonal set = A set of mutually orthogonal vectors. OG LI. Orthogonal Projection of y onto u or onto an OG set {u u
More informationChapter 1 Vector Spaces
Chapter 1 Vector Spaces Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 110 Linear Algebra Vector Spaces Definition A vector space V over a field
More informationLecture 4 Eigenvalue problems
Lecture 4 Eigenvalue problems Weinan E 1,2 and Tiejun Li 2 1 Department of Mathematics, Princeton University, weinan@princeton.edu 2 School of Mathematical Sciences, Peking University, tieli@pku.edu.cn
More information