U.C. Berkeley CS294: Spectral Methods and Expanders Handout 8 Luca Trevisan February 17, 2016
|
|
- Stephany Bishop
- 5 years ago
- Views:
Transcription
1 U.C. Berkeley CS94: Spectral Methods and Expanders Handout 8 Luca Trevsan February 7, 06 Lecture 8: Spectral Algorthms Wrap-up In whch we talk about even more generalzatons of Cheeger s nequaltes, and we analyze the power method to fnd approxmate egenvectors, thus havng a complete descrpton of a polynomal-tme approxmaton algorthm for sparsest cut Irregular Graphs For smplcty, we proved our results on λ and λ k for regular graphs. Those results extend, essentally wth the same proofs, to the case of rregular undrected graphs. In an rregular graph G = (V, E, the noton that generalzes edge expanson s called conductance. If d v s the degree of vertex v, then the conductance of set S of vertces s E(S, V S φ(s := v S d v We wll call the sum of the degrees of a set of vertces the volume of the set, and denote t vol(s := v S d v. The conductance of the graph G s φ(g := mn S:vol(S vol(v φ(s Hgher-order conductance s defned as hgher-order expanson, but wth conductance replacng expanson n the defnton. The Cheeger nequaltes λ φ(g λ stll hold, wth the same proof. Wth some abuse of notaton, we wll call the followng quantty the Raylegh quotent of x R L (x := {u,v} E (x u x v v V d vx v
2 even f, techncally, t s the Raylegh quotent of D / x, where D s the dagonal matrx of degrees. We can also adapt the proof of the hgher-order Cheeger nequalty to show λ k φ k(g O(k 3.5 λ k More Cheeger-type Bounds We proved that f (S F, V S F s the cut found by Fedler s algorthm usng the egenvector of λ, then φ(s F, V S F φ(g whch s a good bound, although t usually underestmates the qualty of the solutons found n practce. (There are graphs, however, for whch the above nequalty s tght wthn a constant factor. One case n whch we can mprove the analyss s when there are not too many egenvalues close to λ Theorem There s a constant c such that, f (S F, V S F s the cut obtaned by Fedler s algorthm usng an egenvector for λ, then, for every k, φ(s F, V S F c k λ λk So we have φ(s F, V S F c φ(g mn k k λk whch s a better bound for famles of graphs n whch, for some k, λ k >> k λ. We wll not have tme to prove Theorem, but we wll state the two man peces of ts proof. Lemma Let x R V 0 be a non-negatve vector. Then, for every k, there s a non-negatve vector y R V 0 whose entres take at most k dstnct values and such that x y R L(x λ k x
3 That s, f R L (x >> λ k, then there are k values such that most entres of x are close to one of those k values. Lemma 3 There s a constant c such that, for every non-negatve vectors x R V 0 and y R V 0, f y s such that ts entrs contan only k dstnct values, then there s a threshold t > 0 such that φ({v : x v t} c k ( R L (x + R L (x x y x The above lemma should be compared to the fact, whch was a major pece n the proof of Cheeger s nequalty, that f x R V 0 s an arbtrary non-negatve vector, then there s a threshold t > 0 such that φ({v : x v t} R L (x One obtans Theorem n the followng. Start from an egenvector x of λ and, usng the frst step n the proof of Cheeger s nequalty, obtan a vector x R V 0 wth non-negatve entres such that R L (x R L (x = λ and such that the support of x contans at most V / vertces. Use Lemma to fnd a vector y wth non-negatve entres and wth at most k dstnct values among ts entres such that x y λ λ k x. Then use Lemma 3 and the fact that λ k to conclude that there exsts at t > 0 such that φ({v : x v t} O(k λ λk The set {v : x v t} contans at most V / vertces, t s one of the cuts consdered by Fedler s algorthm on nput x. Another property of graphs n whch λ k s large for small k s that they contan large expanders as nduced subgraphs. Theorem 4 There s a constant c such that, for every graph G and every k, there exsts a partton of the vertces nto l k sets (S..., S l such that, f we call G the subgraph nduced by the vertex set S, we have φ G c λ k k Theorem 5 If φ k+ > (+ɛφ k, then there s a partton of the vertces nto k subsets (S,..., S k such that ( ɛ {,..., k} : φ G Ω φ k+, φ(s kφ k k 3
4 3 The Power Method Earler n ths class, we showed that, f G = (V, E s a d-regular graph, and L s ts normalzed Laplacan matrx wth egenvalues 0 = λ λ... λ n, gven an egenvector of λ, Fedler s algorthm fnds, n nearly-lnear tme O( E + V log V, a cut (S, V S such that φ(s φ(g. More generally, f, nstead of beng gven an egenvector x such that Lx = λ x, we are gven a vector x such that x T Lx (λ + ɛx T x, then the algorthm fnds a cut such that φ(s 4φ(G + ɛ. We wll now see how to compute such a vector usng O(( V + E ɛ log V ɛ arthmetc operatons. A symmetrc matrx s postve sem-defnte (abbrevated PSD f all ts egenvalues are nonnegatve. We begn by descrbng an algorthm that approxmates the largest egenvalue of a gven symmetrc PSD matrx. Ths mght not seem to help very much because because we want to compute the second smallest, not the largest, egenvalue. We wll see, however, that the algorthm s easly modfed to accomplsh what we want. 3. The Power Method to Approxmate the Largest Egenvalue The algorthm works as follows Algorthm Power Input: PSD matrx M, parameter k Pck unformly at random x 0 {, } n for := to k x := M x return x k That s, the algorthm smply pcks unformly at random a vector x wth ± coordnates, and outputs M k x. Note that the algorthm performs O(k (n + m arthmetc operatons, where m s the number of non-zero entres of the matrx M. Theorem 6 For every PSD matrx M, postve nteger k and parameter ɛ > 0, wth probablty 3/6 over the choce of x 0, the algorthm Power outputs a vector x k such that 4
5 x T k Mx k x T k x k λ ( ɛ where λ s the largest egenvalue of M. + 4n( ɛ k Note that, n partcular, we can have k = O(log n/ɛ and xt k Mx k ( O(ɛ λ x T k x. k Let λ λ n be the egenvalues of M, wth multplctes, and v,..., v n be a system of orthonormal egenvectors such that Mv = λ v. Theorem 6 s mpled by the followng two lemmas Lemma 7 Let v R n be a vector such that v =. Sample unformly x {, } n. Then P [ x, v ] 3 6 Lemma 8 For every x R n, for every postve nteger k and postve ɛ > 0, f we defne y := M k x, we have y T My y T y λ ( ɛ ( + x ( ɛk x, v It remans to prove the two lemmas. Proof: (Of Lemma 7 Let v = (v,..., v n. The nner product x, v s the random varable S := x v Let us compute the frst, second, and fourth moment of S. E S = 0 E S = v = E S 4 = 3 ( v v 4 3 5
6 Recall that the Paley-Zygmund nequalty states that f Z s a non-negatve random varable wth fnte varance, then, for every 0 δ, we have whch follows by notng that P[Z δ E Z] ( δ (E Z E Z ( that E Z = E[Z Z<δ E Z ] + E[Z Z δ E Z ], and that E[Z Z<δ E Z ] δ E Z, E[Z Z δ E Z ] E Z E Z δ E Z = E Z P[Z δ E Z] We apply the Paley-Zygmund nequalty to the case Z = S and δ = /4, and we derve P [ S ] 4 ( = 3 6 Remark 9 The proof of Lemma 7 works even f x {, } n s selected accordng to a 4-wse ndependent dstrbuton. Ths means that the algorthm can be derandomzed n polynomal tme. Proof: (Of Lemma 8 Let us wrte x as a lnear combnaton of the egenvectors x = a v + + a n v n where the coeffcents can be computed as a = x, v. We have and so y = a λ k v + + a n λ k nv n 6
7 and y T My = y T y = a λ k+ a λ k We need to prove a lower bound to the rato of the above two quanttes. We wll compute a lower bound to the numerator and an upper bound to the denomnator n terms of the same parameter. Let l be the number of egenvalues larger than λ ( ɛ. Then, recallng that the egenvalues are sorted n non-ncreasng order, we have y T My l = a λ k+ λ ( ɛ l = a λ k We also see that n =l+ a λ k λ k ( ɛ k n =l+ a λ k ( ɛ k x a λ k ( ɛ t x a x a ( ɛ k l = a λ k So we have gvng y T y ( + x ( ɛ k a l = a y T My y T y λ ( ɛ ( + x ( ɛ k a 7
8 Remark 0 Where dd we use the assumpton that M s postve semdefnte? What happens f we apply ths algorthm to the adjacency matrx of a bpartte graph? 3. Approxmatng the Second Largest Egenvalue Suppose now that we are nterested n fndng the second largest egenvalue of a gven PSD matrx M. If M has egenvalues λ λ λ n, and we know the egenvector v of λ, then M s a PSD lnear map from the orthogonal space to v to tself, and λ s the largest egenvalue of ths lnear map. We can then run the prevous algorthm on ths lnear map. Algorthm Power Input: PSD matrx M, vector v parameter k Pck unformly at random x {, } n x 0 := x v x, v for := to k x := M x return x k If v,..., v n s an orthonormal bass of egenvectors for the egenvalues λ λ n of M, then, at the begnnng, we pck a random vector x = a v + a v + a n v n that, wth probablty at least 3/6, satsfes a /. (Cf. Lemma 7. Then we compute x 0, whch s the projecton of x on the subspace orthogonal to v, that s Note that x = n and that x 0 n. The output s the vector x k x 0 = a v + a n v n x k = a λ k v + a n λ k nv n If we apply Lemma 8 to subspace orthogonal to v, we see that when a / we have that, for every 0 < ɛ <, x T k Mx k x T k x k λ ( ɛ 4n( ɛ k We have thus establshed the followng analyss. 8
9 Theorem For every PSD matrx M, postve nteger k and parameter ɛ > 0, f v s a length- egenvector of the largest egenvalue of M, then wth probablty 3/6 over the choce of x 0, the algorthm Power outputs a vector x k v such that x T k Mx k x T k x k λ ( ɛ + 4n( ɛ k where λ s the second largest egenvalue of M, countng multplctes. 3.3 The Second Smallest Egenvalue of the Laplacan Fnally, we come to the case n whch we want to compute the second smallest egenvalue of the normalzed Laplacan matrx L = I A of a d-regular graph G = (V, E, d where A s the adjacency matrx of G. Consder the matrx M := I L = I + d A. Then f 0 = λ... λ n are the egenvalues of L, we have that = λ λ λ n 0 are the egenvalues of M, and that M s PSD. M and L have the same egenvectors, and so v = n (,..., s a length- egenvector of the largest egenvalue of M. By runnng algorthm Power, we can fnd a vector x such that and so, rearrangng, we have x T Mx T ( ɛ ( λ x T x x T Mx T = x T x x T Lx x T Lx x T x λ + ɛ If we want to compute a vector whose Raylegh quotent s, say, at most λ, then the runnng tme wll be Õ(( V + E /λ, because we need to set ɛ = λ /, whch s not nearly lnear n the sze of the graph f λ s, say O(/ V. For a runnng tme that s nearly lnear n n for all values of λ, one can, nstead, apply the power method to the pseudonverse L + of L. (Assumng that the graph s connected, L + x s the unque vector y such that Ly = x, f x (,...,, and L + x = 0 f x s parallel to (,...,. Ths s because L + has egenvalues 0, /λ,..., /λ n, and so L + s PSD and /λ s ts largest egenvalue. Although computng L + s not known to be doable n nearly lnear tme, there are nearly lnear tme algorthms that, gven x, solve n y the lnear system Ly = x, 9
10 and ths s the same as computng the product L + x, whch s enough to mplement algorthm Power appled to L +. (Such algorthms wll be dscussed n the thrd part of the course. The algorthms wll fnd an approxmate soluton y to the lnear system Ly = x, but ths wll be suffcent. In the followng, we proceed as f the soluton was exact. In tme O((V + E (log V /ɛ O(, we can fnd a vector y such that y = (L + k x, where x s a random vector n {, } n, shfted to be orthogonal to (,..., and k = O(log V /ɛ. What s the Raylegh quotent of such a vector wth respect to L? Let v,..., v n be a bass of orthonormal egenvectors for L and L +. If 0 = λ λ λ n are the egenvalues of L, then we have and, for =,..., n, we have Lv = L + v = 0 Lv = λ L + v = λ Wrte x = a v + a n v n, where a n, and ssume that, as happens wth probablty at least 3/6, we have a. Then 4 y = n = a λ k and the Raylegh quotent of y wth respect to L s y T Ly y T y = a a λ k and the analyss proceeds smlarly to the analyss of the prevous secton. If we let l be the ndex such that λ l (+ɛ λ λ l+ then we can upper bound the numerator as a λ k l l l a a a λ k + λ k ( + ɛ k λ k + n λ k ( + ɛ k λ k + 4na λ k ( + ɛ k λ k >l a 0
11 ( 4n + ( + ɛ k and we can lower bound the denomnator as and the Raylegh quotent s when k = O ( ɛ log n ɛ. a λ k l ( + ɛλ l l a a a λ k λ k λ k ( y T Ly y T y λ 4n ( + ɛ + ( + ɛ λ ( + ɛ k
Stanford University Graph Partitioning and Expanders Handout 3 Luca Trevisan May 8, 2013
Stanford Unversty Graph Parttonng and Expanders Handout 3 Luca Trevsan May 8, 03 Lecture 3 In whch we analyze the power method to approxmate egenvalues and egenvectors, and we descrbe some more algorthmc
More informationStanford University CS359G: Graph Partitioning and Expanders Handout 4 Luca Trevisan January 13, 2011
Stanford Unversty CS359G: Graph Parttonng and Expanders Handout 4 Luca Trevsan January 3, 0 Lecture 4 In whch we prove the dffcult drecton of Cheeger s nequalty. As n the past lectures, consder an undrected
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Luca Trevisan September 5, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 4s Luca Trevsan September 5, 07 Summary of Lecture 4 In whch we ntroduce semdefnte programmng and apply t to Max Cut. Semdefnte Programmng Recall that
More informationU.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan 2/21/2008. Notes for Lecture 8
U.C. Berkeley CS278: Computatonal Complexty Handout N8 Professor Luca Trevsan 2/21/2008 Notes for Lecture 8 1 Undrected Connectvty In the undrected s t connectvty problem (abbrevated ST-UCONN) we are gven
More informationEigenvalues of Random Graphs
Spectral Graph Theory Lecture 2 Egenvalues of Random Graphs Danel A. Spelman November 4, 202 2. Introducton In ths lecture, we consder a random graph on n vertces n whch each edge s chosen to be n the
More informationLectures - Week 4 Matrix norms, Conditioning, Vector Spaces, Linear Independence, Spanning sets and Basis, Null space and Range of a Matrix
Lectures - Week 4 Matrx norms, Condtonng, Vector Spaces, Lnear Independence, Spannng sets and Bass, Null space and Range of a Matrx Matrx Norms Now we turn to assocatng a number to each matrx. We could
More informationLecture 10: May 6, 2013
TTIC/CMSC 31150 Mathematcal Toolkt Sprng 013 Madhur Tulsan Lecture 10: May 6, 013 Scrbe: Wenje Luo In today s lecture, we manly talked about random walk on graphs and ntroduce the concept of graph expander,
More informationFinding Dense Subgraphs in G(n, 1/2)
Fndng Dense Subgraphs n Gn, 1/ Atsh Das Sarma 1, Amt Deshpande, and Rav Kannan 1 Georga Insttute of Technology,atsh@cc.gatech.edu Mcrosoft Research-Bangalore,amtdesh,annan@mcrosoft.com Abstract. Fndng
More informationU.C. Berkeley CS294: Beyond Worst-Case Analysis Handout 6 Luca Trevisan September 12, 2017
U.C. Berkeley CS94: Beyond Worst-Case Analyss Handout 6 Luca Trevsan September, 07 Scrbed by Theo McKenze Lecture 6 In whch we study the spectrum of random graphs. Overvew When attemptng to fnd n polynomal
More informationLecture 2: Gram-Schmidt Vectors and the LLL Algorithm
NYU, Fall 2016 Lattces Mn Course Lecture 2: Gram-Schmdt Vectors and the LLL Algorthm Lecturer: Noah Stephens-Davdowtz 2.1 The Shortest Vector Problem In our last lecture, we consdered short solutons to
More informationStanford University CS254: Computational Complexity Notes 7 Luca Trevisan January 29, Notes for Lecture 7
Stanford Unversty CS54: Computatonal Complexty Notes 7 Luca Trevsan January 9, 014 Notes for Lecture 7 1 Approxmate Countng wt an N oracle We complete te proof of te followng result: Teorem 1 For every
More informationMatrix Approximation via Sampling, Subspace Embedding. 1 Solving Linear Systems Using SVD
Matrx Approxmaton va Samplng, Subspace Embeddng Lecturer: Anup Rao Scrbe: Rashth Sharma, Peng Zhang 0/01/016 1 Solvng Lnear Systems Usng SVD Two applcatons of SVD have been covered so far. Today we loo
More informationSTAT 309: MATHEMATICAL COMPUTATIONS I FALL 2018 LECTURE 16
STAT 39: MATHEMATICAL COMPUTATIONS I FALL 218 LECTURE 16 1 why teratve methods f we have a lnear system Ax = b where A s very, very large but s ether sparse or structured (eg, banded, Toepltz, banded plus
More informationMATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS
MATH 241B FUNCTIONAL ANALYSIS - NOTES EXAMPLES OF C ALGEBRAS These are nformal notes whch cover some of the materal whch s not n the course book. The man purpose s to gve a number of nontrval examples
More informationProblem Set 9 Solutions
Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem
More informationC/CS/Phy191 Problem Set 3 Solutions Out: Oct 1, 2008., where ( 00. ), so the overall state of the system is ) ( ( ( ( 00 ± 11 ), Φ ± = 1
C/CS/Phy9 Problem Set 3 Solutons Out: Oct, 8 Suppose you have two qubts n some arbtrary entangled state ψ You apply the teleportaton protocol to each of the qubts separately What s the resultng state obtaned
More information8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS
SECTION 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS 493 8.4 COMPLEX VECTOR SPACES AND INNER PRODUCTS All the vector spaces you have studed thus far n the text are real vector spaces because the scalars
More informationLecture 4: Universal Hash Functions/Streaming Cont d
CSE 5: Desgn and Analyss of Algorthms I Sprng 06 Lecture 4: Unversal Hash Functons/Streamng Cont d Lecturer: Shayan Oves Gharan Aprl 6th Scrbe: Jacob Schreber Dsclamer: These notes have not been subjected
More information2.3 Nilpotent endomorphisms
s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms
More informationExpected Value and Variance
MATH 38 Expected Value and Varance Dr. Neal, WKU We now shall dscuss how to fnd the average and standard devaton of a random varable X. Expected Value Defnton. The expected value (or average value, or
More informationEdge Isoperimetric Inequalities
November 7, 2005 Ross M. Rchardson Edge Isopermetrc Inequaltes 1 Four Questons Recall that n the last lecture we looked at the problem of sopermetrc nequaltes n the hypercube, Q n. Our noton of boundary
More information= z 20 z n. (k 20) + 4 z k = 4
Problem Set #7 solutons 7.2.. (a Fnd the coeffcent of z k n (z + z 5 + z 6 + z 7 + 5, k 20. We use the known seres expanson ( n+l ( z l l z n below: (z + z 5 + z 6 + z 7 + 5 (z 5 ( + z + z 2 + z + 5 5
More informationInner Product. Euclidean Space. Orthonormal Basis. Orthogonal
Inner Product Defnton 1 () A Eucldean space s a fnte-dmensonal vector space over the reals R, wth an nner product,. Defnton 2 (Inner Product) An nner product, on a real vector space X s a symmetrc, blnear,
More informationLecture 20: Lift and Project, SDP Duality. Today we will study the Lift and Project method. Then we will prove the SDP duality theorem.
prnceton u. sp 02 cos 598B: algorthms and complexty Lecture 20: Lft and Project, SDP Dualty Lecturer: Sanjeev Arora Scrbe:Yury Makarychev Today we wll study the Lft and Project method. Then we wll prove
More informationAPPENDIX A Some Linear Algebra
APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,
More informationLecture 4: Constant Time SVD Approximation
Spectral Algorthms and Representatons eb. 17, Mar. 3 and 8, 005 Lecture 4: Constant Tme SVD Approxmaton Lecturer: Santosh Vempala Scrbe: Jangzhuo Chen Ths topc conssts of three lectures 0/17, 03/03, 03/08),
More informationCalculation of time complexity (3%)
Problem 1. (30%) Calculaton of tme complexty (3%) Gven n ctes, usng exhaust search to see every result takes O(n!). Calculaton of tme needed to solve the problem (2%) 40 ctes:40! dfferent tours 40 add
More informationMATH 5707 HOMEWORK 4 SOLUTIONS 2. 2 i 2p i E(X i ) + E(Xi 2 ) ä i=1. i=1
MATH 5707 HOMEWORK 4 SOLUTIONS CİHAN BAHRAN 1. Let v 1,..., v n R m, all lengths v are not larger than 1. Let p 1,..., p n [0, 1] be arbtrary and set w = p 1 v 1 + + p n v n. Then there exst ε 1,..., ε
More informationCOS 521: Advanced Algorithms Game Theory and Linear Programming
COS 521: Advanced Algorthms Game Theory and Lnear Programmng Moses Charkar February 27, 2013 In these notes, we ntroduce some basc concepts n game theory and lnear programmng (LP). We show a connecton
More information3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X
Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number
More information= = = (a) Use the MATLAB command rref to solve the system. (b) Let A be the coefficient matrix and B be the right-hand side of the system.
Chapter Matlab Exercses Chapter Matlab Exercses. Consder the lnear system of Example n Secton.. x x x y z y y z (a) Use the MATLAB command rref to solve the system. (b) Let A be the coeffcent matrx and
More information763622S ADVANCED QUANTUM MECHANICS Solution Set 1 Spring c n a n. c n 2 = 1.
7636S ADVANCED QUANTUM MECHANICS Soluton Set 1 Sprng 013 1 Warm-up Show that the egenvalues of a Hermtan operator  are real and that the egenkets correspondng to dfferent egenvalues are orthogonal (b)
More informationThe lower and upper bounds on Perron root of nonnegative irreducible matrices
Journal of Computatonal Appled Mathematcs 217 (2008) 259 267 wwwelsevercom/locate/cam The lower upper bounds on Perron root of nonnegatve rreducble matrces Guang-Xn Huang a,, Feng Yn b,keguo a a College
More informationLINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity
LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have
More informationComplete subgraphs in multipartite graphs
Complete subgraphs n multpartte graphs FLORIAN PFENDER Unverstät Rostock, Insttut für Mathematk D-18057 Rostock, Germany Floran.Pfender@un-rostock.de Abstract Turán s Theorem states that every graph G
More informationSection 8.3 Polar Form of Complex Numbers
80 Chapter 8 Secton 8 Polar Form of Complex Numbers From prevous classes, you may have encountered magnary numbers the square roots of negatve numbers and, more generally, complex numbers whch are the
More informationCSCE 790S Background Results
CSCE 790S Background Results Stephen A. Fenner September 8, 011 Abstract These results are background to the course CSCE 790S/CSCE 790B, Quantum Computaton and Informaton (Sprng 007 and Fall 011). Each
More informationLinear Approximation with Regularization and Moving Least Squares
Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...
More informationFeb 14: Spatial analysis of data fields
Feb 4: Spatal analyss of data felds Mappng rregularly sampled data onto a regular grd Many analyss technques for geophyscal data requre the data be located at regular ntervals n space and/or tme. hs s
More informationNOTES ON SIMPLIFICATION OF MATRICES
NOTES ON SIMPLIFICATION OF MATRICES JONATHAN LUK These notes dscuss how to smplfy an (n n) matrx In partcular, we expand on some of the materal from the textbook (wth some repetton) Part of the exposton
More informationLecture Space-Bounded Derandomization
Notes on Complexty Theory Last updated: October, 2008 Jonathan Katz Lecture Space-Bounded Derandomzaton 1 Space-Bounded Derandomzaton We now dscuss derandomzaton of space-bounded algorthms. Here non-trval
More informationNUMERICAL DIFFERENTIATION
NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the
More informationMin Cut, Fast Cut, Polynomial Identities
Randomzed Algorthms, Summer 016 Mn Cut, Fast Cut, Polynomal Identtes Instructor: Thomas Kesselhem and Kurt Mehlhorn 1 Mn Cuts n Graphs Lecture (5 pages) Throughout ths secton, G = (V, E) s a mult-graph.
More informationLecture 12: Discrete Laplacian
Lecture 12: Dscrete Laplacan Scrbe: Tanye Lu Our goal s to come up wth a dscrete verson of Laplacan operator for trangulated surfaces, so that we can use t n practce to solve related problems We are mostly
More information1 Matrix representations of canonical matrices
1 Matrx representatons of canoncal matrces 2-d rotaton around the orgn: ( ) cos θ sn θ R 0 = sn θ cos θ 3-d rotaton around the x-axs: R x = 1 0 0 0 cos θ sn θ 0 sn θ cos θ 3-d rotaton around the y-axs:
More informationMaximizing the number of nonnegative subsets
Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum
More informationInexact Newton Methods for Inverse Eigenvalue Problems
Inexact Newton Methods for Inverse Egenvalue Problems Zheng-jan Ba Abstract In ths paper, we survey some of the latest development n usng nexact Newton-lke methods for solvng nverse egenvalue problems.
More information18.1 Introduction and Recap
CS787: Advanced Algorthms Scrbe: Pryananda Shenoy and Shjn Kong Lecturer: Shuch Chawla Topc: Streamng Algorthmscontnued) Date: 0/26/2007 We contnue talng about streamng algorthms n ths lecture, ncludng
More informationSpectral graph theory: Applications of Courant-Fischer
Spectral graph theory: Applcatons of Courant-Fscher Steve Butler September 2006 Abstract In ths second talk we wll ntroduce the Raylegh quotent and the Courant- Fscher Theorem and gve some applcatons for
More informationFeature Selection: Part 1
CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?
More informationPerron Vectors of an Irreducible Nonnegative Interval Matrix
Perron Vectors of an Irreducble Nonnegatve Interval Matrx Jr Rohn August 4 2005 Abstract As s well known an rreducble nonnegatve matrx possesses a unquely determned Perron vector. As the man result of
More informationRandić Energy and Randić Estrada Index of a Graph
EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS Vol. 5, No., 202, 88-96 ISSN 307-5543 www.ejpam.com SPECIAL ISSUE FOR THE INTERNATIONAL CONFERENCE ON APPLIED ANALYSIS AND ALGEBRA 29 JUNE -02JULY 20, ISTANBUL
More informationChat eld, C. and A.J.Collins, Introduction to multivariate analysis. Chapman & Hall, 1980
MT07: Multvarate Statstcal Methods Mke Tso: emal mke.tso@manchester.ac.uk Webpage for notes: http://www.maths.manchester.ac.uk/~mkt/new_teachng.htm. Introducton to multvarate data. Books Chat eld, C. and
More informationSingular Value Decomposition: Theory and Applications
Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real
More informationREAL ANALYSIS I HOMEWORK 1
REAL ANALYSIS I HOMEWORK CİHAN BAHRAN The questons are from Tao s text. Exercse 0.0.. If (x α ) α A s a collecton of numbers x α [0, + ] such that x α
More informationMore metrics on cartesian products
More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of
More informationExercises. 18 Algorithms
18 Algorthms Exercses 0.1. In each of the followng stuatons, ndcate whether f = O(g), or f = Ω(g), or both (n whch case f = Θ(g)). f(n) g(n) (a) n 100 n 200 (b) n 1/2 n 2/3 (c) 100n + log n n + (log n)
More informationExample: (13320, 22140) =? Solution #1: The divisors of are 1, 2, 3, 4, 5, 6, 9, 10, 12, 15, 18, 20, 27, 30, 36, 41,
The greatest common dvsor of two ntegers a and b (not both zero) s the largest nteger whch s a common factor of both a and b. We denote ths number by gcd(a, b), or smply (a, b) when there s no confuson
More informatione - c o m p a n i o n
OPERATIONS RESEARCH http://dxdoorg/0287/opre007ec e - c o m p a n o n ONLY AVAILABLE IN ELECTRONIC FORM 202 INFORMS Electronc Companon Generalzed Quantty Competton for Multple Products and Loss of Effcency
More informationRandom Walks on Digraphs
Random Walks on Dgraphs J. J. P. Veerman October 23, 27 Introducton Let V = {, n} be a vertex set and S a non-negatve row-stochastc matrx (.e. rows sum to ). V and S defne a dgraph G = G(V, S) and a drected
More informationVapnik-Chervonenkis theory
Vapnk-Chervonenks theory Rs Kondor June 13, 2008 For the purposes of ths lecture, we restrct ourselves to the bnary supervsed batch learnng settng. We assume that we have an nput space X, and an unknown
More informationNotes on Frequency Estimation in Data Streams
Notes on Frequency Estmaton n Data Streams In (one of) the data streamng model(s), the data s a sequence of arrvals a 1, a 2,..., a m of the form a j = (, v) where s the dentty of the tem and belongs to
More informationComputing Correlated Equilibria in Multi-Player Games
Computng Correlated Equlbra n Mult-Player Games Chrstos H. Papadmtrou Presented by Zhanxang Huang December 7th, 2005 1 The Author Dr. Chrstos H. Papadmtrou CS professor at UC Berkley (taught at Harvard,
More informationChapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems
Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons
More informationSpectral Graph Theory and its Applications September 16, Lecture 5
Spectral Graph Theory and ts Applcatons September 16, 2004 Lecturer: Danel A. Spelman Lecture 5 5.1 Introducton In ths lecture, we wll prove the followng theorem: Theorem 5.1.1. Let G be a planar graph
More information10-701/ Machine Learning, Fall 2005 Homework 3
10-701/15-781 Machne Learnng, Fall 2005 Homework 3 Out: 10/20/05 Due: begnnng of the class 11/01/05 Instructons Contact questons-10701@autonlaborg for queston Problem 1 Regresson and Cross-valdaton [40
More informationn α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0
MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector
More informationThe Order Relation and Trace Inequalities for. Hermitian Operators
Internatonal Mathematcal Forum, Vol 3, 08, no, 507-57 HIKARI Ltd, wwwm-hkarcom https://doorg/0988/mf088055 The Order Relaton and Trace Inequaltes for Hermtan Operators Y Huang School of Informaton Scence
More information8.6 The Complex Number System
8.6 The Complex Number System Earler n the chapter, we mentoned that we cannot have a negatve under a square root, snce the square of any postve or negatve number s always postve. In ths secton we want
More informationISSN: ISO 9001:2008 Certified International Journal of Engineering and Innovative Technology (IJEIT) Volume 3, Issue 1, July 2013
ISSN: 2277-375 Constructon of Trend Free Run Orders for Orthogonal rrays Usng Codes bstract: Sometmes when the expermental runs are carred out n a tme order sequence, the response can depend on the run
More informationMMA and GCMMA two methods for nonlinear optimization
MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons
More informationHowever, since P is a symmetric idempotent matrix, of P are either 0 or 1 [Eigen-values
Fall 007 Soluton to Mdterm Examnaton STAT 7 Dr. Goel. [0 ponts] For the general lnear model = X + ε, wth uncorrelated errors havng mean zero and varance σ, suppose that the desgn matrx X s not necessarly
More informationLecture 5 Decoding Binary BCH Codes
Lecture 5 Decodng Bnary BCH Codes In ths class, we wll ntroduce dfferent methods for decodng BCH codes 51 Decodng the [15, 7, 5] 2 -BCH Code Consder the [15, 7, 5] 2 -code C we ntroduced n the last lecture
More informationFirst day August 1, Problems and Solutions
FOURTH INTERNATIONAL COMPETITION FOR UNIVERSITY STUDENTS IN MATHEMATICS July 30 August 4, 997, Plovdv, BULGARIA Frst day August, 997 Problems and Solutons Problem. Let {ε n } n= be a sequence of postve
More informationErrors for Linear Systems
Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch
More information1 Convex Optimization
Convex Optmzaton We wll consder convex optmzaton problems. Namely, mnmzaton problems where the objectve s convex (we assume no constrants for now). Such problems often arse n machne learnng. For example,
More informationConvergence of random processes
DS-GA 12 Lecture notes 6 Fall 216 Convergence of random processes 1 Introducton In these notes we study convergence of dscrete random processes. Ths allows to characterze phenomena such as the law of large
More informationSpectral Clustering. Shannon Quinn
Spectral Clusterng Shannon Qunn (wth thanks to Wllam Cohen of Carnege Mellon Unverst, and J. Leskovec, A. Raaraman, and J. Ullman of Stanford Unverst) Graph Parttonng Undrected graph B- parttonng task:
More informationSalmon: Lectures on partial differential equations. Consider the general linear, second-order PDE in the form. ,x 2
Salmon: Lectures on partal dfferental equatons 5. Classfcaton of second-order equatons There are general methods for classfyng hgher-order partal dfferental equatons. One s very general (applyng even to
More informationModule 9. Lecture 6. Duality in Assignment Problems
Module 9 1 Lecture 6 Dualty n Assgnment Problems In ths lecture we attempt to answer few other mportant questons posed n earler lecture for (AP) and see how some of them can be explaned through the concept
More informationDISCRIMINANTS AND RAMIFIED PRIMES. 1. Introduction A prime number p is said to be ramified in a number field K if the prime ideal factorization
DISCRIMINANTS AND RAMIFIED PRIMES KEITH CONRAD 1. Introducton A prme number p s sad to be ramfed n a number feld K f the prme deal factorzaton (1.1) (p) = po K = p e 1 1 peg g has some e greater than 1.
More informationDeriving the X-Z Identity from Auxiliary Space Method
Dervng the X-Z Identty from Auxlary Space Method Long Chen Department of Mathematcs, Unversty of Calforna at Irvne, Irvne, CA 92697 chenlong@math.uc.edu 1 Iteratve Methods In ths paper we dscuss teratve
More informationDifference Equations
Dfference Equatons c Jan Vrbk 1 Bascs Suppose a sequence of numbers, say a 0,a 1,a,a 3,... s defned by a certan general relatonshp between, say, three consecutve values of the sequence, e.g. a + +3a +1
More informationLECTURE 9 CANONICAL CORRELATION ANALYSIS
LECURE 9 CANONICAL CORRELAION ANALYSIS Introducton he concept of canoncal correlaton arses when we want to quantfy the assocatons between two sets of varables. For example, suppose that the frst set of
More informationρ some λ THE INVERSE POWER METHOD (or INVERSE ITERATION) , for , or (more usually) to
THE INVERSE POWER METHOD (or INVERSE ITERATION) -- applcaton of the Power method to A some fxed constant ρ (whch s called a shft), x λ ρ If the egenpars of A are { ( λ, x ) } ( ), or (more usually) to,
More informationComposite Hypotheses testing
Composte ypotheses testng In many hypothess testng problems there are many possble dstrbutons that can occur under each of the hypotheses. The output of the source s a set of parameters (ponts n a parameter
More informationDIFFERENTIAL FORMS BRIAN OSSERMAN
DIFFERENTIAL FORMS BRIAN OSSERMAN Dfferentals are an mportant topc n algebrac geometry, allowng the use of some classcal geometrc arguments n the context of varetes over any feld. We wll use them to defne
More informationRepresentation theory and quantum mechanics tutorial Representation theory and quantum conservation laws
Representaton theory and quantum mechancs tutoral Representaton theory and quantum conservaton laws Justn Campbell August 1, 2017 1 Generaltes on representaton theory 1.1 Let G GL m (R) be a real algebrac
More informationprinceton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg
prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there
More informationAn Explicit Construction of an Expander Family (Margulis-Gaber-Galil)
An Explct Constructon of an Expander Famly Marguls-Gaber-Gall) Orr Paradse July 4th 08 Prepared for a Theorst's Toolkt, a course taught by Irt Dnur at the Wezmann Insttute of Scence. The purpose of these
More informationSome basic inequalities. Definition. Let V be a vector space over the complex numbers. An inner product is given by a function, V V C
Some basc nequaltes Defnton. Let V be a vector space over the complex numbers. An nner product s gven by a functon, V V C (x, y) x, y satsfyng the followng propertes (for all x V, y V and c C) (1) x +
More informationQuantum Mechanics for Scientists and Engineers. David Miller
Quantum Mechancs for Scentsts and Engneers Davd Mller Types of lnear operators Types of lnear operators Blnear expanson of operators Blnear expanson of lnear operators We know that we can expand functons
More informationGeneralized Linear Methods
Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set
More informationThe Minimum Universal Cost Flow in an Infeasible Flow Network
Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran
More informationBernoulli Numbers and Polynomials
Bernoull Numbers and Polynomals T. Muthukumar tmk@tk.ac.n 17 Jun 2014 The sum of frst n natural numbers 1, 2, 3,..., n s n n(n + 1 S 1 (n := m = = n2 2 2 + n 2. Ths formula can be derved by notng that
More informationLecture 17: Lee-Sidford Barrier
CSE 599: Interplay between Convex Optmzaton and Geometry Wnter 2018 Lecturer: Yn Tat Lee Lecture 17: Lee-Sdford Barrer Dsclamer: Please tell me any mstake you notced. In ths lecture, we talk about the
More informationLecture 10 Support Vector Machines II
Lecture 10 Support Vector Machnes II 22 February 2016 Taylor B. Arnold Yale Statstcs STAT 365/665 1/28 Notes: Problem 3 s posted and due ths upcomng Frday There was an early bug n the fake-test data; fxed
More informationNorms, Condition Numbers, Eigenvalues and Eigenvectors
Norms, Condton Numbers, Egenvalues and Egenvectors 1 Norms A norm s a measure of the sze of a matrx or a vector For vectors the common norms are: N a 2 = ( x 2 1/2 the Eucldean Norm (1a b 1 = =1 N x (1b
More informationLecture 5 September 17, 2015
CS 229r: Algorthms for Bg Data Fall 205 Prof. Jelan Nelson Lecture 5 September 7, 205 Scrbe: Yakr Reshef Recap and overvew Last tme we dscussed the problem of norm estmaton for p-norms wth p > 2. We had
More informationLinear Algebra and its Applications
Lnear Algebra and ts Applcatons 4 (00) 5 56 Contents lsts avalable at ScenceDrect Lnear Algebra and ts Applcatons journal homepage: wwwelsevercom/locate/laa Notes on Hlbert and Cauchy matrces Mroslav Fedler
More information