Algorithms, Graph Theory, and the Solu7on of Laplacian Linear Equa7ons. Daniel A. Spielman Yale University
|
|
|
- Charla Chandler
- 6 years ago
- Views:
Transcription
1 Algorithms, Graph Theory, and the Solu7on of Laplacian Linear Equa7ons Daniel A. Spielman Yale University Rutgers, Dec 6, 2011
2 Outline Linear Systems in Laplacian Matrices What? Why? Classic ways to solve these systems. Approxima7ng Graphs by Trees Sparse Approxima7ons of Graphs Local Graph Clustering
3 Laplacian Linear Systems Ax = b O(m log c m) Solve in 7me where = number of non zeros entries of A m 7mes for approximate solu7on. log(1/ɛ) ɛ x A 1 b A ɛ A 1 b A Enables solu7on of all symmetric, diagonally dominant systems, including sub matrices of Laplacians.
4 Laplacian Quadra7c Form of G =(V, E) For x : V IR x T L G x = (u,v) E (x (u) x (v)) 2
5 Laplacian Quadra7c Form of G =(V, E) For x : V IR x T L G x = (u,v) E (x (u) x (v)) 2 x :
6 Laplacian Quadra7c Form of G =(V, E) For x : V IR x T L G x = (u,v) E (x (u) x (v)) 2 x : x T L G x =
7 Laplacian Quadra7c Form of G =(V, E) For x : V IR x T L G x = (u,v) E (x (u) x (v)) 2 1 x : 0 x T L G x =
8 Laplacian Quadratic Form, examples When x is the characteristic vector of a set S, counts the edges on the boundary of S x T L G x = bdry(s) 0 1 S
9 Laplacian Quadratic Form, examples When x is the characteristic vector of a set S, counts the edges on the boundary of S x T L G x = bdry(s) x T L G x x T x = bdry(s) S 0 1 S = edge expansion of S
10 Learning on Graphs [Zhu Ghahramani Lafferty 03] Infer values of a func7on at all ver7ces from known values at a few ver7ces. Minimize x T L G x = (u,v) E Subject to known values 0 w (u,v) (x (u) x (v)) 2 1
11 Learning on Graphs [Zhu Ghahramani Lafferty 03] Infer values of a func7on at all ver7ces from known values at a few ver7ces. Minimize x T L G x = (u,v) E Subject to known values 0.5 w (u,v) (x (u) x (v)) Taking deriva,ves, minimize by solving Laplacian
12 Other Applica7ons Compu7ng effec7ve resistances in resistor networks: Solve for current when fix voltages 1V 0V
13 Other Applica7ons Compu7ng effec7ve resistances in resistor networks: Solve for current when fix voltages 0.5V 1V 0.5V 0V 0.375V 0.625V
14 Laplacian Quadra7c Form for Weighted Graphs G =(V, E, w) w : E IR + assigns a posi7ve weight to every edge x T L G x = (u,v) E w (u,v) (x (u) x (v)) 2 Matrix L G is posi7ve semi definite nullspace spanned by const vector, if connected
15 Laplacian Matrix of a Weighted Graph L G (u, v) = w(u, v) if (u, v) E d(u) if u = v 0 otherwise d(u) = (v,u) E w(u, v) the weighted degree of u is a diagonally dominant matrix
16 Classic Applica7ons Compu7ng effec7ve resistances. Solving Ellip7c PDEs. Compu7ng Eigenvectors and Eigenvalues of Laplacians of graphs. Solving Maximum Flow by Interior Point Methods
17 Solving Laplacian Linear Equa7ons Quickly Fast when graph is simple, by elimina7on. Fast approxima7on when graph is complicated*, by Conjugate Gradient * = random graph or high expansion
18 Cholesky Factoriza7on of Laplacians When eliminate a vertex, connect its neighbors. Also known as Y Δ
19 Cholesky Factoriza7on of Laplacians When eliminate a vertex, connect its neighbors. Also known as Y Δ
20 Cholesky Factoriza7on of Laplacians When eliminate a vertex, connect its neighbors. Also known as Y Δ
21
22 1 The order maeers
23 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V )
24 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V )
25 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V )
26 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V )
27 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V )
28 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V )
29 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V ) Planar #ops ~ O( V 3/2 ) Lipton Rose Tarjan 79
30 Complexity of Cholesky Factoriza7on #ops ~ Σ v (degree of v when eliminate) 2 Tree #ops ~ O( V ) Planar #ops ~ O( V 3/2 ) Lipton Rose Tarjan 79 Expander like random, but O( V ) edges #ops Ω( V 3 ) Lipton Rose Tarjan 79
31 Expansion and Cholesky Factoriza7on For S V S Φ(S) = bdry(s) min ( S, V S ) Φ G = min S V Φ(S)
32 Expansion and Cholesky Factoriza7on For S V S Φ(S) = bdry(s) min ( S, V S ) Φ G = min S V Φ(S) Cholesky slow when expansion high Cholesky fast when low for G and all subgraphs
33 Cheeger s Inequality and the Conjugate Gradient Cheeger s inequality (degree d unwted case) 1 2 λ 2 d Φ G d 2 λ 2 d λ 2 = second smallest eigenvalue of L G ~ d/mixing 7me of random walk near d for expanders and random graphs
34 Cheeger s Inequality and the Conjugate Gradient Cheeger s inequality (degree d unwted case) 1 2 λ 2 d Φ G d λ 2 = second smallest eigenvalue of L G ~ d/mixing 7me of random walk Conjugate Gradient finds approx solu7on to L G x = b in mults by L O( d/λ 2 log ɛ 1 ) G is ops O(dmΦ 1 G log ɛ 1 ) 2 λ 2 d
35 Fast solu7on of linear equa7ons Conjugate Gradient fast when expansion high. Elimina7on fast when low for G and all subgraphs.
36 Fast solu7on of linear equa7ons Conjugate Gradient fast when expansion high. Planar graphs Elimina7on fast when low for G and all subgraphs. Want speed of extremes in the middle
37 Fast solu7on of linear equa7ons Conjugate Gradient fast when expansion high. Planar graphs Elimina7on fast when low for G and all subgraphs. Want speed of extremes in the middle Not all graphs fit into these categories!
38 Precondi7oned Conjugate Gradient Solve L G x = b by Approxima7ng L G by L H (the precondi7oner) In each itera7on solve a system in L H mul7ply a vector by L G approx solu7on aser O( κ(l G,L H ) log ɛ 1 ) itera7ons condi,on number/approx quality
39 Inequali7es and Approxima7on L H L G if for all x, x T L H x x T L G x Example: if H is a subgraph of G x T L G x = (u,v) E w (u,v) (x (u) x (v)) 2
40 Inequali7es and Approxima7on L H L G if for all x, x T L H x x T L G x κ(l G,L H ) t if L H L G tl H Call such an H a t approx of G
41 Inequali7es and Approxima7on L H L G if for all x, x T L H x x T L G x κ(l G,L H ) t iff c : cl H L G ctl H Call such an H a t approx of G
42 Vaidya s Subgraph Precondi7oners Precondi7on G by a subgraph H L H L G Just need to know t s.t. L G tl H Easy to bound t if H is a spanning tree H And, easy to solve equa7ons in L H by elimina7on
43 Approximate Laplacian Solvers Conjugate Gradient [Hestenes 51, S7efel 52] Vaidya 90: Augmented MST Boman Hendrickson 01: Using Low Stretch Spanning Trees S Teng 04: Spectral sparsifica7on Kou7s Miller Peng 11: Elegance O(m log c n) Õ(m log n)
44 The Stretch of Spanning Trees Boman Hendrickson 01: L G st T (G)L T Where st T (G) = path-length T (u, v) (u,v) E
45 The Stretch of Spanning Trees Boman Hendrickson 01: L G st T (G)L T Where st T (G) = path-length T (u, v) (u,v) E path len 3
46 The Stretch of Spanning Trees Boman Hendrickson 01: L G st T (G)L T Where st T (G) = path-length T (u, v) (u,v) E path len 5
47 The Stretch of Spanning Trees Boman Hendrickson 01: L G st T (G)L T Where st T (G) = path-length T (u, v) (u,v) E path len 1
48 The Stretch of Spanning Trees Boman Hendrickson 01: L G st T (G)L T Where st T (G) = path-length T (u, v) (u,v) E In weighted case, measure resistances of paths
49 Fundamental Graphic Inequality edge k times path of length k With weights, corresponds to resistors in serial (Poincaré inequality) 49
50 When T is a Spanning Tree G T Every edge of G not in T has unique path in T 50
51 When T is a Spanning Tree 51
52 The Stretch of Spanning Trees Boman Hendrickson 01: L G st T (G)L T Where st T (G) = path-length T (u, v) (u,v) E
53 Low Stretch Spanning Trees For every G there is a T with st T (G) m 1+o(1) (Alon Karp Peleg West 91) where m = E st T (G) O(m log m log 2 log m) (Elkin Emek S Teng 04, Abraham Bartal Neiman 08) Solve linear systems in 7me O(m 3/2 log m)
54 Spectral Sparsifica7on [S Teng 04] Approximate G by a sparse H with κ(l G,L H ) 1+ɛ
55 Cut Sparsifica7on [Benczur Karger 96] Approximate G by a sparse H, approximately preserving all cuts S S
56 Sparsifica7on Goal: find sparse approxima7on for every G S Teng 04: For every G is an H with O(n log 7 n/ɛ 2 ) edges and κ(l G,L H ) 1+ɛ
57 Sparsifica7on Goal: find sparse approxima7on for every G S Teng 04: For every G is an H with O(n log 7 n/ɛ 2 ) edges and κ(l G,L H ) 1+ɛ S Srivastava 08: with edges O(n log n/ɛ 2 ) by random sampling by effec7ve resistances 1/(current flow at one volt) v 1V u 0.27V 0.53V 0V 0.2V 0.33V
58 Sparsifica7on Goal: find sparse approxima7on for every G S Teng 04: For every G is an H with O(n log 7 n/ɛ 2 ) edges and κ(l G,L H ) 1+ɛ S Srivastava 08: with edges O(n log n/ɛ 2 ) Batson S Srivastava 09 determinis7c, poly 7me, and edges O(n/ɛ 2 )
59 Sparsifiers Low Stretch Trees Ultra Sparsifiers [S Teng] Approximate G by a tree plus edges n/ log 2 n L H L G c log 2 nl H
60 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
61 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
62 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
63 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
64 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
65 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
66 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
67 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes
68 Cholesky factor to smaller system Eliminate degree 1 and 2 nodes Get system of size, solve recursively O(n/ log 2 n) [Joshi 97, Reif 98, S Teng 04 09]
69 Ultra Sparsifiers Solve systems in H by: 1. Cholesky elimina7ng degree 1 and 2 nodes 2. recursively solving reduced system Time O(m log c m)
70 Kou7s Miller Peng 11 Solve in 7me O(m log n log 2 log n log(1/ɛ)) Build Ultra Sparsifier by: 1. Construc7ng low stretch spanning tree 2. Adding other edges with probability p u,v path-length T (u, v) Code by Yiannis Kou7s
![Local Graph Clustering [S Teng 04] Given vertex of interest find nearby cluster S](/docs-images/72/66969653/images/71-0.jpg "with small expansion* in 7me O( S ) *Actually, use conductance.")
71 Local Graph Clustering [S Teng 04] Given vertex of interest find nearby cluster S with small expansion* in 7me O( S ) *Actually, use conductance. Count ver7ces by degree.
72 Local Graph Clustering [S Teng 04] Prove: Given a set S of small expansion and a random vertex v of S probably find a set T of small expansion most of T inside S in 7me O( T ) S
73 Local Graph Clustering [S Teng 04] Prove: Given a set S of small expansion and a random vertex v of S probably find a set T of small expansion most of T inside S in 7me O( T ) S
74 Local Graph Clustering [S Teng 04] Prove: Given a set S of small expansion and a random vertex v of S probably find a set T of small expansion most of T inside S in 7me O( T ) S
75 Local Graph Clustering [S Teng 04] Prove: Given a set S of small expansion and a random vertex v of S probably find a set T of small expansion most of T inside S in 7me O( T ) S
76 Local Graph Clustering [S Teng 04] Prove: Given a set S of small expansion and a random vertex v of S probably find a set T of small expansion most of T inside S in 7me O( T ) S v
77 Local Graph Clustering [S Teng 04] Prove: Given a set S of small expansion and a random vertex v of S probably find a set T of small expansion most of T inside S in 7me O( T ) S v T
78 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries of wet paint, half stays put, half to neighbors
79 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries of wet paint, half stays put, half to neighbors dry wet 1 0 0
80 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries (α = 1/3) of wet paint, half stays put, half to neighbors dry wet
81 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries (α = 1/3) of wet paint, half stays put, half to neighbors dry wet
82 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries (α = 1/3) of wet paint, half stays put, half to neighbors dry wet
83 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries (α = 1/3) of wet paint, half stays put, half to neighbors dry wet
84 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries (α = 1/3) of wet paint, half stays put, half to neighbors dry wet
85 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: start at one node at each step, α frac7on dries (α = 1/3) of wet paint, half stays put, half to neighbors dry wet
86 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: Time doesn t maeer, can push asynchronously Approximate: only push when a lot of paint dry wet 1 0 0
87 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: Time doesn t maeer, can push asynchronously Approximate: only push when a lot of paint dry wet 1 0 0
88 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: Time doesn t maeer, can push asynchronously Approximate: only push when a lot of paint dry wet
89 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: Time doesn t maeer, can push asynchronously Approximate: only push when a lot of paint dry wet
90 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: Time doesn t maeer, can push asynchronously Approximate: only push when a lot of paint dry wet
91 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: Time doesn t maeer, can push asynchronously Approximate: only push when a lot of paint dry wet
92 Using Approximate Personal PageRank Vectors Jeh Widom 03, Berkhin 06, Andersen Chung Lang 06 Spilling paint in a graph: Time doesn t maeer, can push asynchronously Approximate: only push when a lot of paint dry wet
93 Volume Biased Evolving Set Markov Chain Walk on sets of ver7ces starts at one vertex, ends at V Dual to random walk on graph [Andersen Peres 09] When start inside set of conductance find set of conductance φ 1/2 log 1/2 n with work S log c n/φ 1/2
94 Volume Biased Evolving Set Markov Chain Walk on sets of ver7ces starts at one vertex, ends at V Dual to random walk on graph [Andersen Peres 09] When start inside set of conductance find set of conductance φ 1/2 log 1/2 n with work S log c n/φ 1/2 can we eliminate this?
95 Open Problems Faster and beeer Low Stretch Spanning Trees. Faster high quality sparsifica7on. Faster local clustering and graph decomposi7on. Other families of linear systems.
96 Conclusions Laplacian Solvers are a powerful primi7ve! Faster Maxflow: Chris7ano Kelner Madry S Teng Faster Random Spanning Trees: Kelner Madry Propp All Effec7ve Resistances: S Srivastava Maybe we can solve all well condi7oned graph problems in nearly linear 7me. Don t fear large constants
Algorithms, Graph Theory, and Linear Equa9ons in Laplacians. Daniel A. Spielman Yale University
Algorithms, Graph Theory, and Linear Equa9ons in Laplacians Daniel A. Spielman Yale University Shang- Hua Teng Carter Fussell TPS David Harbater UPenn Pavel Cur9s Todd Knoblock CTY Serge Lang 1927-2005
Spectral and Electrical Graph Theory. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral and Electrical Graph Theory Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Spectral Graph Theory: Understand graphs through eigenvectors and
Laplacian Matrices of Graphs: Spectral and Electrical Theory
Laplacian Matrices of Graphs: Spectral and Electrical Theory Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale University Toronto, Sep. 28, 2 Outline Introduction to graphs
Algorithmic Primitives for Network Analysis: Through the Lens of the Laplacian Paradigm
Algorithmic Primitives for Network Analysis: Through the Lens of the Laplacian Paradigm Shang-Hua Teng Computer Science, Viterbi School of Engineering USC Massive Data and Massive Graphs 500 billions web
Sparsification of Graphs and Matrices
Sparsification of Graphs and Matrices Daniel A. Spielman Yale University joint work with Joshua Batson (MIT) Nikhil Srivastava (MSR) Shang- Hua Teng (USC) HUJI, May 21, 2014 Objective of Sparsification:
Cutting Graphs, Personal PageRank and Spilling Paint
Graphs and Networks Lecture 11 Cutting Graphs, Personal PageRank and Spilling Paint Daniel A. Spielman October 3, 2013 11.1 Disclaimer These notes are not necessarily an accurate representation of what
Graph Sparsifiers: A Survey
Graph Sparsifiers: A Survey Nick Harvey UBC Based on work by: Batson, Benczur, de Carli Silva, Fung, Hariharan, Harvey, Karger, Panigrahi, Sato, Spielman, Srivastava and Teng Approximating Dense Objects
Lecture: Local Spectral Methods (2 of 4) 19 Computing spectral ranking with the push procedure
Stat260/CS294: Spectral Graph Methods Lecture 19-04/02/2015 Lecture: Local Spectral Methods (2 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
Algorithms, Graph Theory, and Linear Equations in Laplacian Matrices
Proceedings of the International Congress of Mathematicians Hyderabad, India, 200 Algorithms, Graph Theory, and Linear Equations in Laplacian Matrices Daniel A. Spielman Abstract The Laplacian matrices
Personal PageRank and Spilling Paint
Graphs and Networks Lecture 11 Personal PageRank and Spilling Paint Daniel A. Spielman October 7, 2010 11.1 Overview These lecture notes are not complete. The paint spilling metaphor is due to Berkhin
Graphs, Vectors, and Matrices Daniel A. Spielman Yale University. AMS Josiah Willard Gibbs Lecture January 6, 2016
Graphs, Vectors, and Matrices Daniel A. Spielman Yale University AMS Josiah Willard Gibbs Lecture January 6, 2016 From Applied to Pure Mathematics Algebraic and Spectral Graph Theory Sparsification: approximating
Lecture 24: Element-wise Sampling of Graphs and Linear Equation Solving. 22 Element-wise Sampling of Graphs and Linear Equation Solving
Stat260/CS294: Randomized Algorithms for Matrices and Data Lecture 24-12/02/2013 Lecture 24: Element-wise Sampling of Graphs and Linear Equation Solving Lecturer: Michael Mahoney Scribe: Michael Mahoney
Spectral Graph Theory and its Applications. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral Graph Theory and its Applications Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Adjacency matrix and Laplacian Intuition, spectral graph drawing
Graph Sparsification I : Effective Resistance Sampling
Graph Sparsification I : Effective Resistance Sampling Nikhil Srivastava Microsoft Research India Simons Institute, August 26 2014 Graphs G G=(V,E,w) undirected V = n w: E R + Sparsification Approximate
Electrical Flow Primi7ve and Fast Graph Algorithms. Aleksander Mądry
Electrical Flow Primi7ve and Fast Graph Algorithms Aleksander Mądry These are exci+ng +mes for fast graph algorithms Last several years brought faster algorithms for a number of fundamental graph problems
Lecture: Local Spectral Methods (3 of 4) 20 An optimization perspective on local spectral methods
Stat260/CS294: Spectral Graph Methods Lecture 20-04/07/205 Lecture: Local Spectral Methods (3 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
ORIE 6334 Spectral Graph Theory October 13, Lecture 15
ORIE 6334 Spectral Graph heory October 3, 206 Lecture 5 Lecturer: David P. Williamson Scribe: Shijin Rajakrishnan Iterative Methods We have seen in the previous lectures that given an electrical network,
Spectral Algorithms for Graph Mining and Analysis. Yiannis Kou:s. University of Puerto Rico - Rio Piedras NSF CAREER CCF
Spectral Algorithms for Graph Mining and Analysis Yiannis Kou:s University of Puerto Rico - Rio Piedras NSF CAREER CCF-1149048 The Spectral Lens A graph drawing using Laplacian eigenvectors Outline Basic
Physical Metaphors for Graphs
Graphs and Networks Lecture 3 Physical Metaphors for Graphs Daniel A. Spielman October 9, 203 3. Overview We will examine physical metaphors for graphs. We begin with a spring model and then discuss resistor
Fast Linear Solvers for Laplacian Systems UCSD Research Exam
Fast Linear olvers for Laplacian ystems UCD Research Exam Olivia impson Fall 2013 olving a system of linear equations is a fundamental problem that has deep implications in the computational sciences,
Lecture 14: Random Walks, Local Graph Clustering, Linear Programming
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 14: Random Walks, Local Graph Clustering, Linear Programming Lecturer: Shayan Oveis Gharan 3/01/17 Scribe: Laura Vonessen Disclaimer: These
Lecture 13: Spectral Graph Theory
CSE 521: Design and Analysis of Algorithms I Winter 2017 Lecture 13: Spectral Graph Theory Lecturer: Shayan Oveis Gharan 11/14/18 Disclaimer: These notes have not been subjected to the usual scrutiny reserved
An Empirical Comparison of Graph Laplacian Solvers
An Empirical Comparison of Graph Laplacian Solvers Kevin Deweese 1 Erik Boman 2 John Gilbert 1 1 Department of Computer Science University of California, Santa Barbara 2 Scalable Algorithms Department
Graph Sparsifiers. Smaller graph that (approximately) preserves the values of some set of graph parameters. Graph Sparsification
Graph Sparsifiers Smaller graph that (approximately) preserves the values of some set of graph parameters Graph Sparsifiers Spanners Emulators Small stretch spanning trees Vertex sparsifiers Spectral sparsifiers
Iterative solvers for linear equations
Spectral Graph Theory Lecture 15 Iterative solvers for linear equations Daniel A. Spielman October 1, 009 15.1 Overview In this and the next lecture, I will discuss iterative algorithms for solving linear
Graph Partitioning Using Random Walks
Graph Partitioning Using Random Walks A Convex Optimization Perspective Lorenzo Orecchia Computer Science Why Spectral Algorithms for Graph Problems in practice? Simple to implement Can exploit very efficient
Math 307 Learning Goals. March 23, 2010
Math 307 Learning Goals March 23, 2010 Course Description The course presents core concepts of linear algebra by focusing on applications in Science and Engineering. Examples of applications from recent
U.C. Berkeley CS270: Algorithms Lecture 21 Professor Vazirani and Professor Rao Last revised. Lecture 21
U.C. Berkeley CS270: Algorithms Lecture 21 Professor Vazirani and Professor Rao Scribe: Anupam Last revised Lecture 21 1 Laplacian systems in nearly linear time Building upon the ideas introduced in the
Fast Linear Solvers for Laplacian Systems
s Solver for UCSD Research Exam University of California, San Diego osimpson@ucsd.edu November 8, 2013 s Solver 1 2 Matrices Linear in the Graph 3 Overview 4 Solving Linear with Using Computing A of the
Effective Resistance and Schur Complements
Spectral Graph Theory Lecture 8 Effective Resistance and Schur Complements Daniel A Spielman September 28, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened in
Four graph partitioning algorithms. Fan Chung University of California, San Diego
Four graph partitioning algorithms Fan Chung University of California, San Diego History of graph partitioning NP-hard approximation algorithms Spectral method, Fiedler 73, Folklore Multicommunity flow,
Lecture: Local Spectral Methods (1 of 4)
Stat260/CS294: Spectral Graph Methods Lecture 18-03/31/2015 Lecture: Local Spectral Methods (1 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
Data Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings
Data Analysis and Manifold Learning Lecture 3: Graphs, Graph Matrices, and Graph Embeddings Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline
Spectral Analysis of Random Walks
Graphs and Networks Lecture 9 Spectral Analysis of Random Walks Daniel A. Spielman September 26, 2013 9.1 Disclaimer These notes are not necessarily an accurate representation of what happened in class.
Fitting a Graph to Vector Data. Samuel I. Daitch (Yale) Jonathan A. Kelner (MIT) Daniel A. Spielman (Yale)
Fitting a Graph to Vector Data Samuel I. Daitch (Yale) Jonathan A. Kelner (MIT) Daniel A. Spielman (Yale) Machine Learning Summer School, June 2009 Given a collection of vectors x 1,..., x n IR d 1,, n
Math 307 Learning Goals
Math 307 Learning Goals May 14, 2018 Chapter 1 Linear Equations 1.1 Solving Linear Equations Write a system of linear equations using matrix notation. Use Gaussian elimination to bring a system of linear
Graph Partitioning Algorithms and Laplacian Eigenvalues
Graph Partitioning Algorithms and Laplacian Eigenvalues Luca Trevisan Stanford Based on work with Tsz Chiu Kwok, Lap Chi Lau, James Lee, Yin Tat Lee, and Shayan Oveis Gharan spectral graph theory Use linear
Con$nuous Op$miza$on: The "Right" Language for Fast Graph Algorithms? Aleksander Mądry
Con$nuous Op$miza$on: The "Right" Language for Fast Graph Algorithms? Aleksander Mądry We have been studying graph algorithms for a while now Tradi$onal view: Graphs are combinatorial objects graph algorithms
Constant Arboricity Spectral Sparsifiers
Constant Arboricity Spectral Sparsifiers arxiv:1808.0566v1 [cs.ds] 16 Aug 018 Timothy Chu oogle timchu@google.com Jakub W. Pachocki Carnegie Mellon University pachocki@cs.cmu.edu August 0, 018 Abstract
Lecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality
CSE 521: Design and Analysis of Algorithms I Spring 2016 Lecture 12: Introduction to Spectral Graph Theory, Cheeger s inequality Lecturer: Shayan Oveis Gharan May 4th Scribe: Gabriel Cadamuro Disclaimer:
Convergence of Random Walks
Graphs and Networks Lecture 9 Convergence of Random Walks Daniel A. Spielman September 30, 010 9.1 Overview We begin by reviewing the basics of spectral theory. We then apply this theory to show that lazy
eigenvalue bounds and metric uniformization Punya Biswal & James R. Lee University of Washington Satish Rao U. C. Berkeley
eigenvalue bounds and metric uniformization Punya Biswal & James R. Lee University of Washington Satish Rao U. C. Berkeley separators in planar graphs Lipton and Tarjan (1980) showed that every n-vertex
Sparsification by Effective Resistance Sampling
Spectral raph Theory Lecture 17 Sparsification by Effective Resistance Sampling Daniel A. Spielman November 2, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened
Towards Practically-Efficient Spectral Sparsification of Graphs. Zhuo Feng
Design Automation Group Towards Practically-Efficient Spectral Sparsification of Graphs Zhuo Feng Acknowledgements: PhD students: Xueqian Zhao, Lengfei Han, Zhiqiang Zhao, Yongyu Wang NSF CCF CAREER grant
Algorithm Design Using Spectral Graph Theory
Algorithm Design Using Spectral Graph Theory Richard Peng CMU-CS-13-121 August 2013 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Gary L. Miller, Chair Guy
Walks, Springs, and Resistor Networks
Spectral Graph Theory Lecture 12 Walks, Springs, and Resistor Networks Daniel A. Spielman October 8, 2018 12.1 Overview In this lecture we will see how the analysis of random walks, spring networks, and
Iterative solvers for linear equations
Spectral Graph Theory Lecture 17 Iterative solvers for linear equations Daniel A. Spielman October 31, 2012 17.1 About these notes These notes are not necessarily an accurate representation of what happened
Data Analysis and Manifold Learning Lecture 7: Spectral Clustering
Data Analysis and Manifold Learning Lecture 7: Spectral Clustering Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture 7 What is spectral
The Simplest Construction of Expanders
Spectral Graph Theory Lecture 14 The Simplest Construction of Expanders Daniel A. Spielman October 16, 2009 14.1 Overview I am going to present the simplest construction of expanders that I have been able
Bellman s Curse of Dimensionality
Bellman s Curse of Dimensionality n- dimensional state space Number of states grows exponen
Random Walks and Evolving Sets: Faster Convergences and Limitations
Random Walks and Evolving Sets: Faster Convergences and Limitations Siu On Chan 1 Tsz Chiu Kwok 2 Lap Chi Lau 2 1 The Chinese University of Hong Kong 2 University of Waterloo Jan 18, 2017 1/23 Finding
Conductance, the Normalized Laplacian, and Cheeger s Inequality
Spectral Graph Theory Lecture 6 Conductance, the Normalized Laplacian, and Cheeger s Inequality Daniel A. Spielman September 17, 2012 6.1 About these notes These notes are not necessarily an accurate representation
1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
A Local Spectral Method for Graphs: With Applications to Improving Graph Partitions and Exploring Data Graphs Locally
Journal of Machine Learning Research 13 (2012) 2339-2365 Submitted 11/12; Published 9/12 A Local Spectral Method for Graphs: With Applications to Improving Graph Partitions and Exploring Data Graphs Locally
Scientific Computing with Case Studies SIAM Press, Lecture Notes for Unit VII Sparse Matrix
Scientific Computing with Case Studies SIAM Press, 2009 http://www.cs.umd.edu/users/oleary/sccswebpage Lecture Notes for Unit VII Sparse Matrix Computations Part 1: Direct Methods Dianne P. O Leary c 2008
Lecture 10. Lecturer: Aleksander Mądry Scribes: Mani Bastani Parizi and Christos Kalaitzis
CS-621 Theory Gems October 18, 2012 Lecture 10 Lecturer: Aleksander Mądry Scribes: Mani Bastani Parizi and Christos Kalaitzis 1 Introduction In this lecture, we will see how one can use random walks to
Powerful tool for sampling from complicated distributions. Many use Markov chains to model events that arise in nature.
Markov Chains Markov chains: 2SAT: Powerful tool for sampling from complicated distributions rely only on local moves to explore state space. Many use Markov chains to model events that arise in nature.
Complex Social System, Elections. Introduction to Network Analysis 1
Complex Social System, Elections Introduction to Network Analysis 1 Complex Social System, Network I person A voted for B A is more central than B if more people voted for A In-degree centrality index
Facebook Friends! and Matrix Functions
Facebook Friends! and Matrix Functions! Graduate Research Day Joint with David F. Gleich, (Purdue), supported by" NSF CAREER 1149756-CCF Kyle Kloster! Purdue University! Network Analysis Use linear algebra
Pseudospectral Methods For Op2mal Control. Jus2n Ruths March 27, 2009
Pseudospectral Methods For Op2mal Control Jus2n Ruths March 27, 2009 Introduc2on Pseudospectral methods arose to find solu2ons to Par2al Differen2al Equa2ons Recently adapted for Op2mal Control Key Ideas
Lecture Introduction. 2 Brief Recap of Lecture 10. CS-621 Theory Gems October 24, 2012
CS-62 Theory Gems October 24, 202 Lecture Lecturer: Aleksander Mądry Scribes: Carsten Moldenhauer and Robin Scheibler Introduction In Lecture 0, we introduced a fundamental object of spectral graph theory:
Locally-biased analytics
Locally-biased analytics You have BIG data and want to analyze a small part of it: Solution 1: Cut out small part and use traditional methods Challenge: cutting out may be difficult a priori Solution 2:
Spectral Sparsification in Dynamic Graph Streams
Spectral Sparsification in Dynamic Graph Streams Kook Jin Ahn 1, Sudipto Guha 1, and Andrew McGregor 2 1 University of Pennsylvania {kookjin,sudipto}@seas.upenn.edu 2 University of Massachusetts Amherst
Ranking and sparsifying a connection graph
Ranking and sparsifying a connection graph Fan Chung 1, and Wenbo Zhao 1 Department of Mathematics Department of Computer Science and Engineering University of California, San Diego La Jolla, CA 9093,
Diffusion and random walks on graphs
Diffusion and random walks on graphs Leonid E. Zhukov School of Data Analysis and Artificial Intelligence Department of Computer Science National Research University Higher School of Economics Structural
Parallel Graph Algorithms (con4nued)
Parallel Graph Algorithms (con4nued) MaxFlow A flow network G=(V,E): a directed graph, where each edge (u,v) E has a nonnega4ve capacity c(u,v)>=0. If (u,v) E, we assume that c(u,v)=0. Two dis4nct ver4ces
Lecture 1. 1 if (i, j) E a i,j = 0 otherwise, l i,j = d i if i = j, and
Specral Graph Theory and its Applications September 2, 24 Lecturer: Daniel A. Spielman Lecture. A quick introduction First of all, please call me Dan. If such informality makes you uncomfortable, you can
Properties of Expanders Expanders as Approximations of the Complete Graph
Spectral Graph Theory Lecture 10 Properties of Expanders Daniel A. Spielman October 2, 2009 10.1 Expanders as Approximations of the Complete Graph Last lecture, we defined an expander graph to be a d-regular
Iterative solvers for linear equations
Spectral Graph Theory Lecture 23 Iterative solvers for linear equations Daniel A. Spielman November 26, 2018 23.1 Overview In this and the next lecture, I will discuss iterative algorithms for solving
Full text available at: Lx = b
Lx = b Lx = b Nisheeth K. Vishnoi Microsoft Research India nisheeth.vishnoi@gmail.com Boston Delft Foundations and Trends R in Theoretical Computer Science Published, sold and distributed by: now Publishers
1 Adjacency matrix and eigenvalues
CSC 5170: Theory of Computational Complexity Lecture 7 The Chinese University of Hong Kong 1 March 2010 Our objective of study today is the random walk algorithm for deciding if two vertices in an undirected
Detecting Sharp Drops in PageRank and a Simplified Local Partitioning Algorithm
Detecting Sharp Drops in PageRank and a Simplified Local Partitioning Algorithm Reid Andersen and Fan Chung University of California at San Diego, La Jolla CA 92093, USA, fan@ucsd.edu, http://www.math.ucsd.edu/~fan/,
Markov Chains, Random Walks on Graphs, and the Laplacian
Markov Chains, Random Walks on Graphs, and the Laplacian CMPSCI 791BB: Advanced ML Sridhar Mahadevan Random Walks! There is significant interest in the problem of random walks! Markov chain analysis! Computer
Lecture: Flow-based Methods for Partitioning Graphs (1 of 2)
Stat260/CS294: Spectral Graph Methods Lecture 10-02/24/2015 Lecture: Flow-based Methods for Partitioning Graphs (1 of 2) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still
U.C. Berkeley Better-than-Worst-Case Analysis Handout 3 Luca Trevisan May 24, 2018
U.C. Berkeley Better-than-Worst-Case Analysis Handout 3 Luca Trevisan May 24, 2018 Lecture 3 In which we show how to find a planted clique in a random graph. 1 Finding a Planted Clique We will analyze
Mobile Robotics 1. A Compact Course on Linear Algebra. Giorgio Grisetti
Mobile Robotics 1 A Compact Course on Linear Algebra Giorgio Grisetti SA-1 Vectors Arrays of numbers They represent a point in a n dimensional space 2 Vectors: Scalar Product Scalar-Vector Product Changes
Fiedler s Theorems on Nodal Domains
Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A. Spielman September 19, 2018 7.1 Overview In today s lecture we will justify some of the behavior we observed when using eigenvectors
Lower-Stretch Spanning Trees
Lower-Stretch Spanning Trees Michael Elkin Ben-Gurion University Joint work with Yuval Emek, Weizmann Institute Dan Spielman, Yale University Shang-Hua Teng, Boston University + 1 The Setting G = (V, E,
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
Sparse Matrix Theory and Semidefinite Optimization
Sparse Matrix Theory and Semidefinite Optimization Lieven Vandenberghe Department of Electrical Engineering University of California, Los Angeles Joint work with Martin S. Andersen and Yifan Sun Third
A Tutorial on Matrix Approximation by Row Sampling
A Tutorial on Matrix Approximation by Row Sampling Rasmus Kyng June 11, 018 Contents 1 Fast Linear Algebra Talk 1.1 Matrix Concentration................................... 1. Algorithms for ɛ-approximation
Spectral Clustering. Guokun Lai 2016/10
Spectral Clustering Guokun Lai 2016/10 1 / 37 Organization Graph Cut Fundamental Limitations of Spectral Clustering Ng 2002 paper (if we have time) 2 / 37 Notation We define a undirected weighted graph
Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank
Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank David Glickenstein November 3, 4 Representing graphs as matrices It will sometimes be useful to represent graphs
1 9/5 Matrices, vectors, and their applications
1 9/5 Matrices, vectors, and their applications Algebra: study of objects and operations on them. Linear algebra: object: matrices and vectors. operations: addition, multiplication etc. Algorithms/Geometric
Using Local Spectral Methods in Theory and in Practice
Using Local Spectral Methods in Theory and in Practice Michael W. Mahoney ICSI and Dept of Statistics, UC Berkeley ( For more info, see: http: // www. stat. berkeley. edu/ ~ mmahoney or Google on Michael
Approximate Gaussian Elimination for Laplacians
CSC 221H : Graphs, Matrices, and Optimization Lecture 9 : 19 November 2018 Approximate Gaussian Elimination for Laplacians Lecturer: Sushant Sachdeva Scribe: Yasaman Mahdaviyeh 1 Introduction Recall sparsifiers
Testing Cluster Structure of Graphs. Artur Czumaj
Testing Cluster Structure of Graphs Artur Czumaj DIMAP and Department of Computer Science University of Warwick Joint work with Pan Peng and Christian Sohler (TU Dortmund) Dealing with BigData in Graphs
Classification Semi-supervised learning based on network. Speakers: Hanwen Wang, Xinxin Huang, and Zeyu Li CS Winter
Classification Semi-supervised learning based on network Speakers: Hanwen Wang, Xinxin Huang, and Zeyu Li CS 249-2 2017 Winter Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions Xiaojin
8.1 Concentration inequality for Gaussian random matrix (cont d)
MGMT 69: Topics in High-dimensional Data Analysis Falll 26 Lecture 8: Spectral clustering and Laplacian matrices Lecturer: Jiaming Xu Scribe: Hyun-Ju Oh and Taotao He, October 4, 26 Outline Concentration
An Algorithmist s Toolkit September 10, Lecture 1
18.409 An Algorithmist s Toolkit September 10, 2009 Lecture 1 Lecturer: Jonathan Kelner Scribe: Jesse Geneson (2009) 1 Overview The class s goals, requirements, and policies were introduced, and topics
(K2) For sum of the potential differences around any cycle is equal to zero. r uv resistance of {u, v}. [In other words, V ir.]
![(K2) For sum of the potential differences around any cycle is equal to zero. r uv resistance of {u, v}. [In other words, V ir.]](/thumbs/88/116126474.jpg "(K2) For sum of the potential differences around any cycle is equal to zero. r uv resistance of {u, v}. [In other words, V ir.]")
Lectures 16-17: Random walks on graphs CSE 525, Winter 2015 Instructor: James R. Lee 1 Random walks Let G V, E) be an undirected graph. The random walk on G is a Markov chain on V that, at each time step,
Parallel Graph Algorithms (con4nued)
Parallel Graph Algorithms (con4nued) MaxFlow A flow network G=(V,E): a directed graph, where each edge (u,v) E has a nonnega4ve capacity c(u,v)>=0. If (u,v) E, we assume that c(u,v)=0. Two dis4nct ver4ces
Spectral Graph Theory
Spectral Graph Theory Aaron Mishtal April 27, 2016 1 / 36 Outline Overview Linear Algebra Primer History Theory Applications Open Problems Homework Problems References 2 / 36 Outline Overview Linear Algebra
Probability & Computing
Probability & Computing Stochastic Process time t {X t t 2 T } state space Ω X t 2 state x 2 discrete time: T is countable T = {0,, 2,...} discrete space: Ω is finite or countably infinite X 0,X,X 2,...
Solving systems of linear equations
Chapter 5 Solving systems of linear equations Let A R n n and b R n be given. We want to solve Ax = b. What is the fastest method you know? We usually learn how to solve systems of linear equations via
The Push Algorithm for Spectral Ranking
The Push Algorithm for Spectral Ranking Paolo Boldi Sebastiano Vigna March 8, 204 Abstract The push algorithm was proposed first by Jeh and Widom [6] in the context of personalized PageRank computations
Random Walks on Graphs. One Concrete Example of a random walk Motivation applications
Random Walks on Graphs Outline One Concrete Example of a random walk Motivation applications shuffling cards universal traverse sequence self stabilizing token management scheme random sampling enumeration
Fiedler s Theorems on Nodal Domains
Spectral Graph Theory Lecture 7 Fiedler s Theorems on Nodal Domains Daniel A Spielman September 9, 202 7 About these notes These notes are not necessarily an accurate representation of what happened in
Lecture 12 : Graph Laplacians and Cheeger s Inequality
CPS290: Algorithmic Foundations of Data Science March 7, 2017 Lecture 12 : Graph Laplacians and Cheeger s Inequality Lecturer: Kamesh Munagala Scribe: Kamesh Munagala Graph Laplacian Maybe the most beautiful
Graphs in Machine Learning
Graphs in Machine Learning Michal Valko INRIA Lille - Nord Europe, France Partially based on material by: Ulrike von Luxburg, Gary Miller, Doyle & Schnell, Daniel Spielman January 27, 2015 MVA 2014/2015