Multiscale Approach for the Network Compression-friendly Ordering
|
|
- Angelica Short
- 5 years ago
- Views:
Transcription
1 Multiscale Approach for the Network Compression-friendly Ordering Ilya Safro (Argonne National Laboratory) and Boris Temkin (Weizmann Institute of Science) SIAM Parallel Processing for Scientific Computing 2012
2 Motivation Networks can be huge (tera-, peta-, exa-, zetta-, yotta-,... bytes) structurally different at different resolutions collected in parallel, i.e., the data is mixed noisy, irregular, etc. 2
3 Motivation Networks can be huge (tera-, peta-, exa-, zetta-, yotta-,... bytes) structurally different at different resolutions collected in parallel, i.e., the data is mixed noisy, irregular, etc. Challenges How to store the network efficiently? How to design an extremely fast access to nodes and links? How to minimize the number of cache misses? 3
4 Problem Find a compressed representation of a network. 4
5 Network representation: compressed row format Node Sorted list of neighbors with possible edge info 1 2, 5, 6, 12, 18, 23, , 1592, 1600 [KDD09 Chierichetti, Kumar, Lattanzi, Mitzenmacher, Panconesi, Raghavan]: given a sorted list of neighbours (x 1, x 2, x 3,...), represent it by a list of differences (x 1, x 2 x 1, x 3 x 1,...) or (x 1, x 2 x 1, x 3 x 2,...). 5
6 Network representation: compressed row format Node Sorted list of neighbors with possible edge info 1 2, 5, 6, 12, 18, 23, , 1592, 1600 [KDD09 Chierichetti, Kumar, Lattanzi, Mitzenmacher, Panconesi, Raghavan]: given a sorted list of neighbours (x 1, x 2, x 3,...), represent it by a list of differences (x 1, x 2 x 1, x 3 x 1,...) or (x 1, x 2 x 1, x 3 x 2,...). Node Sorted list of neighbors with possible edge info 1 1, 4, 5, 11, 17, 22, , 8, 16 6
7 Network representation: compressed row format Node Sorted list of neighbors with possible edge info 1 2, 5, 6, 12, 18, 23, , 1592, 1600 [KDD09 Chierichetti, Kumar, Lattanzi, Mitzenmacher, Panconesi, Raghavan]: given a sorted list of neighbours (x 1, x 2, x 3,...), represent it by a list of differences (x 1, x 2 x 1, x 3 x 1,...) or (x 1, x 2 x 1, x 3 x 2,...). Node Sorted list of neighbors with possible edge info 1 1, 4, 5, 11, 17, 22, , 8, and then use a compression algorithm (such as γ-encoding). Example: Boldi-Vigna compression. 7
8 The Minimum Logarithmic Arrangement problem Problem MLogA: minimize over all possible π s(n) lg π(i) π(j). ij E 8
9 The Minimum Logarithmic Arrangement problem Problem MLogA: minimize over all possible π s(n) lg π(i) π(j). ij E when we have additional information about link/node importance (or its expected access frequency) then introduce importance coefficients w ij, node volumes v i and formulate the generalized version of MLogA Problem GMLogA : min π s(n) c(g, x π ) = ij E w ij lg x i x j such that i V x i = v i /2 + k,π(k)<π(i) v k 9
10 The Minimum Logarithmic Arrangement problem Goal: minimize the number of bits per link: c(g, x π )/ ij E w ij. MLogA is NP-hard [Chierichetti, Kumar, Lattanzi, Mitzenmacher, Panconesi, Raghavan]. Proof using the inapproximability of MaxCut Solutions of MLogA and MinLA are different for this graph [CKLMPP09] 10
11 Main questions Coarsening Uncoarsening Distance metric between nodes Fine to coarse projection operator Relaxation and Refinement Coarse to fine operator Exact solution S, Temkin Multiscale approach for the network compression-friendly ordering, 2011 Lindstrom The Minimum Edge Product Linear Ordering Problem,
12 AMG coarsening c f Fine(f)-to-coarse(c) interpolation operator i L f Weighted Laplacian of G at level f j iw(i j) fine level vertices b l i k w(k l) i Σ w(i j) a 0 coarse level vertices L c ( c f )T L f c f w IJ = l,k iw(il) w lk iw(kj) 12
13 How to measure the connectivity? Examples of existing approaches Shortest path All/some (weighted) indirect paths Spectral approaches Flow network capacity based approaches Random-walk approaches: commute time, first-passage time, etc. (Fouss, Pirotte, Renders, Saerens,...) Speed of convergence of the compatible relaxation from AMG (Brandt, Ron, Livne,...) Probabilistic interpretation of a diffusion (Nadler, Lafon, Coifman, Kevrekidis,...) Effective resistance of a graph (Ghosh, Boyd, Saberi,...) 13
14 Stationary iterative relaxation Relaxation process that shows which pair of vertices tends to be more connected than other. 1. i V define x i = rand() 2. Do k times step 3 3. i V x k i = (1 ω)x k 1 i + ω j w ijx k 1 j / ij w ij Conjecture If x i x j > x u x v then the local connectivity between u and v is stronger than that between i and j. We will call s (k) ij = x i x j the algebraic distance between i and j after k iterations. 14
15 Toy example: graph mesh 20x40+diagonal edge weights: red=2, black=1 15
16 Mesh 20x40+diagonal, random 2D initialization 16
17 Mesh 20x40+diagonal, after 10 iterations of JOR 17
18 Stationary iterative relaxation Rewrite the iterative process as x (k+1) = Hx (k), where H: H GS = (D L) 1 U, H SOR = (D/ω L) 1 ((1/ω 1)D + U), H JAC = D 1 (L + U), H JOR = (D/ω) 1 ((1/ω 1)D + L + U). Definition Extended p-normed algebraic distance between i and j after k iterations x (k+1) = Hx (k) on R random initializations ρ (k) ij := ( R r=1 x (k,r) i x (k,r) j p ) 1/p 18
19 Stationary iterative relaxation Rewrite the iterative process as x (k+1) = Hx (k), where H: H GS = (D L) 1 U, H SOR = (D/ω L) 1 ((1/ω 1)D + U), H JAC = D 1 (L + U), H JOR = (D/ω) 1 ((1/ω 1)D + L + U). Definition Extended p-normed algebraic distance between i and j after k iterations x (k+1) = Hx (k) on R random initializations ρ (k) ij := ( R r=1 lg x (k,r) i x (k,r) j p ) 1/p 19
20 Applications: multilevel graph coarsening Algorithmic Classical Algebraic component AMG distance-based AMG Future volume for edge weight algebraic distance C-points selection C-points selection edge weight algebraic distance Interpolation edge weight weak edge filtering operator Ron, S, Brandt Relaxation-based coarsening and multiscale graph organization, SIAM Multiscale Modeling and Simulations, 2011 Chen, S Algebraic distance on graphs, SISC, 2011 S, Sanders, Schulz Advanced coarsening schemes for multilevel graph partitioning,
21 Uncoarsening P=P 0 S 0 P 1 A. Coarsening Defines the hierarchical structure (P=P,P,...,P ) 0 1 k P k 1 S k 1 C. Interpolation S and 1 Relaxation Produces an initial solution of P i 1of from the solution S i P i and constructs final solution S i 1 from Pi 1 P k S k B. Exact solution 21
22 w ia w ic w ie Uncoarsening: minimizing the contribution of one node a b c d e i 22
23 w ia w ic w ie Uncoarsening: minimizing the contribution of one node a b c d e i N i the set of ith neighbors with assigned coordinates x j. To minimize the local contribution of i to the total energy, we have to assign to it a coordinate x i that minimizes j N i w ij lg x i x j. (1) j N i, x i = x j x i = x t (1) is, we resolve this by setting t = arg min k N i k j N i w kj lg x k x j. (2) 23
24 Uncoarsening: minimizing the contribution of one node Problem arg min k Ni k j N i w kj lg x k x j Trivial solution complexity: O( N i 2 ) 24
25 Uncoarsening: minimizing the contribution of one node Problem arg min k Ni k j N i w kj lg x k x j Trivial solution complexity: O( N i 2 ) Our linear approach: look for the nearly minimum sum in the point of maximal density. How to do this: Parzen window (or kernel density estimation) method. The density at point x is estimated as d(x) = 1 K N i h j N i ( ) x xj h ˆd(x) = 1 w ij 2 x xj /h N i h j N i where K is a kernel and h is a smoothing parameter. (3) 25
26 Uncoarsening Interpolation is similar to AMG, x f = f c x c fine seeds are projected from x c fine non-seeds are calculated from Kernel Density Estimation legalization of the order by resolving overlaps Compatible relaxation: seeds are invariant, non-seeds are relaxed Gauss-Seidel relaxation: all nodes are relaxed Refinement: nearest-neighbors improvements 26
27 Refinement: window minimization pass W Find π of W that minimizes w ij lg x i x j + w ij lg x i x j ij W i W,j W subject to x i = v i /2 + k,π(k)<π(i) v k 27
28 What are the most competitive algorithms today? Randomized ordering - sometimes is better than parallel network crawling (fast to obtain, bad for performance) Lexicographical - network traversal for some order of neighbours such as BFS and DFS (easy to calculate, can be good for networks with excellent locality) Gray ordering - inspired by Gray coding when two successive vectors differ by exactly one bit (easy to calculate, good for Web-like (or good locality) networks) Shingle ordering - brings nodes with similar neighborhoods together, uses Jaccard coefficient J(A, B) = A B / A B to measure the similarity (works good in preferential attachment models when rich gets richer). LayeredLPA - label propagation algorithm is similar to the algebraic distance (usually better than previous methods) Spectral methods - based on Fiedler vector 28
29 ms-gmloga vs min(rnd, lex, nat) ratios (lg or la)/min(nat,lex,rnd) la/min(nat,lex,rnd) lg/min(nat,lex,rnd) ordered graphs 29
30 ms-gmloga vs min(rnd, lex, nat) 1 no-n-n-lg/min(nat,lex,rnd) 1 no-n-n-lg/min(nat,lex,rnd) k25-n-n-lg/min(nat,lex,rnd) k25-n-n-lg/min(nat,lex,rnd) ratios lg/min(nat,lex,rnd) ratios lg/min(nat,lex,rnd) ordered graphs (a) directed ordered graphs (b) undirected Comparison of the fastest and the slowest versions of ms-gmloga. Notations no-n-n-lg and k25-n-n lg correspond to the fast and slow versions of ms-gmloga, respectively. 30
31 ms-gmloga vs Gray/Shingle ratios ms-gmloga/gray directed: ms-gmloga/gray undirected: ms-gmloga/gray ordered graphs (a) Gray ordering vs ms-gmloga ratios ms-gmloga/shingle directed: ms-gmloga/shingle undirected: ms-gmloga/shingle ordered graphs (b) Double shingle vs ms-gmloga 31
32 Comparison of LayeredLPA vs ms-gmloga ratios ms-gmloga/layeredlpa undirected: ms-gmloga/layeredlpa ordered graphs 32
33 Scalability: time vs graph size lg( V + E ) lg(running time in sec.) 33
34 Heavy-tailed degree distribution Compressed BPL/Native BPL Networks with heavy-tailed degree distribution 34
35 Cache misses: ParMETIS, Graclus, hmetis, Power method, SVD Cache misses: GMLogA/Native Networks with heavy-tailed degree distribution 35
36 Further compression. Boldi-Vigna algorithm 5 4 Compression Improvement Compression Improvement Order Improvement Order Improvement a) initial ordering is natural b) initial ordering is Gray 36
37 MUSKETEER - Mulltiscale Entropic Network Generator First release (next week)! safro/musindex.html 37
38 Conclusions Linear time method for the network compression-friendly ordering Computational results: (heavy-tailed) networks can be compressed; number of cache misses can be minimized; running time can be better Questions: directed graph coarsening; collective refinement of nodes; more sophisticated gap encodings Thank you! 38
39 Conclusions Linear time method for the network compression-friendly ordering Computational results: (heavy-tailed) networks can be compressed; number of cache misses can be minimized; running time can be better Questions: directed graph coarsening; collective refinement of nodes; more sophisticated gap encodings Looking for a position Thank you! 39
ALGEBRAIC DISTANCE ON GRAPHS
ALGEBRAIC DISTANCE ON GRAPHS JIE CHEN AND ILYA SAFRO Abstract. Measuring the connection strength between a pair of vertices in a graph is one of the most vital concerns in many graph applications. Simple
More informationSolving PDEs with Multigrid Methods p.1
Solving PDEs with Multigrid Methods Scott MacLachlan maclachl@colorado.edu Department of Applied Mathematics, University of Colorado at Boulder Solving PDEs with Multigrid Methods p.1 Support and Collaboration
More informationAlgebraic Multigrid as Solvers and as Preconditioner
Ò Algebraic Multigrid as Solvers and as Preconditioner Domenico Lahaye domenico.lahaye@cs.kuleuven.ac.be http://www.cs.kuleuven.ac.be/ domenico/ Department of Computer Science Katholieke Universiteit Leuven
More informationSpectral element agglomerate AMGe
Spectral element agglomerate AMGe T. Chartier 1, R. Falgout 2, V. E. Henson 2, J. E. Jones 4, T. A. Manteuffel 3, S. F. McCormick 3, J. W. Ruge 3, and P. S. Vassilevski 2 1 Department of Mathematics, Davidson
More informationLecture 13 Spectral Graph Algorithms
COMS 995-3: Advanced Algorithms March 6, 7 Lecture 3 Spectral Graph Algorithms Instructor: Alex Andoni Scribe: Srikar Varadaraj Introduction Today s topics: Finish proof from last lecture Example of random
More informationNew Multigrid Solver Advances in TOPS
New Multigrid Solver Advances in TOPS R D Falgout 1, J Brannick 2, M Brezina 2, T Manteuffel 2 and S McCormick 2 1 Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, P.O.
More informationBootstrap AMG. Kailai Xu. July 12, Stanford University
Bootstrap AMG Kailai Xu Stanford University July 12, 2017 AMG Components A general AMG algorithm consists of the following components. A hierarchy of levels. A smoother. A prolongation. A restriction.
More informationAspects of Multigrid
Aspects of Multigrid Kees Oosterlee 1,2 1 Delft University of Technology, Delft. 2 CWI, Center for Mathematics and Computer Science, Amsterdam, SIAM Chapter Workshop Day, May 30th 2018 C.W.Oosterlee (CWI)
More informationAdaptive algebraic multigrid methods in lattice computations
Adaptive algebraic multigrid methods in lattice computations Karsten Kahl Bergische Universität Wuppertal January 8, 2009 Acknowledgements Matthias Bolten, University of Wuppertal Achi Brandt, Weizmann
More information1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
More informationEntropy for Sparse Random Graphs With Vertex-Names
Entropy for Sparse Random Graphs With Vertex-Names David Aldous 11 February 2013 if a problem seems... Research strategy (for old guys like me): do-able = give to Ph.D. student maybe do-able = give to
More informationDiffusion/Inference geometries of data features, situational awareness and visualization. Ronald R Coifman Mathematics Yale University
Diffusion/Inference geometries of data features, situational awareness and visualization Ronald R Coifman Mathematics Yale University Digital data is generally converted to point clouds in high dimensional
More informationSpectral Graph Theory and its Applications. Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity
Spectral Graph Theory and its Applications Daniel A. Spielman Dept. of Computer Science Program in Applied Mathematics Yale Unviersity Outline Adjacency matrix and Laplacian Intuition, spectral graph drawing
More informationCommunities, Spectral Clustering, and Random Walks
Communities, Spectral Clustering, and Random Walks David Bindel Department of Computer Science Cornell University 26 Sep 2011 20 21 19 16 22 28 17 18 29 26 27 30 23 1 25 5 8 24 2 4 14 3 9 13 15 11 10 12
More informationUsing an Auction Algorithm in AMG based on Maximum Weighted Matching in Matrix Graphs
Using an Auction Algorithm in AMG based on Maximum Weighted Matching in Matrix Graphs Pasqua D Ambra Institute for Applied Computing (IAC) National Research Council of Italy (CNR) pasqua.dambra@cnr.it
More informationA greedy strategy for coarse-grid selection
A greedy strategy for coarse-grid selection S. MacLachlan Yousef Saad August 3, 2006 Abstract Efficient solution of the very large linear systems that arise in numerical modelling of real-world applications
More information1 Matrix notation and preliminaries from spectral graph theory
Graph clustering (or community detection or graph partitioning) is one of the most studied problems in network analysis. One reason for this is that there are a variety of ways to define a cluster or community.
More informationSpectral Clustering. Guokun Lai 2016/10
Spectral Clustering Guokun Lai 2016/10 1 / 37 Organization Graph Cut Fundamental Limitations of Spectral Clustering Ng 2002 paper (if we have time) 2 / 37 Notation We define a undirected weighted graph
More informationUniform Convergence of a Multilevel Energy-based Quantization Scheme
Uniform Convergence of a Multilevel Energy-based Quantization Scheme Maria Emelianenko 1 and Qiang Du 1 Pennsylvania State University, University Park, PA 16803 emeliane@math.psu.edu and qdu@math.psu.edu
More informationBackground. Background. C. T. Kelley NC State University tim C. T. Kelley Background NCSU, Spring / 58
Background C. T. Kelley NC State University tim kelley@ncsu.edu C. T. Kelley Background NCSU, Spring 2012 1 / 58 Notation vectors, matrices, norms l 1 : max col sum... spectral radius scaled integral norms
More informationAMG for a Peta-scale Navier Stokes Code
AMG for a Peta-scale Navier Stokes Code James Lottes Argonne National Laboratory October 18, 2007 The Challenge Develop an AMG iterative method to solve Poisson 2 u = f discretized on highly irregular
More informationGlobal vs. Multiscale Approaches
Harmonic Analysis on Graphs Global vs. Multiscale Approaches Weizmann Institute of Science, Rehovot, Israel July 2011 Joint work with Matan Gavish (WIS/Stanford), Ronald Coifman (Yale), ICML 10' Challenge:
More informationLecture: Local Spectral Methods (2 of 4) 19 Computing spectral ranking with the push procedure
Stat260/CS294: Spectral Graph Methods Lecture 19-04/02/2015 Lecture: Local Spectral Methods (2 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough. They provide
More informationStabilization and Acceleration of Algebraic Multigrid Method
Stabilization and Acceleration of Algebraic Multigrid Method Recursive Projection Algorithm A. Jemcov J.P. Maruszewski Fluent Inc. October 24, 2006 Outline 1 Need for Algorithm Stabilization and Acceleration
More informationDrawing Large Graphs by Multilevel Maxent-Stress Optimization
Drawing Large Graphs by Multilevel Maxent-Stress Optimization Henning Meyerhenke, Martin Nöllenburg, Christian Schulz 23rd International Symposium on Graph Drawing & Network Visualization, Karlsruhe 1
More informationLecture 24: Element-wise Sampling of Graphs and Linear Equation Solving. 22 Element-wise Sampling of Graphs and Linear Equation Solving
Stat260/CS294: Randomized Algorithms for Matrices and Data Lecture 24-12/02/2013 Lecture 24: Element-wise Sampling of Graphs and Linear Equation Solving Lecturer: Michael Mahoney Scribe: Michael Mahoney
More informationPreface to the Second Edition. Preface to the First Edition
n page v Preface to the Second Edition Preface to the First Edition xiii xvii 1 Background in Linear Algebra 1 1.1 Matrices................................. 1 1.2 Square Matrices and Eigenvalues....................
More informationKasetsart University Workshop. Multigrid methods: An introduction
Kasetsart University Workshop Multigrid methods: An introduction Dr. Anand Pardhanani Mathematics Department Earlham College Richmond, Indiana USA pardhan@earlham.edu A copy of these slides is available
More informationMarkov Chains and Web Ranking: a Multilevel Adaptive Aggregation Method
Markov Chains and Web Ranking: a Multilevel Adaptive Aggregation Method Hans De Sterck Department of Applied Mathematics, University of Waterloo Quoc Nguyen; Steve McCormick, John Ruge, Tom Manteuffel
More informationData Analysis and Manifold Learning Lecture 7: Spectral Clustering
Data Analysis and Manifold Learning Lecture 7: Spectral Clustering Radu Horaud INRIA Grenoble Rhone-Alpes, France Radu.Horaud@inrialpes.fr http://perception.inrialpes.fr/ Outline of Lecture 7 What is spectral
More informationGraph Metrics and Dimension Reduction
Graph Metrics and Dimension Reduction Minh Tang 1 Michael Trosset 2 1 Applied Mathematics and Statistics The Johns Hopkins University 2 Department of Statistics Indiana University, Bloomington November
More informationCS224W: Social and Information Network Analysis Jure Leskovec, Stanford University
CS224W: Social and Information Network Analysis Jure Leskovec Stanford University Jure Leskovec, Stanford University http://cs224w.stanford.edu Task: Find coalitions in signed networks Incentives: European
More informationCertifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering
Certifying the Global Optimality of Graph Cuts via Semidefinite Programming: A Theoretic Guarantee for Spectral Clustering Shuyang Ling Courant Institute of Mathematical Sciences, NYU Aug 13, 2018 Joint
More informationHarmonic Analysis and Geometries of Digital Data Bases
Harmonic Analysis and Geometries of Digital Data Bases AMS Session Special Sesson on the Mathematics of Information and Knowledge, Ronald Coifman (Yale) and Matan Gavish (Stanford, Yale) January 14, 2010
More informationMLCC Clustering. Lorenzo Rosasco UNIGE-MIT-IIT
MLCC 2018 - Clustering Lorenzo Rosasco UNIGE-MIT-IIT About this class We will consider an unsupervised setting, and in particular the problem of clustering unlabeled data into coherent groups. MLCC 2018
More informationSpectral Algorithms I. Slides based on Spectral Mesh Processing Siggraph 2010 course
Spectral Algorithms I Slides based on Spectral Mesh Processing Siggraph 2010 course Why Spectral? A different way to look at functions on a domain Why Spectral? Better representations lead to simpler solutions
More informationNumerical Programming I (for CSE)
Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let
More informationChapter 7 Iterative Techniques in Matrix Algebra
Chapter 7 Iterative Techniques in Matrix Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematics University of California, Berkeley Math 128B Numerical Analysis Vector Norms Definition
More informationData Mining and Analysis: Fundamental Concepts and Algorithms
Data Mining and Analysis: Fundamental Concepts and Algorithms dataminingbook.info Mohammed J. Zaki 1 Wagner Meira Jr. 2 1 Department of Computer Science Rensselaer Polytechnic Institute, Troy, NY, USA
More informationOPERATOR-BASED INTERPOLATION FOR BOOTSTRAP ALGEBRAIC MULTIGRID
OPERATOR-BASED INTERPOLATION FOR BOOTSTRAP ALGEBRAIC MULTIGRID T. MANTEUFFEL, S. MCCORMICK, M. PARK, AND J. RUGE Abstract. Bootstrap Algebraic Multigrid (BAMG) is a multigrid-based solver for matrix equations
More informationIMPLEMENTATION OF A PARALLEL AMG SOLVER
IMPLEMENTATION OF A PARALLEL AMG SOLVER Tony Saad May 2005 http://tsaad.utsi.edu - tsaad@utsi.edu PLAN INTRODUCTION 2 min. MULTIGRID METHODS.. 3 min. PARALLEL IMPLEMENTATION PARTITIONING. 1 min. RENUMBERING...
More informationAlgorithms for Data Science: Lecture on Finding Similar Items
Algorithms for Data Science: Lecture on Finding Similar Items Barna Saha 1 Finding Similar Items Finding similar items is a fundamental data mining task. We may want to find whether two documents are similar
More informationAn Introduction to Algebraic Multigrid (AMG) Algorithms Derrick Cerwinsky and Craig C. Douglas 1/84
An Introduction to Algebraic Multigrid (AMG) Algorithms Derrick Cerwinsky and Craig C. Douglas 1/84 Introduction Almost all numerical methods for solving PDEs will at some point be reduced to solving A
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Abstract. We consider the problem of choosing the edge weights of an undirected graph so as to maximize or minimize some function of the
More informationClustering. SVD and NMF
Clustering with the SVD and NMF Amy Langville Mathematics Department College of Charleston Dagstuhl 2/14/2007 Outline Fielder Method Extended Fielder Method and SVD Clustering with SVD vs. NMF Demos with
More informationBeyond Scalar Affinities for Network Analysis or Vector Diffusion Maps and the Connection Laplacian
Beyond Scalar Affinities for Network Analysis or Vector Diffusion Maps and the Connection Laplacian Amit Singer Princeton University Department of Mathematics and Program in Applied and Computational Mathematics
More informationParallel Iterative Methods for Sparse Linear Systems. H. Martin Bücker Lehrstuhl für Hochleistungsrechnen
Parallel Iterative Methods for Sparse Linear Systems Lehrstuhl für Hochleistungsrechnen www.sc.rwth-aachen.de RWTH Aachen Large and Sparse Small and Dense Outline Problem with Direct Methods Iterative
More informationEXAMPLES OF CLASSICAL ITERATIVE METHODS
EXAMPLES OF CLASSICAL ITERATIVE METHODS In these lecture notes we revisit a few classical fixpoint iterations for the solution of the linear systems of equations. We focus on the algebraic and algorithmic
More informationIntroduction to Scientific Computing
(Lecture 5: Linear system of equations / Matrix Splitting) Bojana Rosić, Thilo Moshagen Institute of Scientific Computing Motivation Let us resolve the problem scheme by using Kirchhoff s laws: the algebraic
More informationRobust solution of Poisson-like problems with aggregation-based AMG
Robust solution of Poisson-like problems with aggregation-based AMG Yvan Notay Université Libre de Bruxelles Service de Métrologie Nucléaire Paris, January 26, 215 Supported by the Belgian FNRS http://homepages.ulb.ac.be/
More informationAlgebraic Multigrid (AMG) for saddle point systems from meshfree discretizations
Algebraic Multigrid (AMG) for saddle point systems from meshfree discretizations K. H. Leem 1, S. Oliveira 2, and D. E. Stewart 3 1 Applied Mathematics and Computational Sciences, The University of Iowa,
More informationR ij = 2. Using all of these facts together, you can solve problem number 9.
Help for Homework Problem #9 Let G(V,E) be any undirected graph We want to calculate the travel time across the graph. Think of each edge as one resistor of 1 Ohm. Say we have two nodes: i and j Let the
More informationCME342 Parallel Methods in Numerical Analysis. Matrix Computation: Iterative Methods II. Sparse Matrix-vector Multiplication.
CME342 Parallel Methods in Numerical Analysis Matrix Computation: Iterative Methods II Outline: CG & its parallelization. Sparse Matrix-vector Multiplication. 1 Basic iterative methods: Ax = b r = b Ax
More informationLaplacian Filters. Sobel Filters. Laplacian Filters. Laplacian Filters. Laplacian Filters. Laplacian Filters
Sobel Filters Note that smoothing the image before applying a Sobel filter typically gives better results. Even thresholding the Sobel filtered image cannot usually create precise, i.e., -pixel wide, edges.
More informationFokker-Planck Equation on Graph with Finite Vertices
Fokker-Planck Equation on Graph with Finite Vertices January 13, 2011 Jointly with S-N Chow (Georgia Tech) Wen Huang (USTC) Hao-min Zhou(Georgia Tech) Functional Inequalities and Discrete Spaces Outline
More informationConditioning of the Entries in the Stationary Vector of a Google-Type Matrix. Steve Kirkland University of Regina
Conditioning of the Entries in the Stationary Vector of a Google-Type Matrix Steve Kirkland University of Regina June 5, 2006 Motivation: Google s PageRank algorithm finds the stationary vector of a stochastic
More informationOn nonlinear adaptivity with heterogeneity
On nonlinear adaptivity with heterogeneity Jed Brown jed@jedbrown.org (CU Boulder) Collaborators: Mark Adams (LBL), Matt Knepley (UChicago), Dave May (ETH), Laetitia Le Pourhiet (UPMC), Ravi Samtaney (KAUST)
More informationADAPTIVE ALGEBRAIC MULTIGRID
ADAPTIVE ALGEBRAIC MULTIGRID M. BREZINA, R. FALGOUT, S. MACLACHLAN, T. MANTEUFFEL, S. MCCORMICK, AND J. RUGE Abstract. Efficient numerical simulation of physical processes is constrained by our ability
More informationRandom Surfing on Multipartite Graphs
Random Surfing on Multipartite Graphs Athanasios N. Nikolakopoulos, Antonia Korba and John D. Garofalakis Department of Computer Engineering and Informatics, University of Patras December 07, 2016 IEEE
More informationSelf-Tuning Semantic Image Segmentation
Self-Tuning Semantic Image Segmentation Sergey Milyaev 1,2, Olga Barinova 2 1 Voronezh State University sergey.milyaev@gmail.com 2 Lomonosov Moscow State University obarinova@graphics.cs.msu.su Abstract.
More informationA Generalized Eigensolver Based on Smoothed Aggregation (GES-SA) for Initializing Smoothed Aggregation Multigrid (SA)
NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS Numer. Linear Algebra Appl. 2007; 07: 6 [Version: 2002/09/8 v.02] A Generalized Eigensolver Based on Smoothed Aggregation (GES-SA) for Initializing Smoothed Aggregation
More informationLECTURE NOTE #11 PROF. ALAN YUILLE
LECTURE NOTE #11 PROF. ALAN YUILLE 1. NonLinear Dimension Reduction Spectral Methods. The basic idea is to assume that the data lies on a manifold/surface in D-dimensional space, see figure (1) Perform
More informationMini-project in scientific computing
Mini-project in scientific computing Eran Treister Computer Science Department, Ben-Gurion University of the Negev, Israel. March 7, 2018 1 / 30 Scientific computing Involves the solution of large computational
More informationTHE HIDDEN CONVEXITY OF SPECTRAL CLUSTERING
THE HIDDEN CONVEXITY OF SPECTRAL CLUSTERING Luis Rademacher, Ohio State University, Computer Science and Engineering. Joint work with Mikhail Belkin and James Voss This talk A new approach to multi-way
More informationCompressed Sensing and Linear Codes over Real Numbers
Compressed Sensing and Linear Codes over Real Numbers Henry D. Pfister (joint with Fan Zhang) Texas A&M University College Station Information Theory and Applications Workshop UC San Diego January 31st,
More informationMining of Massive Datasets Jure Leskovec, AnandRajaraman, Jeff Ullman Stanford University
Note to other teachers and users of these slides: We would be delighted if you found this our material useful in giving your own lectures. Feel free to use these slides verbatim, or to modify them to fit
More informationIterative Methods for Solving A x = b
Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http
More informationConvex Optimization of Graph Laplacian Eigenvalues
Convex Optimization of Graph Laplacian Eigenvalues Stephen Boyd Stanford University (Joint work with Persi Diaconis, Arpita Ghosh, Seung-Jean Kim, Sanjay Lall, Pablo Parrilo, Amin Saberi, Jun Sun, Lin
More informationAMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences)
AMS526: Numerical Analysis I (Numerical Linear Algebra for Computational and Data Sciences) Lecture 19: Computing the SVD; Sparse Linear Systems Xiangmin Jiao Stony Brook University Xiangmin Jiao Numerical
More information6. Iterative Methods for Linear Systems. The stepwise approach to the solution...
6 Iterative Methods for Linear Systems The stepwise approach to the solution Miriam Mehl: 6 Iterative Methods for Linear Systems The stepwise approach to the solution, January 18, 2013 1 61 Large Sparse
More informationMachine Learning for Data Science (CS4786) Lecture 11
Machine Learning for Data Science (CS4786) Lecture 11 Spectral clustering Course Webpage : http://www.cs.cornell.edu/courses/cs4786/2016sp/ ANNOUNCEMENT 1 Assignment P1 the Diagnostic assignment 1 will
More informationIntroduction. Marcel Radermacher Algorithmen zur Visualisierung von Graphen
Algorithms for Graph Visualization INSTITUT FÜR THEORETISCHE INFORMATIK FAKULTÄT FÜR INFORMATIK Marcel Radermacher 30.11.2017 1 Introduction Before: always based on some properties: tree, series-parallel
More informationAn Algebraic Multigrid Method for Eigenvalue Problems
An Algebraic Multigrid Method for Eigenvalue Problems arxiv:1503.08462v1 [math.na] 29 Mar 2015 Xiaole Han, Yunhui He, Hehu Xie and Chunguang You Abstract An algebraic multigrid method is proposed to solve
More informationRobust and Adaptive Multigrid Methods: comparing structured and algebraic approaches
NUMERICAL LINEAR ALGEBRA WITH APPLICATIONS Numer. Linear Algebra Appl. 0000; 00:1 25 Published online in Wiley InterScience www.interscience.wiley.com). Robust and Adaptive Multigrid Methods: comparing
More information9. Iterative Methods for Large Linear Systems
EE507 - Computational Techniques for EE Jitkomut Songsiri 9. Iterative Methods for Large Linear Systems introduction splitting method Jacobi method Gauss-Seidel method successive overrelaxation (SOR) 9-1
More informationToday s class. Linear Algebraic Equations LU Decomposition. Numerical Methods, Fall 2011 Lecture 8. Prof. Jinbo Bi CSE, UConn
Today s class Linear Algebraic Equations LU Decomposition 1 Linear Algebraic Equations Gaussian Elimination works well for solving linear systems of the form: AX = B What if you have to solve the linear
More informationAdaptive Multigrid for QCD. Lattice University of Regensburg
Lattice 2007 University of Regensburg Michael Clark Boston University with J. Brannick, R. Brower, J. Osborn and C. Rebbi -1- Lattice 2007, University of Regensburg Talk Outline Introduction to Multigrid
More informationThe Generalized Haar-Walsh Transform (GHWT) for Data Analysis on Graphs and Networks
The Generalized Haar-Walsh Transform (GHWT) for Data Analysis on Graphs and Networks Jeff Irion & Naoki Saito Department of Mathematics University of California, Davis SIAM Annual Meeting 2014 Chicago,
More informationLecture: Some Practical Considerations (3 of 4)
Stat260/CS294: Spectral Graph Methods Lecture 14-03/10/2015 Lecture: Some Practical Considerations (3 of 4) Lecturer: Michael Mahoney Scribe: Michael Mahoney Warning: these notes are still very rough.
More informationSpectral Clustering. Zitao Liu
Spectral Clustering Zitao Liu Agenda Brief Clustering Review Similarity Graph Graph Laplacian Spectral Clustering Algorithm Graph Cut Point of View Random Walk Point of View Perturbation Theory Point of
More informationLearning on Graphs and Manifolds. CMPSCI 689 Sridhar Mahadevan U.Mass Amherst
Learning on Graphs and Manifolds CMPSCI 689 Sridhar Mahadevan U.Mass Amherst Outline Manifold learning is a relatively new area of machine learning (2000-now). Main idea Model the underlying geometry of
More informationMULTILEVEL ADAPTIVE AGGREGATION FOR MARKOV CHAINS, WITH APPLICATION TO WEB RANKING
MULTILEVEL ADAPTIVE AGGREGATION FOR MARKOV CHAINS, WITH APPLICATION TO WEB RANKING H. DE STERCK, THOMAS A. MANTEUFFEL, STEPHEN F. MCCORMICK, QUOC NGUYEN, AND JOHN RUGE Abstract. A multilevel adaptive aggregation
More informationDedicated to J. Brahms Symphony No. 1 in C minor, Op. 68
LEAN ALGEBRAIC MULTIGRID (LAMG): FAST GRAPH LAPLACIAN LINEAR SOLVER OREN E. LIVNE AND ACHI BRANDT Dedicated to J. Brahms Symphony No. in C minor, Op. 68 Abstract. Laplacian matrices of graphs arise in
More informationAlgorithms for Graph Visualization Force-Directed Algorithms
Algorithms for Graph Visualization INSTITUT FÜR THEORETISCHE INFORMATIK FAKULTÄT FÜR INFORMATIK Tamara Mchedlidze 21.12.2016 1 Introduction Before: always based on some properties: tree, series-parallel
More informationVon Neumann Analysis of Jacobi and Gauss-Seidel Iterations
Von Neumann Analysis of Jacobi and Gauss-Seidel Iterations We consider the FDA to the 1D Poisson equation on a grid x covering [0,1] with uniform spacing h, h (u +1 u + u 1 ) = f whose exact solution (to
More informationINTRODUCTION TO MULTIGRID METHODS
INTRODUCTION TO MULTIGRID METHODS LONG CHEN 1. ALGEBRAIC EQUATION OF TWO POINT BOUNDARY VALUE PROBLEM We consider the discretization of Poisson equation in one dimension: (1) u = f, x (0, 1) u(0) = u(1)
More informationParallel Numerics, WT 2016/ Iterative Methods for Sparse Linear Systems of Equations. page 1 of 1
Parallel Numerics, WT 2016/2017 5 Iterative Methods for Sparse Linear Systems of Equations page 1 of 1 Contents 1 Introduction 1.1 Computer Science Aspects 1.2 Numerical Problems 1.3 Graphs 1.4 Loop Manipulations
More informationGraph and Controller Design for Disturbance Attenuation in Consensus Networks
203 3th International Conference on Control, Automation and Systems (ICCAS 203) Oct. 20-23, 203 in Kimdaejung Convention Center, Gwangju, Korea Graph and Controller Design for Disturbance Attenuation in
More informationMath 471 (Numerical methods) Chapter 3 (second half). System of equations
Math 47 (Numerical methods) Chapter 3 (second half). System of equations Overlap 3.5 3.8 of Bradie 3.5 LU factorization w/o pivoting. Motivation: ( ) A I Gaussian Elimination (U L ) where U is upper triangular
More informationAlgorithms and Data Structures
Algorithms and Data Structures, Divide and Conquer Albert-Ludwigs-Universität Freiburg Prof. Dr. Rolf Backofen Bioinformatics Group / Department of Computer Science Algorithms and Data Structures, December
More informationAlgebraic Representation of Networks
Algebraic Representation of Networks 0 1 2 1 1 0 0 1 2 0 0 1 1 1 1 1 Hiroki Sayama sayama@binghamton.edu Describing networks with matrices (1) Adjacency matrix A matrix with rows and columns labeled by
More informationReview: From problem to parallel algorithm
Review: From problem to parallel algorithm Mathematical formulations of interesting problems abound Poisson s equation Sources: Electrostatics, gravity, fluid flow, image processing (!) Numerical solution:
More informationUsing Local Spectral Methods in Theory and in Practice
Using Local Spectral Methods in Theory and in Practice Michael W. Mahoney ICSI and Dept of Statistics, UC Berkeley ( For more info, see: http: // www. stat. berkeley. edu/ ~ mmahoney or Google on Michael
More informationNetwork Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec
Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec Jiezhong Qiu Tsinghua University February 21, 2018 Joint work with Yuxiao Dong (MSR), Hao Ma (MSR), Jian Li (IIIS,
More informationNon-Equidistant Particle-In-Cell for Ion Thruster Plumes
Non-Equidistant Particle-In-Cell for Ion Thruster Plumes IEPC-213-67 Presented at the 33 rd International Electric Propulsion Conference, The George Washington University, Washington, D.C., USA October
More informationLecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo. Department of Mathematics Iowa State University
Lecture 16 Methods for System of Linear Equations (Linear Systems) Songting Luo Department of Mathematics Iowa State University MATH 481 Numerical Methods for Differential Equations Songting Luo ( Department
More informationThe Removal of Critical Slowing Down. Lattice College of William and Mary
The Removal of Critical Slowing Down Lattice 2008 College of William and Mary Michael Clark Boston University James Brannick, Rich Brower, Tom Manteuffel, Steve McCormick, James Osborn, Claudio Rebbi 1
More informationOverlapping Communities
Overlapping Communities Davide Mottin HassoPlattner Institute Graph Mining course Winter Semester 2017 Acknowledgements Most of this lecture is taken from: http://web.stanford.edu/class/cs224w/slides GRAPH
More informationA short course on: Preconditioned Krylov subspace methods. Yousef Saad University of Minnesota Dept. of Computer Science and Engineering
A short course on: Preconditioned Krylov subspace methods Yousef Saad University of Minnesota Dept. of Computer Science and Engineering Universite du Littoral, Jan 19-30, 2005 Outline Part 1 Introd., discretization
More informationFaster quantum algorithm for evaluating game trees
Faster quantum algorithm for evaluating game trees x 7 x 8 Ben Reichardt xx1 1 x 2 x 3 x 4 x 6 OR x 9 x1 1 x 5 x 9 University of Waterloo x 5 AND OR AND AND OR AND ϕ(x) x 7 x 8 x 1 x 2 x 3 x 4 x 6 OR x
More information