Fast Linear Solvers for Laplacian Systems

Transcription

1 s Solver for UCSD Research Exam University of California, San Diego November 8, 2013

2 s Solver 1 2 Matrices Linear in the Graph 3 Overview 4 Solving Linear with Using Computing A of the Problem Solving the System 5

3 s Solver Definitions Let G = (V, E) be an undirected edge-weighted graph on n vertices and m edges. We can generalize an unweighted graph to the case where all edge weights are equal to 1. The degree of a vertex v V is the sum of the weights of the edges adjacent to it, d v = u v w(u, v). The degree matrix is the diagonal matrix whose entries are the degrees of the vertices, (D) vv = d v.

4 Definitions s Solver The adjacency matrix is the n n matrix which is the weight w(e) in the entries corresponding to an edge, { w(u, v) if {u, v} E, (A) uv = 0 otherwise. The is the matrix L = D A.

5 Example s Solver For the following simple graph, L = =

6 Properties of the s Solver Symmetric Diagonally dominant All zero row-sums Non-positive off-diagonal entries Clean quadratic form: x T Lx = (x(u) x(v)) 2 u v

7 A network of n decision-making agents Consensus s Solver edges are communication channels x i is the decision value of each agent values can be influenced by communicating neighbors two agents v i, v j are said to agree when x i = x j Goal: all agents reach a common decision value, called the consensus

8 A network of n decision-making agents Consensus s Solver The potential is a measure of disagreement in the system, Ψ G (x) = 1 2 x T Lx = 1 (x i x j ) 2. 2 v i v j all agents agree Ψ G (x) = 0. Goal: find the vector x which minimizes the potential [OM03]

9 Why? s Solver arise in a number of natural contexts, Characterizing the motion of coupled oscillators [HS08] Approximating Fiedler eigenvectors [ST04]... Computing effective resistance in an electrical network [Kir1847]

10 Electrical Networks s Solver Edges are wires in the network Edge weights correspond to the conductance of each wire Goal: set voltages x 1, x 2, x 3 on the vertices to create a flow of electrical current

11 Electrical Networks s Solver Ohm s law: take conductance(e) = 1/resistance(e), then current(e) = conductance(e) voltage(u) voltage(v) Consider the flow of current from V 1. By Ohm s law: current(v 1, V 2) = 1(voltage(V 1) voltage(v 2)) current(v 1, V 3) = 2(voltage(V 1) voltage(v 3))

12 Electrical Networks s Solver Kirchhoff s law [Kir1847]: For every point in the network, netflow(v) := flow in (v) flow out (v) = 0, except at the injection point, netflow = 1, and the extraction point, netflow = 1.

13 Electrical Networks s Solver So, at vertex V 1 the voltage x 1 must satisfy: netflow(v 1) = current(v 1, V 2) + current(v 1, V 3) 1 = 1(x 1 x 2 ) + 2(x 1 x 3 ) 1 = 3x 1 x 2 2x 3

14 Electrical Networks s Solver Applying the same rules at V 2 and V 3 yields the following system of equations: 3x 1 x 2 2x 3 = 1 x 1 + 2x 2 x 3 = 0 2x 1 x 2 + 3x 3 = 1

15 Electrical Networks s Solver Or, in matrix form, x 1 x 2 x 3 = 1 0 1

16 Electrical Networks s Solver Or, in matrix form, x 1 x 2 x 3 = A system in the of the network

17 Electrical Networks s Solver When current is injected at V 1 and extracted at V 3, the solution vector x can be used to compute the effective resistance of the edge (V 1, V 3), R (V 1,V 3) = x 1 x 3

18 s Solver [OM03] A connected network of decision-making agents Consensus Again edges are communication channels x i is the decision value of each agent values can be influenced by communicating neighbors two agents v i, v j are said to agree when x i = x j Goal: all agents reach a common decision value, called the consensus

19 s Solver [OM03] A connected network of decision-making agents Consensus Again Suppose each agent evolves their decision value according to the distributed linear protocol, ẋ i (t) = v j v i x j (t) x i (t). Then the solution to the system ẋ = Lx, x(0) R n is the vector of decision values as a function of t. Goal: all agents reach a common decision value, called the consensus

20 s Solver 1 2 Matrices Linear in the Graph 3 Overview 4 Solving Linear with Using Computing A of the Problem Solving the System 5

21 Solving Directly s Solver Given a system of linear equations, Ax = b, we can solve the system directly using Gaussian elimination. Gaussian elimination takes O(n 3 ) time in general. - [CW90] show the exponent can be as small as [Wil12] improves this to

22 Iterative s Solver Iterative methods for solving systems Ax = b are based on a sequence of increasingly better approximations. The idea is to construct a sequence x (0), x (1),..., x (i),..., x (N),... that converges to the true solution, x, lim k x (k) = x. In practice, the iterative process is stopped after N iterations when x (N) x < ɛ for any vector norm and a prescribed error parameter ɛ.

23 Iterative s Solver Richardson s Method([Young53],[GV61]) An iterative method that improves approximations using the residual error, b Ax (i), at each step: x (i+1) = x (i) + (b Ax (i) ) = b + (I A)x (i)

24 Preconditioned Iterative s Solver To accelerate the iterative process, instead use an approximation of the matrix, called a preconditioner. This method will produce a solution to the preconditioned system: B 1 Ax = B 1 b.

25 Preconditioned Iterative s Solver To accelerate the iterative process, instead use an approximation of the matrix, called a preconditioner. This method will produce a solution to the preconditioned system: B 1 Ax = B 1 b. What makes a matrix B a good preconditioner for A?

26 Preconditioned Iterative s Solver To accelerate the iterative process, instead use an approximation of the matrix, called a preconditioner. This method will produce a solution to the preconditioned system: B 1 Ax = B 1 b. What makes a matrix B a good preconditioner for A? 1 It is a very good approximation of the matrix

27 Preconditioned Iterative s Solver To accelerate the iterative process, instead use an approximation of the matrix, called a preconditioner. This method will produce a solution to the preconditioned system: B 1 Ax = B 1 b. What makes a matrix B a good preconditioner for A? 1 It is a very good approximation of the matrix 2 It reduces the number of iterations

28 Preconditioned Iterative s Solver To accelerate the iterative process, instead use an approximation of the matrix, called a preconditioner. This method will produce a solution to the preconditioned system: B 1 Ax = B 1 b. What makes a matrix B a good preconditioner for A? 1 It is a very good approximation of the matrix 2 It reduces the number of iterations 3 B can be computed quickly

29 Preconditioned Iterative s Solver To accelerate the iterative process, instead use an approximation of the matrix, called a preconditioner. This method will produce a solution to the preconditioned system: B 1 Ax = B 1 b. What makes a matrix B a good preconditioner for A? 1 It is a very good approximation of the matrix 2 It reduces the number of iterations 3 B can be computed quickly 4 in B can be solved quickly

30 Preconditioned Iterative s Solver Preconditioned Richardson s Method([Young53],[GV61]) x (i+1) = b + (I A)x (i)

31 Preconditioned Iterative s Solver Preconditioned Richardson s Method([Young53],[GV61]) x (i+1) = B 1 b + (I B 1 A)x (i)

32 Preconditioned Iterative s Solver Preconditioned Richardson s Method([Young53],[GV61]) x (i+1) = B 1 b + (I B 1 A)x (i) The term B 1 b involves solving a system in B.

33 Convergence of Iterative s Solver Since the solution x is not known, different criteria must be used to determine the value N for which x (N) x < ɛ.

34 Convergence of Iterative s Solver Since the solution x is not known, different criteria must be used to determine the value N for which x (N) x < ɛ. - One criterion uses the residual vector, r (i) = b Ax (i) - The minimum iterations N should satisfy x x (N) x κ(a) r (N) b ɛκ(a), where κ(a) = A 1 A is the condition number of A for any matrix norm [QRS07].

35 Convergence of Iterative s Solver Since the solution x is not known, different criteria must be used to determine the value N for which x (N) x < ɛ. In the case of preconditioned methods, this becomes B 1 r (N) B 1 r (0) ɛ. In particular, this means the rate of convergence will depend on how quickly systems in B can be solved.

36 Convergence of Iterative s Solver Preconditioned Chebyshev method will find solutions with absolute error ɛ in time O(mS(B) log(κ(a)/ɛ) κ(a, B)), [GO88] where S(B) is the time required so solve a system in B and ( )( ) x T Ax x T Bx κ(a, B) = max x:ax 0 x T max Bx x:ax 0 x T. Ax

37 Convergence of Iterative s Solver Factors: minimum number of iterations N, depends on κ(a) time to solve the system in B In general, worst case time bounds are O(mn).

38 s Solver 1 2 Matrices Linear in the Graph 3 Overview 4 Solving Linear with Using Computing A of the Problem Solving the System 5

39 A First Nearly-Linear Time Solver s Solver Spielman and Teng [ST04] presented the first nearly-linear time algorithm for solving systems of equations in symmetric, diagonally-dominant (SDD) systems. The success of the ST-solver can be ascribed to two innovations:

40 A First Nearly-Linear Time Solver s Solver Spielman and Teng [ST04] presented the first nearly-linear time algorithm for solving systems of equations in symmetric, diagonally-dominant (SDD) systems. The success of the ST-solver can be ascribed to two innovations: 1 enhancing a one-level iterative solver using recursion

41 A First Nearly-Linear Time Solver s Solver Spielman and Teng [ST04] presented the first nearly-linear time algorithm for solving systems of equations in symmetric, diagonally-dominant (SDD) systems. The success of the ST-solver can be ascribed to two innovations: 1 enhancing a one-level iterative solver using recursion 2 using a preconditioning matrix based on a subgraph

42 s Solver The Recursive Solver The ST-solver addresses the problem of solving systems in the preconditioning matrix B with recursion: solve(a,b): if dimension-check(a) and sparse(a): return PrecChebyshev(A,b) B = precondition(a) A = reduce(b) solve(a,b) Compute B, the preconditioner for A Reduce the system in B to a system in A 1, a refined version of A Recursively solve the system in A 1 At the base, use the Preconditioned Chebyshev method

43 s Solver The Recursive Solver The ST-solver addresses the problem of solving systems in the preconditioning matrix B with recursion: A B A 1 B 1 A k Compute B, the preconditioner for A Reduce the system in B to a system in A 1, a refined version of A Recursively solve the system in A 1 At the base, use the Preconditioned Chebyshev method

44 The Recursive Solver s Solver Two methods are used at each level of the recursion. reduce(b): B i A i+1 - Reduce the preconditioned matrix by greedily removing rows and columns with at most two non-zero entries - Done with a partial Cholesky decomposition precondition(a): A i B i - Sparsify the graph associated to A to obtain a subgraph H - Set B to be a matrix of H

45 Graph Preconditioners s Solver [GMZ95]: There is a linear time transformation: Ax = b Lx = b SDD system system For the recursive sparsifying procedures, Spielman and Teng use the underlying graphs of the matrices to produce a chain of progressively sparser graphs.

46 Graph Preconditioners s Solver [GMZ95]: There is a linear time transformation: Ax = b Lx = b SDD system system For the recursive sparsifying procedures, Spielman and Teng use the underlying graphs of the matrices to produce a chain of progressively sparser graphs. A B A 1 B 1 A k

47 Graph Preconditioners s Solver [GMZ95]: There is a linear time transformation: Ax = b Lx = b SDD system system For the recursive sparsifying procedures, Spielman and Teng use the underlying graphs of the matrices to produce a chain of progressively sparser graphs. L A L B L A1 L B1 L Ak

48 Graph Preconditioners s Solver [GMZ95]: There is a linear time transformation: Ax = b Lx = b SDD system system For the recursive sparsifying procedures, Spielman and Teng use the underlying graphs of the matrices to produce a chain of progressively sparser graphs. G H G 1 H 1 G k

49 Spanning Trees s Solver [Vaidya91]: subgraphs serve as good preconditioners maximum weight spanning tree as a preconditioning base solves SDD systems with non-positive off-diagonal entries of degree d in time O(dn) 1.75 log(κ(a)/ɛ) [ST04]: subgraphs serve as good preconditioners a better base tree uses an edge measure called stretch

50 s Solver The Stretch of an Edge The detour forced by traversing T instead of G

51 s Solver The Stretch of an Edge The detour forced by traversing T instead of G e = {V 1, V 4}, w(e) = 1

52 s Solver The Stretch of an Edge The detour forced by traversing T instead of G A spanning tree, T

53 s Solver The Stretch of an Edge The detour forced by traversing T instead of G e = {V 1, V 4}, Tree path: {V 1, V 3}, {V 3, V 5}, {V 5, V 4}

54 The Stretch of an Edge s Solver Let T be a spanning tree of a weighted graph G, and let the weight of an edge e = {u, v} be denoted by w(e). Define w (e) = 1/w(e), which is the resistance of the edge, r e. Let e 1, e 2,..., e k be the unique path in T from u to v. Then the stretch of the edge by T is defined k i=1 stretch T (e) = w (e 1 ) k w = 1/r e (e) The total stretch of a graph G by the tree T is the sum of the stretch of all the off-tree edges. i=1 r ei

55 Low Stretch Spanning Trees s Solver Low stretch spanning tree (LSST) for the preconditioning base A spine that keeps resistance low

56 Low Stretch Spanning Trees s Solver Low stretch spanning tree (LSST) for the preconditioning base A spine that keeps resistance low LSSTs for preconditioners had been implemented before ([AKPW95, BH01, BH03]), but combining the high quality preconditioners with a recursive solver amounted to an algorithm for solving SDD systems in nearly-linear time.

57 s Solver [ST04]: Given a system of equations Ax = b in a symmetric, diagonally dominant matrix, the ST-solver computes a vector ˆx satisfying Aˆx b < ɛ and ˆx x ɛ in time O(m log O(1) m), where the exponent is a large constant.

58 s Solver of the ST-solver 1 Compute an LSST T of the graph G 2 Reweight the edges of T by a constant factor k, call the reweighted tree T 3 Replace T in G by T 4 H T 5 Add off-tree edges to H by sampling with probabilities related to vertex degree

59 s Solver of Koutis et al. [KMP10, KMP11] 1 Compute an LSST T of the graph G 2 Reweight the edges of T by a constant factor k, call the reweighted tree T 3 Replace T in G by T 4 H T 5 Add off-tree edges to H by sampling with probabilities related to effective resistance

60 s Solver pro: Probabilities related to effective resistance yield sparsifiers with few edges [SS11] con: Computing effective resistances involves solving a system of linear equations This is problem is avoided by instead using upper bounds for edge probabilities, p e w e R e, where R e is the effective resistance of the edge.

61 s Solver pro: Probabilities related to effective resistance yield sparsifiers with few edges [SS11] con: Computing effective resistances involves solving a system of linear equations This is problem is avoided by instead using upper bounds for edge probabilities, p e w e R e, where R e is the effective resistance of the edge. The sparsifier of [KMP11] improved the expected time bound to O(m log 2 n log(1/ɛ)) for an approximate solution with absolute error bounded by ɛ.

62 Summary of Discussed s Solver Gaussian elimination O(n 3 ) [CW90] O(n ) [Wil12] O(n ) Prec Chebyshev [GO88] O(mS(B) log(κ(a)/ɛ) κ(a, B)) [Vaidya91] O(dn) 1.75 log(κ(a)/ɛ) [ST04] O(m log O(1) m) [KMP11] O(m log 2 n log(1/ɛ))

63 s Solver 1 2 Matrices Linear in the Graph 3 Overview 4 Solving Linear with Using Computing A of the Problem Solving the System 5

64 The of a Subset s Solver Let S be a subset of vertices in a graph. S = {S1, S2, S3}

65 The of a Subset s Solver Let S be a subset of vertices in a graph. The vertex boundary of S is the set of vertices not in S which border S δ(s) = {v / S : {v, u} E for some u S}.

66 The of a Subset s Solver Let b be a vector over the vertices of a graph. Then a vector x over the satisfies the boundary condition of b for a subset S when x(v) = b(v) v δ(s). δ(s) = {v / S : {v, u} E for some u S}.

67 Leader-Following Formation s Solver a network with a leader, and a group of agents values x i correspond to position leader moves independently of agents Goal: design a distributed protocol for agents to follow the leader

68 Leader-Following Formation s Solver The leader is the boundary of the subset of agents. So our solution should: - Respect the leader s position (fix the solution on the boundary) - Use local information among the agents (compute the solution on the subset) Goal: design a distributed protocol for agents to follow the leader

69 s Solver [NC10]: leader. Leader-Following Formation Consider a multi-agent system of n agents and one 1 The dynamics of the leader is ẋ 0 = Ax 0, which is independent. 2 The dynamics of each agent is ẋ i = v j v i x j x i, and the vector x of agent positions is given by ẋ = Lx.

70 s Solver Satisfying the Condition Consider a linear system Lx = b. Suppose there exists a subset S such that the induced subgraph on S is connected the boundary δ(s) is non-empty support(b) δ(s) x satisfies the boundary condition b: x(v) = b(v), v δ(s). Goal: find the solution x restricted to S. That is, the solution will satisfy { 1 x(v) = d v u v x(u) if v S b(v) if v δ(s).

71 Solving Linear with s Solver Main tools and techniques: Fast computation of a heat kernel pagerank vector Expressing a linear system with boundary conditions in terms of the heat kernel of the graph Approximate the solution by a sum of heat kernel pagerank vectors

72 s Solver on a Graph Consider P = D 1 A as a random walk matrix /2 0 1/ /2 0 1/2 P = 1/4 1/4 0 1/4 1/ /3 1/3 1/3 0 0

73 s Solver on a Graph Consider P = D 1 A as a random walk matrix /2 0 1/ /2 0 1/2 P = 1/4 1/4 0 1/4 1/ /3 1/3 1/3 0 0

74 s Solver on a Graph Consider P = D 1 A as a random walk matrix /2 0 1/ /2 0 1/2 P = 1/4 1/4 0 1/4 1/ /3 1/3 1/3 0 0 When f is a probability distribution vector, f T P k is the distribution after k random walk steps. Define to be the Laplace operator, defined = I P.

75 s Solver The heat kernel pagerank vector is determined by parameters t R + and f R n, ρ t,f = f T e t = e t k=0 t k k! f T P k.

76 s Solver The heat kernel pagerank vector is determined by parameters t R + and f R n, ρ t,f = f T e t = e t k=0 t k k! f T P k. If f is a starting distribution over the vertices, then f T P k will be the distribution after k random walk steps

77 s Solver The heat kernel pagerank vector is determined by parameters t R + and f R n, ρ t,f = f T e t = e t k=0 t k k! f T P k. If f is a starting distribution over the vertices, then f T P k will be the distribution after k random walk steps The sum of the coefficients satisfies k=0 e t tk k! = e t e t = 1

78 s Solver The heat kernel pagerank vector is determined by parameters t R + and f R n, ρ t,f = f T e t = e t k=0 t k k! f T P k. If f is a starting distribution over the vertices, then f T P k will be the distribution after k random walk steps The sum of the coefficients satisfies k=0 t tk e k! = e t e t = 1 Then if k steps of a P random walk are taken with probability e t tk k!

79 s Solver The heat kernel pagerank vector is determined by parameters t R + and f R n, ρ t,f = f T e t = e t k=0 t k k! f T P k. If f is a starting distribution over the vertices, then f T P k will be the distribution after k random walk steps The sum of the coefficients satisfies k=0 t tk e k! = e t e t = 1 Then if k steps of a P random walk are taken with probability e t tk k! ρ t,f is the expected distribution of the process

80 s Solver Computing Q: A good way to compute the heat kernel pagerank? A: Draw enough samples of the random process to get the expected value.

81 s Solver Computing Q: A good way to compute the heat kernel pagerank? A: Draw enough samples of the random process to get the expected value. Avoid an exponential sum by taking samples.

82 s Solver Computing Q: A good way to compute the heat kernel pagerank? A: Draw enough samples of the random process to get the expected value. Avoid an exponential sum by taking samples. Control error by drawing enough samples r(ɛ)

83 s Solver Computing Q: A good way to compute the heat kernel pagerank? A: Draw enough samples of the random process to get the expected value. Avoid an exponential sum by taking samples. Control error by drawing enough samples r(ɛ) Avoid long walk processes by sampling truncated random walks.

84 s Solver Computing Q: A good way to compute the heat kernel pagerank? A: Draw enough samples of the random process to get the expected value. Avoid an exponential sum by taking samples. Control error by drawing enough samples r(ɛ) Avoid long walk processes by sampling truncated random walks. Control contribution lost in later step by stopping after enough steps K(ɛ)

85 s Solver Computing Q: A good way to compute the heat kernel pagerank? A: Draw enough samples of the random process to get the expected value. Let p k be the probability of taking k random walk steps, t tk p k = e k!. Let X be the distribution over values of k. Then obtain a good approximation by taking the average of r random walks of length at most K. The variables r, K are both in terms of a prescribed error parameter, ɛ.

86 s Solver Computing Let the output of the algorithm be ˆρ t,f. Then, to achieve we choose (1 ɛ)ρ t,f [v] ɛ ˆρ t,f [v] (1 + ɛ)ρ t,f [v], r 16 log s, ɛ3 K log(ɛ 1 ) log log(ɛ 1 ). Then, assuming that (1) taking a random walk step and (2) sampling from a distribution take constant time, the runtime of the heat kernel pagerank computation is r K = 16 ɛ 3 log s log(ɛ 1 ) log log(ɛ 1 ) = O ( log s log(ɛ 1 ) ɛ 3 log log ɛ 1 ).

87 Solving Linear with s Solver Main tools and techniques: Fast computation of a heat kernel pagerank vector Expressing a linear system with boundary conditions in terms of the heat kernel of the graph Approximate the solution by a sum of heat kernel pagerank vectors

88 Linear with s Solver Recall what it means for a solution vector x to satisfy the boundary condition in the system Lx = b. S = {v V : b[v] = 0}, { 1 x(v) = d v u v x(u) b(v) if v S if v δ(s).

89 Linear with s Solver Given a graph G and a vector b of boundary conditions, Let S be the subset of s = S vertices S = {v V : b[v] = 0}. Let A δs be the s δ(s) matrix A with columns restricted to the vertices of δ(s) and rows to vertices of S. Let L S, A S and D S be L, A and D with rows and columns restricted to vertices of S. Let x S be the solution vector over the vertices of S, and let b δs be b over δ(s).

90 Linear with s Solver Then we would like to find a vector x R s satisfying: or, equivalently: D S x S = A S x S + A δs b δs x S = (D S A S ) 1 (A δs b δs ) = L 1 S (A δsb δs ),

91 Linear with s Solver Then we would like to find a vector x R s satisfying: or, equivalently: D S x S = A S x S + A δs b δs x S = (D S A S ) 1 (A δs b δs ) = L 1 S (A δsb δs ), The inverse L 1 S exists when the induced subgraph on S is connected and the boundary of S is non-empty.

92 s Solver The Normalized The normalized is defined 1 if u = v, (L) uv = 1 dud if u v, v 0 otherwise. Degree-nomalized version of L, L = D 1/2 LD 1/2 Symmetric version of = I P, L = D 1/2 D 1/2

93 s Solver The Normalized When G has no isolated vertex and the matrix D is invertible, Lx = b and Lx = b the system in L can be deduced from the system in L by setting x = D 1/2 x and b = D 1/2 b. Then the solution x R s satisfying the boundary condition should satisfy x S = L 1 S (A δsb δs )

94 s Solver The Normalized When G has no isolated vertex and the matrix D is invertible, Lx = b and Lx = b the system in L can be deduced from the system in L by setting x = D 1/2 x and b = D 1/2 b. Then the solution x R s satisfying the boundary condition should satisfy x S = L 1 S (A δsb δs ) x S = L 1 S b 1. And computing b 1 takes time proportional to the sum of vertex degrees in δ(s) := vol(δ(s)).

95 Dirichlet s Solver The Dirichlet heat kernel is defined H S,t = e tl S = D 1/2 S e t S D 1/2 S.

96 s Solver The Dirichlet heat kernel is defined Lemma ([CS13]) Dirichlet H S,t = e tl S = D 1/2 S e t S D 1/2 S. Let S be a strict subset of vertices of a graph G and let the induced subgraph on S be connected. Then, L 1 S = 0 H S,t dt.

97 s Solver The Dirichlet heat kernel is defined Lemma ([CS13]) Dirichlet H S,t = e tl S = D 1/2 S e t S D 1/2 S. Let S be a strict subset of vertices of a graph G and let the induced subgraph on S be connected. Then, L 1 S = x S = L 1 S b 1 = 0 0 H S,t dt. H S,t b 1 dt.

98 Solving Linear with s Solver Main tools and techniques: Fast computation of a heat kernel pagerank vector Expressing a linear system with boundary conditions in terms of the heat kernel of the graph Approximate the solution by a sum of heat kernel pagerank vectors

99 Solving Linear with s Solver Approximate the solution 1 express the solution as an improper integral of heat kernel pagerank 2 approximate with a definite integral by limiting the range 3 approximate by taking a finite sum of heat kernel pagerank vectors 4 approximate each term by sampling truncated random walks

100 Approximating with s Solver Claim For Lx = b, ( ) x S = ρ t,f dt 0 D 1/2 S, f = b T 1 D 1/2 S.

101 s Solver Claim For Lx = b : Proof. x S = H S,t b 1 dt 0 x S = b1 T H S,t dt 0 x S = 0 x S = 0 Approximating with ( ) x S = 0 ρ t,f dt D 1/2 S, f = b1 T D1/2 S. from Lemma H S,t symmetric b T 1 (D 1/2 S e t S D 1/2 S ) dt definition of H S,t b1 T D 1/2 S e t S dt D 1/2 S

102 s Solver Claim For Lx = b : Proof. x S = H S,t b 1 dt 0 x S = b1 T H S,t dt 0 x S = 0 x S = 0 Approximating with ( ) x S = 0 ρ t,f dt D 1/2 S, f = b1 T D1/2 S. from Lemma H S,t symmetric b T 1 (D 1/2 S e t S D 1/2 S ) dt definition of H S,t b1 T D 1/2 S e t S dt D 1/2 S

103 s Solver Claim For Lx = b : Proof. x S = H S,t b 1 dt 0 x S = b1 T H S,t dt 0 x S = 0 x S = 0 Approximating with ( ) x S = 0 ρ t,f dt D 1/2 S, f = b1 T D1/2 S. from Lemma H S,t symmetric b T 1 (D 1/2 S e t S D 1/2 S ) dt definition of H S,t b1 T D 1/2 S e t S dt D 1/2 S

104 s Solver Claim For Lx = b : Proof. x S = H S,t b 1 dt 0 x S = b1 T H S,t dt 0 x S = 0 Approximating with ( ) x S = 0 ρ t,f dt D 1/2 S, f = b1 T D1/2 S. from Lemma H S,t symmetric b T 1 (D 1/2 S e t S D 1/2 S ) dt definition of H S,t x S = b1 T D 1/2 S e t S }{{} 0 dt D 1/2 S ρ t,f

105 Finding the Solution s Solver Let x 1 = x S D1/2 S. We can compute x 1 = by a number of approximations, 0 ρ t,f dt, f = b T 1 D 1/2 S. 1. Ignore the tail, take the integral to a finite T, x 1 T 0 ρ t,f dt

106 Finding the Solution s Solver Let x 1 = x S D1/2 S. We can compute x 1 = by a number of approximations, 0 ρ t,f dt, f = b T 1 D 1/2 S. 2. Discretize with a finite Riemann sum for small intervals T /N, x 1 N ρ jt /N,f T /N j=1

107 Finding the Solution s Solver Let x 1 = x S D1/2 S. We can compute x 1 = by a number of approximations, 0 ρ t,f dt, f = b T 1 D 1/2 S. 3. Obtain a good approximation by finitely many samples of ρ jt /N,f over values of t. for r times: draw j from the interval [1,N] compute ApproxHKPR(G, jt/n, f)

108 Solving the System with s Solver [CS13] show that to achieve an approximation vector ˆx satisfying x ˆx O(ɛ(1 + b )), the number of samples needed is r = ɛ 2 (log s + log(ɛ 1 )), where s = S.

109 Solving the System with s Solver [CS13] show that to achieve an approximation vector ˆx satisfying x ˆx O(ɛ(1 + b )), the number of samples needed is r = ɛ 2 (log s + log(ɛ 1 )), where s = S. Then using that the time to compute a heat kernel pagerank vector is ( log s log(ɛ 1 ) ) O ɛ 3 log log ɛ 1, and assuming additional preprocessing time proportional to vol(δs), the main result of [CS13] is the following.

110 Solving the System with s Solver Theorem ( Solver [CS13]) For a graph G and a linear system Lx = b, assume: x is required to satisfy the boundary condition b S = V \ support(b) and s = S the boundary of S is nonempty the induced subgraph on S is connected.

111 Solving the System with s Solver Theorem ( Solver [CS13]) Then the approximate solution ˆx output by the boundary solver satisfies the following with probability 1 ɛ. 1 x ˆx O(ɛ(1 + b )) 2 The running time is ( (log s) 2 (log(ɛ 1 )) 2 ) O ɛ 5 log log(ɛ 1 ) with additional preprocessing time O(vol(δ(S))).

112 s Solver 1 2 Matrices Linear in the Graph 3 Overview 4 Solving Linear with Using Computing A of the Problem Solving the System 5

113 s Solver Capitalize on speed. Use fast linear solvers to improve existing graph algorithms Interior point algorithms [SD08] Learning problems [ZHS05] Graph partitioning [ST04],[ACL06] Sparsification [ST04],[SS11] Find more applications for these linear solvers Find more applications for heat kernel pagerank Expected distribution of random walks Vertex similarity graph distance

114 s Solver References Noga Alon, Richard M. Karp, David Peleg, and Douglas West (1995) A Graph-Theoretic Game and its Applications to the k-sever Problem. SIAM J. Computing, 21(1), Reid Andersen, Fan Chung, and Kevin Lang (2006) Local Graph Partitioning Using PageRank Vectors. FOCS 06, Erik Boman and Bruce Hendrickson (2001) On Spanning Tree Preconditioners. Sandia National Laboratories Manuscript. Erik Boman and Bruce Hendrickson (2003) Support Theory for Preconditioning. SIAM J. Matrix Analysis and Applications, 25(3), Sergey Brin and Lawrence Page (1998) The Anatomy of a Large-Scale Hypertextual Web Search Engine. Computer Networks and ISDN, 30(1),

115 s Solver References Fan Chung and (2013) Solving Linear with using. WAW 13. Don Coppersmith and Shmuel Winograd (1990) Matrix Multiplication via Arithmetic Progressions. J. Symbolic Computation, 9(3), Samuel I. Daitch and Daniel A. Spielman (2008) Faster Approximate Lossy Generalized Flow Via Interior Point Algorithms. STOC 08, Gene H. Golub and Michael L. Overton (1988) The Convergence of Inexact Chebyshev and Richardson Iterative for Solving Linear. Numerische Mathematik, 53(5),

116 s Solver References Gene H. Golub and Richard S. Varga (1961) Chebyshev Semi-Iterative, Successive Overrelaxation Iterative, and Second Order Richardson Iterative. Numerical Math. 3, Keith D. Gremban, Gary L. Miller, and Marco Zagha (1995) Performance Evaluation of a New Parallel Preconditioner. International Parallel Processing Symposium 95, Aric Hagberg and Daniel A. Schult (2008) Rewiring Networks for Synchronization. Chaos: An Interdisciplinary Journal of Nonlinear Science, 18(3). Gustav Kirchhoff (1847) Über die Auflösung der Gleichungen, auf welche man bei der Untersuchung der linearen Vertheilung galvanischer Ströme geführt wird. Annalen der Physik, 148(12),

117 s Solver References Ioannis Koutis, Gary L. Miller, and Richard Peng (2010) Approaching Optimality for Solving SDD Linear. FOCS 10, Ioannis Koutis, Gary L. Miller, and Richard Peng (2011) A Nearly-m log n Time Solver for SDD Linear. FOCS 11, Ioannis Koutis, Gary L. Miller, and Richard Peng (2012) A Fast Solver for a Class of Linear. Communications of the ACM, 55(10), Wei Ni and Daizhan Cheng (2010) Leader-Following Consensus of Multi-Agent Under Fixed and Switching Topologies. & Control Letters, 59(3), Reza Olfati-Saber and Richard Murray (2003) Consensus Protocols for Networks of Dynamic Agents. American Control Conference 03,

118 s Solver Daniel A. Spielman and Nikhil Srivastava (2011) Graph Sparsification by Effective Resistances. SIAM J. Computing, 40(6), References Daniel A. Spielman and Shang-Hua Teng (2004) Nearly-Linear Time Algorithms for Graph Partitioning, Graph Sparsification, and Solving Linear. STOC 04, Pravin M. Vaidya (1991) Solving Linear Equations with Symmetric Diagonally Dominant Matrices by Constructing Good Preconditioners. UIUC manuscript based on a talk. Alfio Quarteroni, Riccardo Sacco, and Fausto Saleri (2007) Numerical Mathematics, vol 37. Virginia Vassilevska Williams (2012) Multiplying Matrices Faster than Coppersmith-Winograd. STOC 12,

119 References s Solver David Young (1950) On Richardson s Method for Solving Linear with Positive Definite Matrices. J. Math. and Physics 32, Denyong Zhou, Jiayuan Huang, and Bernhard Sch olkopf (2005) Learning From Labeled and Unlabeled Data on a Directed Graph International Conference on Machine Learning

120 s Solver Thank You