Methods for estimating the diagonal of matrix. functions. functions. Jesse Laeuchli Andreas Stathopoulos CSC 2016

Transcription

1 Jesse Laeuchli Andreas Stathopoulos CSC / 24

2 The Problem Given a large A, and a function f, find diag(f(a)),or trace(f(a)) = n i=1 f(a) ii = n i=1 f(λ i) Some important examples of f f (A) = A 1 f (A) = A k f (A) = exp(a) f (A) = log(a) 2 / 24

3 Applications Compute Diagonals of A k /exp(a) Count Triangles/Polygons Higher distance paths, higher powers of A Network Centrality 3 / 24

4 Applications Compute Diagonals of A y =.3*x *x + 5 Observation LSQ Statistics 4 / 24

5 Applications Compute Diagonals of A 1 Uncertainty Quantification 5 / 24

6 Exact Methods What if we compute it exactly? LU Decomposition Eigendecomposition Recursive Factorizations Takahashi s Equations All too slow for large matrices 6 / 24

7 Statistical Trace Estimator Exact methods infeasible, so we resort to statistical approaches i=m trace(f(a)) = E[z T i= f(a)z] zt i f(a)z i m m m diag(f(a)) = E[z f(a)z] ( z i f(a)z i ) ( z i z i ) [ ] a b [ ] c = d [ a c b d ], i=1 [ a b ] [ ] c = d [ a/c b/d i=1 ] 7 / 24

8 How to pick z? Random Random Methods Gaussian{z = from Gaussian Distribution } Hutchinson{z = 1, 1 probability 1/2 } Canoncial basis e i with random i Mixing of diagonals DFT n e i,hadamard n e i with random i Estimator Gaussian Hutchinson s Unit Vector Variance of the Sample 2 A 2 F 2( A 2 F n i=1 A2 ii ) n n i=1 A2 ii trace2 (A) 8 / 24

9 How to pick z? Deterministic Deterministic Hadamard n e i with i = 1 : n H 1 = [ 1 ] H 2 = nz = 124 [ nz = 512 ] [ ] H2 k 1 H H 2 k = 2 k 1 = H H 2 k 1 H 2 H 2 k 1 2 k 1 9 / 24

10 Probing Color the associated graph of A, reorder nodes with the same color to be adjacent Can then recover the diagonal { of an m-colorable with exactly m vectors xi m 1, if i C m =, otherwise. If we color graph of f (A), then we can recover diag(f(a)) / 24

11 Probing f (A) Computing f (A) is hard. If we had it, we would be done f (A) is dense. It takes too many colors to probe Look at structure of polynomial p k (A) approximating f (A) 11 / 24

12 Problems With Probing-What We Address Even lower order A k expensive to compute and store 1 x 16 Powers of L 8 NNZ Powers of L 12 / 24

13 Problems With Probing-What We Address Probing vectors for A k not guaranteed to be a subset of vectors for A k+1 13 / 24

14 Hierarchical Coloring What is Hierarchical Coloring? Colors must be nested Colors split into the same number of new colors Ensures reusability of probing vectors 14 / 24

15 Hierarchical Coloring via Multilevel How to color a arbitrary graph? Use multi-level strategy. Problems if merging is done naively Same color neighbor at distance 8 Two green nodes still at distance 2 after 3 levels 15 / 24

16 Hierarchical Coloring via Multilevel Our strategy, merge distance 1 nodes, and distance 2 neighbourhoods N 2 N 1 N 1 N 2 16 / 24

17 Hierarchical Coloring via Multilevel Example Graph. One level of merging 17 / 24

18 Ensure Hierarchical Coloring Create Mixed-Radix Coordinate Interpret as color at each level Ensures colorings are hierarchical 18 / 24

19 Statistical Considerations How well should we expect to do with HP? Pure statistical is O( 1 s ) Hierarchical Probing depends on structure, g(x) describes fall-off of elements g(x) = c O( 1 s ) g(x) = 1 x O( 1 s ) If there is no structure Hierarchical Probing does as well as the statistical methods 19 / 24

20 Different Coloring Methods Top eigenvectors can be used to divide graph into bipartite groups in multiple ways Eigenvector Coloring From Probing 2 / 24

21 Covariance Matrix Variance Merge/Greedy Color Statistical Probing Colors 21 / 24

22 Uncertainty Quantification 1 2 Merge/Greedy Color Statistical Probing Variance Colors / 24

23 Wiki-Vote Merge/Greedy Color Merge/Spectral Color Statistical Probing Variance Colors / 24

24 Conclusion Exploits structure of Statistical bounds More efficient than classical probing 24 / 24