Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank

Similar documents
A Note on Google s PageRank

ORIE 6334 Spectral Graph Theory September 8, Lecture 6. In order to do the first proof, we need to use the following fact.

Topics in Graph Theory

Math 413/513 Chapter 6 (from Friedberg, Insel, & Spence)

1 Counting spanning trees: A determinantal formula

Application. Stochastic Matrices and PageRank

DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

Class President: A Network Approach to Popularity. Due July 18, 2014

Laplacian eigenvalues and optimality: II. The Laplacian of a graph. R. A. Bailey and Peter Cameron

Spectral Graph Theory and You: Matrix Tree Theorem and Centrality Metrics

Ma/CS 6b Class 23: Eigenvalues in Regular Graphs

Kernels of Directed Graph Laplacians. J. S. Caughman and J.J.P. Veerman

0.1 Naive formulation of PageRank

Introduction to Search Engine Technology Introduction to Link Structure Analysis. Ronny Lempel Yahoo Labs, Haifa

Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Recitation 8: Graphs and Adjacency Matrices

The Matrix-Tree Theorem

EIGENVALUES AND EIGENVECTORS 3

SVD, Power method, and Planted Graph problems (+ eigenvalues of random matrices)

Lecture 1 and 2: Random Spanning Trees

Eigenvalues and Eigenvectors

This section is an introduction to the basic themes of the course.

4.3 - Linear Combinations and Independence of Vectors

Lecture 14: Random Walks, Local Graph Clustering, Linear Programming

Fiedler s Theorems on Nodal Domains

Root systems and optimal block designs

CS 246 Review of Linear Algebra 01/17/19

An Introduction to Spectral Graph Theory

Krylov Subspace Methods to Calculate PageRank

Using Laplacian Eigenvalues and Eigenvectors in the Analysis of Frequency Assignment Problems

Linear algebra and applications to graphs Part 1

Linear Algebra Review

Laplacian Integral Graphs with Maximum Degree 3

Dot Products, Transposes, and Orthogonal Projections

Linear Algebra. James Je Heon Kim

Ir O D = D = ( ) Section 2.6 Example 1. (Bottom of page 119) dim(v ) = dim(l(v, W )) = dim(v ) dim(f ) = dim(v )

1 Last time: determinants

Math 314H Solutions to Homework # 3

5 Flows and cuts in digraphs

A lower bound for the Laplacian eigenvalues of a graph proof of a conjecture by Guo

HOMEWORK #2 - MATH 3260

1 Matrix notation and preliminaries from spectral graph theory

A Brief Outline of Math 355

Lab 8: Measuring Graph Centrality - PageRank. Monday, November 5 CompSci 531, Fall 2018

Graph fundamentals. Matrices associated with a graph

MATH 315 Linear Algebra Homework #1 Assigned: August 20, 2018

(a) If A is a 3 by 4 matrix, what does this tell us about its nullspace? Solution: dim N(A) 1, since rank(a) 3. Ax =

1.10 Matrix Representation of Graphs

Solution Set 7, Fall '12

Lecture 11: Eigenvalues and Eigenvectors

PageRank: The Math-y Version (Or, What To Do When You Can t Tear Up Little Pieces of Paper)

Lecture 10. Lecturer: Aleksander Mądry Scribes: Mani Bastani Parizi and Christos Kalaitzis

Fiedler s Theorems on Nodal Domains

Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space

Algebraic Representation of Networks

Lecture: Modeling graphs with electrical networks

Walks, Springs, and Resistor Networks

2 Two-Point Boundary Value Problems

Problem Set 1. Homeworks will graded based on content and clarity. Please show your work clearly for full credit.

Linear Algebra 2 Spectral Notes

1 9/5 Matrices, vectors, and their applications

Ma/CS 6b Class 20: Spectral Graph Theory

Course : Algebraic Combinatorics

Conditioning of the Entries in the Stationary Vector of a Google-Type Matrix. Steve Kirkland University of Regina

Conceptual Questions for Review

Google PageRank. Francesco Ricci Faculty of Computer Science Free University of Bozen-Bolzano

CSI 445/660 Part 6 (Centrality Measures for Networks) 6 1 / 68

MATH 583A REVIEW SESSION #1

2: LINEAR TRANSFORMATIONS AND MATRICES

MAT2342 : Introduction to Applied Linear Algebra Mike Newman, fall Projections. introduction

Spectral radius, symmetric and positive matrices

IMPORTANT DEFINITIONS AND THEOREMS REFERENCE SHEET

Random Walks on Graphs. One Concrete Example of a random walk Motivation applications

1 T 1 = where 1 is the all-ones vector. For the upper bound, let v 1 be the eigenvector corresponding. u:(u,v) E v 1(u)

1 Matrix notation and preliminaries from spectral graph theory

REVIEW FOR EXAM III SIMILARITY AND DIAGONALIZATION

Chapter 6: Orthogonality

Math 291-2: Lecture Notes Northwestern University, Winter 2016

Calculating Web Page Authority Using the PageRank Algorithm

18.312: Algebraic Combinatorics Lionel Levine. Lecture 19

Spectral Graph Theory Lecture 2. The Laplacian. Daniel A. Spielman September 4, x T M x. ψ i = arg min

2.1 Laplacian Variants

UNDERSTANDING THE DIAGONALIZATION PROBLEM. Roy Skjelnes. 1.- Linear Maps 1.1. Linear maps. A map T : R n R m is a linear map if

Markov Chains Handout for Stat 110

Math 304 Handout: Linear algebra, graphs, and networks.

Central Groupoids, Central Digraphs, and Zero-One Matrices A Satisfying A 2 = J

CS168: The Modern Algorithmic Toolbox Lectures #11 and #12: Spectral Graph Theory

Refined Inertia of Matrix Patterns

642:550, Summer 2004, Supplement 6 The Perron-Frobenius Theorem. Summer 2004

eigenvalues, markov matrices, and the power method

8.1 Concentration inequality for Gaussian random matrix (cont d)

Math 113 Practice Final Solutions

A Glimpse of Quantum Computation

In particular, if A is a square matrix and λ is one of its eigenvalues, then we can find a non-zero column vector X with

Math 307 Learning Goals. March 23, 2010

Homework For each of the following matrices, find the minimal polynomial and determine whether the matrix is diagonalizable.

Math 115A: Homework 5

Graph Models The PageRank Algorithm

Choose three of: Choose three of: Choose three of:

Some notes on Coxeter groups

Transcription:

Math 443/543 Graph Theory Notes 5: Graphs as matrices, spectral graph theory, and PageRank David Glickenstein November 3, 4 Representing graphs as matrices It will sometimes be useful to represent graphs as matrices. This section is taken from C-.. Let G be a graph of order p: We denote the vertices by v ; : : : ; v p : We can then nd an adjacency matrix A = A (G) = [a ij ] de ned to be the p p matrix such that a ij = if v i v j E (G) : This matrix will be symmetric for an undirected graph. We can easily consider the generalization to directed graphs and multigraphs. Note that two isomorphic graphs may have di erent adjacency matrices. However, they are related by permutation matrices. De nition A permutation matrix is a matrix gotten from the identity by permuting the columns (i.e., switching some of the columns). Proposition The graphs G and G are isomorphic if and only if their adjacency matrices are related by for some permutation matrix P. A = P T A P Proof (sketch). Given isomorphic graphs, the isomorphism gives a permutation of the vertices, which leads to a permutation matrix. Similarly, the permutation matrix gives an isomorphism. Now we see that the adjacency matrix can be used to count uv-walks. Theorem 3 Let A be the adjacency matrix of a graph G, where V (G) = fv ; v ; : : : ; v p g : Then the (i; j) entry of A n ; where n ; is the number of di erent v i v j -walks of length n in G:

Proof. We induct on n: Certainly this is true for n = : Now suppose A n = gives the number of v i v j -walks of length n: We can consider the entries a (n) ij of A n+ = A n A: We have a (n+) ij = px k= a (n) ik a kj: This is the sum of all walks of length n between v i and v k followed by a walk from v k to v j of length : All walks of length n + are generated in this way, and so the theorem is proven. Spectral graph theory Recall that an eigenvalue of a matrix M is a number such that there is a vector v (called the corresponding eigenvector) such that Mv = v: It turns out that symmetric n n matrices have n eigenvalues. Since adjacency matrices of two isomorphic graphs are related by permutation matrices as above, and so the set of eigenvalues of A is an invariant of a graph. We will actually use the Laplacian matrix instead of the adjacency matrix. The Laplacian matrix is de ned to be L = A D where D is the diagonal matrix whose entries are the degrees of the vertices (called the degree matrix). The Laplacian matrix is also symmetric, and thus it has a complete set of eigenvalues. The set of these eigenvalues is called the spectrum of the Laplacian. Notice the following. Proposition 4 Let G be a nite graph. The eigenvalues of the matrix L are all nonpositive. Moreover, the constant vector ~ = (; ; ; : : : ; ) is an eigenvector with eigenvalue zero. Proof. It is clear that ~ is an eigenvector with eigenvalue since the sum of

the entries in each row must be zero. Now, notice that we can write v T Lv = X v i (Lv) i = X X v i L ij v j i j = X v i (v j v i ) v iv je = 4 X = v iv je X v iv je v i (v j v i ) + X (v i v j ) : v iv je 3 v j (v i v j ) 5 (The sums over i are over all vertices, but the sums over v i v j E is the sum over the edges.) Now note that if v is an eigenvector of L with eigenvalue ; then Lv = v; and v T Lv = v T v = X vi : i Thus we have that = Pv (v iv je i v j ) P : i v i De nition 5 The eigenvalues of L can be arranged = p ; where p is the order of the graph. The collection ( ; ; : : : ; p ) is called the spectrum of the Laplacian. Remark 6 Sometimes the Laplacian is taken to be D = LD = : If there are no isolated vertices, these are essentially equivalent. Remark 7 Note that the Laplacian matrix, much like the adjacency matrix, depends on the ordering of the vertices and must be considered up to conjugation by permutation matrices. Since eigenvalues are independent of conjugation by permutation matrices, the spectrum is an isomorphism invariant of a graph. The following is an easy fact about the spectrum: Proposition 8 For a graph G of order p; Xp i = q: i= Proof. The sum of the eigenvalues is equal to the trace, which is the sum of the degrees. We will be able to use the eigenvalues to determine some geometric properties of a graph. 3

3 Connectivity and spanning trees Recall that = ; which means that the matrix L is singular and its determinant is zero. Recall the de nition of the adjugate of a matrix. h i De nition 9 If M is a matrix, the adjugate is the matrix M y = M y ij where M y ij is equal to ( )i+j det ^Mij ; where ^M ij is the matrix with the ith row and jth column removed. The adjugate has the property that M M y T = (det M) I; where I is the identity matrix. Applying this to L (which is symmetric) gives that LL y = : Now, the p p matrix L has rank less than p: If it is less than or equal to p ; then all determinants of (p ) (p ) submatrices are zero, and hence L y = : If L has rank p ; then it has only one zero eigenvalue, which must be (; ; : : : ; ) T : Since LL y = ; all columns of L y must be a multiple of (; ; : : : ; ) T : But L is symmetric, so that means that L y must be a multiple of the matrix of all ones. This motivates the following de nition. De nition We de ne t (G) by t (G) = ( ) i+j det ^Lij for any i and j (it does not matter since all are the same). Remark It follows that t (G) is an integer. Proposition t (G) = p p : Proof. In general for a matrix A with eigenvalues ; : : : ; p we have that Xp k= p Xp = k i= det ^A ii : In our case, = and the right sum is the sum of p of the same entries, so the result follows. Remark 3 It follows that t (G) : Recall that a spanning tree of G is a subgraph containing all of the vertices of G and is a tree. 4

Theorem 4 (Matrix Tree Theorem) The number t (G) is equal to the number of spanning trees of G: Proof. Omitted, for now. We can apply this, however, as follows. Example, consider the graph K 3 : Clearly this has Laplacian matrix L (G) = @ The number of spanning trees are equal to det = 3: It is clear that each spanning tree is given by omitting one edge, so it is clear there are 3. Example : Consider the following graph. A : Its Laplacian matrix is L (G) = B @ 4 3 C A : 5

The number of spanning trees are equal to t (G) = det B 3 C @ A = det 3 A @ = (6 ) + ( ) = 8 One can check directly that it has eight spanning trees. Corollary 5 6= if and only if G is connected. Proof. = if and only if t (G) = since t (G) is the product of the eigenvalues p and is the minimal eigenvalue after : But t (G) = means that there are no spanning trees, so G is not connected. Now we can consider the di erent components. De nition 6 The disjoint union of two graphs G = G t G is the graph gotten by taking V (G) = V (G ) t V (G ) and E (G) = E (G ) t E (G ) where t is the disjoint union of sets. It is not hard to see that if we number the vertices in G by rst numbering the vertices of G and then numbering the vertices of G ; that the Laplacian matrix takes the form L (G) = L (G ) L (G ) This means that the eigenvalues of L (G) are the union of the eigenvalues of L (G ) and L (G ) : This implies the following. Corollary 7 If n = ; then there are at least n + connected components of G: Proof. Induct on n: We already know this is true for n = : Suppose n = : We know there must be at least n components, since n = implies n = : We can then write the matrix L (G) in the block diagonal form with L (G i ) along the diagonal for some graphs G i : Since n = ; one of these graphs must have (G i ) = : But that means that there is another connected component, completing the induction. 4 PageRank problem and idea of solution We will generally follow the paper by Bryan and Leise, denoted BL. Search engines generally do three things: : 6

. Locate all webpages on the web.. Index the data so that it can be searched e ciently for relevent words. 3. Rate the importance of each page so that the most important pages can be shown to the user rst. We will discuss this third step. We will assign a nonnegative score to each webpage such that more important pages have higher scores. The rst idea is: Derive the score for a page by the number of links to that page from other pages (called the backlinks for the page). In this sense, other pages vote for the page. The linking of pages produces a digraph. Denote the vertices by v k and the score of vertex v k by x k : Approach : Let x k equal the number of backlinks for page v k : See example in BL Figure reproduced here: We see that x = ; x = ; x 3 = 3; and x 4 = : Here are two problems with this ranking: Problem : Links from more important pages should increase the score more. For instance, the scores of v and v 4 are the same, but v has a link from x 3 ; which is a more important page, so maybe it should be ranked higher. We will deal with this by, instead of letting x i equal the total number of links to it, we will have it be equal to the sum of the scores of the pages linking to it, so more important pages count more. Thus we get the relations x = x 3 + x 4 x = x x 3 = x + x + x 4 x 4 = x + x : 7

This doesn t quite work as stated, since to solve this linear system, we see that we get x = x = x 4 = 4 x 3; which means that if we look at the rst equality, we must have that they are all equal to zero. However, a slight modi cation in regard to the next problem will x this. Problem : One site should not be able to signi cantly a ect the rankings by creating lots of links. Of course, creating links should a ect the rankings, but by creating thousands of links from one site, one should not be able to boost the importance too much. So instead of giving one vote for each link out, we will give equal votes to each outlink from a particular page, but the total votes is equal to one. This changes the above system to x = x 3 + x 4 x = 3 x x 3 = 3 x + x + x 4 x 4 = 3 x + x : This can be solved as follows. x = 3 x x 4 = 3 x + x = x x 3 = 3 x + x + x 4 = 3 x + 6 x + 4 x = 3 4 x : Thus we can have a score of x = ; x = 3 ; x 3 = 3 4 ; x 4 = : Notice that x has the highest ranking! This is because x 3 threw its whole vote to x and so that even though x 3 got votes from three di erent sites, they still do not total as much as what x gets. Note, usually we will rescale so that the sum is equal to ; and so we get 5 General formulation x = 3 ; x = 4 3 ; x 3 = 9 3 ; x 4 = 6 3 : First, we introduce some terminology for directed graphs. De nition 8 If e = (v; w) E + is a directed edge (we use E + to denote the edges in a directed graph, and the ordered pair to denote that the edge is from v to w; denoted in a picture as an arrow from v to w), we say v is adjacent to (or links to) w and w is adjacent from (or links from) v: De nition 9 The indegree of a vertex v; denoted i deg (v) ; is the number of vertices adjacent to v: The outdegree of v; denoted o deg (v), is the number of vertices adjacent from v: 8

We can state the Page Rank algorithm in a more general way. We want to assign scores so that x i = X x j n j jl i where L i are the indices such that v j links to v i if j L i ; and n j is equal to outdegree of v j : Note that L i contains i deg (v i ) elements. The set L i is called the set of backlinks of vertex v i : This can be rewritten as a vector equation x = Ax; where A is the matrix A = (a ij ) given by n a ij = j if j L i otherwise : This matrix is called the link matrix. We note that in the example, the matrix A was 3 3 A = 6 4 3 3 The problem of solving for the scores x then amounts to nding an eigenvector with eigenvalue for the matrix A: We can consider the link matrix as giving the probabilities of traversing a link from the page represented by the column to the page representing the row. Thus it makes sense that the sum of the values of the columns are equal to one. De nition A matrix is called a column stochastic matrix if all of its entries are positive and the sum of the elements in each column are equal to : Now the question is whether we can nd an eigenvector for a column stochastic matrix, and the answer is yes. Proposition If A is a column stochastic matrix, then is an eigenvalue. Proof. Let e be the column vector of all ones. Since A is column stochastic, we clearly have that e T A = e T : Thus A T e = e and e is an eigenvector with eigenvalue for A T : However, A and A T have the same eigenvalues (not eigenvectors, though), so A must have an eigenvalue, too. Remark Do you remember why A and A T have the same eigenvalues? The eigenvalues of A are the solutions of det (A I) = det A T I : 7 5 : 9

6 Challenges to the algorithm See the worksheet. 7 Computing the ranking The basic idea is that we can try to compute an eigenvector iteratively like x k+ = Mx k = M k x : Certainly, if Mx = x ; then this procedure xes x : In general, if we replace this method with x k+ = Mx k kmx k k for any vector norm, we will generally nd an eigenvector for the largest eigenvalue. Before we proceed, let s de ne the one-norm of a vector. De nition 3 The one-norm of a vector v = (v i ) R n is equal to kvk = jv i j ; i= where jv i j is the absolute value of the ith component of v: Proposition 4 Let M be a positive column-stochastic n n matrix and let V denote the subspace of R n consisting of vectors v such that v i = : i= Then for any v V we have Mv V and where c < : kmvk c kvk ; Corollary 5 In the situation in the proposition, M k v c k kvk : Proof. This is a simple induction on k; using the fact that Mv V and M k v c M k v : This is essentially showing that the iteration is a contraction mapping, and that will allow us to show that the method works.

Proposition 6 Every positive column-stochastic matrix M has a unique vector q with positive components such that Mq = q and kqk = : The vector can be computed as q = lim k! M k x for any initial guess x with positive componets such that kx k = : Proof. We already know that M has as an eigenvalue and that the subspace V (M) is one-dimensional. All eigenvectors have all positive or all negative components, so we can choose a unique representative q with positive components and norm by rescaling. Now let x be any vector in R n with positive components and kx k = : We can write x = q + v for some vector v: We note that if we sum the components of x or the components of q; we get one since both have positive components and -norm equal to one. Thus v V as in the previous proposition. Now we see that M k x = M k q + M k v = q + M k v: Thus M k x q = M k v c k kvk : Since c < ; we get that M k x q! as k! : We now go back and prove Proposition 4. Proof of Proposition 4. It is pretty clear that Mv V since X (Mv)j = X X M ji v i j = X i = X i i X M ji v i j v i = since M is column-stochastic. Now we consider kmvk = X X M ji v i j i = X X e j M ji v i j i = X i a i v i where e j = sgn X i M ji v i!

and a i = X j e j M ji : Note that if ja i j c for all i; then We can see that kmvk c kvk : X ja i j = e j M ji j X = M ji + X (e j ) M ji j j = + X (e j ) M ji j : Each term in the sum is nonpositive, and since M ji are positive and e j are not all the same sign, the largest this can be is if most e j are except for a single e j which is negative and corresponds to the smallest M ji : Thus we see that ja i j min j M ji min i;j M ji < : 8 Appendix: Approximating the Laplacian on a lattice Recall that the Laplacian is the operator 4 = @ @x + @ @y + @ @z acting on functions f (x; y; z) ; with analogues in other dimension. Let s rst consider a way to approximate the one-dimensional Laplacian. Suppose f (x) is a function and I want to approximate the second derivative d f dx (x) : We can take a centered di erence approximation to the get this as d f dx (x) f x x + x x f (x + x) x f x x f (x) f (x) f (x x) x = [f (x + x) f (x) + (f (x x) f (x))] (x) = [f (x + x) + f (x x) f (x)] (x)

Note that if we take x = ; then this only depends on the value of the function at the integer points. Now consider the graph consisting of vertices on the integers of the real line and edges between consectutive integers. Give an function f on the vertices, we can compute the Laplacian as 4f (v i ) = f (v i+ ) + f (v i ) f (v i ) for any vertex v i : Notice that the Laplacian is an in nite matrix of the form f (v ) f (v ) 4f = f (v ) : B C B f (v ) C @ A @ f (v ) A Also note that that matrix is exactly equal to the adjacency matrix minus twice the identity. The number is the degree of each vertex, so we can write the matrix, which is called the Laplacian matrix, as L = A where A is the adjacency matrix and D is the diagonal matrix consisting of degrees (called the degree matrix). Note that something similar can be done for a two-dimensional grid. We see that @ @x + @ @y f (x; y) f x x + x; y f x x x; y x D + f y x; y + y f y x; y y y f (x + x; y) f (x; y) [f (x; y) f (x x; y)] x x + f (x; y + y) f (x; y) [f (x; y) f (x; y y)] y y = [f (x + x; y) (x) f (x; y) + (f (x x; y) f (x; y))] + [f (x; y + y) (y) f (x; y) + (f (x; y y) f (x; y))] = [f (x + x; y) + (f (x (x) x; y) f (x; y))] + [f (x; y + y) + (f (x; y (y) y) f (x; y))] : If we let x = y = ; then we get @ @x + @ @y f (x; y) f (x + ; y)+f (x ; y)+f (x; y + )+f (x; y ) 4f (x; y) : 3

Note that on the integer grid, this translates to the sum of the value of f for the four vertices neighboring the vertex, minus 4 times the value at the vertex. This is precisely the same as the last time, and we see that this operator can again be written as L = A D: In general we will call this matrix the Laplacian matrix. It can be thought of as a linear operator on functions on the vertices. Sometimes the Laplacian will denote the negative of this operator (which gives positive eigenvalues instead of negative ones), and sometimes a slight variation is used in the graph theory literature. 9 Appendix: Electrical networks One nds applications of the Laplacian in the theory of electrical networks. Recall that the current through a circuit is proportional to the change in voltage, and that constant of proportionality is called the conductance (or resistance, depending on where the constant is placed. Thus, for a wire with conductance C between points v and w with voltages f (v) and f (w) respectively, the current from v to w is C (f (v) f (w)) : Kircho s law says that if we have a network of wires, each with conductance C; the total current through any given point is zero. Thus, we get that X C (f (w) f (v)) = v adjacent to w which is the same as 4f = : Note that if the conductances are di erent, then we get an equation like X c vw (f (w) f (v)) = ; v adjacent to w which is quite similar to the Laplacian. Hence we can use the Laplacian to understand graphs by attaching voltages to some of the vertices and seeing what happens at the other vertices. This is very much like solving a boundary value problem for a partial di erential equation! Appendix: Proof of the matrix tree theorem In order to prove the Matrix Tree Theorem, we need another characterization of the Laplacian. De nition 7 Let G be a directed (p; q)-graph. The oriented vertex-edge incidence graph is a p q matrix Q = [q ir ] ; such that q ir = if e r = (v i ; v j ) for some j and q ir = if e r = (v j ; v i ) for some j: 4

Proposition 8 The Laplacian matrix L satis es L = QQ T for any oriented vertex-edge incidence graph (so, given an undirected graph, we can take any orientation), i.e., L ij = qx q ir q jr : r= Proof. If i = j; then we see that L ii = deg v i : Notice that q ir is nonzero if v i is in edge e r : Thus qx (q ir ) = deg v i : r= Now for i 6= j; we have that q ir q jr = if e r = v i v j ; giving the result. Remark 9 This observation can be used to give a di erent proof that L ij has all nonpositive eigenvalues. The proof of the matrix tree theorem proceeds as follows. We need only consider the case det ^L : A property of determinants allows us to use the fact that L = QQ T to compute this determinant in terms of determinants of submatrices of Q: Proposition 3 (Binet-Cauchy Formula) Let A be an nm matrix and B be an n m matrix (usually we think n < m): We can compute det (AB) = X S det A S det B T S where A S is matrix consisting of the columns of A speci ed by S; and S ranges over all choices of n columns. We will not prove this theorem, but it is quite geometric in the case that B = A T ; which is our case. If we consider the rows of A; then det AA T is equal to the square of the volume of the parallelopiped determines by these rows (since we can rotate A to be in R n ). The formula says that the volume squared is equal to the sum of the squares of the volumes when projected onto the coordinate planes. These are generalizations of the pythagorean theorem! Thus, we have that L = QQ T for a choice of directions. To compute t (G) ; we have that t (G) = det ^L = det ^Q ^QT = X det ^Q S S 5

where ^Q is Q with the rst row removed and S ranges over collections of p edges in G. We need to understand what det ^Q S represents. The claim is that det Q S = if the edges represented by S form a spanning tree for G and zero otherwise. If S forms a disconnected graph, then ^Q S looks like M M and since M consists of columns with exactly one and one ; we see that (; ; : : : ; ; ; ; ; : : : ) T (respecting the form above) is a zero eigenvector, and hence the determinant is equal to : Now, if S contains a cycle, then since it has p edges, the corresponding graph must be disconnected, and the previous comment follows.the problem of showing that det ^Q S = if the edges represented by S form a spanning tree will be part of your next homework assignment. It follows that t (G) = X det ^Q S S is equal to the number of spanning trees. Appendix: Challenges to the PageRank algorithm There are two issues we will have to deal with.. Nonuniqueness We would like our ranking to be unique, which means that we should have only one eigenvector representing the eigenvalue. It turns out that this is true if the web is a strongly connected digraph. We will show this later. However, if the web is disconnected, then we can have a higher dimensional eigenspace for eigenvalue : Consider the web in BL Figure.. The link matrix is A = 6 4 It is easy to see that the vectors ; ; ; ; T and ; ; ; ; T have eigenvalue : However, we also have that any linear combination of these have eigenvalue ; and so we have vectors like 3 8 ; 3 8 ; 8 ; 8 ; T as well as 8 ; 8 ; 3 8 ; 3 8 ; T ; which give di erent rankings! Proposition 3 Let W be a web with r components W ; W ; : : : ; W r : Then the eigenspace of the eigenvalue is at least r-dimensional. 3 7 5 : 6

Proof (Sketch). A careful consideration shows that if we label the web by assigning the vertices in W rst, then the vertices in W ; etc., then the link matrix will have a block diagonal form like 3 A A A = 6 4... 7 5 ; A r where A k is the link matrix for the web W k : If each is column stochastic, each has an eigenvector v k with eigenvalue, and that can be expanded into a eigenvector w k for A by letting w = B @ v. ; w C = B A @ etc. Each of these is linearly independent and part of the eigenspace V of eigenvalue. We will gure out a way to deal with this soon.. Dangling nodes De nition 3 A dangling node is a vertex in the web with outdegree zero (i.e., with no links). The problem with a dangling node is that it produces a column of zeroes. This means that the resulting link matrix is not column-stochastic, since some columns may sum to zero. This means that we may not use our theorem that is an eigenvalue. In fact, it may not be true. We will sketch how to deal with this later. Appendix: Solving the problems of the PageRank algorithm v. Dealing with multiple eigenspaces Recall that we seemed to be okay if we had a strongly connected graph (web). We will now take our webs that are not strongly connected and make them strongly connected by adding a little bit of an edge between any two vertices. From a probabilistic perspective, we are adding on a possibility of randomly jumping to any page on the entire web. We will make this probability small compared with the probability to navigate from a page.. ; C A 7

Let S be the matrix n n matrix with all entries =n: Notice that this matrix is column stochastic. In terms of probabilities, this matrix represents equal probabilities of jumping to any page on the web (including the one you are already on). Also notice that if A is a column stochastic matrix, then M = ( m) A + ms is column stochastic for all values of m between zero and one. Supposedly the original value for m used by Google was :5: We will show that the matrix M has a one-dimensional eigenspace V (M) for the eigenvalue as long as m > : Note that the matrix M has all positive entries. This motivates: De nition 33 A matrix M = (M ij ) is positive if M ij > for every i and j: For future use, we de ne the following. De nition 34 Given a matrix M; we write V (M) for the eigenspace of eigenvalue : We will show that a positive column-stochastic matrix has a one dimensional eigenspace V for eigenvalue : Proposition 35 If M is a positive, column-stochastic matrix, then V (M) has dimension : Proof. Suppose v and w are in V (M) : Then we know that sv + tw V (M) for any real numbers s and t: We will now show that () any eigenvector in V (M) has all positive or all negative components and that () if x and y are any two linearly independent vectors, then there is some s and some t such that sx + ty has both positive and negative components. This would imply that sv + tw has all positive or all negative components, and thus v and w must be linearly dependent. First we prove the following: Proposition 36 Any eigenvector in V (M) has all positive or all negative components. Proof. Suppose Mv = v: Since M is column-stochastic, we know that M ij = i= for each j; and since M is positive, we know that jm ij j = M ij 8

for each i and j: Therefore, we see that kvk = kmvk = M ij v j i= j= jm ij j jv j j = = i= j= j= i= M ij jv j j jv j j = kvk : That means that the inequality must be an equality, meaning that M ij v j = jm ij j jv j j : j= j= This is only true if M ij v j for each i and j (or M ij v j for each i and j). However, since M ij > ; this implies that v j for each j (or v j for each j). Furthermore, since v i = M ij v j j= with v j (v j )and M ij > ; we must have that either all v are zero or all are positive (negative). Since v is an eigenvector, it is not the zero vector. Remark 37 A similar argument shows that for any positive, column-stochastic matrix, all eigenvalues satisfy jj : Proposition 38 For any linearly independent vectors x and y R n ; there are real values of s and t such that sx+ty has both negative and positive components. Proof. Certainly this is true if either x or y have both postive and negative components, so we may assume both have only positive components (the other cases of both negative or one positive and one negative are handled by adjusting the signs of s and t appropriately). We may now consider the vector x =! w i v i= j=! v i w: Both the sums in the above expression are nonzero by assumption (in fact, positive). Also x is nonzero since v and w are linearly independent. Notice that x i = : i= 9 i=

Since x is not the zero vector, this implies that x must have both positive and negative components. Thus, the matrix M can be used to produce unique rankings if there no dangling nodes.. Dealing with Dangling nodes For dangling nodes, we have the following theorem of Perron: Theorem 39 If A is a matrix with all positive entries, then A contains a real, positive eigenvalue such that. For any other eigenvalue ; we have jj < (recall that could be complex).. The eigenspace of is one-dimensional and there is a unique eigenvector x = [x ; x ; : : : ; x p ] T with eigenvalue such that x i > for all i and px x i = : i= This eigenvector is called the Perron vector. Thus, if we had a matrix with all postive entries, as we got in the last section, we can use the Perron vector as the ranking.