U.C. Berkeley CS78: Computatioal Complexity Hadout N Professor Luca Trevisa 3/4/008 Notes for Lecture Eigevalues, Expasio, ad Radom Walks As usual by ow, let G = (V, E) be a udirected d-regular graph with vertices, M be its trasitio matrix, = λ λ λ be the eigevalues of M, ad x,..., x be a system of orthoormal eigevectors. Let p be a probability distributio over vertices V, ad cosider the followig process: pick at radom a vertex v accordig to p, the perform a t-step radom walk i G startig from v. We would like to address the followig questios:. Is there a clea formula that specifies the distributio of the fial vertex of the walk i terms of p, M ad t?. For large t, does the distributio of the fial vertex coverge to a fixed distributio idepedet of p? 3. If so, what is this distributio? 4. Ad how fast is the covergece? As we will see, the aswers are. Yes, if we write distributios as row vectors, the it s pm t.. Yes (provided the graph is coected ad ot bipartite, both ecessary coditios). 3. The uiform distributio. 4. It depeds o max{λ, λ }, or just o λ if we take a lazy radom walk. Let us begi with the first questio. If p R V is a row vector that represets a probability distributio over vertices (hece, p(v) 0 for all v, ad v p(v) = ), the cosider the vector pm. Its v-th etry is (pm)(v) = u p(u)m(u, v) which clearly represets the probability of reachig v by first pickig a vertex u accordig to distributio p, ad the movig to a radom eighbor of u. I particular, pm is itself a probability distributio. Reasoig iductively, we see that, for every t, pm t is a probability distributio, ad it represets the distributio of the fial vertex i a t-step radom walk i G that starts at a vertex selected accodig to p.
It follows that, i order to uderstad radom walks i G, we eed to uderstad the actio of the matrix M t. Fortuately, we already kow the eigevalues ad the eigevectors of M t : the matrix M t has eigevalues λ t,..., λt, ad eigevectors x,..., x. If p is a probability distributio, let us write p = α x + α x. The pm t = α x + α λ t x +... + α λ t x Where x = (,..., ) ad α = p x T = v p(v) =, so that α x = (,..., ) is the uiform distributio, which we shall deote by p U from ow o. Now we would like to argue that pm t coverges to p U for large t. Ideed cosider the vector pm t p U, which measures the o-uiformity of pm t ; its legth is pm t p U = α λ t x +... + α λ t x = α λt + α λ t max λ i t i=,..., max λ i t p i=,..., max λ i t i=,..., α + + α Where we use the fact that if p is a probability distributio the p = p (v) p(v) = v Let λ := max i=,..., λ i = max{λ, λ } be the secod largest eigevalue i absolute value. Our calculatio shows that v pm t p U λ t ad so if choose t = O( λ log ) the we ca have, say pm t p U 00 ad, i particular, pm t (v) 00 for every v. This meas that it oly takes t = O( log ) steps for a radom walk to coverge to the uiform distributio, ad that λ the diameter of G is upper bouded by O( log ) for a stroger reaso. λ It is istructive to see that ay boud o the covergece of radom walks must deped both o λ ad o λ. Suppose, for starters, that G has two coected compoets S, V S ad that p is uiform o S. The pm = p, pm t = p, ad a radom walk will ever coverge to the uiform distributio. (Algebraically, we have λ =, α 0.) This demostrates the ecessity of a depedece o λ.
Suppose kow that G = (V, E) is a complete bipartite graph with bipartitio A, V A, such that / vertices are o each side. (Hece G is regular of degree /.) Let us costruct the eigevectors ad eigevalues of G. We ow that x := (,..., ) is a eigevector for the eigevalue. Cosider ow the vector x such that x (v) = if v A ad x (v) = if v A. The we ca verify that x x, ad that x is a eigevector of eigevalue -. Fially, we ote that a vector x is orthogoal to both x ad x if ad oly if v A x(v) = v A x(v) = 0, ad that ay such vector is a eigevector of eigevalue 0. Choose x,..., x to be a orthoormal basis for the above-described space of dimesio, ad so we have costructed a orthoormal set of eigevectors x,..., x for the eigevalues λ =, λ = λ = 0, λ =. Cosider the distributio p that is uiform o A. The pm is uiform o V A, pm is uiform o A, pm 3 is uiform o V A, ad so o, ad the radom walk does ot coverge to the uiform distributio. To see what happes algebraically, p = x + x, so pm t p U = λ t x = ( ) t x. The theory that we have developed so far coects edge expasio with λ, but ot with λ. There is, however, a simple trick that allows to relate edge expasio to the behavior of radom walks. For a d-regular graph G with trasitio matrix M, defie the lazy radom walk o G as the radom walk of trasitio matrix M L := I + M; equivaletly, we ca thik of it as a stadard radom walk o the d-regular graph G L that is idetical to G except that every vertex has d self-loops. If M has eigevalues λ =,..., λ ad eigevectors x,..., x, the it is easy to see that x,..., x are also eigevectors for M L, ad that the eigevalues of M L are + λ =, + λ,..., + λ. I particular, all the eigevalues of M L are o-egative, ad so λ (M L ) = λ (M L ) = + λ (M). It follows that if G has ormalized edge expasio h, the λ (M L ) h 4. If G is coected, the every cut is crossed by at least oe edge, ad so the ormalized edge expasio is at least iverse-polyomial. h(g) = edges(s, V S) mi S V, S V d S d = d From Cheeger s iequality, we have that the gap betwee largest ad secod-largest eigevalue is also iverse-polyomial Ad so, i M L, we have λ h d λ (M L ) = λ (M L ) = λ (M) d This meas that, i a coected graph, a lazy radom walk of legth O( d log ) reaches a early uiformly distributed vertex. The ame lazy radom walk refers to the fact that the radom walk M L behaves essetially like the radom walk M, except that at every step there is a probability of doig othig. 3
( ) Aother coclusio that we ca reach is that the diameter of G L is at most O λ log, ad the same boud applies to G, sice G ad G L have the same diameter. If G has ormalized edge expasio h, the G has diameter at most O(h log ). The Expader Mixig Lemma So far we have studied properties (ad characterizatios) of graphs i which λ ad λ are bouded away from. Graphs satisfyig the stroger requiremet that λ is close to zero ejoy a umber of additioal, sometimes surprisig, properties. Such graphs are also useful i a umber of applicatios, ad the questio of explicitly costructig such graphs havig bouded degree is well studied (ad it will be the subject of the ext two lectures). For ow we ote that oce we have costructed a family of arbitrary large d-regular graphs satisfyig λ ɛ for some fixed ɛ > 0, the we also immediately get a family of k d k -regular graphs satisfyig λ ( ) ɛ k, because give a d-regular graph G satisfyig λ (G) ɛ, we ca first covert it ito the d-regular graph G L satisfyig λ (G L ) ɛ ad the take the k-th power of G L. (The k-th power of a graph G with trasio matrix M is the graph G k whose trasitio matrix is M k : G k has oe edge for every legth-k path i G.) Oe of the mai results about graphs with small λ is the Expader Mixig Lemma. Lemma (Expader Mixig Lemma) Let G be a d-regular graph with vertices ad secod largest eigevalue i absolute value λ, let A, B be two disjoit sets of vertices. The edges(a, B) d A B λ d A B λ d () Thus, i a very good expader, the umber of edges betwee ay two sufficietly large sets of vertices is approximately what it would be i a radom d-regular graph. Although there is a short direct proof, it is istructive to use the proof of the lemma as a opportuity to itroduce the followig otio. Defiitio (Matrix Norm) Let M be a m matrix, the we defie its orm as xm M := max = max xm () x R x x R, x = Note that if M is the trasitio matrix of a udirected regular graph, the M =. Defie J to be the matrix that has a i each etry. We drop the subscript whe it s clear from the cotext. Claim Let M be the trasitio matrix of a udirected graph G, ad let λ be the secod largest eigevalue of M i absolute value. The λ = M J (3) 4
The poit of the claim is that whe λ is small the M is close to J, which is the trasitio matrix of the complete graph. Note that a radom walk i J reaches, already at the first step, a radom vertex, so this is cosistet with the ituitio that a radom walk i a graph with small λ coverges rapidly to the uiform distributio. Let us ow tur to the proof of the Expader Mixig Lemma. Let A R be the vector such that A (x) = if x A ad A (x) = 0 otherwise; defie B similarly. We have ad so that edges(a, B) = A dm T B (4) A JT B = A B (5) ( edges(a, B) d A B = d A M ) J T B ad, applyig Caucy-Schwarz ad the defiitio of orm, we have A (M ) J Ad ow we see that T B A (M J ) B A M J B A = A B = B 3 Exercises M J = λ. Prove Claim [Hit: the maximizig vector is the eigevector of λ ] 5