Lectures on Spectral Graph Theory. Fan R. K. Chung

Transcription

1 Lectures on Spectral Graph Theory Fan R. K. Chung Author address: Uniersity of Pennsylania, Philadelphia, Pennsylania address:

2 Contents Chapter 1. Eigenalues and the Laplacian of a graph Introduction The Laplacian and eigenalues Basic facts about the spectrum of a graph Eigenalues of weighted graphs Eigenalues and random walks 14 Chapter 2. Isoperimetric problems History The Cheeger constant of a graph The edge expansion of a graph The ertex expansion of a graph A characterization of the Cheeger constant Isoperimetric inequalities for cartesian products 36 Chapter 3. Diameters and eigenalues The diameter of a graph Eigenalues and distances between two subsets Eigenalues and distances among many subsets Eigenalue upper bounds for manifolds 50 Chapter 4. Paths, flows, and routing Paths and sets of paths 59 iii

3 i CONTENTS 4.2. Flows and Cheeger constants Eigenalues and routes with small congestion Routing in graphs Comparison theorems 68 Chapter 5. Eigenalues and quasi-randomness Quasi-randomness The discrepancy property The deiation of a graph Quasi-random graphs 85 Bibliography 91

4 CHAPTER 1 Eigenalues and the Laplacian of a graph 1.1. Introduction Spectral graph theory has a long history. In the early days, matrix theory and linear algebra were used to analyze adjacency matrices of graphs. Algebraic methods are especially effectie in treating graphs which are regular and symmetric. Sometimes, certain eigenalues hae been referred to as the algebraic connectiity of a graph [126]. There is a large literature on algebraic aspects of spectral graph theory, well documented in seeral sureys and books, such as Biggs [25], Cetkoić, Doob and Sachs [90, 91], and Seidel [224]. In the past ten years, many deelopments in spectral graph theory hae often had a geometric flaor. For example, the explicit constructions of expander graphs, due to Lubotzky-Phillips-Sarnak [193] and Margulis [195], are based on eigenalues and isoperimetric properties of graphs. The discrete analogue of the Cheeger inequality has been heaily utilized in the study of random walks and rapidly mixing Marko chains [224]. New spectral techniques hae emerged and they are powerful and well-suited for dealing with general graphs. In a way, spectral graph theory has entered a new era. Just as astronomers study stellar spectra to determine the make-up of distant stars, one of the main goals in graph theory is to deduce the principal properties and structure of a graph from its graph spectrum (or from a short list of easily computable inariants). The spectral approach for general graphs is a step in this direction. We will see that eigenalues are closely related to almost all major inariants of a graph, linking one extremal property to another. There is no question that eigenalues play a central role in our fundamental understanding of graphs. The study of graph eigenalues realizes increasingly rich connections with many other areas of mathematics. A particularly important deelopment is the interaction between spectral graph theory and differential geometry. There is an interesting analogy between spectral Riemannian geometry and spectral graph theory. The concepts and methods of spectral geometry bring useful tools and crucial insights to the study of graph eigenalues, which in turn lead to new directions and results in spectral geometry. Algebraic spectral methods are also ery useful, especially for extremal examples and constructions. In this book, we take a broad approach with emphasis on the geometric aspects of graph eigenalues, while including the algebraic aspects as well. The reader is not required to hae special background in geometry, since this book is almost entirely graph-theoretic. 1

5 2 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH From the start, spectral graph theory has had applications to chemistry [27]. Eigenalues were associated with the stability of molecules. Also, graph spectra arise naturally in arious problems of theoretical physics and quantum mechanics, for example, in minimizing energies of Hamiltonian systems. The recent progress on expander graphs and eigenalues was initiated by problems in communication networks. The deelopment of rapidly mixing Marko chains has intertwined with adances in randomized approximation algorithms. Applications of graph eigenalues occur in numerous areas and in different guises. Howeer, the underlying mathematics of spectral graph theory through all its connections to the pure and applied, the continuous and discrete, can be iewed as a single unified subject. It is this aspect that we intend to coer in this book The Laplacian and eigenalues Before we start to define eigenalues, some explanations are in order. The eigenalues we consider throughout this book are not exactly the same as those in Biggs [25] orcetkoić, Doob and Sachs [90]. Basically, the eigenalues are defined here in a general and normalized form. Although this might look a little complicated at first, our eigenalues relate well to other graph inariants for general graphs in a way that other definitions (such as the eigenalues of adjacency matrices) often fail to do. The adantages of this definition are perhaps due to the fact that it is consistent with the eigenalues in spectral geometry and in stochastic processes. Many results which were only known for regular graphs can be generalized to all graphs. Consequently, this proides a coherent treatment for a general graph. For definitions and standard graph-theoretic terminology, the reader is referred to [31]. In a graph G, letd denote the degree of the ertex. We first define the Laplacian for graphs without loops and multiple edges (the general weighted case with loops will be treated in Section 1.4). To begin, we consider the matrix L, defined as follows: d if u =, L(u, ) = 1 if u and are adjacent, 0 otherwise. Let T denote the diagonal matrix with the (, )-th entry haing alue d. The Laplacian of G is defined to be the matrix We can write L(u, ) = 1 if u = and d 0, 1 du d if u and are adjacent, 0 otherwise. L = T 1/2 LT 1/2 with the conention T 1 (, ) =0ford =0. Wesay is an isolated ertex if d = 0. A graph is said to be nontriial if it contains at least one edge.

6 1.2. THE LAPLACIAN AND EIGENVALUES 3 L canbeiewedasanoperatoronthespaceoffunctionsg : V (G) R which satisfies Lg(u) = 1 ( g(u) g() ) du du d u When G is k-regular, it is easy to see that L = I 1 k A, where A is the adjacency matrix of G,( i. e., A(x, y) =1ifx is adjacent to y, and 0otherwise,)andI is an identity matrix. All matrices here are n n where n is the number of ertices in G. For a general graph, we hae L = T 1/2 LT 1/2 = I T 1/2 AT 1/2. We note that L can be written as L = SS, where S is the matrix whose rows are indexed by the ertices and whose columns are indexed by the edges of G such that each column corresponding to an edge e = {u, } has an entry 1/ d u in the row corresponding to u, anentry 1/ d in the row corresponding to, and has zero entries elsewhere. (As it turns out, the choice of signs can be arbitrary as long as one is positie and the other is negatie.) Also, S denotes the transpose of S. For readers who are familiar with terminology in homology theory, we remark that S can be iewed as a boundary operator mapping 1-chains defined on edges (denoted by C 1 ) of a graph to 0-chains defined on ertices (denoted by C 0 ). Then, S is the corresponding coboundary operator and we hae C 1 S S C 0 Since L is symmetric, its eigenalues are all real and non-negatie. We can use the ariational characterizations of those eigenalues in terms of the Rayleigh quotient of L (see, e.g. [162]). Let g denote an arbitrary function which assigns to each ertex of G arealalueg(). We can iew g as a column ector. Then g, Lg = g, T 1/2 LT 1/2 g g, g g, g f,lf = T 1/2 f,t 1/2 f (f(u) f()) 2 (1.1) = u f() 2 d

7 4 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH where g = T 1/2 f and u denotes the sum oer all unordered pairs {u, } for which u and are adjacent. Here f,g = x f(x)g(x) denotes the standard inner product in R n.thesum u (f(u) f()) 2 is sometimes called the Dirichlet sum of G and the ratio on the left-hand side of (1.1) is often called the Rayleigh quotient. (We note that we can also use the inner product f,g = f(x)g(x) for complex-alued functions.) From equation (1.1), we see that all eigenalues are non-negatie. In fact, we can easily deduce from equation (1.1) that 0 is an eigenalue of L. Wedenotethe eigenalues of L by 0 = λ 0 λ 1 λ n 1. The set of the λ i s is usually called the spectrum of L (or the spectrum of the associated graph G.) Let 1 denote the constant function which assumes the alue 1 on each ertex. Then T 1/2 1 is an eigenfunction of L with eigenalue 0. Furthermore, (1.2) λ G = λ 1 = inf f T 1 (f(u) f()) 2 u. f() 2 d The corresponding eigenfunction is g = T 1/2 f as in (1.1). It is sometimes conenient to consider the nontriial function f achieing (1.2), in which case we call f a harmonic eigenfunction of L. The aboe formulation for λ G corresponds in a natural way to the eigenalues of the Laplace-Beltrami operator for Riemannian manifolds: f 2 M λ M = inf, f 2 M where f ranges oer functions satisfying f =0. M We remark that the corresponding measure here for each edge is 1 although in the general case for weighted graphs the measure for an edge is associated with the edge weight (see Section 1.4.) The measure for each ertex is the degree of the ertex. A more general notion of ertex weights will be considered in Section 2.5.

8 1.2. THE LAPLACIAN AND EIGENVALUES 5 (1.3) (1.4) We note that (1.2) has seeral different formulations: (f(u) f()) 2 λ 1 = inf f = inf f sup t u (f() t) 2 d (f(u) f()) 2 u (f() f), 2 d where f()d f = ol G, and ol G denotes the olume of the graph G, gienby ol G = d. By substituting for f N and using the fact that N i a) i=1(a 2 = (a i a j ) 2 i<j N for a = a i /N, we hae the following expression (which generalizes the one in [126]): (1.5) i=1 λ 1 = ol G inf f (f(u) f()) 2 u, (f(u) f()) 2 d u d u, where u, denotes the sum oer all unordered pairs of ertices u, in G. We can characterize the other eigenalues of L in terms of the Rayleigh quotient. largest eigenalue satisfies: (f(u) f()) 2 (1.6) For a general k, wehae (1.7) (1.8) λ n 1 = sup f λ k = inf f sup g P k 1 = inf f TP k 1 u. f 2 ()d (f(u) f()) 2 u (f() g()) 2 d (f(u) f()) 2 u f() 2 d The

9 6 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH where P i is the subspace generated by the harmonic eigenfunctions corresponding to λ i,fori k 1. The different formulations for eigenalues gien aboe are useful in different settings and they will be used in later chapters. Here are some examples of special graphs and their eigenalues. Example 1.1. For the complete graph K n on n ertices, the eigenalues are 0andn/(n 1) (with multiplicity n 1). Example 1.2. For the complete bipartite graph K m,n on m + n ertices, the eigenalues are 0, 1 (with multiplicity m + n 2), and 2. Example 1.3. For the star S n on n ertices, the eigenalues are 0, 1(with multiplicity n 2), and 2. Example 1.4. For the path P n on n ertices, the eigenalues are 1 cos πk n 1 for k =0, 1,,n 1. Example 1.5. For the cycle C n on n ertices, the eigenalues are 1 cos 2πk n for k =0,,n 1. Example 1.6. For the n-cube Q n on 2 n ertices, the eigenalues are 2k n (with multiplicity ( n k) )fork =0,,n. More examples can be found in Chapter 6on explicit constructions Basic facts about the spectrum of a graph Roughly speaking, half of the main problems of spectral theory lie in deriing bounds on the distributions of eigenalues. The other half concern the impact and consequences of the eigenalue bounds as well as their applications. In this section, we start with a few basic facts about eigenalues. Some simple upper bounds and lower bounds are stated. For example, we will see that the eigenalues of any graph lie between 0 and 2. The problem of narrowing the range of the eigenalues for special classes of graphs offers an open-ended challenge. Numerous questions can be asked either in terms of other graph inariants or under further assumptions imposed on the graphs. Some of these will be discussed in subsequent chapters. Lemma 1.7. For a graph G on n ertices, we hae (i): λ i n i with equality holding if and only if G has no isolated ertices. (ii): For n 2, λ 1 n n 1 with equality holding if and only if G is the complete graph on n ertices. Also, for a graph G without isolated ertices, we hae λ n 1 n n 1.

10 1.3. BASIC FACTS ABOUT THE SPECTRUM OF A GRAPH 7 (iii): For a graph which is not a complete graph, we hae λ 1 1. (i): If G is connected, then λ 1 > 0. Ifλ i =0and λ i+1 0,thenG has exactly i +1 connected components. (): For all i n 1, we hae λ i 2. with λ n 1 =2if and only if a connected component of G is bipartite and nontriial. (i): The spectrum of a graph is the union of the spectra of its connected components. Proof. (i) follows from considering the trace of L. The inequalities in (ii) follow from (i) and λ 0 =0. Suppose G contains two nonadjacent ertices a and b, and consider d b if = a, f 1 () = d a if = b, 0 if a, b. (iii) then follows from (1.2). If G is connected, the eigenalue 0 has multiplicity 1 since any harmonic eigenfunction with eigenalue 0 assumes the same alue at each ertex. Thus, (i) follows from the fact that the union of two disjoint graphs has as its spectrum the union of the spectra of the original graphs. () follows from equation (1.6) and the fact that (f(x) f(y)) 2 2(f 2 (x)+f 2 (y)). Therefore (f(x) f(y)) 2 λ i sup f x y 2. f 2 (x)d x x Equality holds for i = n 1whenf(x) = f(y) for eery edge {x, y} in G. Therefore, since f 0,G has a bipartite connected component. On the other hand, if G has a connected component which is bipartite, we can choose the function f so as to make λ n 1 =2. (i) follows from the definition. For bipartite graphs, the following slightly stronger result holds: Lemma 1.8. The following statements are equialent: (i): G is bipartite. (ii): G has i +1 connected components and λ n j =2for 1 j i. (iii): For each λ i,thealue2 λ i is also an eigenalue of G.

11 8 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH Proof. It suffices to consider a connected graph. Suppose G is bipartite graph with ertex set consisting of two parts A and B. For any harmonic eigenfunction f with eigenalue λ, we consider the function g { f(x) if x A, g(x) = f(x) if x B. It is easy to check that g is a harmonic eigenfunction with eigenalue 2 λ. For a connected graph, we can immediately improe the lower bound of λ 1 in Lemma 1.7. For two ertices u and, thedistance between u and is the number of edges in a shortest path joining u and. Thediameter of a graph is the maximum distance between any two ertices of G. Here we will gie a simple eigenalue lower bound in terms of the diameter of a graph. More discussion on the relationship between eigenalues and diameter will be gien in Chapter 3. Lemma 1.9. For a connected graph G with diameter D, we hae λ 1 1 D ol G Proof. Suppose f is a harmonic eigenfunction achieing λ 1 in (1.2). Let 0 denote a ertex with f( 0 ) =max f(). Since f() = 0, there exists a ertex u 0 satisfying f(u 0 )f( 0 ) < 0. Let P denote a shortest path in G joining u 0 and 0. Then by (1.2) we hae λ 1 = (f(x) f(y)) 2 x y f 2 (x)d x x {x,y} P by using the Cauchy-Schwarz inequality. (f(x) f(y)) 2 ol Gf 2 ( 0 ) 1 D (f( 0) f(u 0 )) 2 ol Gf 2 ( 0 ) 1 D ol G Lemma Let f denote a harmonic eigenfunction achieing λ G in (1.2). Then, for any ertex x V, we hae 1 (f(x) f(y)) = λ G f(x). d x y y x

12 1.3. BASIC FACTS ABOUT THE SPECTRUM OF A GRAPH 9 Proof. We use a ariational argument. For a fixed x 0 V,weconsiderf ɛ such that f(x 0 )+ ɛ if y = x 0, d f ɛ (y) = x0 ɛ f(y) otherwise. ol G d x0 We hae = x,y V x y x,y V x y (f ɛ (x) f ɛ (y)) 2 fɛ 2 (x)d x x V (f(x) f(y)) 2 + 2ɛ(f(x 0 ) f(y)) y d x0 y y x y 0 y x 0 y y f 2 2ɛ (x)d x +2ɛf(x 0 ) f(y)d y ol G d x0 x V y x 0 +O(ɛ 2 ) 2ɛ(f(y) f(y )) ol G d x0 = (f(x) f(y)) 2 + x,y V x y +O(ɛ 2 ) x V 2ɛ (f(x 0 ) f(y)) 2ɛ (f(x 0 ) f(y)) y y y x 0 y x 0 + d x0 ol G d x0 f 2 (x)d x +2ɛf(x 0 )+ 2ɛf(x 0)d x0 ol G d x0 since x V implies that f(x)d x =0,and y (f ɛ (x) f ɛ (y)) 2 x,y V x y fɛ 2 (x)d x x V y (f(y) f(y )) = 0. The definition in (1.2) (f(x) f(y)) 2 x,y V x y f 2 (x)d x If we consider what happens to the Rayleigh quotient for f ɛ as ɛ 0fromaboe, or from below, we can conclude that 1 d x0 and the Lemma is proed. x V (f(x 0 ) f(y)) = λ G f(x 0 ). y y x 0

13 10 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH One can also proe the statement in Lemma 1.10 by recalling that f = T 1/2 g, where Lg = λ G g.then T 1 Lf = T 1 (T 1/2 LT 1/2 )(T 1/2 g)=t 1/2 λ G g = λ G f, and examining the entries gies the desired result. With a little linear algebra, we can improe the bounds on eigenalues in terms of the degrees of the ertices. We consider the trace of (I L) 2.Wehae Tr(I L) 2 = (1 λ i ) 2 i (1.9) where 1+(n 1) λ 2, λ =max i. i 0 On the other hand, (1.10) Tr(I L) 2 = Tr(T 1/2 AT 1 AT 1/2 ) = 1 A(x, y) 1 1 A(y, x) dx d x,y y dx = 1 ( 1 1 ) 2, d x x d x y x d y where A is the adjacency matrix. From this, we immediately deduce Lemma For a k-regular graph G on n ertices, we hae (1.11) max i i 0 n k (n 1)k This follows from the fact that max 1 λ i 2 1 i 0 n 1 (tr(i L)2 1). Let d H denote the harmonic mean of the d s, i.e., 1 = 1 1. d H n d It is tempting to consider generalizing (1.11) with k replaced by d H. This, howeer, is not true as shown by the following example due to Elizabeth Wilmer. Example Consider the m-petal graph on n =2m+1ertices, 0, 1,, 2m with edges { 0, i } and { 2i 1, 2i },fori 1. This graph has eigenalues 0, 1/2 (with multiplicity m 1), and 3/2 (with multiplicity m + 1). So we hae max i 0 1 λ i =1/2. Howeer, n d H m 1/2 = (n 1)d H 2m 1 2 as m.

14 1.4. EIGENVALUES OF WEIGHTED GRAPHS 11 Still, for a general graph, we can use the fact that ( 1 1 ) 2 d x y x d y (1.12) ( 1 1 λ n 1 1+ λ. ) 2 d x d x d H x V Combining (1.9), (1.10) and (1.12), we obtain the following: Lemma For a graph G on n ertices, λ =max i 0 1 λ i satisfies 1+(n 1) λ 2 n (1 (1 + d λ)( k 1)), H d H where k denotes the aerage degree of G. There are relatiely easy ways to improe the upper bound for λ 1. From the characterization in the preceding section, we can choose any function f : V (G) R, and its Rayleigh quotient will sere as an upper bound for λ 1. Here we describe an upper bound for λ 1 (see [204]). Lemma Let G be a graph with diameter D 4, andletk denote the maximum degree of G. Then ( k 1 λ ) + 2 k D D. One way to bound eigenalues from aboe is to consider contracting the graph G intoaweightedgraphh (which will be defined in the next section). Then the eigenalues of G can be upper-bounded by the eigenalues of H or by arious upper bounds on them, which might be easier to obtain. We remark that the proof of Lemma 1.14 proceeds by basically contracting the graph into a weighted path. We will proe Lemma 1.14 in the next section. We note that Lemma 1.14 gies a proof (see [5]) that for any fixed k and for any infinite family of regular graphs with degree k, k 1 lim sup λ k This bound is the best possible since it is sharp for the Ramanujan graphs (which will be discussed in Chapter??). We note that the cleaner ersion of λ k 1/k is not true for certain graphs (e.g., 4-cycles or complete bipartite graphs). This example also illustrates that the assumption in Lemma 1.14 concerning D 4 is essential Eigenalues of weighted graphs Before defining weighted graphs, we will say a few words about two different approaches for giing definitions. We could hae started from the ery beginning with weighted graphs, from which simple graphs arise as a special case in which the weights are 0 or 1. Howeer, the unique characteristics and special strength of

15 12 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH graph theory is its ability to deal with the {0, 1}-problems arising in many natural situations. The clean formulation of a simple graph has conceptual adantages. Furthermore, as we shall see, all definitions and subsequent theorems for simple graphs can usually be easily carried out for weighted graphs. A weighted undirected graph G (possibly with loops) has associated with it a weight function w : V V R satisfying and w(u, ) =w(, u) w(u, ) 0. We note that if {u, } E(G),thenw(u, ) = 0. Unweighted graphs are just the special case where all the weights are 0 or 1. In the present context, the degree d of a ertex is defined to be: d = u w(u, ), ol G = d. We generalize the definitions of preious sections, so that d w(, ) if u =, L(u, ) = w(u, ) if u and are adjacent, 0 otherwise. In particular, for a function f : V R, wehae Lf(x) = (f(x) f(y))w(x, y). y x y Let T denote the diagonal matrix with the (, )-th entry haing alue d.the Laplacian of G is defined to be In other words, we hae L = T 1/2 LT 1/2. w(, ) 1 if u =, andd 0, d L(u, ) = w(u, ) if u and are adjacent, du d 0 otherwise.

16 1.4. EIGENVALUES OF WEIGHTED GRAPHS 13 We can still use the same characterizations for the eigenalues of the generalized ersions of L. For example, (1.13) g, Lg λ G := λ 1 = inf g T 1/2 1 g, g f(x)lf(x) x V = inf! f f(x)dx=0 f 2 (x)d x x V (f(x) f(y)) 2 w(x, y) = inf f! f(x)dx=0 x y. f 2 (x)d x x V A contraction of a graph G is formed by identifying two distinct ertices, say u and, intoasingleertex. The weights of edges incident to are defined as follows: w(x, ) = w(x, u)+w(x, ), w(, ) = w(u, u)+w(, )+2w(u, ). Lemma If H is formed by contractions from a graph G, then λ G λ H The proof follows from the fact that an eigenfunction which achiees λ H for H can be lifted to a function defined on V (G) such that all ertices in G that contract to the same ertex in H share the same alue. Now we return to Lemma SKETCHED PROOF OF LEMMA 1.14: Let u and denote two ertices that are at distance D 2t +2 in G. We contract G into a path H with 2t + 2 edges, with ertices x 0,x 1,...x t,z,y t,...,y 2,y 1,y 0 such that ertices at distance i from u, 0 i t, are contracted to x i, and ertices at distance j from, 0 j t, are contracted to y j. The remaining ertices are contracted to z. To establish an upper bound for λ 1, it is enough to choose a suitable function f, defined as follows: f(x i ) = a(k 1) i/2, f(y j ) = b(k 1) j/2, f(z) = 0, where the constants a and b are chosen so that f(x)d x =0. x

17 14 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH It can be checked that the Rayleigh quotient satisfies (f(u) f()) 2 w(u, ) u 1 2 ( k ) + 1 f() 2 d k t +1 t +1, since the ratio is maximized when w(x i,x i+1 )=k(k 1) i 1 = w(y i,y i+1 ). This completes the proof of the lemma Eigenalues and random walks In a graph G, a walk is just a sequence of ertices ( 0, 1,, s )with { i 1, i } E(G) for all 1 i s. A random walk is determined by the transition probabilities P (u, ) =Prob(x i+1 = x i = u), which are independent of i. Clearly, for each ertex u, P (u, ) =1. For any initial distribution f : V R with f() = 1, the distribution after k steps is just fp k (i.e., a matrix multiplication with f iewed as a row ector where P is the matrix of transition probabilities). The random walk is said to be ergodic if there is a unique stationary distribution π() satisfying lim s fps () =π(). It is easy to see that necessary conditions for the ergodicity of P are (i) irreducibility, i.e., for any u, V,thereexistssomessuch that P s (u, ) > 0 (ii) aperiodicity, i.e., g.c.d. {s : P s (u, ) > 0} = 1. As it turns out, these are also sufficient conditions. A major problem of interest is to determine the number of steps s required for P s to be close to its stationary distribution, gien an arbitrary initial distribution. We say a random walk is reersible if π(u)p (u, ) =π()p (, u). An alternatie description for a reersible random walk can be gien by considering a weighted connected graph with edge weights satisfying w(u, ) =w(, u) =π()p (, u)/c where c can be any constant chosen for the purpose of simplifying the alues. (For example, we can take c to be the aerage of π()p (, u) oerall(, u) with P (, u) 0, so that the alues for w(, u) are either 0 or 1 for a simple graph.) The random walk on a weighted graph has as its transition probabilities w(u, ) P (u, ) =, d u where d u = z w(u, z) is the (weighted) degree of u. The two conditions for ergodicity are equialent to the conditions that the graph be (i) connected and (ii) non-bipartite. From Lemma 1.7, we see that (i) is equialent to λ 1 > 0and

18 1.5. EIGENVALUES AND RANDOM WALKS 15 (ii) implies λ n 1 < 2. As we will see later in (1.15), together (i) and (ii) deduce ergodicity. We remind the reader that an unweighted graph has w(u, ) equal to either 0 or 1. The usual random walk on an unweighted graph has transition probability 1/d of moing from a ertex to any one of its neighbors. The transition matrix P then satisfies { 1/du if u and are adjacent, P (u, ) = 0 otherwise. In other words, fp() = 1 f(u) u d u u for any f : V (G) R. It is easy to check that P = T 1 A = T 1/2 (I L)T 1/2, where A is the adjacency matrix. In a random walk with an associated weighted connected graph G, thetransition matrix P satisfies 1TP = 1T where 1 is the ector with all coordinates 1. Therefore the stationary distribution is exactly π = 1T/ol G, We want to show that when k is large enough, for any initial distribution f : V R, fp k conerges to the stationary distribution. First we consider conergence in the L 2 (or Euclidean) norm. Suppose we write ft 1/2 = i a i φ i, where φ i denotes the orthonormal eigenfunction associated with λ i. Recall that φ 0 = 1T 1/2 / ol G and denotes the L 2 -norm, so a 0 = ft 1/2, 1T 1/2 1T 1/2 since f,1 =1. Wethenhae = 1 ol G fp s π = fp s 1T/ol G = fp s a 0 φ 0 T 1/2 = ft 1/2 (I L) s T 1/2 a 0 φ 0 T 1/2 = λ i ) i 0(1 s a i φ i T 1/2 (1.14) (1 λ ) s max x dx min y dy e max sλ x dx min y dy

19 16 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH where { λ = λ 1 if 1 λ 1 λ n λ n 1 otherwise. So, after s 1/λ log(max x dx /ɛ min y dy ) steps, the L 2 distance between fp s and its stationary distribution is at most ɛ. Although λ occurs in the aboe upper bound for the distance between the stationary distribution and the s-step distribution, in fact, only λ 1 is crucial in the following sense. Note that λ is either λ 1 or 2 λ n 1. Suppose the latter holds, i.e., λ n λ 1. We can consider a modified random walk, called the lazy walk, on the graph G formed by adding a loop of weight d to each ertex. The new graph has Laplacian eigenalues λ k = λ k /2 1, which follows from equation (1.13). Therefore, 1 λ 1 1 λ n 1 0, and the conergence bound in L 2 distance in (1.14) for the modified random walk becomes 2/λ 1 log( max x dx ). ɛ min y dy In general, suppose a weighted graph with edge weights w(u, ) has eigenalues λ i with λ n λ 1. We can then modify the weights by choosing, for some constant c, { w w(, )+cd if u = (1.15) (u, ) = w(u, ) otherwise. The resulting weighted graph has eigenalues where Then we hae λ k = λ k 1+c = 2λ k λ n 1 + λ k c = λ 1 + λ n λ 1 = λ n 1 1=λ n 1 λ 1. λ n 1 + λ 1 Since c 1/2 and we hae λ k 2λ k/(2 + λ k ) 2λ k /3forλ k 1. In particular we set λ = λ 2λ 1 1 = 2 λ n 1 + λ 1 3 λ 1. Therefore the modified random walk corresponding to the weight function w has an improed bound for the conergence rate in L 2 distance: 1 λ log max x dx. ɛ min y dy

20 1.5. EIGENVALUES AND RANDOM WALKS 17 We remark that for many applications in sampling, the conergence in L 2 distance seems to be too weak since it does not require conergence at each ertex. There are seeral stronger notions of distance seeral of which we will mention. A strong notion of conergence that is often used is measured by the relatie pointwise distance (see [224]): After s steps, the relatie pointwise distance (r.p.d.) of P to the stationary distribution π(x) is gien by (s) =max x,y P s (y, x) π(x). π(x) Let ψ x denote the characteristic function of x defined by: { 1 if y = x, ψ x (y) = 0 otherwise. Suppose ψ x T 1/2 = i α i φ i, ψ y T 1/2 = i β i φ i. where φ i s denote the eigenfunction of the Laplacian L of the weighted graph associated with the random walk. In particular, α 0 = d x ol G, β 0 = 1 ol G. Let A denote the transpose of A. Wehae (t) = max x,y = max x,y max x,y ψ y P t ψx π(x) π(x) ψ y T 1/2 (I L) t T 1/2 ψx π(x) π(x) (1 λ i ) t α i β i i 0 λ t max x,y = λ t max x,y d x /ol G α i β i i 0 d x /ol G λ t ol G min x,y dx d y e t(1 λ) ol G min x d x ψ x T 1/2 ψ y T 1/2 d x /ol G

21 18 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH where λ =max i 0 1 λ i.soifwechooset such that then, after t steps, we hae (t) ɛ. t 1 1 λ log ol G ɛ min x d x, When 1 λ 1 λ, we can improe the aboe bound by using a lazy walk as described in (1.15). The proof is almost identical to the aboe calculation except for using the Laplacian of the modified weighted graph associated with the lazy walk. This can be summarized by the following theorem: Theorem For a weighted graph G, we can choose a modified random walk P so that the relatie pairwise distance (t) is bounded aboe by: (t) e tλ ol G exp 2tλ1/(2+λ1) ol G. min x d x min x d x where λ = λ 1 if 2 λ n 1 + λ 1 and λ =2λ 1 /(λ n 1 + λ 1 ) otherwise. Corollary For a weighted graph G, we can choose a modified random walk P so that we hae if (t) e c t 1 λ log ol G min x d x where λ = λ 1 if 2 λ n 1 + λ 1 and λ =2λ 1 /(λ n 1 + λ 1 ) otherwise. We remark that for any initial distribution f : V R with f,1 =1and f(x) 0, we hae, for any x, fp s (x) π(x) π(x) y y f(y) P s (y, x) π(x) π(x) f(y) (s) (s). Another notion of distance for measuring conergence is the so-called total ariation distance, which is just half of the L 1 distance: TV (s) = max max (P s (y, x) π(x)) A V (G) y V (G) x A = 1 2 max P s (y, x) π(x). y V (G) x V (G)

22 1.5. EIGENVALUES AND RANDOM WALKS 19 The total ariation distance is bounded aboe by the relatie pointwise distance, since TV (s) = max max (P s (y, x) π(x)) A V (G) y V (G) ola olg x A 2 max A V (G) ola olg 2 π(x) (s) x A 1 2 (s). Therefore, any conergence bound using relatie pointwise distance implies the same conergence bound using total ariation distance. There is yet another notion of distance, sometimes called χ-squared distance, denoted by (s) and defined by: (s) = max y V (G) max x V (G) y V (G) x V (G) = 2 TV (s), (P s (y, x) π(x)) 2 π(x) P s (y, x) π(x) using the Cauchy-Schwarz inequality. (s) is also dominated by the relatie pointwise distance (which we will mainly use in this book). (s) = max x V (G) y V (G) (P s (x, y) π(y)) 2 π(y) max ( ( (s)) 2 π(y)) 1 2 x V (G) y V (G) (s). 1/2 1/2 We note that (s) 2 x π(x) y (P s (x, y) π(y)) 2 π(y) = x = x ψ x T 1/2 (P s I 0 )T 1 (P s I 0 )T 1/2 ψ x ψ x ((I L) 2s I 0 )ψ x, where I 0 denotes the projection onto the eigenfunction φ 0, φ i denotes the i-th orthonormal eigenfunction of L and ψ x denotes the characteristic function of x. Since ψ x = i φ i (x)φ i,

23 20 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH we hae (1.16) (s) 2 x ψ x ((I L) 2s I 0 )ψ x = ( φ i (x)φ i )((I L) 2s I 0 )( x i i = φ 2 i (x)(1 λ i) 2s x i 0 = φ 2 i (x)(1 λ i) 2s i 0 x = λ i ) i 0(1 2s. φ i (x)φ i ) Equality in (1.16) holds if, for example, G is ertex-transitie, i.e., there is an automorphism mapping u to for any two ertices in G, (for more discussions, see Chapter 7 on symmetrical graphs). Therefore, we conclude Theorem Suppose G is a ertex transitie graph. Then a random walk after s steps conerges to the uniform distribution under total ariation distance or χ-squared distance in a number of steps bounded by the sum of (1 λ i ) 2s,whereλ i ranges oer the non-triial eigenalues of the Laplacian: (1.17) TV (s) 1 2 (s) = 1 2 ( i 0(1 λ i ) 2s ) 1/2. The aboe theorem is often deried from the Plancherel formula. Herewehae employed a direct proof. We remark that for some graphs which are not ertextransitie, a somewhat weaker ersion of (1.17) can still be used with additional work (see [81] and the remarks in Section 4.6). Here we will use Theorem 1.18 to consider random walks on an n-cube. Example For the n-cube Q n, our (lazy) random walk (as defined in (1.15)) conerges to the uniform distribution under the total ariation distance, as estimated as follows: From Example (1.6), the eigenalues of the Q n are 2k/n of multiplicity ( n k) for k =0,,n. The adjusted eigenalues for the weighted graph corresponding to the lazy walk are λ k =2λ k/(λ n 1 + λ 1 )=λ k n/(n +1). By using Theorem 1.18 (also see [104]), we hae TV (s) 1 2 (s) 1 n 2 ( ( ) n (1 2k k n +1 )2s ) 1/2 if s 1 4n log n + cn. k=1 1 2 ( n k=1 e c 4ks k log n e n+1 ) 1/2 We can also compute the rate of conergence of the lazy walk under the relatie pointwise distance. Suppose we denote ertices of Q n by subsets of an n-set

24 1.5. EIGENVALUES AND RANDOM WALKS 21 {1, 2,,n}. The orthonormal eigenfunctions are φ S for S {1, 2,,n} where φ S (X) = ( 1) S X 2 n/2 for any X {1, 2,,n}. For a ertex indexed by the subset S, the characteristic function is denoted by { 1 if X = S, ψ S (X) = 0 otherwise. Clearly, Therefore, This implies if ψ X = S P s (X, Y ) π(y ) π(y ) (s) = n k=1 n k=1 e c s n log n 2 ( 1) S X 2 n/2 φ S. = 2 n ψ X P s ψ Y 1 2 n ψ X P s ψx 1 = (1 2 S n +1 )s S n ( ) n = (1 2k k n +1 )s k=1 ( ) n (1 2k k n +1 )s 2ks k log n e n+1 + cn. So, the rate of conergence under relatie pointwise distance is about twice that under the total ariation distance for Q n. In general, TV (s), (s) and (s) can be quite different [81]. Neertheless, a conergence lower bound for any of these notions of distance (and the L 2 -norm) is λ 1. This we will leae as an exercise. We remark that Aldous [4] hasshownthat if TV (s) ɛ, thenp s (y, x) c ɛ π(x) for all ertices x, wherec ɛ depends only on ɛ. Notes For an induced subgraph of a graph, we can define the Laplacian with boundary conditions. We will leae the definitions for eigenalues with Neumann boundary conditions and Dirichlet boundary conditions for Chapter??.

25 22 1. EIGENVALUES AND THE LAPLACIAN OF A GRAPH The Laplacian for a directed graph is also ery interesting. The Laplacian for a hypergraph has ery rich structures. Howeer, in this book we mainly focus on the Laplacian of a graph since the theory on these generalizations and extensions is still being deeloped. In some cases, the factor log ol G min x d x in the upper bound for (t) canbefurther reduced. Recently, P. Diaconis and L. Saloff-Coste [100] introduced a discrete ersion of the logarithmic Sobole inequalities which can reduce this factor further for certain graphs (for (t)). In Chapter 12, we will discuss some adanced techniques for further bounding the conergence rate under the relatie pointwise distance.