Expander and Derandomization

Transcription

1 Expander and Derandomization

2 Many derandomization results are based on the assumption that certain random/hard objects exist. Some unconditional derandomization can be achieved using explicit constructions of pseduorandom objects. Computational Complexity, by Fu Yuxi Expander and Derandomization 1 / 92

3 Synopsis 1. Basic Linear Algebra 2. Random Walk 3. Expander Graph 4. Explicit Construction of Expander Graph 5. Reingold s Theorem Computational Complexity, by Fu Yuxi Expander and Derandomization 2 / 92

4 Basic Linear Algebra Computational Complexity, by Fu Yuxi Expander and Derandomization 3 / 92

5 Three Views All capital lower case letters will denote column vectors. Matrix = Linear transformation : Q n Q m 1. f (u + v) = f (u) + f (v), f (cu) = cf (u) 2. the matrix corresponding to f has f (e j ) i as the (i, j)-th entry Interpretation of v = Au 1. Dynamic view: u is transformed to v, movement in one basis 2. Static view: u in the column basis is the same as v in the standard basis, one point in two bases Equation, Geometry (row picture), Algebra (column picture) Linear equation, hyperplane, linear combination Computational Complexity, by Fu Yuxi Expander and Derandomization 4 / 92

6 Inner Product, Projection, Orthogonality 1. Inner product u v measures the degree of colinearity of u, v u = u u where u is the conjugate transpose of u is the normalization of u u u cos θ = u v u v u v u u u is the projection of v onto u u and v are orthogonal if u v = 0 2. Row space null space, column space left null space 3. Basis, orthogonal/orthonormal basis 4. Orthogonal matrix Q 1 = Q Gram-Schmidt orthogonalization, A = QR Cauchy-Schwartz Inequality. cos θ = u v u v 1. Computational Complexity, by Fu Yuxi Expander and Derandomization 5 / 92

7 Fixpoints for Linear Transformation We look for fixpoints of a linear transformation A : R n R n. Av i = λ i v i. If there are n linear independent fixpoints v 1,..., v n, every v R n is some linear combination c 1 v c n v n. By linearity, Av = c 1 Av c n Av n = c 1 λ 1 v c n λ n v n. If we think of v 1,..., v n as a basis, the effect of the transform A is to stretch the coordinates in the directions of the axes. Computational Complexity, by Fu Yuxi Expander and Derandomization 6 / 92

8 Eigenvalue, Eigenvector, Eigenmatrix If A λi is singular, an eigenvector x is such thst Ax=λx, and λ is the eigenvalue. 1. S = [x 1,..., x n ] is the eigenmatrix. By definition AS = SΛ. 2. If λ 1,..., λ n are different, x 1,..., x n are linearly independent. 3. If x 1,..., x n are linearly independent, then A = SΛS 1. We shall write the spectrum λ 1, λ 2,..., λ n of a matrix in the order that λ 1 λ 2... λ n. The value ρ(a) = λ 1 is called spectral radius. Computational Complexity, by Fu Yuxi Expander and Derandomization 7 / 92

9 Hermitian Matrix and Symmetric Matrix real matrix complex matrix length x = i [n] x 2 i x = i [n] x i 2 transpose A A inner product x y = i [n] x iy i x y = i [n] x iy i orthogonality x y = 0 x y = 0 symmetric/hermitian A = A A = A diagonalization A = QΛQ A = UΛU orthogonal/unitary Q Q = I U U = I Fact. If A = A, then x Ax = (x Ax) is real for all complex x. Fact. If A = A, the eigenvalues are real since v Av = λv v = λ v 2. Fact. If A = A, the eigenvectors of different eigenvalues are orthogonal. Fact. Ux 2 = x 2 and Qx 2 = x 2. Computational Complexity, by Fu Yuxi Expander and Derandomization 8 / 92

10 Similarity Transformation Similarity Transformation = Change of Basis 1. A is similar to B if A = MBM 1 for some invertible M. 2. v is an eigenvector of A iff M 1 v is an eigenvector of B. A and B describe the same transformation using different bases. 1. The basis of B consists of the column vectors of M. 2. A vector x in the basis of A is transformed into the vector M 1 x in the basis of B, that is x = M(M 1 x). 3. B then transforms M 1 x into some y in the basis of B. 4. In the basis of A the vector Ax is My. Fact. Similar matrices have the same eigenvalues. Computational Complexity, by Fu Yuxi Expander and Derandomization 9 / 92

11 Triangularization Diagonalization transformation is a special case of similarity transformation. In diagonalization Q provides an orthogonal basis. Question. Is every matrix similar to a diagonal matrix? Schur s Lemma. For each matrix A there is a unitary matrix U such that T = U 1 AU is triangular. The eigenvalues of A appear in the diagonal of T. If A is Hermitian, T must be diagonal. Computational Complexity, by Fu Yuxi Expander and Derandomization 10 / 92

12 Spectral Theorem Theorem. Every Hermitian matrix A can be diagonalized by a unitary matrix U. Every symmetric matrix A can be diagonalized by an orthogonal matrix Q. U AU = Λ, Q AQ = Λ. The eigenvalues are in Λ; the orthonormal eigenvectors are in Q/U. Corollary. Every Hermitian matrix A has a spectral decomposition. A = UΛU = λ 1 v 1 v λ nv n v n. Computational Complexity, by Fu Yuxi Expander and Derandomization 11 / 92

13 Diagonalization We still need to answer the Question. What are the matrices that are similar to diagonal matrices? A matrix N is normal if NN = N N. (A normal A Hermitian) Theorem. A matrix N is normal iff T = U 1 NU is diagonal iff N has a complete set of orthonormal eigenvectors. Proof. If N is normal, T is normal. It follows from T = T that T is diagonal. If T is diagonal, it is the eigenvalue matrix of N, and NU = UT says that the column vectors of U are precisely the eigenvectors. Computational Complexity, by Fu Yuxi Expander and Derandomization 12 / 92

14 Jordan Form What if a matrix is not diagonalizable? Every matrix is similar to a Jordan Form. λ i 1 1 J = M 1 AM = J, where J i = λ i 1. J s λ i Here s is the number of independent eigenvectors. The same eigenvalue λ i will appear in several Jordan blocks if it has several independent eigenvectors. Theorem. A, B are similar iff they have the same Jordan Form. Theorem. If an n n matrix A is of rank r, then it has r nonzero eigenvalues and n r zero eigenvalues. Computational Complexity, by Fu Yuxi Expander and Derandomization 13 / 92

15 Rayleigh Quotient Suppose A is an n n Hermitian matrix, (λ 1, v 1 ),..., (λ n, v n ) are the eigenpairs. The Rayleigh quotient of A and nonzero x is defined as follows: R(A, x) = x Ax i [n] x x = λ i v i x 2. (1) i [n] v i x 2 It is clear from (1) that if λ 1... λ n, then λ i = max x v1,...,x v i 1 R(A, x), and if λ 1... λ n, then λ i = max x v1,...,x v i 1 R(A, x). One can use Rayleigh quotient to derive lower bound for λ i. Computational Complexity, by Fu Yuxi Expander and Derandomization 14 / 92

16 Positive Definite Matrix A symmetric matrix A is positive definite if x Ax > 0 for all x 0. Theorem. Suppose A is symmetric. The following are equivalent. 1. x Ax > 0 for all x λ i > 0 for all the eigenvalues λ i. 3. A i > 0 for all the upper left sub-matrices A i. 4. d i > 0 for all the pivots d i. 5. A = R R for some matrix R with independent columns. If we replace > by, we get positive semidefinite matrices. Computational Complexity, by Fu Yuxi Expander and Derandomization 15 / 92

17 Singular Value Decomposition Consider an m n matrix. Both AA and A A are symmetric. 1. AA is positive semidefinite since x AA x = A x AA = UΣU, where U consists of the orthonormal eigenvectors u 1,..., u m and Σ is the diagonal matrix made up from the eigenvalues σ 2 1,..., σ2 r. We assume σ 1... σ r. 3. Similarly A A = V Σ V. 4. Now AA u i = σ 2 i u i implies that σ 2 i is an eigenvalue of A A and A u i is the corresponding eigenvector. So v i = 5. Observe that u i AA u i = u i σ2 i u i = σ 2 i. 6. Hence Av i = A A u i A u i = σ2 i u i σ i = σ i u i. Conclude AV = UΣ. A u i A u i. Computational Complexity, by Fu Yuxi Expander and Derandomization 16 / 92

18 Singular Value Decomposition 1. σ 1,..., σ r are called the singular values of A. 2. A = UΣV is the singular value decomposition, or SVD, of A. Lemma. If A is normal, then σ i = λ i for all i [n]. Proof. Since A is normal, A = UΛU. Now A A = AA = UΛ 2 U. So the spectrum of A A/AA is λ 2 1,..., λ2 n. Computational Complexity, by Fu Yuxi Expander and Derandomization 17 / 92

19 Vector Norm The norm of a vector is a measure of its magnitude/size/length. A norm on F n is a function : F n R 0 satisfying the following: 1. v = 0 iff v = av = a v. 3. v + w v + w. A vector space with a norm is called a normed vector space. 1. L 1 -norm. v 1 = v v n. 2. L 2 -norm. v 2 = v v n 2 = v v. 3. L p -norm. v p = p v 1 p v n p. 4. L -norm. v = max{ v 1,..., v n }. Computational Complexity, by Fu Yuxi Expander and Derandomization 18 / 92

20 Matrix Norm We define matrix norm in compatible with vector norm. Suppose F n is a normed vector space over filed F. An induced matrix norm is a function : F n n R + satisfying the following properties. 1. A = 0 iff A = aa = a A. 3. A + B A + B. 4. AB A B. Computational Complexity, by Fu Yuxi Expander and Derandomization 19 / 92

21 Matrix Norm A matrix norm measures the amplifying power of a matrix. Define A = max v 0 Av v. It satisfies (1-4). Additionally Ax A x for all x. Lemma. ρ(a) A. A 1 = max 1 j n A = max 1 i n n A i,j, i=1 n A i,j. j=1 Computational Complexity, by Fu Yuxi Expander and Derandomization 20 / 92

22 Spectral Norm A 2 is called the spectral norm of A. 1 n A 1 A 2 n A 1. Lemma. A 2 = σ 1. Corollary. If A is a normal matrix, then A 2 = λ 1. Let A A = V ΣV be the decomposition, let v 1,..., v n be the orthonormal eigenvectors, and let x = a 1 v a n v n. Then Ax 2 2 = x (A Ax) = x ( σi 2 a i v i ) σ1 x i [n] The equality holds when x = v 1. Therefore A 2 = σ 1. Computational Complexity, by Fu Yuxi Expander and Derandomization 21 / 92

23 MIT Open Course Computational Complexity, by Fu Yuxi Expander and Derandomization 22 / 92

24 Random Walk Computational Complexity, by Fu Yuxi Expander and Derandomization 23 / 92

25 Graphs are the prime objects of study in combinatorics. The matrix representation of graphs lends itself to an algebraic treatment to these combinatorial objects. It is especially effective in the treatment of regular graph. Computational Complexity, by Fu Yuxi Expander and Derandomization 24 / 92

26 Our digraph admit both self-loops and parallel edges. An undirected edge is seen as two directed edges in opposite directions. In this chapter whenever we say graph, we mean undirected graph. Computational Complexity, by Fu Yuxi Expander and Derandomization 25 / 92

27 Random Walk Matrix The reachability matrix M of a digraph G is defined by M i,j = 1 if there is an edge from vertex j to vertex i; M i,j = 0 otherwise. The random walk matrix A of a d-regular digraph G is 1 d M. Let p be a probability distribution over the vertices of G and A is the random walk matrix of G. Then A k p is the distribution after k-step random walk. Computational Complexity, by Fu Yuxi Expander and Derandomization 26 / 92

28 Random Walk Matrix Consider the following periodic graph with d n vertices. The vertices are arranged in n layers, each consisting of d vertices. There is an edge from every vertex in the i-th layer to every vertex in the j-th layer, where j = i + 1 mod n. Does A k p converge to a stationary state? Computational Complexity, by Fu Yuxi Expander and Derandomization 27 / 92

29 Spectral Graph Theory In spectral graph theory graph properties are characterized by graph spectrums. Suppose G is a d-regular graph and A is the random walk matrix of G is an eigenvalue of A and its associated eigenvector is the stationary distribution vector 1 = ( 1 n,..., 1 n ). In other words A1 = All eigenvalues have absolute values G is disconnected iff 1 is an eigenvalue of multiplicity If G is connected, G is bipartite iff 1 is an eigenvalue of A. 3( ). 4( ). Computational Complexity, by Fu Yuxi Expander and Derandomization 28 / 92

30 Rate of Convergence For a regular graph G with random walk matrix A, we define def Ap 1 2 λ G = max p p 1 2 = max v 1 where p is over all probability distribution vectors. The two definitions are equivalent. 1. (p 1) 1 and Ap 1 = A(p 1). Av 2 v 2 = max v 1, v 2 =1 Av 2, 2. For each v 1, p = αv + 1 is a probability distribution for a sufficiently small α. By definition Av 2 λ G v 2 for all v. Computational Complexity, by Fu Yuxi Expander and Derandomization 29 / 92

31 Lemma. λ G = λ 2. Let v 2,..., v n be the eigenvectors corresponding to λ 2,..., λ n. Given x 1, let x = c 2 v c n v n. Then Ax 2 2 = λ 2 c 2 v λ n c n v n 2 2 = λ 2 2c 2 2 v λ 2 nc 2 n v n 2 2 λ 2 2(c 2 2 v c 2 n v n 2 2) = λ 2 2 x 2 2. So λ 2 G λ2 2. The equality holds since Av = λ2 2 v Computational Complexity, by Fu Yuxi Expander and Derandomization 30 / 92

32 The spectral gap γ G of a graph G is defined by γ G = 1 λ G. A graph has spectral expansion γ, where γ (0, 1), if γ G γ. In an expander G, γ G provides a bound on the expansion ratio. In terms of random walk, λ G bounds the speed of mixing time. Computational Complexity, by Fu Yuxi Expander and Derandomization 31 / 92

33 Lemma. Let G be an n-vertex regular graph and p a probability distribution over the vertices of G. Then A l p 1 2 λ l G p 1 2 < λ l G. The first inequality holds because The second inequality holds because A l p 1 2 p 1 2 = Al p 1 2 A l 1 p Ap 1 2 p 1 2 λ l G. p = p p, n 2 1 n < 1. Computational Complexity, by Fu Yuxi Expander and Derandomization 32 / 92

34 Lemma. If G is an n-vertex regular graph with self-loops at each vertex, γ G 1 12n 2. Let u be the unit vector such that u 1 and λ G = Au 2, and let v = Au. If we can prove 1 v n 2, we will get λ G = v n 2, hence the lemma. It s easy to show 1 v 2 2 = u 2 2 v 2 2 = u Au, v + v 2 2 = i,j A i,j(u i v j ) 2. By the assumption u i u j 1 n for some i, j [n]. Let u i u i1... u ik u j be a shortest path. Then u i u j = (u i v i ) + (v i u i1 ) (v ik u j ) u i v i + v i u i v ik u j (u i v i ) 2 + (v i u i1 ) (v ik u j ) 2 2D + 1, (2) where D is the diameter of G. Thus i,j A i,j(u i v j ) 2 1 dn(2d+1) by (2) and A h,h, A h,h+1 1 d. The inequality then follows from D 3n d+1. Computational Complexity, by Fu Yuxi Expander and Derandomization 33 / 92

35 Randomized Algorithm for Undirected Connectivity Corollary. Let G be a d-degree n-vertex graph with self-loop on every vertex. Let s, t be connected. Let l > 24n 2 log n and let X l denote the vertex distribution after l step random walk from s. Then Pr[X l = t] > 1 2n. Graphs with self-loops are not bipartite. According to the Lemmas, A l e s 1 2 < It follows that ( A l e s )t 1 n > 1 n 2. ( 1 1 ) 24n 2 log n 12n 2 < 1 n 2. If the walk is repeated for 2n 2 times, the error probability is reduced to below 1 2 n. Computational Complexity, by Fu Yuxi Expander and Derandomization 34 / 92

36 Randomized Algorithm for Undirected Connectivity Theorem. UPATH (Undirected Connectivity) is in RL. An undirected graph can be turned into a non-bipartite regular graph by introducing enough self-loops. Computational Complexity, by Fu Yuxi Expander and Derandomization 35 / 92

37 Can the random algorithm for UPATH be derandomized? Recall that L RL NL. Computational Complexity, by Fu Yuxi Expander and Derandomization 36 / 92

38 Expander Graph Computational Complexity, by Fu Yuxi Expander and Derandomization 37 / 92

39 Expansion graphs, defined by Pinsker in 1973, are sparse and well connected. They behave approximately like complete graphs. Sparsity should be understood in an asymptotic sense. 1. Fan Chung. Spectral Graph Theory. American Mathematical Society, Hoory, Linial, and Wigderson. Expander Graphs and their Applications. Bulletin of the AMS, 43, , Computational Complexity, by Fu Yuxi Expander and Derandomization 38 / 92

40 Well-connectedness can be characterized in a number of manners. 1. Algebraically, expanders are graphs whose second largest eigenvalue is bounded away from 1 by a constant. 2. Combinatorially, expanders are highly connected. Every set of vertices of an expander has a large boundary geometrically. 3. Probabilistically, expanders are graphs in which a random walk converges to the stationary distribution quickly. Computational Complexity, by Fu Yuxi Expander and Derandomization 39 / 92

41 Algebraic Property Intuitively the faster random walk converges, the better the graph is connected. According to Lemma, the smaller λ G is, the faster random walk converges to the uniform distribution. Suppose d N and λ (0, 1) are constants. A d-regular graph G with n vertices is an (n, d, λ)-graph if λ G λ. {G n } n N is an (d, λ)-expander graph family if G n is an (n, d, λ)-graph for all n N. Computational Complexity, by Fu Yuxi Expander and Derandomization 40 / 92

42 Probabilistic Property In expanders random walk converges to the uniform distribution in logarithmic steps. A 2 log 1 λ (n) p 1 2 < λ 2 log 1 λ (n) = 1 n 2. In other words, the mixing time of an expander is logarithmic. The mixing time of an n-vertex graph G is the minimal l such that for any vertex distribution p, A l p 1 < 1 2n, where A is the random walk matrix of G. The diameter of an n-vertex expander graph is Θ(log n). Computational Complexity, by Fu Yuxi Expander and Derandomization 41 / 92

43 Combinatorial Property Suppose G = (V, E) is an n-vertex d-regular graph. Let S stand for V \ S for S V. Let E(S, T ) be the set of edges i j with i S and j T. Let S = E(S, S) for S n 2. The expansion constant h G of G is defined as follows: h G = min S S S. Suppose ρ > 0 is a constant. An n-vertex d-regular graph G is an (n, d, ρ)-edge expander if h G ρd. In other words G is an (n, d, ρ)-edge expander if S ρ (d S ) for all S. Computational Complexity, by Fu Yuxi Expander and Derandomization 42 / 92

44 Existence of Expander Theorem. Let ɛ > 0. There exists d = d(ɛ) and N N such that for every n > N there exists an (n, d, 1 2 ɛ) edge expander. Computational Complexity, by Fu Yuxi Expander and Derandomization 43 / 92

45 Expansion and Spectral Gap Theorem. Let G = (V, E) be a finite, connected, d-regular graph. Then d γg 2 h G d 2γ G. 1. J. Dodziuk. Difference Equations, Isoperimetric Inequality and Transience of Certain Random Walks. Trans. AMS, N. Alon and V. Milman. λ 1, Isoperimetric Inequalities for Graphs, and Superconcentrators. J. Comb. Theory, N. Alon. Eigenvalues and Expanders. Combinatorica, Computational Complexity, by Fu Yuxi Expander and Derandomization 44 / 92

46 d 1 λ G 2 h G Let S be such that S n 2 { (S) S, i S, and S = h G. Define x 1 by x i = S, i S. x 2 2 = n S S, x Ax = ( S 1 S S 1 S ) A( S 1 S S 1 S ) = 1 ( S 2 E(S, S) + S 2 E(S, S) 2 S S E(S, S) ) d = 1 d ( dn S S n 2 E(S, S) ), where = is due to d S = E(S, S) + E(S, S) and E(S, S) + E(S, S) = d S. The Rayleigh quotient R(A, x) provides a lower bound for λ G. λ G x Ax x 2 2 = 1 dn S S n 2 E(S, S) d n S S = 1 1 d S (S) n 1 2h G d. Computational Complexity, by Fu Yuxi Expander and Derandomization 45 / 92

47 h G d 2(1 λ G ) Let u 1 be such that Au = λ 2 u. Write u = v + w, where v/w is defined from u by replacing the negative/positive components by 0. Wlog, assume that the number of positive components of v is n 2. Computational Complexity, by Fu Yuxi Expander and Derandomization 46 / 92

48 h G d 2(1 λ G ) Wlog, assume v 1 v 2... v n. Then A i,j vi 2 vj 2 = 2 i,j i<j n/2 (vk 2 vk+1) 2 A i,j j 1 k=i = 2 [k] (vk 2 v 2 d k+1) k=1 n/2 2 h G k(vk 2 v 2 d k+1) k=1 = 2h G d v 2 2, where the second equality holds because S n 2 and because for every fixed k the term vk 2 v2 k+1 appears once for every edge i j such that i k < j. Computational Complexity, by Fu Yuxi Expander and Derandomization 47 / 92

49 h G d 2(1 λ G ) Av, v > Av, v + Aw, v = λ 2 v 2 2 because Au = λ 2u, v, w = 0 and Aw, v < 0. 1 λ 2 1 Av, v v 2 2 = v 2 2 Av, v i,j v 2 = A i,j(v i v j ) v 2. (3) 2 Let brown = i,j A i,j(v i + v j ) 2. Using Cauchy-Schwartz Inequality, blue brown A i,j vi 2 vj 2. (4) i,j 2 Now Av 2 σ 1 v 2 = v 2 implies Av, v v 2 2. Therefore red brown 2 v 2 2 (2 v Av, v ) 8 v 4 2. (5) (3)+(4)+(5)+the previous inequality implies 8(1 λ 2 ) 2h G d. Computational Complexity, by Fu Yuxi Expander and Derandomization 48 / 92

50 Combinatorial definition and algebraic definition are equivalent. 1. The inequality d 1 λ G 2 h G implies that if G is an (n, d, λ)-expander graph, then it is an (n, d, 1 λ 2 ) edge expander. 2. The inequality h G d 2(1 λ G ) implies that if G is an (n, d, ρ) edge expander, then it is an (n, d, 1 ρ2 2 )-expander graph. Computational Complexity, by Fu Yuxi Expander and Derandomization 49 / 92

51 Convergence in Entropy Rényi 2-Entropy: H 2 (p) = 2 log( p 2 ). Fact. If A is the random walk matrix of an (n, d, λ)-expander, then H 2 (Ap) H 2 (p). The equality holds if and only if p is uniform. Proof. Let p = 1 + w such that w 1. Then Aw, 1 = w A 1 = w A1 = w 1 = 0. Therefore ( ) Ap 2 2 = Aw λ w 2 2 = λ( p 2 2) = γ p 2 + λ p 2 2 p The equality holds when p = 1. Random walks do not decrease randomness. Computational Complexity, by Fu Yuxi Expander and Derandomization 50 / 92

52 The smaller the spectral gap, or the larger the spectral expansion, the more expander graphs behave like random graphs. This is what the next lemma says. Computational Complexity, by Fu Yuxi Expander and Derandomization 51 / 92

53 Expander Mixing Lemma Lemma. Let G = (V, E) be an (n, d, λ)-expander graph. Let S, T V. Then E(S, T ) d n S T λd S T. (6) Notice that (6) implies E(S, T ) S T dn n n λ. (7) The edge density approximates the product of the vertex densities. 1. N. Alon and F. Chung. Explicit Construction of Linear Sized Tolerant Networks. Discrete Mathematics, Computational Complexity, by Fu Yuxi Expander and Derandomization 52 / 92

54 Proof of Expander Mixing Lemma Let [v 1,..., v n ] be the eigenmatrix of G with v 1 = ( 1 n,..., 1 n ). Let 1 S = i α iv i and 1 T = j β jv j be the characteristic vectors of S, T respectively. E(S, T ) = (1 S ) (da)1 T = ( i α i v i ) (da)( j β j v j ) = i dλ i α i β i. Since α 1 = (1 S ) v 1 = S n and β 1 = (1 T ) v 1 = T n, E(S, T ) d n S T = n dλ i α i β i dλ Finally observe that α 2 β 2 = 1 S 2 1 T 2 = S T. i=2 n α i β i dλ α 2 β 2. i=2 Computational Complexity, by Fu Yuxi Expander and Derandomization 53 / 92

55 Error Reduction for Random Algorithm Suppose A(x, r) is a random algorithm with error probability 1/3. The algorithm uses r(n) random bits on input x with x = n. 1. We can reduce the error probability exponentially by repeating the algorithm t(n) times. 2. The resulting algorithm uses r(n)t(n) random bits. The goal is to achieve the same error reduction rate using far fewer random bits (in fact r(n) + O(t(n)) random bits). The key observation is that a t-step random walk in an expander graph looks like t vertices sampled uniformly and independently. Confer the inequality (7). Computational Complexity, by Fu Yuxi Expander and Derandomization 54 / 92

56 A Decomposition Lemma for Random Walk K n is perfect from the viewpoint of random walk. No matter what distribution it starts with, a random walk reaches the uniform distribution in one step. Let J n = [1,..., 1] be the random walk matrix of K n with self-loop. Lemma. Suppose G is an (n, d, λ)-expander and A is its random walk matrix. Then A = γj n + λe for some E such that E 1. We may think of a random walk on an expander as a convex combination of two random walks of different type, a walk with probability γ on a complete graph, and a walk with probability λ according to an error matrix that does not amplify the distance to the uniform distribution. Computational Complexity, by Fu Yuxi Expander and Derandomization 55 / 92

57 A Decomposition Lemma for Random Walk We need to prove that Ev 2 v 2 for all v, where E is defined by E = 1 λ (A (1 λ)j n). The following proof methodology should now be familiar. Decompose v into v = u + w = α1 + w with w 1. A1 = 1 and J n 1 = 1. Consequently Eu = u. J n w = 0, w def = Aw 1 and w u. Hence Ew = 1 λ w. w 2 λ w 2. So Ev 2 2 = u + 1 λ w 2 2 = u λ w 2 2 u w 2 2 = v 2 2. Computational Complexity, by Fu Yuxi Expander and Derandomization 56 / 92

58 Expander Random Walk Theorem Theorem. Let G be an (n, d, λ) expander graph, and let B [n] satisfy B βn for some β (0, 1). Let X 1 be a random variable denoting the uniform distribution on [n] and let X k be a random variable denoting a k 1 step random walk from X 1. Then Pr X i B i [k] ( γ β + λ) k 1. Computational Complexity, by Fu Yuxi Expander and Derandomization 57 / 92

59 Expander Random Walk Theorem Let B i stand for X i B. We need to bound the following. Pr X i B = Pr[B 1 ] Pr[B 2 B 1 ]... Pr[B k B 1... B k 1 ]. (8) i [k] By seeing B as a diagonal matrix, we define the distribution p i by p i = BA Pr[B i B 1... B i 1 ]... BA Pr[B 2 B 1 ] B1 Pr[B 1 ]. So the probability in (8) is bounded by (BA) k 1 B1 1. We ll prove (BA) k 1 B1 2 1 n ((1 λ) β + λ) k 1. Computational Complexity, by Fu Yuxi Expander and Derandomization 58 / 92

60 Expander Random Walk Theorem Using Lemma, Therefore BA = B((1 λ)j n + λe) (1 λ) BJ n + λ BE (BA) k 1 B1 2 = (1 λ) β + λ BE (1 λ) β + λ B E (1 λ) β + λ. β ((1 λ) ) k 1 1 β + λ n ((1 λ) k 1 β + λ). n BJ n v 2 = BJ n α1 2 = α B1 2 n B1 2 = n β n = β for v 2 = 1. Therefore BJ n = max{ BJ n v 2 v 2 = 1} = β. Computational Complexity, by Fu Yuxi Expander and Derandomization 59 / 92

61 Error Reduction for RP Suppose A(x, r) is a random algorithm with error probability β. Given input x with n = x, let k = r(n). Choose an explicit (2 k, d, λ)-graph G = (V, E) with V = {0, 1} k. Algorithm A. 1. Pick v 0 R V. 2. Generate a random walk v 0,..., v t. 3. Output t i=0 A(x, v i). By the Theorem, the error probability of A is ( γ β + λ ) k 1. Here B is the set of r s for which A errs on x. Computational Complexity, by Fu Yuxi Expander and Derandomization 60 / 92

62 Error Reduction for BPP Algorithm A. 1. Pick v 0 R V. 2. Generate a random walk v 0,..., v t. 3. Output Maj{A(x, v i )} i [t]. Let K [t] be the set of samples for which A errs and K t+1 2. Pr[ i K.v i B] assuming γ β + λ 1/16. By union bound, ( γ ) t 1 2 β + λ ( ) 1 t 1, 4 Pr[A fails] 2 t ( 1 4) t 1 = O(2 t ). Computational Complexity, by Fu Yuxi Expander and Derandomization 61 / 92

63 Explicit Construction of Expander Graph Computational Complexity, by Fu Yuxi Expander and Derandomization 62 / 92

64 Explicit Construction In some applications the size of expander graph can be handled. An expander family {G n } n N is mildly explicit if there is a P-time algorithm that outputs the random walk matrix of G n whenever the input is 1 n. In some other applications a huge expander graph must be used. An expander family {G n } n N is strongly explicit if there is a P-time algorithm that on input n, v, i outputs the index of the i-th neighbor of v. Computational Complexity, by Fu Yuxi Expander and Derandomization 63 / 92

65 We will look at several graph product operations. We then show how to use these operations to construct explicit expander graphs. 1. O. Reingold, S. Vadhan, and A. Wigderson. Entropy Waves, the Zig-Zag Graph Product, and New Constant-Degree Expanders and Extractors. FOCS, Computational Complexity, by Fu Yuxi Expander and Derandomization 64 / 92

66 Rotation Map Suppose G is an n-vertex d-degree graph. A rotation map Ĝ of G is a function of type [n] [d] [n] [d] satisfying the following. Ĝ(u, i) = (v, j) if vertex v is the i-th neighbor of vertex u and u is the j-th neighbor of v. Clearly Ĝ is a permutation. The rotation matrix  is defined by  (v,m),(u,l) = { 1, if Ĝ(u, l) = (v, m), 0, otherwise. Walks described in  are deterministic, meaning that no random bits are necessary. Computational Complexity, by Fu Yuxi Expander and Derandomization 65 / 92

67 Path Product Suppose G, G are n-vertex graphs with degree d respectively d. Let A, A be their random walk matrices. The path product G G is defined by the random walk matrix A A. G G is n-vertex dd -degree. Lemma. λ G G λ G λ G. Proof. A λ G G = max Av 2 A v 1 v 2 = max v 2 v 1 v 2 Av 2 A v 2 max Av 2 Av v 1 Av 2 max 2 v 1 λ G λ G using the fact that Av 1 whenever v 1. Lemma. λ G k = (λ G ) k. Proof. (λ G ) k is the second largest eigenvalue of G k. v 2 = Computational Complexity, by Fu Yuxi Expander and Derandomization 66 / 92

68 Tensor Product Suppose G is an n-vertex d-degree graph and G is an n -vertex d -degree graph. The random walk matrix of the tensor product G G is a 11 A a 12 A a 1n A A A a 21 A a 22 A a 2n A =.... a n1 A a n2 A a nn A (u, u ) (v, v ) in G G iff u v in G and u v in G. G G is nn -vertex dd -degree. Computational Complexity, by Fu Yuxi Expander and Derandomization 67 / 92

69 Tensor Product Lemma. λ G G = max{λ G, λ G }. If λ is an eigenvalue of A and v is the associated eigenvector, and λ is an eigenvalue of A and v is the associated eigenvector, then (A A )(v v ) = λλ (v v ). Computational Complexity, by Fu Yuxi Expander and Derandomization 68 / 92

70 Zig-Zag Product G is an n-vertex D-degree graph. H is a D-vertex d-degree graph. The zig-zag product G z H is the nd-vertex d 2 -degree graph constructed as follows: 1. The vertex set is [n] [D]. 2. For i, j [d] and (u, l) [n] [D], the (i, j)-th neighbor of (u, l) is the vertex (v, m) [n] [D] computed as follows: 2.1 Let l be the i-th neighbor of l in H. 2.2 v is the l -th neighbor of u and u is the m -th neighbor of v. 2.3 Let m be the j-th neighbor of m in H. Typically d D. A t-step random walk uses O(t log d) rather than O(t log D) random bits. Computational Complexity, by Fu Yuxi Expander and Derandomization 69 / 92

71 Zig-Zag Product We need a picture here. Computational Complexity, by Fu Yuxi Expander and Derandomization 70 / 92

72 Zig-Zag Product Lemma. (I n J D )Â(I n J D ) = A J D. Consider the left matrix. Starting from (u, l), a random neighbor l of l is chosen with probability 1 D, then a walk from (u, l ) to some (v, m ) with probability 1; and finally a random neighbor m of m is picked up with probability 1 D. According to the right matrix the random walk from (u, l) to (v, m) occurs with the probability 1 D 1 D. Â is the rotation matrix of G, and I n is the n n diagonal matrix. Computational Complexity, by Fu Yuxi Expander and Derandomization 71 / 92

73 Claim. If C 2 1 then λ C 1. Proof. λ C = max v 1 Cv 2 v 2 max v 1 C 2 v 2 v 2 1. Claim. λ A+B λ A + λ B for symmetric stochastic matrices A, B. Proof. λ A+B = max v 1 (A+B)v 2 v 2 max v 1 Av 2 + Bv 2 v 2 λ A + λ B. Computational Complexity, by Fu Yuxi Expander and Derandomization 72 / 92

74 Zig-Zag Product Lemma. λ G z H λ G + 2λ H and γ M γ G γh 2. Let A, B and M be the random walk matrices of G, H and G z H. Â is the (nd) (nd) rotation matrix of G. B = (1 λ H )J D + λ H E for some E with E 2 1. This is Lemma. Now M = (I n B)Â(I n B) = ((1 λ H )I n J D + λ H I n E) Â ((1 λ H)I n J D + λ H I n E) = (1 λ H ) 2 (I n J D )Â(I n J D ) +... = (1 λ H ) 2 (A J D ) +..., where = is due to Lemma. Using Lemma and the Claims, one gets λ M (1 λ H ) 2 λ A JD + 1 (1 λ H ) 2 max{λ G, λ JD } + 2λ H = λ G + 2λ H. Computational Complexity, by Fu Yuxi Expander and Derandomization 73 / 92

75 Zig-Zag Product The lemma is useful when both λ G and λ H are small. If not, a different upper bound can be derived. Both upper bounds are discussed in the following paper. 1. O. Reingold, S. Vadhan, and A. Wigderson. Entropy Waves, the Zig-Zag Graph Product, and New Constant Degree Expanders and Extractors. FOCS, Computational Complexity, by Fu Yuxi Expander and Derandomization 74 / 92

76 Expander Construction I The crucial point of the zig-zag construction is that we can use a constant graph to build a constant degree graph family. Let H be a (D 4, D, 1/8)-graph constructed by brute force. Define Fact. G k is a (D 4k, D 2, 1/2)-graph. G 1 = H 2, G k+1 = Gk 2 z H. Proof. The base case is clear from Lemma, and the induction step is taken care of by the previous lemma. Computational Complexity, by Fu Yuxi Expander and Derandomization 75 / 92

77 Expander Construction I The time to access to a neighbor of a vertex is given by the following recursive equation time(g k+1 ) = 2 time(g k ) + poly( vertex ) = poly( G k+1 ). The expander family is mildly explicit but not strongly explicit. Computational Complexity, by Fu Yuxi Expander and Derandomization 76 / 92

78 Expander Construction I Both the size of the graph and the time to compute a neighbor grow exponentially. This suggests to use tensor product to expand the size of graph doubly exponential. We will explain the idea using a variant of zig-zag product. Computational Complexity, by Fu Yuxi Expander and Derandomization 77 / 92

79 Replacement Product G is an n-vertex D-degree graph. H is a D-vertex d-degree graph. The replacement product G RH is the nd-vertex 2d-degree graph constructed in the following manner: 1. Every vertex w of G is replaced by a copy H w of H. 2. If Ĝ(u, l) = (v, m), place d parallel edges from the l-th vertex of H u to the m-th vertex of H v. The replacement product is well known in graph theory. It is often used to reduce vertex degree without loosing connectivity. Computational Complexity, by Fu Yuxi Expander and Derandomization 78 / 92

80 Replacement Product We need a picture here. Computational Complexity, by Fu Yuxi Expander and Derandomization 79 / 92

81 Replacement Product The rotation map on [n] [D] [d] {0, 1} is defined by { (u, Ĥ(m, i), b), if b = 0, ĜRH(u, m, i, b) = (Ĝ(u, m), i, b), if b = 1. Computational Complexity, by Fu Yuxi Expander and Derandomization 80 / 92

82 Replacement Product Lemma. λ GRH 1 (1 λ G )(1 λ H ) 2 24 and γ GRH 1 24 γ G γ 2 H. Let A, B be the random walk matrices of G, H respectively. Â = the (nd) (nd) permutation matrix corresponding to Ĝ. By Lemma, B = λ H E + γ H J D for some E with E 2 1. The (nd) (nd) random walk matrix of GRH is ARB = 1 2Â (I n B). It suffices to prove λ 3 ARB 1 γ G γ 2 H 8, which is λ (ARB) 3 1 γ G γ 2 H 8. Computational Complexity, by Fu Yuxi Expander and Derandomization 81 / 92

83 Replacement Product We have (ARB) 3 = = ( 1 2Â + 1 ) 3 2 (I n B) ( 1 2Â (I n (λ H E + γ H J D )) ) 3 ) 3 = 1 (Â + λh (I n E) + γ H (I n J D ) 8 = 1 (Â3 ) γ 2 8 H(I n J D )Â(I n J D ) = 1 (Â3 ) γ 2 8 H(A J D ), where the last equality is due to Lemma. Applying the two Claims, we get λ (ARB) 3 1 γ2 H 8 + γ2 H 8 λ A J D 1 γ2 H 8 + γ2 H 8 λ G = 1 γ2 H 8 γ G. Computational Complexity, by Fu Yuxi Expander and Derandomization 82 / 92

84 Expander Construction II Theorem. There exists a strongly explicit (4, λ)-expander family for some λ < 1. As a first step we prove that we can efficiently construct a family {G k } k of graphs where each G k has (2d) 100k vertices. 1. Let H be a ((2d) 100, d, 0.01)-expander graph, G 1 a ((2d) 100, 2d, 0.5)-expander graph, and G 2 a ((2d) 100 2, 2d, 0.5)-expander graph. 2. For k > 2 define G k = ( G k 1 2 G k 1 2 ) 50 RH. Tensor product increases graph size. Path product improves spectral expansion. Replacement product reduces degree. Computational Complexity, by Fu Yuxi Expander and Derandomization 83 / 92

85 Expander Construction II G k is a ((2d) 100k, 2d, 0.98)-expander graph. 1. Let n k be the number of vertices of G k. n k k 1 = n k 1 n 100 k (2d)100 = (2d) 2 k (2d) 2 (2d) 100 = (2d) 100k. 2. G k 1, G k 1 degree 2d G k 1 G k 1 degree (2d) 2 (G k 1 G k 1 ) 50 degree (2d) 100 G k degree 2d. 3. λ G k 1, λ G k λ G k 1 G k λ (G k 1 G k 1 ) λ Gk 1 0.5(0.99) 2 /24 < Computational Complexity, by Fu Yuxi Expander and Derandomization 84 / 92

86 Expander Construction II There is a poly(k)-time algorithm that upon receiving a label i of a vertex in G k and an index j in [2d] finds the j-th neighbor of i. 1. A search for a neighborhood of the input vertex of G k recursively calls 50 times on G k 1 and 50 times on G k The depth of the recursive calls is bounded by t = O(log k). 3. The overall time is bounded by O(2 O(log k) ) = poly(k). Computational Complexity, by Fu Yuxi Expander and Derandomization 85 / 92

87 Expander Construction II Suppose (2d) 100k < i < (2d) 100(k+1). Let (2d) 100(k+1) = xi + r. Divide the (2d) 100(k+1) vertices into i classes among which r classes being of size x + 1 and i r classes being of size x. Contract every class into a mega-vertex. Add 2d self-loops to each of the i r mega-vertices. This is a (i, 2d(x + 1), (2d)0.01/(x + 1)) edge expander. We get a ((2d) 101, 0.01/(2d) 99 ) edge expander family. Computational Complexity, by Fu Yuxi Expander and Derandomization 86 / 92

88 Reingold s Theorem Computational Complexity, by Fu Yuxi Expander and Derandomization 87 / 92

89 Theorem. UPATH L. 1. O. Reingold. Undirected ST-Connectivity in Log-Space. STOC Computational Complexity, by Fu Yuxi Expander and Derandomization 88 / 92

90 The Idea Connectivity Algorithm for d-degree expander graph is easy. The diameter of an expander graph is of length O(log(n)). An exhaustive search can be carried out in logspace. Reingold s idea is to transform conceptually a graph G to a graph G so that a connected component in G becomes an expander in G and unconnected vertices in G remain unconnected in G. Moreover finding a neighbor of a given vertex in the conceptual G can be done in O(log G ) space. Computational Complexity, by Fu Yuxi Expander and Derandomization 89 / 92

91 The Algorithm 1. Fix a (d 50, d/2, 0.01)-expander graph H for d = Convert the input graph G to a d 50 -degree graph on the fly. 2.1 Add self-loops to increase degree. 2.2 Replace a large degree vertex by a cycle to decrease degree. 3. G 0 = G; G k = (G k 1 RH) 50 is constructed on the fly. 4. Apply Connectivity Algorithm to the expander G 10 log n. If G k 1 is an N-vertex d 50 -degree, G k 1 RH is an d 50 N-vertex d-degree, and (G k 1 RH) 50 is an d 50 N-vertex d 50 -degree. So G 10 log n contains (d 50 ) 10 log n n = n 1001 vertices. Computational Complexity, by Fu Yuxi Expander and Derandomization 90 / 92

92 The Complexity Only paths of length 2 log 1 n 1001 need be considered Each vertex, except s, t, is coded up by 50 log(d) bits, say x, declaring that it is the 2 x -th neighbor of the previous vertex. 2. The algorithm keeps the current vertex. When backtracking it starts from s all over again to get the previous vertex. Step 2 and Step 3 of the algorithm can be carried out on the fly using this mechanism. We cannot record all the vertices in a path. [log(n) 2 ] Although the Expander Construction I is only mildly explicit, computing the i-th neighbor of a given vertex is in logspace. Computational Complexity, by Fu Yuxi Expander and Derandomization 91 / 92

93 Lewis and Papadimitriou introduced SL as the class of problems solvable in logspace by an NTM that satisfies the following. 1. If the answer is yes, one or more computation paths accept. 2. If the answer is no, all paths reject. 3. If the machine can make a transition from configuration C to configuration D, then it can also goes from D to C. Theorem. UPATH is SL-complete. Corollary. UPATH is L-complete. Proof. Reingold Theorem implies that L = SL. Computational Complexity, by Fu Yuxi Expander and Derandomization 92 / 92