Lecture 9: Expaders Part, Extractors Topics i Complexity Theory ad Pseudoradomess Sprig 013 Rutgers Uiversity Swastik Kopparty Scribes: Jaso Perry, Joh Kim I this lecture, we will discuss further the pseudoradomess properties of expader graphs. The we will begi o the topic of extractors. 1 More results o expaders I the previous lecture we saw the defiitios of eigevalue expaders, edge expaders, ad vertex expaders for d-regular graphs. To recap, i a eigevalue expader, all except the first eigevalue of the graph s adjacecy matrix are bouded below d. I a edge expader, every subset of vertices of size below a certai threshold has a large umber of edges goig out of the subset. Ad i a vertex expader, every such subset of vertices has a large eighborhood. We will show that λ eigevalue expaders are edge ad vertex expaders. The coverse is also true: edge ad vertex expaders are eigevalue expaders. All these objects are strictly weaker tha absolute eigevalue expaders. For more theorems o the relatios betwee expader defiitios, see the lecture otes from the professor s graph theory course at http://math.rutgers.edu/ ~sk133/courses/graphtheory-f11/. 1.1 Eigevalue expaders are edge expaders Let es, T deote the umber of edges betwee subsets S ad T of vertices i a graph, ad let S C = V \S. Theorem 1. If GV, E is a d-regular λ-absolute eigevalue expader o vertices, the for all S V where S /, es, S C d λ S. Which is to say that G is a d λ S, / edge expader. As a aside, we might try to get this theorem from the expader mixig lemma, which bouds the umber of edges betwee ay two subsets of vertices i G. We ca cosider es, S, the umber of edges iteral to S, ad subtract it from the umber of outgoig edges, which has a upper boud of d S. So, by the expader mixig lemma: d S es, S λ S es, S d S + λ S. 1
The, d S es, S C d S + λ S d d S λ S d 1 S d λ S λ S We did ot reach our desired lower boud, but it s close if λ is much smaller tha d say d. We ow proceed with the proof proper. We use a similar idicator fuctio approach as i the proof of the expader mixig lemma. Proof. Let 1 S, 1 S C write be the 0/1 idicator vectors for vertex sets S ad S C, respectively. We ca es, S = 1 T S A1 S es, S C = 1 T S A1 S C = d S 1 T S A1 S. Now let s write 1 S i the eigebasis. Say v i are the eigevectors, ad we kow the followig facts: 1 S = α i v i α 1 = 1 S, v 1 = S αi = S. Now expad out the expressio for es, S: 1 T S A1 S = α i v i A α j v j = = j=1 j=1 α i α j vi T Av j αi λ i, sice oly the terms where i = j survive. We already kow i α i. We isolate the first term,
recallig that λ 1 = d, ad boud the rest: 1 T S A1 S S d + λ i= α i S S d + λ S d S S + λ 1 S Observe that this is a covex combiatio of d ad λ with weight S /. Let s relate this back to the expressio for outgoig edges: d S es, S C d + λ 1 S S = 1 S d λ S. Sice 1 S / is at least 1/, es, S C d λ S.. I fact, we could have gotte this result from the expader mixig lemma, by usig our precise value i α i = S S /, istead of upper-boudig it by S. 1. Aside: How good ca a eigevalue expader be? The ext result shows that a bouded degree graph caot be too good a eigevalue expader. I fact the same result holds for o-absolute eigevalue expasio but we will ot show that here. Lemma. For all d-regular λ-absolute eigevalue expaders, λ = Ω d. Proof. To show that the adjacecy matrix A of λ-absolute eigevalue expader G has a large secod eigevalue, we look at its trace. However, the trace, which is equal to the sum of the eigevalues, could be egative, so istead we take: TrA = λ i d + λ 1. Note that the etries of A, A ij = k A ika kj, are the umber of legth- paths from i to j i G. The trace of A is the sum over all vertices i of legth- paths from i to i. The oly such paths are those that go out ad come back alog oe edge, so the value of the trace i our d-regular graph is d. So d d + λ 1 λ Ω d. 3
1.3 Eigevalue expaders are vertex expaders Defie ΓS as the eighborhood of S i G. Let us take G to be a d-regular, λ-absolute eigevalue expader o vertices, ad see what lower boud we get o the size of ΓS. First, we eed to express ΓS algebraically. Agai takig the vector 1 S, ote that the etries of A 1 S are the umber of eighbors of each vertex i S. Therefore, SupportA 1 S = ΓS Our go-to trick for showig that support is large is Cauchy-Schwarz. SupportA 1 S A 1 S 1 A 1 S. For the L 1 orm, observe that each vertex cotributes 1 to each of its d eighbors: A 1 S 1 = d S ad So, A 1 S = α i λ i v i = αi λ i i S d + λ S S. d S ΓS = SupportA 1 S S d + λ S S = = d S 1 S 1 S, + λ 1 S d S d + λ which looks like a expasio factor of about d /λ. If S α, the expasio factor is S d αd + 1 αλ, which gives us 1+Ω1-expasio whe λ < d, α < 1. If α = o1 the this gives expasio early d, which caot be larger tha d λ 4 this is a sharper versio of the boud o absolute eigevalue expasio that we showed earlier. Later we will see costructios of expaders such that all small sets have vertex expasio early equal to d. 4
1.4 Expader Tricks Is there a way to make a eigevalue a expader a absolute eigevalue expader? Yes, by usig the somewhat dirty trick of addig lots of self-loops at every vertex. Trick 1. If G is a d-regular λ-eigevalue expader with adjacecy matrix A, the the graph give by A + di is a d-regular, d + λ-eigevalue expader. To see this, ote that the eigevalues of A + di are λ 1 + d, λ + d,..., λ + d. Sice the smallest eigevalue of A is at least d, A + di has o egative eigevalues. What we lose here is the ratio betwee λ ad d. It s more like d/ istead of the d we had origially. However, there is aother trick that amplifies that gap: Trick. The graph G k, give by adjacecy matrix A k, is a d k -regular, λ k -absolute eigevalue expader. This is the case sice the eigevalues of A k are λ k 1, λk,..., λk. 1.5 Cheroff bouds for expader walks We already saw i the last class that radom walks o expaders give good samplig properties. I fact, expader walk samplig obeys Cheroff-like bouds. Cosider GV, E, a d-regular λ-absolute eigevalue expader, ad a set S V. Pick x 0 V uiformly, ad let x 0, x 1,..., x D be a radom walk o G. Let X i be the idicator of whether vertex i is i S. the µ = S / V. The result, due to Gillma, is that [ ] # of i with x i S Pr µ x D > ɛ exp ɛ D 1 λ /4 α The proof of this will be developed i the homework. Extractors Extractors are also pseudoradom objects, ad seem to bear some similarities to expaders we have see, but their properties are differet. They address the questio, Where do we obtai uiform radom bits i the first place? Physical pheomea seem to be good sources of radomess, but they do ot provide radomess i the form of strigs of idepedet, uiformly distributed bits. So we desire a way to take umbers from a ukow distributio all that is kow is that it has sufficietly high etropy ad produce sequeces of uiform radom bits. This is the job of extractors. Usig extractors, we have the result due to Zuckerma that determiistic polyomial time algorithms ca simulate radomized algorithms usig just a weak source of radomess. I complexity laguage, P teacup = BP P. As a toy example of the applicatios of extractors, cosider a procedure for geeratig a radom -bit prime umber. The obvious thig to do is to geerate a radom -bit iteger, test it for 5
primality, ad repeat util success. By the prime umber theorem, we expect to have to do this about times, thus requirig about uiform radom bits. Is there ay way to decrease this? Oe way is that if, after a failure fidig that x is composite, we could somehow use the radomess left over i x. It has the etropy of a sample from the uiform distributio over all -bit composite itegers. This is what extractors will allow us to do with some limitatios..1 Mi Etropy Let X be a radom variable over some domai W. Defiitio 3. The mi etropy of X is defied by: I particular, ote that: H X = mi w W log 1 P rx = w. 1. H X k All elemets i W occur with probability at most k.. H U m = m where U m is the uiform distribuitio o m bits.. Data Processig Iequalities for Mi Etropy Let Y be a radom variable over a domai W. Let f be ay fuctio o W ad let g be ay fuctio o W W. The: 1. H fx H X. H gx, Y H X + H Y.3 A Wishful Extractor A ideal extractor would take ay radom variable with sufficietly large mi etropy to a uiform distributio. More precisely, Defiitio 4. A k-extractor is a fuctio f : 0, 1 0, 1 m such that for every X distributed o 0, 1 with H X k, fx is uiform o 0, 1 m. Ufortuately, k-extractors do ot exist. This is for trivial reasos: askig for exact uiformity turs out to be too much. For practical purposes, we do t eed extractors to map exactly to the uiform distributio. Perhaps there are extractors that take radom variables with large mi etropy to distributios that are close to uiform. 6
Defiitio 5. A k, ɛ-extractor is a fuctio f : 0, 1 0, 1 m such that for every X distributed o 0, 1 with H X k, fx is ɛ-close to uiform o 0, 1 m. such k, ɛ-extractors also do ot exist! To see this, take m = 1. The either f 1 0 or f 1 1 has size at least 1. WLOG, suppose it is f 1 0. Let X be the uiform distribuitio o f 1 0. The H X 1. However, fx=0, so the statistical distace from fx to U 1 is 1..4 Extractors ad Their Existece To get a useful extractor that actually exists, we eed to add a small umber of seed radom bits as a ivestmet to our extractor. Defiitio 6 the real thig. A k, ɛ-extractor is a fuctio f : 0, 1 0, 1 d 0, 1 m such that for every X distributed o 0, 1 with H X k, fx, Y is ɛ-close i L 1 to uiform o 0, 1 m, where Y = U d is a radom variable idepedet of X. The mai theorem is that k, ɛ-extractors exist. Theorem 7. Radom fuctios are good k, ɛ-extractors. I fact, they are amazig extractors. Oe ca take d = Olog + log 1 ɛ so there are oly about log bits of radomess used as seed ivestmet, ad, more surpisigly, we ca have m = d + k log 1 ɛ O1 so we get early as may output bits as imagiable: all the d bits of seed, ad early all the k bits of weak radomess i the weak radom source!. Directly applyig the probabilistic method the usual way is tricky: the umber of potetial weak radom sources X the extractor should work with is ifiite ad so it wo t support a uio boud across all weak radom sources. Thus, before startig with our use of the probabilistic method, we eed a few simple observatios. Observatio 8. Every radom variable X with H X k is a covex combiatio of uiform distributios over sets of size k requires k Z. Observatio 9. If fx i, Y is ɛ-close to uiform i ad X is a covex combiatio of the X i, the fx, Y is ɛ-close to uiform. Proof. Write X = i α i X i where i α i = 1. The fx, Y = i α i fx i, Y. So we have: 7
fx, Y U m = α i fx i, Y U m i i α i fx i, Y U m i α i ɛ = ɛ. Proof. of Theorem 7 For simplicity of otatio, we write N =, K = k, D = d, M = m. Let X be ay uiform distributio o some set of size K. It follows from the two observatios that it suffices to prove that for a radom fuctio f, fx, Y is ɛ-close to U m i L 1 with high probability. If fx, Y is ot ɛ-close to U m the S {0, 1} m such that Pr [fx, Y S] Pr[U m S] ɛ. X,Y So for a radom fuctio f, we defie bad evets B S = {f : Pr X,Y [fx, Y S] Pr[U m S] ɛ}. For fixed S {0, 1} m, the Cheroff Boud gives: Pr f B S e ɛ KD. Takig the uio boud over all N K choices of X ad M choices of S, we have: Pr[f is ot a k, ɛ-extractor] Pr[ X, S such that B S occurs] f N M e ɛ KD K en K M e ɛ KD K = c+k log N K +M cɛ KD where c = log e. We eed this probability to be less tha 1 for a radom f to be a k, ɛ-extractor. That is, we require: c + K log N K + M cɛ KD < 0. 8
This may be writte as: D > 1 cɛ log N K + M + c cɛ K, which is satisfied if each of the two parts of the sum is less tha D/. Thus, as log as both d > log k + log 1 ɛ + O1 d > m k + log 1 ɛ + O1 are satisfied, the f is a good extractor with positive probability. Choosig d ad m accordigly completes the proof. Fidig explicit extractors matchig the above parameters has bee a major research directio for the last two decades..5 Extractors from Pairwise Idepedece We ow give a example of a explicit extractor. This extractor has very poor seed legth, but the amout of etropy extracted is as high as possible. Let H be a pairwise idepedet family of fuctios {h : {0, 1} {0, 1} m, ad let d = log H. Defie f : {0, 1} {0, 1} d {0, 1} m+d by: Theorem 10. f is a d + k log 1 ɛ. fx, h = h, hx. M k, DK -extractor. I particular, we have optimal output legth m = Proof. We boud the L distace from f to U m+d : 9
= α {0,1} m+d α {0,1} m+d 1 Pr [fx, h = α] x X,h H m+d Pr [fx, h = α] 1 x X,h H = Pr x 1,h 1,x,h [fx 1, h 1 = fx, h ] 1 m+d m+d = Pr x 1,h 1,x,h [h 1, h 1 x 1 = h, h x ] 1 m+d = Pr [h 1 = h ] Pr [h 1 x 1 = h x ] 1 h 1,h x 1,h 1,x,h = 1 d Pr [hx 1 = hx ] 1 h,x 1,x 1 1 d k + 1 m 1 m+d 1 = d+k. m+d m+d We ca ow use Cauchy-Schwarz ad the L distace to boud the L 1 distace: fx, H U m 1 DK M fx, H U m 1 DK m = d + k log 1 ɛ. 10