CS71 Randomness & Computation Spring 018 Instructor: Alistair Sinclair Lecture 5: January 30 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They may be distributed outside this class only with the permission of the Instructor. 5.1 The Probabilistic Method The probabilistic method is a powerful mathematical tool used in solving many combinatorial problems. The method was first introduced by Paul Erdős in his seminal 1947 paper [E47]. The probabilistic method works as follows: to show that an object with a specific property exists, we choose an object randomly from an appropriate collection of objects and show that it has the desired property with positive probability, which proves the existence of at least one such object. It turns out that it is often easier to show the existence of certain objects by this method than by constructing them explicitly. Moreover, the probabilistic method often suggests efficient randomized algorithms, which can sometimes subsequently be derandomized. 5.1.1 Ramsey Theory As a warm-up, we state and prove a result from Erdős s original paper on Ramsey theory [E47]. Definition: The k-th (diagonal) Ramsey number, R k, is the smallest number n such that any -coloring of the edges of the complete graph on n vertices, K n, must contain a monochromatic k-clique. As a useful Exercise, you should verify that R 3 = 6 by showing that K 5 can be -colored with no monochromatic triangles, while K 6 cannot. The existence of Ramsey numbers can be established by a simple inductive argument, which also provides an upper bound of k 3 for R k. The following theorem provides a lower bound for R k via the probabilistic method. Theorem 5.1 R k > k/. Proof: Let n = k/. We show that there exists a -coloring of the complete graph K n, such that there is no monochromatic k-clique. Randomly color each edge of K n red or blue with probability 1, independently. Let C be any k-clique in the graph K n. Then, Pr[C is monochromatic] = (k ) = 1 ( k ). Since the total number of k-cliques in K n is ( n k), by the union bound, we get Pr[K n has a monochromatic k-clique] ( n k) Pr[a given k-clique is monochromatic] = ( ) n 1 ( k ) 5-1 k nk k! 1 (k ).
5- Lecture 5: January 30 Putting n = k/, we get Pr[K n has a monochromatic k-clique] 1 k! k +1 k k = 1 k! k+ < 1 for k 3. So, the probability of a good coloring, i.e., a -coloring such that there is no monochromatic k-clique, is more than zero. Thus, there must exist a good coloring. Ramsey numbers have been the object of extensive study in combinatorics. Despite this, improvements in bounds on R k have been slow. In fact, for general k the above upper and lower bounds remain essentially the best known, in the sense that no asymptotic lower bound of the form R k (1/+ɛ)k or upper bound of the form R k ( ɛ)k for ɛ > 0 has been found. In particular, it is surprising that a completely random coloring of K n provides a lower bound which is almost as good as bounds obtained by the cleverest deterministic arguments. 5.1. The Max Cut Problem We now consider the problem of finding a partition of a graph into two parts such that the number of cut edges (i.e., those between the two parts of the partition) is maximized. Input: A graph G = (V, E). Goal: Find a partition V = V 1 V of G such that the number of edges between V 1 and V is maximized. This is a well-known NP-hard optimization problem. However, as the following lemma shows, a simple application of the probabilistic method gives us a lower bound on the size of the maximum cut. Lemma 5. There exists a cut containing at least E edges. Proof: Pick a random partition, by assigning each vertex to V 1 or V independently with probability 1. For each edge e, define the indicator variable X e as X e = { 1 if e is cut; 0 otherwise Thus, if X is the number of cut edges, then X = e E X e. Moreover, we have E[X e ] = Pr[e is cut] = Pr[the endpoints of e lie on different sides of the partition] = 1. Now by linearity of expectation, E[X] = e E E[X e ] = E. This implies that there must exist a cut such that X E (since any random variable must achieve a value at least as large as its expectation at some sample point). Exercise: Improve the above bound slightly by showing that there must always exist a cut with strictly more than E edges.
Lecture 5: January 30 5-3 5.1.3 Independent Set In a graph G = (V, E), a subset of vertices V V is said to be an independent set if no two vertices u, v V are adjacent in G. It is NP-hard to determine the size of a largest independent set in G. However, an elementary but clever application of the probabilistic method will allow us to establish a good lower bound on the size of the maximum independent set for any given graph. Claim 5.3 Any graph G = (V, E) contains an independent set V V such that V v V where deg(v) is the number of vertices adjacent to v in G. 1 deg(v)+1, As an example, this claim implies that every 3-regular graph contains an independent set of size at least V /4. Proof: Assign a random weight w v to every vertex v V, choosing these weights independently and uniformly at random from the interval [0, 1]. Call v V a local minimum if w v < w u for every vertex u adjacent to v. Since no two adjacent vertices can both be local minima, it is clear that the set of local minima forms an independent set. Moreover, since weights are assigned uniformly from a continuous set, any vertex among v and its neighbors is equally likely to have minimum weight, so we have Pr[v is a local minimum] = 1 deg(v) + 1. Now let X be the number of local minima, and write X = v X v, where X v is an indicator r.v. defined by By linearity of expectation, { 1 if v is a local minimum; X v = 0 otherwise. E[X] = v V E[X v ] = v V 1 deg(v) + 1. Hence G must contain an independent set of the claimed size. 5.1.4 Graph Crossing Number Consider an arbitrary graph G = (V, E); let n denote the number of vertices in G and let m denote the number of edges. We define the crossing number c(g) as the minimum number of edge crossings in any planar embedding of G. (Thus a graph is planar iff c(g) = 0.) Most graphs cannot be embedded in the plane without crossing edges. In fact, those that can are necessarily quite sparse. Euler s formula says that if G is planar then m 3n 6. (5.1) We begin with a preliminary lower bound on the crossing number that generalizes Euler s formula: Claim 5.4 For any graph G with n vertices and m edges, we have c(g) m 3n + 6. Proof: The proof is a simple deterministic argument. Consider an optimal planar embedding of G, i.e., an embedding that achieves c = c(g) edge crossings. It is easy to verify (Exercise) that such an embedding satisfies the following properties:
5-4 Lecture 5: January 30 Figure 5.1: Construction of planar graph G from G. 1. No edge crosses itself.. No two edges cross more than once. 3. No two edges that share a vertex cross. Construct a new graph G = (V, E ) from G by replacing each edge crossing in the optimal embedding of G with a vertex, as shown in Figure 5.1. More precisely, we start with G = G and, for every pair of edges e 1 = (v a, v b ) and e = (v c, v d ) that cross, transform G as follows: 1. Add a new vertex v to V.. Remove e 1 and e from E and insert four new edges: e 1 = (v a, v ), e = (v b, v ), e 3 = (v c, v ), and e 4 = (v d, v ). From the construction and the properties of optimal planar embeddings, we have: V n = n + c and E m = m + c. (To verify the second equality, observe that the number of edges increases by two for every edge crossing eliminated from the original graph). Furthermore, G is a planar graph, so by Euler s formula we have: as required. m 3n 6 m + c 3n + 3c 6 c m 3n + 6, The above result gives us an estimate of the crossing number that is reasonably tight for sparse graphs (when m is not much larger than 3n). We now apply the probabilistic method to demonstrate a much stronger lower bound for large values of m. This beautiful proof is attributed to Chazelle, Sharir and Welzl [CSW], though the result (with a different constant) had previously been proved using much heavier deterministic tools in [L83] and [ACNS8]. Claim 5.5 For any graph G with n vertices and m edges with m 4n, we have c(g) m3 64n. Proof: Fix an optimal planar embedding of G with c = c(g) crossings. Construct a random induced subgraph G p of G, by including each vertex in G p independently with probability p (and retaining precisely those edges both of whose endpoints are in G p ). The value of p will be specified later. Now let n p and m p be random variables denoting the number of vertices and edges in G p, respectively. Additionally, let c p denote the number of edge crossings from the original embedding that remain in G p. Using the result of Claim 5.4, we have: c p m p 3n p + 6 m p 3n p.
Lecture 5: January 30 5-5 Taking expectations then gives E[c p ] E[m p 3n p ] = E[m p ] 3E[n p ]. Now, since each vertex in G is included in G p with probability p, we have E[n p ] = np. Similarly, since every edge in G p has probability p of remaining in G p, we have E[m p ] = mp. Lastly, E[c p ] = cp 4, since every edge crossing involves four distinct vertices of G. Plugging these values into the above inequality gives cp 4 mp 3pn, and hence c m p 3n p 3. Now we choose p to maximize the right-hand side of this inequality. Specifically, setting p = 4n/m yields as claimed. c m3 64n, 5. Some more sophisticated examples 5..1 Unbalancing lights Consider a square n n array of lights (see Figure 5.). There is one switch corresponding to each row and each column (i.e., n switches). Throwing a switch changes the state of all the lights in the corresponding row or column. We now consider the problem of setting the switches so as to maximize the number of lights that are on, starting from an arbitrary initial on/off configuration of the lights. Light Switch Figure 5.: A 4 4 array of lights with switches. Note first that, if we set all the switches independently and uniformly at random, then each light is on with probability 1 and the states of the lights are pairwise indepedent. Thus the expectation of the quantity #on #off is O(n), so we could hope to achieve about n + O(n) lights on. (Note that we can always ensure that at least half are on simply by flipping all the row switches if necessary.) Here is an argument using the probabilistic method that it is always possible to get a much better advantage.
5-6 Lecture 5: January 30 Claim: For any initial configuration of the lights, there exists a setting of the switches for which the number of lights that are on is asymptotically n + 1 π n3/ as n. Proof: We first set the column switches randomly and independently. Define the indicator random variable X ij as { 1 if light (i,j) is on X ij = 1 if light (i,j) is off We now look at row i. Let Z i = j X ij. Note that, as a result of our random flipping of the column switches, X i1, X i,..., X in are iid uniform on {±1} and hence the random variable Z i is the sum of n independent Bernoulli trials with outcomes ±1. Therefore, E[ Z i ] π n. 1 Finally, for each row i, we set the row switch to get the majority of lights on. By linearity of expectation, E[#on #off] = i E[ Z i ] π n3/. Hence there must exist some setting of switches that achieves at least this difference, and hence at least n + 1 π n3/ lights on (as n ). 5.. Graphs with large girth and chromatic number Let G = (V, E) be a graph with vertex set V and edge set E. The girth of G is the length of the shortest cycle in G. The chromatic number of G is the minimum number of colors to properly color the vertices of G (i.e., so that no two adjacent vertices receive the same color). One may expect that a graph with large girth should have a small chromatic number, but it turns out that this is not the case. A classical result of Erdős shows that there exist graphs for which both the girth and chromatic number are arbitrarily large, provided the number of vertices is sufficiently large [E59]. The proof is another application of the probabilistic method. Theorem 5.6 For any positive integers k, l, there exists a graph with girth l and chromatic number k. Proof: Pick a random graph G from G(n, p), where n is the number of vertices and each of the ( n ) possible edges is included independently with probability p. We will choose the value p = n 1/l 1. (Note that the resulting graphs are rather sparse; the expected number of edges is on the order of n 1+1/l.) Let X be the number of cycles of length < l in G. First observe that the number of possible cycles of length i is ( ) n i! i i since for every subset of size i of the n vertices, there are i! i possible cycles (direction and starting position do not matter). Note that a path is only present if all the corresponding edges are present, which happens 1 This is a standard result about the positive part of the binomial distribution; it is equivalent to saying q that the expected distance traveled from the origin in n steps of a symmetric random walk with reflecting barrier at 0 is π n.
Lecture 5: January 30 5-7 with probability p i for each possible cycle. Thus, l 1 ( ) n E[X] = i! i i pi i=3 l 1 i=3 = o(n), n i/l i (5.) where (5.) follows from the fact that ( n i) i! n i and n i p i = n i n i/l i = n i/l. Thus, by Markov s inequality, Pr[X n/] = o(1). Hence the probability that G contains n/ cycles of length l or more goes to zero. Turning now to the chromatic number, note that the number of colors needed to properly color a graph can be lower bounded as # vertices # colors max. independent set size since the set of vertices that receive any given color must form an independent set. Let Y be the size of a maximal independent set in G. Then by a union bound, ( ) n Pr[Y y] (1 p) ) (y y ( n e p(y 1)/) y, (5.3) = o(1) if we set y = 3 p ln n where (5.3) follows from ( n y) n y and the inequality 1 + x e x. Putting the above two calculations together, by taking n large enough we can ensure that the probabilities Pr[X n/] and Pr[Y 3 p ln n] are both less than 1, so by a union bound there must exist a graph G with the following properties: 1. G has n cycles of length < l;. G has maximum independent set size < 3 p ln n. Finally, we can remove from G one vertex from every cycle of length l, resulting in a graph G with the following properties: 1. G has girth l;. G has n vertices; 3. G has maximum independent set size < 3 p ln n. (Note that removing vertices from G can only decrease the size of the maximum independent set.) Hence, G has chromatic number n/ as n. So again by taking n large enough we can make n1/l 3 = p ln n 6 ln n
5-8 Lecture 5: January 30 this at least k, as desired. The previous two examples illustrate the technique of sample and modify, which arises in many applications of the probabilistic method. Recall that in the examples we saw in the first section, we simply constructed an object at random (a -coloring, a truth assignment etc.) and then argued that it has the desired property with non-zero probability. In the above two examples, we first construct a random object (the initial setting of the column switches, or the random graph G), and then modify it so that it has the desired property (by setting the row switches, or by deleting vertices from G). References [ACNS8] [CSW] M. Ajtai, V. Chvátal, M. Newborn and E. Szemerédi, Crossing-free subgraphs, Annals of Discrete Mathematics 1 (198), pp. 9 1. B. Chazelle, M. Sharir and E. Welzl, folklore. [E47] P. Erdős, Some remarks on the theory of graphs, Bulletin of the American Math. Society 53 (1947), pp. 9 94. [E59] P. Erdős, Graph Theory and Probability, Canad. J. Math 11 (1959), pp. 34 38. [L83] F.T. Leighton, Complexity issues in VLSI, MIT Press, Cambridge, 1983.