Independence and chromatic number (and random k-sat): Sparse Case. Dimitris Achlioptas Microsoft

Transcription

1 Independence and chromatic number (and random k-sat): Sparse Case Dimitris Achlioptas Microsoft

2 Random graphs W.h.p.: with probability that tends to 1 as n.

3 Hamiltonian cycle Let τ 2 be the moment all vertices have degree 2

4 τ 2 Hamiltonian cycle Let be the moment all vertices have degree 2 G(n, m = τ 2 ) W.h.p. has a Hamiltonian cycle

5 τ 2 Hamiltonian cycle Let be the moment all vertices have degree 2 G(n, m = τ 2 ) W.h.p. has a Hamiltonian cycle W.h.p. it can be found in time O(n 3 log n) [Ajtai, Komlós, Szemerédi 85] [Bollobás, Fenner, Frieze 87]

6 Hamiltonian cycle τ 2 Let be the moment all vertices have degree 2 G(n, m = τ 2 ) W.h.p. has a Hamiltonian cycle W.h.p. it can be found in time O(n 3 log n) [Ajtai, Komlós, Szemerédi 85] [Bollobás, Fenner, Frieze 87] In G(n, 1/2) Hamiltonicity can be decided in expected time. O(n) [Gurevich, Shelah 84]

7 Cliques in random graphs G(n, 1/2) The largest clique in has size 2log 2 n 2log 2 log 2 n±1 [Bollobás, Erdős 75] [Matula 76]

8 Cliques in random graphs G(n, 1/2) The largest clique in has size 2log 2 n 2log 2 log 2 n±1 [Bollobás, Erdős 75] [Matula 76] No maximal clique of size < log 2 n

9 Cliques in random graphs G(n, 1/2) The largest clique in has size 2log 2 n 2log 2 log 2 n±1 [Bollobás, Erdős 75] [Matula 76] No maximal clique of size < log 2 n (1 + ²)log 2 n Can we find a clique of size? [Karp 76]

10 Cliques in random graphs G(n, 1/2) The largest clique in has size 2log 2 n 2log 2 log 2 n±1 [Bollobás, Erdős 75] [Matula 76] No maximal clique of size < log 2 n (1 + ²)log 2 n Can we find a clique of size? What if we hide a clique of size n 1=2 ²?

11 Two problems for which we know much less. Chromatic number of sparse random graphs Random k-sat

12 Two problems for which we know much less. Chromatic number of sparse random graphs Random k-sat Canonical for random constraint satisfaction: Binary constraints over k-ary domain k-ary constraints over binary domain Studied in: AI, Math, Optimization, Physics,

13 A factor-graph representation of k-coloring Each vertex is a variable with domain {1,2,,k}. Each edge is a constraint on two variables. Vertices v 1 e 1 e 2 v 2 All constraints are not-equal Random graph = each constraint picks two variables at random. Edges

14 SAT via factor-graphs (x 12 x 5 x 9 ) (x 34 x 21 x 5 ) (x 21 x 9 x 13 )

15 SAT via factor-graphs (x 12 x 5 x 9 ) (x 34 x 21 x 5 ) (x 21 x 9 x 13 ) Edge between x and c iff x occurs in clause c. Variable nodes x c Edges are labeled +/- to indicate whether the literal is negated. Constraints are at least one literal must be satisfied. Random k-sat = constraints pick k literals at random Clause nodes

16 Diluted mean-field spin glasses Small, discrete domains: spins Conflicting, fixed constraints: quenched disorder Variables x c Random bipartite graph: lack of geometry, mean field Sparse: diluted Hypergraph coloring, random XOR-SAT, error-correcting codes Constraints

17 Random graph coloring: Background

18 A trivial lower bound For any graph, the chromatic number is at least: Number of vertices Size of maximum independent set

19 A trivial lower bound For any graph, the chromatic number is at least: Number of vertices Size of maximum independent set For random graphs, use upper bound for largest independent set. µ n s (1 p) (s 2) 0

20 An algorithmic upper bound Repeat Pick a random uncolored vertex Assign it the lowest allowed number (color) Uses 2 x trivial lower bound number of colors

21 An algorithmic upper bound Repeat Pick a random uncolored vertex Assign it the lowest allowed number (color) Uses 2 x trivial lower bound number of colors No algorithm is known to do better

22 The lower bound is asymptotically tight G(n, d/n) As d grows, can be colored using independent sets of essentially maximum size [Bollobás 89] [Łuczak 91]

23 The lower bound is asymptotically tight G(n, d/n) As d grows, can be colored using independent sets of essentially maximum size [Bollobás 89] [Łuczak 91]

24 Only two possible values Theorem. For every d>0, there exists an integer k = k(d) such that w.h.p. the chromatic number of G(n, p = d/n) is either k or k +1 [Łuczak 91]

25 The Values Theorem. For every d>0, there exists an integer k = k(d) such that w.h.p. the chromatic number of G(n, p = d/n) is either k or k +1 where k is the smallest integer s.t. d<2k log k.

26 Examples If d =7, w.h.p. the chromatic number is 4 or 5.

27 Examples d =7 4 5 If, w.h.p. the chromatic number is or. d =10 60 If, w.h.p. the chromatic number is or

28 One value Theorem. If (2k 1) ln k < d < 2k ln k then w.h.p. the chromatic number of G(n, d/n) is k +1.

29 One value Theorem. If (2k 1) ln k < d < 2k ln k then w.h.p. the chromatic number of G(n, d/n) is k +1. If d =10 100, then w.h.p. the chromatic number is

30 Random k-sat: Background

31 Fix a set of n variables Random k-sat 2 k µ n k X = {x 1,x 2,...,x n } Among all possible k-clauses select m uniformly and independently. Typically m = rn. k =3 Example ( ) : (x 12 x 5 x 9 ) (x 34 x 21 x 5 ) (x 21 x 9 x 13 )

32 Generating hard 3-SAT instances [Mitchell, Selman, Levesque 92]

33 Generating hard 3-SAT instances [Mitchell, Selman, Levesque 92] The critical point appears to be around r 4.2

34 The satisfiability threshold conjecture For every k 3, there is a constant r k such that lim n!1 Pr[F k (n, rn) issatisfiable] = ½ 1 if r = rk ² 0 if r = r k + ²

35 The satisfiability threshold conjecture For every k 3, there is a constant r k such that lim n!1 Pr[F k (n, rn) issatisfiable] = ½ 1 if r = rk ² 0 if r = r k + ² For every k 3, 2 k k < r k < 2 k ln 2

36 Repeat Unit-clause propagation Pick a random unset variable and set it to 1 While there are unit-clauses satisfy them If a 0-clause is generated fail

37 Repeat Unit-clause propagation Pick a random unset variable and set it to 1 While there are unit-clauses satisfy them If a 0-clause is generated fail UC finds a satisfying truth assignment if r< 2k k [Chao, Franco 86]

38 An asymptotic gap The probability of satisfiability it at most 2 n µ k m = 2 µ 1 12 k r n 0 for r 2 k ln 2

39 An asymptotic gap Since mid-80s, no asymptotic progress over 2 k k < r k < 2 k ln 2

40 Getting to within a factor of 2 Theorem: For all k 3and r<2 k 1 ln 2 1. a random k-cnf formula with m = rn clauses w.h.p. has a complementary pair of satisfying truth assignments.

41 The trivial upper bound is the truth! Theorem: For all k 3, a random k-cnf formula with m = rn clauses is w.h.p. satisfiable if r 2 k ln 2 k 2 1

42 Some explicit bounds for the k-sat threshold k Upper bound , 817 1, 453, 635 Our lower bound , 809 1, 453, 626 Algorithmic lower bound , , 453

43 The second moment method For any non-negative r.v. X, E[X ]2 Pr[X > 0] E[X 2 ] Pro of: Let Y =1ifX>0, and Y = 0 otherwise. By Cauchy-Schwartz, E[X] 2 = E[XY ] 2 E[X 2 ]E[Y 2 ] = E[X 2 ]Pr[X>0].

44 Ideal for sums If X = X 1 +X 2 + then E[X] 2 = X i;j E[X i ]E[X j ] E[X 2 ] = X i;j E[X i X j ] X X

45 Ideal for sums If X = X 1 +X 2 + then E[X] 2 = X i;j E[X i ]E[X j ] E[X 2 ] = X i;j E[X i X j ] Example: The X i correspond to the n q potential q-cliques in G(n, 1/2) X X Dominant contribution from non-ovelapping cliques

46 General observations Method works well when the X i are like needles in a haystack

47 General observations Method works well when the X i are like needles in a haystack Lack of correlations = rapid drop in influence around solutions

48 General observations Method works well when the X i are like needles in a haystack Lack of correlations = rapid drop in influence around solutions Algorithms get no hints

49 The second moment method for random k-sat Let X be the # of satisfying truth assignments

50 The second moment method for random k-sat Let X be the # of satisfying truth assignments For every clause-density r>0, there is β = β(r) > 0 such that E[X] 2 E[X 2 ] < (1 β)n

51 The second moment method for random k-sat Let X be the # of satisfying truth assignments For every clause-density r>0, there is β = β(r) > 0 such that E[X] 2 E[X 2 ] < (1 β)n The number of satisfying truth assignments has huge variance.

52 The second moment method for random k-sat Let X be the # of satisfying truth assignments For every clause-density r>0, there is β = β(r) > 0 such that E[X] 2 E[X 2 ] < (1 β)n The number of satisfying truth assignments has huge variance. The satisfying truth assignments do not form a uniformly random mist in σ {0, 1} n

53 To prove 2 k ln2 k/2-1 H(σ,F) Let be the number of satisfied literal occurrences in F under σ where γ < 1 satisfies (1 + γ 2 ) k 1 (1 γ 2 )=1.

54 To prove 2 k ln2 k/2-1 H(σ,F) Let be the number of satisfied literal occurrences in F under σ Let X = X(F ) be defined as X(F ) = X ¾ = X ¾ 1 ¾j= F γ H (¾;F ) Y 1 ¾j= c γ H (¾;c ) c where γ < 1 satisfies (1 + γ 2 ) k 1 (1 γ 2 )=1.

55 To prove 2 k ln2 k/2-1 H(σ,F) Let be the number of satisfied literal occurrences in F under σ Let X = X(F ) be defined as X(F ) = X ¾ = X ¾ 1 ¾j= F γ H (¾;F ) Y 1 ¾j= c γ H (¾;c ) c where γ < 1 satisfies (1 + γ 2 ) k 1 (1 γ 2 )=1.

56 General functions Given any t.a. σ and any k-clause c let v = v(σ,c) { 1, +1} k be the values of the literals in c under σ.

57 General functions Given any t.a. σ and any k-clause c let v = v(σ,c) { 1, +1} k be the values of the literals in c under σ. We will study random variables of the form X = X Y f(v(σ,c)) ¾ c where f : { 1, +1} k R is an arbitrary function

58 X = X ¾ Y c f(v(σ,c)) f (v) = 1 for all v = 2 n f (v) = ( 0 if v =( 1, 1,..., 1) 1 otherwis e = # of satisfying truth assignments f (v) = 8 >< >: 0 if v =( 1, 1,..., 1) or if v = (+1, +1,...,+1) 1 otherwis e = # of Not All Equal truth assignments (NAE)

59 X = X ¾ Y c f(v(σ,c)) f (v) = 1 for all v = 2 n f (v) = ( 0 if v =( 1, 1,..., 1) 1 otherwis e = # of satisfying truth assignments f (v) = 8 >< >: 0 if v =( 1, 1,..., 1) or if v = (+1, +1,...,+1) 1 otherwis e = # of satisfying truth assignments whose complement is also satisfying

60 Overlap parameter = distance Overlap parameter is Hamming distance

61 Overlap parameter = distance Overlap parameter is Hamming distance For any f, if σ, τagree on z = n/2 variables h i E f(v(σ,c))f(v(τ,c)) = E f(v(σ,c)) E f(v(τ,c))

62 Overlap parameter = distance Overlap parameter is Hamming distance For any f, if σ, τagree on z = n/2 variables h i E f(v(σ,c))f(v(τ,c)) = E f(v(σ,c)) E f(v(τ,c)) σ, τ z For any f, if agree on variables, let C f (z/n) E h i f(v(σ,c))f(v(τ,c))

63 Contribution according to distance Independence

64 Entropy vs. correlation For every function f : Xn E[X 2 ] = 2 n z=0 µ n z C f (z/n) m Xn E[X] 2 = 2 n z=0 µ n z C f (1/2) m Recall: µ n αn = µ n 1 α (1 α) 1 poly(n)

65 Contribution according to distance Independence

66 =z/n

67 The importance of being balanced An analytic condition: C 0 f (1=2) = 0 = the s.m.m. fails

68 =z/n

69 NAE 5-SAT 7 <r < 11

70 NAE 5-SAT 7 <r < 11

71 The importance of being balanced An analytic condition: C 0 f (1=2) = 0 = the s.m.m. fails

72 The importance of being balanced An analytic condition: C 0 f (1=2) = 0 = the s.m.m. fails A geometric criterion: C 0 f (1=2) = 0 X v { 1;+1} k f (v)v =0

73 The importance of being balanced C 0 f (1=2) = 0 X v { 1;+1} k f (v)v =0 (-1,-1,,-1) (+1,+1,,+1) Constant SAT Complementary

74 Balance & Information Theory Want to balance vectors in optimal way

75 Balance & Information Theory Want to balance vectors in optimal way Information theory = f(v) maximize the entropy of the subject to X and f(v)v =0 v 2f 1;+1 g k f( 1, 1,..., 1) = 0

76 Balance & Information Theory Want to balance vectors in optimal way Information theory = f(v) maximize the entropy of the subject to X and f(v)v =0 v 2f 1;+1 g k f( 1, 1,..., 1) = 0 Lagrange multipliers the optimal f is f(v) = γ #of+1sinv γ = for the unique that satisfies the constraints

77 Balance & Information Theory Want to balance vectors in optimal way Information theory f(v) maximize the entropy of the subject to X and f(v)v =0 v 2f 1;+1 g k f( 1, 1,..., 1) = 0 = (-1,-1,,-1) (+1,+1,,+1) Heroic

78 Random graph coloring

79 Threshold formulation Theorem. A random graph with n vertices and m = cn edgesis w.h.p. k-colorable if c k log k log k 1. and whp non-k-colorable if c k log k 1 2 log k.

80 Main points Non-k-colorability: Pro of. The probability that there exists any k-coloring is at most µ k n 1 1 cn 0 k k-colorability: Pro of. Apply second moment method to the number of balanced k-colorings of G(n, m).

81 Setup Let X σ be the indicator that the balanced k-partition σ is a proper k-coloring. We will prove that if then for all c k log k log k 1 D = D(k) > 0 such that X = P ¾ X ¾ E[X 2 ] <DE[X] 2 G(n, cn) there is a constant This implies that is k-colorable w.h.p.

82 E[X 2 ]= Setup σ, τ E[X ¾ X ] sum over all of. For any pair of balanced k-partitions let a ij n be the # of vertices having color i in σ and color j in τ. Pr[σ and τ are proper] 2 k + X ij 0 σ, τ a 2 ij 1 A cn

83 Examples Balance = A is doubly-stochastic. σ, τ When are uncorrelated, A is 1/k the flat matrix 1 k

84 Examples Balance = A is doubly-stochastic. σ, τ When are uncorrelated, A is 1/k the flat matrix As align, A σ, τ tends to the identity matrix I 1 k

85 So, A matrix-valued overlap E[X 2 ]= X k µ n An µ1 2 k + 1 k 2 X a 2ij cn A2B k

86 So, A matrix-valued overlap E[X 2 ]= X k µ n An µ1 2 k + 1 k 2 X a 2ij cn A2B k which is controlled by the maximizer of X µ a ij log a ij + c log 1 2 k + 1 X a 2ij k 2 over k k doubly-stochastic matrices A =(a ij ).

87 So, A matrix-valued overlap E[X 2 ]= X k µ n An µ1 2 k + 1 k 2 X a 2ij cn A2B k which is controlled by the maximizer of X µ a ij log a ij + c log 1 2 k + 1 X a 2ij k 2 over k k doubly-stochastic matrices A =(a ij ).

88 X i;j a ij log a ij + c 1 X 2 k i;j a 2 ij Entropy decreases away from the flat matrix c For small, this loss overwhelms the sum of squares gain c But for large enough,. 1/k The maximizer jumps instantaneously from flat, to a matrix where vertices capture the majority of mass k Proved this happens only for c >klog k log k 1

89 X i;j a ij log a ij + c 1 X 2 k i;j a 2 ij Entropy decreases away from the flat matrix c For small, this loss overwhelms the sum of squares gain c But for large enough,. 1/k The maximizer jumps instantaneously from flat, to a matrix where vertices capture the majority of mass k Proved this happens only for c >klog k log k 1

90 X i;j a ij log a ij + c 1 X 2 k i;j a 2 ij Entropy decreases away from the flat matrix c For small, this loss overwhelms the sum of squares gain c But for large enough,. 1/k The maximizer jumps instantaneously from flat, to a matrix where vertices capture the majority of mass k Proved this happens only for c >klog k log k 1

91 X i;j a ij log a ij + c 1 X 2 k i;j a 2 ij Entropy decreases away from the flat matrix c For small, this loss overwhelms the sum of squares gain c But for large enough,. 1/k The maximizer jumps instantaneously from flat, to a matrix where entries capture majority of mass k

92 X i;j a ij log a ij + c 1 X 2 k i;j a 2 ij Entropy decreases away from the flat matrix c For small, this loss overwhelms the sum of squares gain c But for large enough,. 1/k The maximizer jumps instantaneously from flat, to a matrix where entries capture majority of mass k This jump happens only after c >klog k log k 1

93 Proof overview Proof. Compare the value at the flat matrix with upper bound for everywhere else derived by:

94 Proof overview Proof. Compare the value at the flat matrix with upper bound for everywhere else derived by: 1. Relax to singly stochastic matrices.

95 Proof overview Proof. Compare the value at the flat matrix with upper bound for everywhere else derived by: 1. Relax to singly stochastic matrices. 2. Prescribe the L 2 norm of each row ρ i.

96 Proof overview Proof. Compare the value at the flat matrix with upper bound for everywhere else derived by: 1. Relax to singly stochastic matrices. 2. Prescribe the L 2 norm of each row ρ i. 3. Find max-entropy, f(ρ i ), of each row given ρ i.

97 Proof overview Proof. Compare the value at the flat matrix with upper bound for everywhere else derived by: 1. Relax to singly stochastic matrices. 2. Prescribe the L 2 norm of each row ρ i. 3. Find max-entropy, f(ρ i ), of each row given ρ i. 4. Prove that f > 0.

98 Proof overview Proof. Compare the value at the flat matrix with upper bound for everywhere else derived by: 1. Relax to singly stochastic matrices. 2. Prescribe the L 2 norm of each row ρ i. 3. Find max-entropy, f(ρ i ), of each row given ρ i. 4. Prove that f > Use (4) to determine the optimal distribution of the ρ i given their total ρ.

99 Proof overview Proof. Compare the value at the flat matrix with upper bound for everywhere else derived by: 1. Relax to singly stochastic matrices. 2. Prescribe the L 2 norm of each row ρ i. 3. Find max-entropy, f(ρ i ), of each row given ρ i. 4. Prove that f > Use (4) to determine the optimal distribution of the ρ i given their total ρ. 6. Optimize over ρ.

100 Random regular graphs Theorem. For every integer d>0, w.h.p. the chromatic number of a random d-regular graph is either k, k +1,ork +2 where k is the smallest integer s.t. d<2k log k.

101 Maximize X k i =1 subject to X k i =1 X k i =1 A vector analogue (optimizing a single row) a i log a i a i = 1 a 2 i = ρ for some 1/k < ρ < 1

102 Maximize X k i =1 subject to X k i =1 X k i =1 A vector analogue (optimizing a single row) a i log a i a i = 1 a 2 i = ρ for some 1/k < ρ < 1 k

103 Maximize X k i =1 subject to X k i =1 X k i =1 a i log a i a i = 1 a 2 i = ρ for some 1/k < ρ < 1 A vector analogue (optimizing a single row) For k =3 the maximizer is (x, y, y) wherex>y

104 Maximize X k i =1 subject to X k i =1 X k i =1 a i log a i a i = 1 a 2 i = ρ for some 1/k < ρ < 1 A vector analogue (optimizing a single row) For k =3 the maximizer is (x, y, y) wherex>y

105 Maximize X k i =1 subject to X k i =1 X k i =1 a i log a i a i = 1 a 2 i = ρ for some 1/k < ρ < 1 A vector analogue (optimizing a single row) For k =3 the maximizer is (x, y, y) wherex>y

106 Maximize X k i =1 subject to X k i =1 X k i =1 a i log a i a i = 1 a 2 i = ρ for some 1/k < ρ < 1 A vector analogue (optimizing a single row) For k =3 the maximizer is (x, y, y) wherex>y

107 Maximize X k i =1 subject to X k i =1 X k i =1 a i log a i a i = 1 a 2 i = ρ for some 1/k < ρ < 1 A vector analogue (optimizing a single row) For k =3 the maximizer is (x, y, y) wherex>y For k>3 the maximizer is (x, y,..., y)

108 Maximum entropy image restoration Create a composite image of an object that: Minimizes empirical error Typically, least-squares error over luminance Maximizes plausibility Typically, maximum entropy

109 Maximum entropy image restoration Structure of maximizer helps detect stars in astronomy

110 The End