The cavity method. Vingt ans après

Transcription

1 The cavity method Vingt ans après

2 Les Houches lectures 1982 Early days with Giorgio

3 SK model E = J ij s i s j i<j P (s) = W α P α (s) α Many pure states, organized in a hierarchical structure: phase transition without a clear symmetry breaking 1983 Ultrametricity Non self-averageness (M, Parisi, Sourlas, Toulouse, Virasoro) State α, free energyf α

4 SK model 1983 Ultrametricity Non selfaveraging (M, Parisi, Sourlas, Toulouse, Virasoro)

5 The cavity method for the SK model M, Parisi, Virasoro, 1985 E = i<j J ij s i s j Motivation: understanding replica symmetry breaking David Sherrington: Replica Symmetry Breaking and the conceptual, mathematical and physical challenges it raised have been a rich and fruitful source from which new knowledge and application have flowed profusely since it was invented 30 years ago by Giorgio Parisi and show no sign of abating. abate verb [ intrans. ] (of something perceived as hostile, threatening, or negative) become less intense or widespread : the storm suddenly abated.

6 The cavity method for the SK model E = i<j J ij s i s j N N + 1 s 0 Hyp. F N Nf f = F N+1 F N New spin s 0 sees a local magnetic field J 0i s i which has a Gaussian distribution... within one pure state i

7 SK model N N + 1 Distribution of free energies W α = Ce βf α e βx(f G N ) F α F Two main ingredients: Ultrametricity and exponential distribution of low-lying free energies e βx(f G N+1) f = F N+1 F N = G N+1 G N F α + δf α Only exp. distribution is stable

8 The cavity method beyond SK Diluted systems (where the number of variables interacting with a given one is finite) at the RS level: simple optimization problems (assignment) and Bethe lattice spin glasses ( ) Diluted systems at the 1RSB level (one level of hierarchy in the ultrametric tree): RSB effects in Bethe lattice spin glasses, phase diagram and new algorithms in hard optimization problems (satisfiability, coloring) ( )

9 Optimisation problems Assignment ( easy, in P) Isabelle Vincent Marc Cleaning Supermarket Washing Travelling salesman ( hard, NPC) Hamiltonian path ( hard, NPC)

10 costs: O(1/N) Simple (RS) optimization pbs Random cost assignment and TSP (M, Parisi, 86) Assign job a to person j : cost E aj iid on [0, 1] Pb: find the lowest cost assignment (permutation) Selected costs: O(1/N) Diluted system 1 N 1 N 1 N N + 1 Finite number of possibilities for the newly connected job (or person)

11 The cavity method for simple (RS) optimization pbs e.g.: Random cost assignment (M,Parisi 86) Field theoretic representation with spin variables. Local field: sum of a finite number of fluctuating terms, non-gaussian. Cavity integral equations for local field Distribution of rescaled edges d/n in the ground state Optimal cost: ζ(2) = π 2 /6 P (d) = d e d sinh d sinh 2 d (Rigorous proof with cavity method: Aldous 2001)

12 Finite connectivity spin glasses Bethe lattice, usually: lim L lim M L M Spin glass: boundary conditions?? i.i.d: only replica symmetric

13 Finite connectivity spin glasses Bethe lattice for spin glasses Random regular graph (fixed degree k + 1 ) Locally tree-like Loops of length O(log N)

14 Finite connectivity spin glasses Cavity recursion: local field on s 0 : h 0 n +1 n n s s s s s 0 0 s h 0 = f(h 1, h 2 ) s 0 Large n : h 0, h 1, h 2 i.i.d. from P (h) Only true if one pure state Replica symmetric approximation Self consistent

15 Finite connectivity spin glasses n+1 n n s s s s h 0 = f(h 1, h 2 ) s 0 0 Connectivity k + 1 Ground state energy s s 0 E 0 / k = RS result for SK. Correct RSB result:.763 Replica symmetric approximation (M, Parisi 87): Maybe the most interesting perspectives are linked to the problem of replica symmetry breaking effects [] on the ground state energy of optimization problems years later:

16 1RSB in finite connectivity spin glasses (M, Parisi, 2001): - State crossing - Non gaussian local fields e βxf F α + δf α (h α 1, h α 2 ) n+1 n n s s s s α α α h 0 = f(h 1, h 2 ) s 0 0 s s 0 s 0 P (h, δf ) On a given site : Distribution of fields h for states at a given free energy F: integral equation with reweighting P 0 (h) = dp 1 (h 1 )dp 2 (h 2 )e βxδf (h 1,h 2 ) δ(h f(h 1, h 2 ))

17 1RSB in finite connectivity spin glasses Integral equation with reweighting P 0 (h) = dp 1 (h 1 )dp 2 (h 2 )e βxδf (h 1,h 2 ) δ(h f(h 1, h 2 )) Instead of the RS recursion: h 0 = f(h 1, h 2 ) GS energy (k=4): instead of NB: one probability distribution on each edge P 0 (h) = ν 0 4 (h) ν 0 4 = G (ν 1 0, ν 2 0 ) 4

18 A broad class of problems Spin glasses Structural glasses (lattice models) Information theory decoding/compression Coloring Satisfiability of logical propositions... Constraint satisfaction problems: Many simple variables, related by local constraints... finite connectivity P (x 1,..., x N ) = C M a=1 ψ a (X a ) X a = {x i1 (a),..., x ik (a)}

19 A broad class of problems P (x 1,..., x N ) = C M a=1 ψ a (X a ) X a = {x i1 (a),..., x ik (a)} Compute marginals? Sample from P? Compute free-energy 1/C?... efficiently: not in O ( e an) operations, but O(N c )

20 Cavity method 1 Factor graph representation f e a 4 c 2 5 g d 3 b P (x 1,..., x 5 ) = ψ a (x 1, x 2, x 4 )ψ b (x 2, x 3 )ψ c (x 1, x 2, x 3 )...

21 Locally tree-like: OK for large random factor graphs Loop: length O(log N)

22 RS cavity method 1 Dig a cavity f e a Compute probability of x 1 in absence of a m 1 a (x 1 ) Message= local field 5 4 g h 1 a d m 1 a (x 1 ) = exp (h 1 ax 1 ) 2 cosh (h 1 a ) c 3 b 2

23 RS cavity method 1 Dig a cavity f e a Compute probability of x 1 when it is connected only to Message c h c g d c 3 b 2

24 Replica symmetric cavity equations e f d 1 m 1 c (x 1 ) = Cm d 1 (x 1 )m e 1 (x 1 )m f 1 (x 1 ) c 3 2 m c 2 (x 2 ) = x 1,x 3 ψ c (x 1, x 2, x 3 )m 1 c (x 1 )m 3 c (x 3 ) Closed set of equations: two messages propagate on each edge of the factor graph

25 Replica symmetric cavity equations = Belief Propagation = Bethe Peierls with disorder Successful in some problems (fast decoding of LDPC codes)... Only RS phases Modification in presence of glassy phase: 1RSB cavity: Statistical analysis: 1RSB phase diagram (M, Parisi) Algorithm on a single instance (M, Zecchina)

26 1RSB cavity method 1 Message= survey of the local fields in the various states P 1 a (h) : Probability that h α 1 a is equal to h, when α is picked up at random f 5 4 P 1 a = G (P c 1, P d 1, P e 1, P f 1 ) g e d c 3 a b 2

27 Application: satisfiability 1 c 3 2 x 1 x 2 x 3 Constraint = clause like The grand father of hard problems (Cook 71) variables, M clauses, density of constraints α = M/N N Large N limit: 100 %SAT N=100 N=200 Computer time SAT for α < α c 50 UNSAT for α > α c Phase transition α α=μ/ν

28 Application: satisfiability 1 c 3 2 x 1 x 2 x 3 Constraint = clause like The grand father of hard problems (Cook 71) variables, M clauses, density of constraints α = M/N N Phase diagram: SAT (E = 0 ) UNSAT (E >0) state Many states E=0 E>0 Many states E>0 α α = d c α =M/N

29 Application: satisfiability 1 state SAT (E = 0 ) UNSAT (E >0) 0 0 Many states E=0 E>0 Many states E>0 Phase diagram= properties of almost all samples α α = d c α =M/N On a single instance/sample: use the 1RSB cavity equations P 1 a = G (P c 1, P d 1, P e 1, P f 1 ) as a message passing algorithm: survey propagation (M,Zecchina) solves systems of variables at 10 7 α = 4.25

30 The cavity method as a powerful message passing algorithm Local exchange of messages along a factor graph Simple computations at each node Solves very complicated global constraint satisfaction /optimization problems (in spite of anarchy!) in a distributed way What are the problems it does not solve? Local structures... Need more work UNSAT phase (does solve, but no signature) Point-like clusters from locked constraint satisfaction problems

31 Spin glasses: Totally useless (few grams) of boring material... Intellectual interest. Tens of thousands of papers over the last 30 years. Some of the most fascinating developments in statistical physics A beautiful conceptual scheme, for many topics ranging from distributed computing (neural networks) to portfolio optimization in finance, to basic computer science problems like coloring, satisfiability,... Unexpected offsprings and applications

32 Spin glasses: Totally useless (few grams) of boring material... Intellectual interest. Tens of thousands of papers over the last 30 years. Some of the most fascinating developments in the statistical physics A beautiful conceptual scheme, for many topics ranging from distributed computing (neural networks) to portfolio optimization in finance, to basic computer science problems like coloring, satisfiability,... Unexpected offsprings and applications Information, Physics, and Computation Marc Mézard, Andrea Montanari

33 With Giorgio: 34 papers, on spin glasses, structural glasses, polymers, manifolds, interfaces, vortices, random matrices, Burgers turbulence, optimization, computer science... one book, a lot of disorder many cavities many replicas... zero...and an infinite amount of thanks! Let s keep wandering with curiosity in complex landscapes...

34 Many thanks to Enzo and all the local organizers, for this And all the staff for the perfect organization: Manuela Marchetti Adriana Vescera Angelo Campus And the whole team working with them wonderful conference