Spin glasses and Stein s method

Transcription

1 UC Berkeley

2 Spin glasses Magnetic materials with strange behavior. ot explained by ferromagnetic models like the Ising model. Theoretically studied since the 70 s. An important example: Sherrington-Kirkpatrick model. High temperature phase: Thouless-Anderson-Palmer. Low temperature phase: Mézard and Parisi. Mathematically almost intractable until the late 90 s. Early results due to Aizenman, Lebowitz, Ruelle ( 87), Fröhlich and Zegarliński ( 87). otable papers due to Comets and eveu ( 95), Shcherbina ( 99) etc. Series of breakthroughs from Talagrand ( present). Groundbreaking contributions from Guerra & Toninelli ( 02) and Guerra ( 03), taken to completion by Talagrand in The Parisi formula. Ann. Math. (2) 163 no 1, Still, a lot of mysteries.

3 The Sherrington-Kirkpatrick model spins. State space: Σ = { 1, 1}. Gibbs measure on Σ for the SK model: where otation: ( β G (σ) = Z 1 exp i<j g ij σ i σ j + h i σ i ) (gij ) i<j is a fixed realization of independent standard gaussian random variables (the disorder), β and h are parameters, Z is the normalizing constant (partition function). f (σ) := σ Σ f (σ)g (σ).

4 High temperature phase Let z be a standard gaussian r.v. Then log Z lim where q is determined by = log 2 + E log cosh(βz q + h) + β2 4 (1 q)2, q = E tanh 2 (βz q + h). Rigorously proven to hold for β < 1/2, all h (Talagrand). Conjectured description of the full high temperature regime: All (β, h) such that β 2 1 E cosh 4 (βz q + h) < 1. The line where equality holds is called the Almeida-Thouless line.

5 The Gibbs measure at high temperature Spins are approximately independent, i.e. with high probability, σi1 σ i2 σ ik σi1 σi2 σik. This is a deep fact and surprisingly hard to prove. Both sides are nondegenerate random variables in the limit. Moreover, σ i1,..., σik are approximately i.i.d. First proved by Talagrand using his cavity argument. If h = 0, then σ i 0 for all i. What if h 0? o simple formulas for σ 1,..., σ.

6 The TAP equations When (β, h) the high temperature regime, the random quantities σ1,..., σ satisfy ( β σi tanh g ij σj + h β 2 (1 q) ) σ i, j i where q solves q = E tanh 2 (βz q + h), z being a standard gaussian r.v. Discovered by Thouless, Anderson, and Palmer ( 77). Rigorous proof by Talagrand ( 03) using the cavity method. Unique solution if β is smaller than a constant. Talagrand also shows that σ i converges in law to tanh(βz q + h).

7 The cavity method Basically, a very complex induction over. Coefficient of σ in the Hamiltonian is 1 β g j σ j + h. j=1 Replace this by an independent gaussian r.v. to get a new Hamiltonian. Determine the mean and variance of this r.v. such that the annealed measures remain approximately the same. Using gaussian interpolation and a certain recursive argument, Talagrand shows that it is possible to do this if β is sufficiently small. This is the foundation of the cavity induction. We will follow a different route, starting from the next slide.

8 Explaining the TAP equations: (I) Local fields Local field at site i: l i = 1 g ij σ j. j i Easy to show: The conditional expectation of σ i given (σ j ) j i is exactly tanh(βl i + h). This gives the equations σi = tanh(βli + h). Thus, if we knew the limiting distribution of l i, σ i could be computed directly. But this was not known previously.

9 Explaining the TAP equations: (II) The Onsager correction If X is a random variable with small variance, then E tanh(ax + b) tanh(ae(x ) + b). This is the usual mean-field approximation. Applying the naïve mean-field logic (although Var(l i ) 0), we may wonder whether σi = tanh(βli + h)? tanh ( β li + h ) ot surprisingly, this is incorrect. In the TAP equations, we have ( β = tanh g ij σj + h ). ( β σi tanh g ij σj + h β 2 (1 q) ) σ i. j i The extra term is called the Onsager correction term in physics. j i

10 Onsager correction and mixture gaussians Let φ µ,σ 2 be the gaussian density with mean µ and variance σ 2. Let ψ p,µ1,µ 2,σ 2 be the mixture gaussian density ψ p,µ1,µ 2,σ 2 = pφ µ 1,σ 2 + (1 p)φ µ 2,σ 2. Suppose µ 2 > µ 1, and we define a = µ 2 µ 1 2σ 2, b = 1 2 log p 1 p µ2 2 µ2 1 4σ 2. Then, if X ψ p,µ1,µ 2,σ2, then E tanh(ax + b) = tanh(ae(x ) + b (2p 1)a 2 σ 2 ). This is what happens! The local fields have mixture gaussian laws and the highlighted term is the Onsager correction term.

11 Our main result: Limit law of local fields Recall: The local field at site i is defined as l i := 1 g ij σ j. For each i, let r i be the random variable r i := 1 g ij σj β(1 q) σi. j i Let ν i be the (random) mixture gaussian probability measure with density function j i p i φ ri +β(1 q),1 q + (1 p i )φ ri β(1 q),1 q, where p i = e βr i +h e βr i +h + e βr i h. Under the Gibbs measure, ν i approximates the law of l i for large.

12 Our main result: Limit law of local fields Theorem Suppose (β, h) is in the high temperature regime and q satisfies q = E tanh 2 (βz q + h), where z is a standard gaussian r.v. Let l 1,..., l be the local fields and let ν 1,..., ν be defined as in the previous slide. Then for any bounded measurable f : R R and any 1 i, we have f E( (li ) ) 2 f (x)ν i (dx) C(β, h) f 2, R where C(β, h) is a constant depending only on β and h. Recall: σ i = tanh(βli + h). Taking f (x) = tanh(βx + h), the TAP equations follow easily. Proof is by Stein s method.

13 Stein s method Let X and Z be two random variables. Suppose we want to show that they approximately have the same distribution. Basic steps in Stein s method: 1. Identify an operator T such that for all functions h, E`Th(Z) = 0. (T is called a Stein characterizing operator.) For example, if Z is standard gaussian, then Th(x) = h (x) xh(x) is a characterizing operator. 2. Given a function f, find h such that Th(x) = f (x) E(f (Z)). Relate the properties of h to those of f. 3. By the definition of h it follows that Ef (X ) Ef (Z) = E(Th(X )). Compute a bound on E(Th(X )) by whatever means possible.

14 Stein s method for gaussian approximation Let Z be a standard gaussian r.v. Then for all h, E(h (Z) Zh(Z)) = 0. Thus, Th(x) := h (x) xh(x) is a characterizing operator for the standard gaussian distribution. Thus, to show that a r.v. W is approximately standard gaussian, one has to show that for all h, E(h (W ) Wh(W )) 0.

15 Stein s method for mixture gaussians For the mixture gaussian density pφ µ1,σ 2 + (1 p)φ µ 2,σ 2 the characterizing operator is ( ) x µ Th(x) = h (x) σ 2 a tanh(ax + b) h(x), where µ, a, b are defined as µ = µ 1 + µ 2, a = µ 2 µ 1 2 2σ 2, b = 1 2 log p 1 p µ2 2 µ2 1 4σ 2. Appears to be naturally connected to physical models. Does not occur in the literature on Stein s method.

16 The Approximation Lemma Lemma Suppose g = (g 1,..., g n ) is a collection of independent standard gaussian random variables, and h 1,..., h n are absolutely continuous functions of g. Then ( n n ) 2 h i E g i h i g i i=1 i=1 n n ( = E(hi 2 hi ) + E i=1 i,j=1 g j h j g i ). Idea: If the right hand side is small, then the lemma generates the equation n n h i g i h i. g i i=1 Can be used to obtain Stein characterizing equations for highly complex functions of gaussian random variables. i=1

17 Example Consider the SK model with zero external field (the simple case): ( β G (σ) = Z 1 exp i<j g ij σ i σ j ). We wish to generate a characterizing equation for the local field at site 1: l 1 = 1 g 1j σ j. j=2 Fix a smooth function f. For each j = 2,...,, let h j = 1 σj f (l 1 ). Then g 1j h j = l 1 f (l 1 ). j=2

18 Example contd. On the other hand, an easy computation gives h j f (l 1 ) + β σ 1 f (l 1 ) β σ j f (l 1 ) σ 1 σ j =. g 1j The 2nd term. ote that l 1 does not depend on σ 1, and the conditional expectation of σ 1 given σ 2,..., σ is tanh(βl 1 ). Thus, σ1 f (l 1 ) = tanh(βl 1 )f (l 1 ). The 3rd term. It follows from the high temperature theory for β < 1 that for 2 j, σ1 σ j σ1 σj = 0.

19 Example contd. Thus, if we can apply the Approximation Lemma, we get the approximation h j g 1j h j j=2 g 1j j=2 which is equivalent to l1 f (l 1 ) f (l 1 ) β tanh(βl 1 )f (l 1 ) 0. ow, the operator Tf (x) = xf (x) f (x) β tanh(βx)f (x) is a Stein characterizing operator for the mixture gaussian density 1 2 φ β, φ β,1. This procedure discovers the limiting distribution of l 1.

20 When h 0 When h 0, l 1 does not have a nonrandom limiting distribution. The situation becomes more complex. Given a function u, we start with a solution f (x, y) of the p.d.e. f (x, y) x = u(x) Then, defining ( ) x y σ 2 β tanh(βx + h) f (x, y) R cosh(βt + h)e (t y) 2 2(1 q) u(t) 1 2π(1 q) cosh(βµ + h)e 2 β2 (1 q) dt. we let r 1 = 1 j=2 g 1j σj β(1 q) σ1, h j = 1 ( σj σ j ) f (l1, r 1 ). The proof is completed by an application of the Approximation Lemma with these h j s.

21 Distribution of σ 1 By the TAP equations, σ1 tanh(βr1 + h), where r 1 = 1 j=2 g 1j σj β(1 q) σ1. Thus, it suffices to find the limiting distribution of r 1. Repeatedly integrating by parts, we can get where η 1 = 1 + βσ 1 E(r 1 f (r 1 )) qe ( f (r 1 ) η 1 ) j=2 g 1j ( σj σ j ) β 2 (1 q) ( 1 σ 1 2 ).

22 Distribution of σ 1 contd. Applying the Approximation Lemma with h j = β ( σ1 σ j σ1 σj ), we can show that η 1 1. Combined with the earlier approximation E(r 1 f (r 1 )) qe ( f (r 1 ) η 1 ), this shows that E(r 1 f (r 1 )) qe(f (r 1 )). Thus, r 1 is a gaussian r.v. with mean zero and variance q in the large limit.

23 Summary and future directions Often in spin glasses and other models we have the situation that for some random quantity X, E tanh(ax + b) = tanh(ae(x ) + b + a correction term). We show that this happens if the distribution of X is a mixture of two gaussian densities. Following this line, we derive the TAP equations for the SK model by showing that the local field is indeed a mixture of two gaussians in the limit. Gives total variation error bounds. The key tool is the Approximation Lemma that generates characterizing equations for Stein s method. The Approximation Lemma can possibly be used to derive/discover limiting distributions of other objects. Results apply only to the high temperature phase. Possible to extend to low temperature???