Simulated Annealing. Simulated Annealing

Transcription

1 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing What is siulate annealing? Siulate annealing an probabilistic inversion Exaples he proble: we nee to efficiently search a possibly ulti-oal function in orer to either saple the function or fin the axiulikely point of of that function. We raw on an analogy fro soli state physics: the annealing process.

2 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing Annealing is the process of heating a soli until theral stresses are release. hen in cooling it very slowly to the abient teperature until perfect crystals eerge. he quality of the results strongly epens on the cooling teperature. he final state can be interprete as an energy state crystaline potential energy which is lowest if a perfectly crystal eerge. But where s the connection to inverse probles?

3 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing Our goal is to saple a ulti-oal function efficiently. We use an analogy between the physical process of annealing an the atheatical proble of obtaining a global iniu of a function. it ay also be turne aroun so that we try an fin the axiu of for exaple a probability ensity l Siulate annealing really ais at fining the axiu likelihoo point l axiu for l We first efine an energy function: S 0 log µ

4 Nonlinear Inverse Probles Siulate Annealing k µ θ Siulate Annealing Siulate Annealing l 0 is a fixe positive nuber tere the abient teperature e.g. 1. We obtain the probability ensity function S exp written out... o log exp 0 he probability thus efine has interesting properties: 0 0 l const δ

5 Nonlinear Inverse Probles Siulate Annealing k µ θ he Heat Bath he Heat Bath o log exp 0 -> peaks as pf -> constant 0 1 S eperature ecreasing

6 Nonlinear Inverse Probles Siulate Annealing θ µ he Heat Bath S eperature ecreasing

7 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing For constant prior enstiy this function resebles he Gibbs istribution giving the probability of State with energ S of a statistical syste at teperature. l he proceure woul be: 1. ake a high teperature heat the syste an generate rano oels this eans we re effectively sapling the prior istribution 2. Cool the syste slowly while continuing to generate rano oels until 0. you shoul now be in the global iniu. he efficienvcy strongly epens on the cooling proceure. If too fast you ay en up in sconary inia. If too slow you will waste a lot of forwar calculations.

8 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing Here is a pseuo-coe. It is only a slight oifiction to the etripolis algorith l Siulate annealing: Define a high teperature Define a cooling scheule it e.g. alpha Define an energy function S Define current_oel initial state While not converge new_oel rano Delta_S Snew_oel-Scurrent_oel If Delta_S < 0 current_oel new_oel Else with probability P e^-delta_s/ : current_oel new_oel alpha

9 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing Cooling scheules coul be: alpha a 0.8 < a < alpha +b b close to 0 alpha c/log1+k k is the iteration nuber an c is a constant

10 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing We will aopt a special approach Rothann 1986: Define a high teperature Define a cooling scheule it e.g. alpha Define an energy function S an the associate pf Define current_oel initial state While not converge new_oel rano calculate Pnew_oel generate rano nuber x in 01 accept with etroplis rule upate

11 Nonlinear Inverse Probles Siulate Annealing θ µ Siulate Annealing Exaple with peaks function: 0 1 a iterations ca. 340 accepte oel upates A posteriori probability y Paccepte oels x # accepte oels

12 Nonlinear Inverse Probles Siulate Annealing θ µ Suary Siulate annealing is an atheatical analogy to a cooling syste which can be use to saple highly nonlinear ultiiensional functions. here are any flavors aroun an the efficiency strongly epens on the particular function to saple. herefore it is extreely ifficult to ake general stateents as to what paraeters work best.