Biitlli Biointelligence Laboratory Lb School of Computer Science and Engineering. Seoul National University

Size: px

Start display at page:

Download "Biitlli Biointelligence Laboratory Lb School of Computer Science and Engineering. Seoul National University"

Maude Jennings
5 years ago
Views:

1 Monte Carlo Samling Chater Course on Probabilistic Grahical Models Artificial Neural Networs, Studies in Artificial i lintelligence and Cognitive i Process Biitlli Biointelligence Laboratory Lb School of Comuter Science and Engineering Seoul National University

2 Overview Introduction ti Transformation Method Rejection Samling Imortance Samling Samling-Imortance-Resamling SIR Marov Chain Monte Carlo Methods Metroolis Algorithm Metroolis-Hastings Algorithm Gibbs Samling Evolutionary MCMC c 2009 SNU CSE Biointelligence Lab. 2

3 Introduction: ti The Problem 1/4 1. Generate samles from a given robability distribution Px. 2. Finding the exectation of some function fx with resect to a robability distribution Px. Can be aroximated by samling indeendent oints from the distribution P and summation. The accuracy does not deend on the dimensionality of the sace to be samled. c 2009 SNU CSE Biointelligence Lab. 3

4 Intro: Monte Carlo Samling 2/4 General tools for aroximating robability distributions Comutational techniques using random numbers Two main alications of Monte Carlo methods 1. Generate samles { x = 1,..., K} from a given.d.f. x 2. Estimate exectation of a function fx under x using random samles { x = 1,..., K} E [ f x] < f x> x f x dx = = K 1 f x K = 1 c 2009 SNU CSE Biointelligence Lab. 4

5 Introduction ti 3/4 Why is samling from Px hard? We tyically do not now the normaliing constant. There is no obvious way to samle from P without enumerating most or all of the ossible states. c 2009 SNU CSE Biointelligence Lab. 5

6 Introduction ti 4/4 Uniform samling Drawing random samles uniformly from the state sace and evaluating P * x at those oints This will only stand a chance of giving ing a good estimate if we mae the number of samles R sufficiently large that we are liely to hit the tyical set at least once or twice. Uniform samling is unliely to be useful in most high-dimensional distribution roblems. c 2009 SNU CSE Biointelligence Lab. 6

7 Transformation Method Transform samles from uniform distribution. P ~ U 0,1 = h y y P yˆ dyˆ y = h 1 = f Can we get h -1 always? c 2009 SNU CSE Biointelligence Lab. 7

8 Rj Rejection Samling 1/2 Assumtions Samling directly from target distribution Px is difficult. Estimating Qx is easy for any value of x. How to choose q?? 1. Generate 0 from qx. 2. Generate u 0 from uniform distribution [0, q 0 ]. 3. If u 0 > x, reject 0, u Otherwise, accet 0, u 0. c 2009 SNU CSE Biointelligence Lab. 8

9 Rj Rejection Samling 2/2 In high hdimensions, i If is large, the accetances will be very rare. In general grows exonentially with the dimensionality N, so the accetance rate is exected to be exonentially small in N. c 2 N / 2 q σ q = ex ln 2 N / 2 σ 2πσ = N 2πσ When σ N = 1000, q = 1.01, c σ 20,000 High rejection ratio Accetance ratio is exected to be exonentially small with the dimensionality N. c 2009 SNU CSE Biointelligence Lab. 9

10 Imortance Samling 1/3 A method for estimating the exectation of a function A generaliation of the uniform samling We have a simler density Qx from which we can generate samles and which we can evaluate to within a multilicative constant. E [ f ] = f d = f q d q L l 1 l f l q L i = 1 Samle from q weighted by. c 2009 SNU CSE Biointelligence Lab. 10

11 Imortance Samling 2/3 Deends on the choice of Q. Gaussian Samler Cauchy Samler In high dimensions, We need to obtain samles that lies in the tyical set of P,, and this may be tae a long time unless Q is a good aroximation to P. The variation of weights q/ is large order of exn 1/2, unless Q is a near-erfect aroximation to P. c 2009 SNU CSE Biointelligence Lab. 11

12 Imortance Samling 3/3 Samling-Imortance-Resamling SIR It is difficult to set in rejection samling 1. Samling from q. 2. Set weight on each samle as in imortance samling. 3. Resamle from the samles. Final samles aroximate as the samle sie increases. c 2009 SNU CSE Biointelligence Lab. 12

13 Marov Chain Monte Carlo Method 1/5 Allows samling from a large class of distributions. Scales well with the dimensionality of the samle sace. Basic Metroolis Algorithm a simle MCMC Maintain a record of state t : forms a Marov chain. Next state is samled from q t q must be symmetric. Candidate state from q is acceted with robability of ~ t A, = min1, ~ t If rejected, current state is added to the record and becomes the next state. Distribution of tends to in the infinity. The original sequence is autocorrelated and get every M th samle to get indeendent samles. For large M, the retained samles will be indeendent. c 2009 SNU CSE Biointelligence Lab. 13

14 Marov Chain Monte Carlo Method 2/5 c 2009 SNU CSE Biointelligence Lab. 14

15 Marov Chain Monte Carlo Method 3/5 Under what conditions will a Marov chain converge to the desired distribution? If we set u a Marov chain such that the desired distribution is invariant with resect to the chain. Transition robabilities do not change over time. For a homogeneous Marov chain with transition robabilities T,, a distribution * is invariant with resect to the chain if, ~ = T, ~ The marginal distribution for each state does not change after transitions. A sufficient i not necessary condition for ensuring the required distribution ib ti is invariant is to choose the transition robabilities to satisfy the roerty of detailed balance: reversible Marov chain. ~ T, = ~ T, ~ T, = ~ T, = ~ = ~ For ergodic Marov chain, the distribution t converges to the required invariant distribution irresective to initial distributions 0. c 2009 SNU CSE Biointelligence Lab. 15

16 Marov Chain Monte Carlo Marov Chain Monte Carlo Ma o C a Mo te Ca o Ma o C a Mo te Ca o Method Method Metroolis-Hastings algorithm Generaliation of Metroolis algorithm Q can be non-symmetric Modified accet robability: P f fd t il db l = ~ ~ 1, min, * * * * t t t t q q A Proof of detailed balance ~ ~ min ', ' ~ ', ~ q q A q T = = ~ 1 i ~ ~, ~ min, min q q q q q = = c 2009 SNU CSE Biointelligence Lab. 16 ', ' ~, ~ ~ 1, min ~ T A q q q q = = =

17 Marov Chain Monte Carlo Method 5/5 The common choice for q is Gaussian Trade-off between ste sie and convergence time Small variance high accetance ratio slow random wal. c 2009 SNU CSE Biointelligence Lab. 17

18 Gibb S li 1/3 Gibb S li 1/3 Gibbs Samling 1/3 Gibbs Samling 1/3 Simle and widely alicable MCMC algorithm. Secial case of Metroolis-Hastings algorithm. Each ste relaces the value of one of the variables by a value drawn from the distribution of that variable conditioned on the values of the remaining variables. values of the remaining variables. The rocedure 1. Initialie i, i=1,, M 2. For t=1,,t Samle,, ~ t M t i t +,,,,, ~ t M t i t i t i t i ,,, ~ t M t t i t + +,,,,, M i i i i + c 2009 SNU CSE Biointelligence Lab. 18,, ~ t M t i t M

19 Gibb S li 2/3 Gibb S li 2/3 Gibbs Samling 2/3 Gibbs Samling 2/3 1. = \i i \i is an invariant of each of Gibbs samling stes and whole Marov chain. At each ste, the marginal distribution \i is invariant. Each ste correctly samles from the conditional distribution i \i. 2 The Marov chain defined is ergodic 2. The Marov chain defined is ergodic. The conditional distribution must be non-ero. The Gibbs samling correctly samles from. The Gibbs samling correctly samles from. Gibbs samling as an instance of Metroolis-Hastings algorithm. A ste involving in which \ remain fixed. \ Transition robability q * = * \, 1 * \ * \ * \ * * * * = = = q A \ * \ = 1, \ * \ \ * = = = q A c 2009 SNU CSE Biointelligence Lab. 19

20 Gibbs Samling 3/3 Random wal behavior The number of stes needed to obtain indeendent samles is of order L/l 2. Over-relaxation The ractical alicability deends on the ease of samling from the conditional distributions c 2009 SNU CSE Biointelligence Lab. 20

21 Rf References Pattern Recognition and Machine Learning Ch. 11 Samling Methods C. Bisho, Sringer, E-Boo available from library. Information Theory, Inference, and Learning Algorithms Ch. 29 Monte Carlo Methods D. MacKay, Cambridge University Press, Pdf file available from htt:// hy cam ac c 2009 SNU CSE Biointelligence Lab. 21

22 Evolutionary MCMC & Estimation of Distribution Algorithms EDA Course on Probabilistic bili Grahical Models Artificial Neural Networs, Studies in Artificial Intelligence and Cognitive Process

23 Evolutionary Comutation ti Basic concet Use of Darwinian-lie evolutionary rocesses to solve difficult comutational roblems. Hence the name, Evolutionary Comutation Biological i l evolutionary rocess Poulation of arents roduce descendants Descendants are changed from their arents Selective survival of descendants for next generation EVOLUTION Individual Fitness Environment PROBLEM SOLVING Candidate Solution Quality Problem c 2009 SNU CSE Biointelligence Lab. 23

24 Evolutionary Comutation ti General Framewor Generate Initial Poulation Fitness Function Evaluate Fitness Termination Condition? Yes Best Individual No Select Parents Crossover, Mutation Generate New Offsring 24

25 Evolutionary Comutation ti Tyical run c 2009 SNU CSE Biointelligence Lab. 25

26 Paradigms aadg in Evolutionary outo ay Comutation Evolutionary Programming EP [L. Fogel et al., 1966] FSMs, mutation only, tournament selection Evolution Strategy ES [I. Rechenberg, 1973] Real values, mainly mutation, raning selection Genetic Algorithm GA [J. Holland, 1975] Bitstrings, mainly crossover, roortionate selection Genetic Programming GP [J. Koa, 1992] Trees, mainly crossover, roortionate selection C SNU CSE Biointelligence Lab 26

27 Evolutionary Comutation ti Reresentations Candidate solutions individuals exist in henotye sace. They are encoded in chromosomes, which exist in genotye sace. Encoding : henotye genotye not necessarily one to one Decoding : genotye henotye must tbe one to one In order to find the global otimum, every feasible solution must be reresented in genotye sace. Poulation Usually has a fixed sie and is a multiset of genotyes Some sohisticated EAs also assert a satial structure on the oulation e.g., a grid. Diversity of a oulation refers to the number of different fitnesses / henotyes / genotyes resent note not the same thing. 27

28 Evolutionary Comutation ti Fitness function Reresents the requirements that the oulation should adat to a..a. quality function or objective function Assigns a single real-valued fitness to each henotye which forms the basis for selection So the more discrimination different values the better Tyically we tal about fitness being maximised Some roblems may be best osed as minimisation roblems, but conversion is trivial. c 2009 SNU CSE Biointelligence Lab. 28

29 Evolutionary Comutation ti Parent selection mechanism Assigns robabilities for an individual to be selected as arents. Selection robabilities are relative to current oulation. Different robabilities can be assigned to the same individuals. Usually deends on the individual s fitness and robabilistic. High quality solutions more liely to become arents than low quality ones Selection ressure the degree of correlation between the individual s fitness and its selection robability. High selection ressure results in reducing search scoe. Even worst in current oulation usually has non-ero robability of becoming a arent This stochastic nature can aid escae from local otima. c 2009 SNU CSE Biointelligence Lab. 29

30 Evolutionary Comutation ti Crossover Recombination Mix information from arents into offsring in stochastic way. Hoe is that some are better by combining elements of genotyes that lead to good traits. Mutation It is alied to one genotye and delivers a slightly modified mutant, the child or offsring of it. New elements can be introduced to the oulation. The role of crossover/mutation is different in various subtyes. c 2009 SNU CSE Biointelligence Lab. 30

31 Evolutionary Comutation ti Examles of crossover/mutation c 2009 SNU CSE Biointelligence Lab. 31

32 Estimation of Distribution Algorithms EDAs EDAs relace crossover and mutation with building and samling robabilistic model. Junction between een evolutionary comutation and machine learning grahical models. c 2009 SNU CSE Biointelligence Lab. 32

33 Estimation of Distribution Algorithms EDAs Comarison with conventional genetic algorithms Both erform similar tas Generate new solutions using robability distribution based on selected solutions. Genetic algorithms Probability distribution of selected solutions is built and samled imlicitly by crossover/mutation. EDAs Exlicit robabilistic model of selected candidate solutions is built and samled. c 2009 SNU CSE Biointelligence Lab. 33

34 Estimation of Distribution Algorithms EDAs Objective: finding aroriate model that reresent the distribution of good candidate solutions. EDAs are classified by the robabilistic models they use. Univariate models Assumes no deendency between variables. PBIL Baluja, 94, UMDA Mühlenbein & Paass, 96, cga Hari et al., 97 Bivariate models Assumes binary deendency between air of variables. MIMIC debonet et al, 97, COMIT Baluja and Davis, 97, BMDA Pelian and Muhlenbein, 99 Multivariate models Assumes most general deendency between grous of variables. ECGA Hari, 99, FDA Mühlenbein &Mahnig, 99, BOA Pelian et al, 99 c 2009 SNU CSE Biointelligence Lab. 34

35 EDAs with Univariate i Models Bits that erform get more coies. The context of each bit is ignored. c 2009 SNU CSE Biointelligence Lab. 35

36 EDAs with Univariate i Models Examle 1: OneMax function. f X = X i= 1 X i c 2009 SNU CSE Biointelligence Lab. 36

5 Tra5 ones = 4 ones if ones = 5 otherwise 1 1 0 1 0 0

37 EDAs with Univariate i Models Examle 2: Tra5 function. Inut string is artitioned into disjoint grous of 5 bits. 5 Tra5 ones = 4 ones if ones = 5 otherwise = 3 c 2009 SNU CSE Biointelligence Lab. 37

38 EDAs with Univariate i Models Why failure in Tra5 function? Tra50****=2 Tra51****=1.375 Single bits are misleading. Consider a grou of related bits as one. Bivariate models Multivariate models Requires measure for model comarison. c 2009 SNU CSE Biointelligence Lab. 38

39 EDAs with Bivariate i Models c 2009 SNU CSE Biointelligence Lab. 39

EDAs with Bivariate i Models Tree model in

Start with tree with no edges Add edge with

40 EDAs with Bivariate i Models Tree model in COMIT. How to learn a tree model? Find tree that maximies mutual information between connected nodes. Start with tree with no edges Add edge with large mutual information greedy search. c 2009 SNU CSE Biointelligence Lab. 40

41 EDAs with Multivariate i t Models c 2009 SNU CSE Biointelligence Lab. 41

42 EDAs with Multivariate i t Models Probability Model in ECGA Extended Comact GA. Disjoint grous of variables c 2009 SNU CSE Biointelligence Lab. 42

43 EDAs with Multivariate i t Models Learning Model in ECGA. Start with each bit in a searate grou. Each iteration merges two grous for best imrovement in minimum descrition length. MDL M, D D D Mdl Model Data = g G = N 2 = D g 1 Model log N + X log Samling in ECGA x D Data X Samle grou of bits at a time. Based on the observed frequencies. c 2009 SNU CSE Biointelligence Lab. 43

44 EDAs with Multivariate i t Models Bayesian networ model in BOA Bayesian Otimiation Algorithm. Directed acyclic grah. Learning BN model. Start with emty networ. Add edge / Delete edge / Reverse edge that imroves the scoring metric the most until no more imrovement is ossible. c 2009 SNU CSE Biointelligence Lab. 44

45 EDAs with Multivariate i t Models Scoring metric for Bayesian networ. Bayesian metric: Bayesian-Dirichlet with lielihood equivalence Minimum descrition length metric: Bayesian information criterion BIC c 2009 SNU CSE Biointelligence Lab. 45

46 Rf References Introduction to Evolutionary Comuting A. E. Eiben and J. E Smith, Sringer, Estimation of Distribution Algorithms P. Larranaga and J. A. Loano Eds., Kluwer, Further information Conferences IEEE Congress on Evolutionary Comutation ti CEC Genetic and Evolutionary Comutation Conference GECCO Parallel Problem Solving from Nature PPSN Foundation of Genetic Algorithms FOGA Journals IEEE Transactions on Evolutionary Comutation Evolutionary Comutation Genetic Programming and Evolvable Machines c 2009 SNU CSE Biointelligence Lab. 46

Estimation-of-Distribution Algorithms. Discrete Domain.

Estimation-of-Distribution Algorithms. Discrete Domain. Petr Pošík Introduction to EDAs 2 Genetic Algorithms and Epistasis.....................................................................................