Introduction of Expectation-Maximization Algorithm, Cross-Entropy Method and Genetic Algorithm

Transcription

1 Itroductio of Expectatio-Maximizatio Algorithm, Cross-Etropy Method ad Geetic Algorithm Wireless Iformatio Trasmissio System Lab. Istitute of Commuicatios Egieerig Natioal Su Yat-se Uiversity 2012/07/23 顏愷緯

2 Outlie Itroductio of Expectatio-Maximizatio Algorithm Itroductio of Geetic Algorithm Itroductio of Cross-Etropy Method 2

3 Itroductio of Expectatio-Maximum Algorithm Wireless Iformatio Trasmissio System Lab. Istitute of Commuicatios Egieerig Natioal Su Yat-se Uiversity 2010/08/03 顏愷緯

4 Expectatio-Maximizatio Algorithm Expectatio-maximizatio (EM) algorithm is foud ad give its ame i 1970s. EM algorithm is a algorithm for fidig maximum likelihood (ML) estimates of parameters i probabilistic models, where the model depeds o uobserved latet variables. It is a efficiet ad quite useful algorithm for fidig the ML estimatio of parameters whe the data model is easy to preset.

5 Expectatio-Maximizatio Algorithm EM alterates betwee performig the followig two steps: expectatio (E) step, which computes a expectatio of the likelihood by icludig the latet variables as if they were observed. maximizatio (M) step, which computes the maximum likelihood estimates of the parameters by maximizig the expected likelihood foud o the E step. The parameters foud o the M step are the used to begi aother E step, ad the process is repeated.

6 Expectatio-Maximizatio Algorithm Assume that X is the observable radom vector, z is the missig radom vector. I maximum likelihood (ML) estimatio for θ, we wat to fid θ such that p X is a maximum. The log likelihood fuctio is kow as L p X l. The EM algorithm is a iterative process for maximizig L.

7 Expectatio-Maximizatio Algorithm We would like to fid a ew θ such that where L L, meas the value of the th iteratio. The probability p X ca be writte i terms of the hidde variables z as X X z z p p, p. z Usig Jese s iequality, it was show that, l v l v. i i i i i1 i1

8 Expectatio-Maximizatio Algorithm The the equatio L L ca be expressed as L L p p p z l X z, z l X p zx, l p, p l p X z z X z p, zx X z, pz pzx, p l p, l p z X z p X z, pz pzx, p z X, l l p X z, pz pzx, X p pz X, l p, l p z X X z z z z p p X z, p z zx, l. p, p z X X X

9 For coveiece, we defie the followig equatio Sice L, it ca be said that is bouded by the likelihood fuctio L ad that the value of the fuctios ad X z, pz p L p zx, l L., p p z z X X L Expectatio-Maximizatio Algorithm are equal at curret estimate because that X z, pz z X, p X p L pzx, l L. z p

10 Therefore ay θ which icreases also icrease the. I order to achieve the greatest possible icrease i the value of L, the EM algorithm calls for selectig θ such that is maximized. Formally, we have Expectatio-Maximizatio Algorithm 1 arg max X z, pz L p arg max, L p zx l., p p z z X X

11 Expectatio-Maximizatio Algorithm Drop the term L which is costat with respect to θ. The we ca obtai z X X z z 1 arg max p, l p, p z arg max p z X, l p X, z z arg max E l p,., X z zx

12 Expectatio-Maximizatio Algorithm Thus, the EM algorithm cosists of iteratig the: E-step: Determie the coditioal expectatio 1 Xz Q E l p, zx, M-step: Maximize this expressio with respect to θ. We ca cosider the set of observatios X as beig icomplete i relatio to the complete set {X, z}.

13 Expectatio-Maximizatio Algorithm 1 If the likelihood fuctio is bouded, ad if Q is cotiuous, the log-likelihood fuctio l p X coverges to a statioary poit (a local maximum). Whe the likelihood fuctio has several maxima, the EM algorithm is ot guarateed to coverge to the global maximum. EM is particularly useful whe maximum likelihood estimatio of a complete data model is easy. If Closed form estimators exist, the M step is ofte trivial.

14 Expectatio-Maximizatio Algorithm

15 Expectatio-Maximizatio Algorithm (Example) We trasmit the complex symbol s t ad we receive y tm o receive ateas. The received sigal ca be expressed as y h s, t 1,2,, T; m 1,2,, M. tm m t where h m is the complex-valued fadig coefficiet betwee the trasmitter ad the mth receiver. The fadig coefficiets are assumed to be idepedet ad CN(0, 1) distributed. Let T be the legth durig which chael state iformatio (CSI) remais costat.

16 Expectatio-Maximizatio Algorithm (Example) If oe block of T symbols are set, the received sigal blocks by M receive ateas ca be expressed by a matrix Y of M T. It ca be writte as Y hs N H The oise is assumed to be white, i.e., E Nt N, 1 t Σ t 2 1t2 MM where Σ is the oise covariace matrix.

17 Expectatio-Maximizatio Algorithm (Example) We defie a complete data, X = (Y,h), for the parameter that we wat to estimate. E-step: The E-step of EM algorithm requires the calculatio of where k Q s s E f [ ] [ k l X s Y, s ]. Q s s [ k ] M-step: M-step is to fid the s that maximize the Q fuctio as s arg max Q s s. [ k1] [ k] s

18 Itroductio of Cross-Etropy Method Wireless Iformatio Trasmissio System Lab. Istitute of Commuicatios Egieerig Natioal Su Yat-se Uiversity 2010/08/03 顏愷緯

19 Cross-Etropy Method The Cross-Etropy Method was origially developed as a simulatio method for the estimatio of rare evet probabilities: Estimate P S X X : radom vector/process takig values i some set χ. S: real-valued fuctio o χ. It was soo realized that the CE Method could also be used as a optimizatio method: Determie max S x X 19

20 Cross-Etropy Method May real-world optimizatio problems ca be formulated mathematically as either (or both) Locate some elemet i a set χ such that where S is a objective fuctio or performace measure defied o χ. x * * S x S x for all x * * Fid S x, the globally maximal value of the fuctio. The CE method ivolves a iterative procedure where each iteratio ca be broke ito two phase: Geerate a radom, data sample accordig to specified mechaism. Update the parameters of the radom mechaism based o the data to produce a better sample i the ext iteratio. 20

21 Geeral CE Algorithm: 1) Iitialize the parameters Cross-Etropy Method 2) Geerate the radom sample from the desity fuctio which is updated i previous iteratio ad calculate the fitess value. 3) Update the probability desity fuctio of the radom sample based o the acquired iformatio to produce a better solutio i the ext iteratio. 4) Retur to Step 2 util some stoppig criterio is met. 21

22 Cross-Etropy Method (Example 1) Cosider a biary vector y y1, y2,, y. Suppose that we do t kow which compoets of y are 0 ad which are 1. We have a oracle which for each biary iput vector returs the performace or respose, j j1 S x x y represetig the umber of matches betwee the elemets of x ad y. j, x x1, x2,, x 22

23 Cross-Etropy Method (Example 1) 23

24 Cross-Etropy Method (Example 1) 1) Start with iteratio umber t 1 ad probability 2) Draw a sample X,, 1 XN of Beroulli vectors with success probability vector. p t1 3) Calculate the performaces S X i for all i, ad order them from the smallest to biggest. Let t be 1 sample quatile of the performaces: t S. 1 N p0 0.5,,

25 Cross-Etropy Method (Example 1) 4) Use the same sample to calculate pt pt,1, pt,2,, pt, via p t, j N I j1 N j 1 S Xi t Xij 1 I S X i I t, j = 1,,, where X X, X, X, ad icrease t by 1. i i1 i2 i 5) If the stoppig criterio is met, the stop; otherwise set t = t + 1 ad reiterate from step 2 ad step 3. 25

26 Cross-Etropy Method (Example 1) As a example cosider the case = 10, N = 50 ad ρ = 0.1 where y 1,1,1,1,1,0,0,0,0,0. 26

27 Cross-Etropy Method (Example 2) 27

28 Cross-Etropy Method (Example 2) S = ilie( exp(-(x - 2).^2)) * exp(-(x + 2).^2) ); mu = -10; sigma = 10; rho = 0.1; N = 100; eps = 1E-3; t = 0; while sigma > eps t = t + 1; x = mu + sigma * rad(n,1); SX = S(x); % compute the performace mu = mea(sortsx((1 - rho) * N:N,1)); sigma = std(sortsx((1 - rho) * N:N,1)); ed 28

29 Cross-Etropy Method (Example 2) 29

30 Cross-Etropy Method Some strategies to improve the performace of CE method: Elite reservatio. Smoothig factor. The umber of ρ. 30

31 Itroductio of Geetic Algorithm Wireless Iformatio Trasmissio System Lab. Istitute of Commuicatios Egieerig Natioal Su Yat-se Uiversity 2010/08/03 顏愷緯

32 Geetic Algorithm Geetic Algorithm (GA), first itroduced by Joh Hollad i the early seveties, is the powerful stochastic algorithm based o the priciples of atural selectio ad atural geetics. They are used as optimizatio methods ad have show their effectiveess i various problems. GAs evolves solutios i a iterative maer by applyig geetic operators to a pool of cadidate solutios. To solve a problem, a GA maitais a populatio of idividuals ad probabilistically modifies the populatio by some geetic operators such as selectio, crossover ad mutatio, with the itet of seekig a suboptimal solutio to the problem. 32

33 Geetic Algorithm Start 1 Iitial populatio Geetic Operators (mutatio ad crossover) Compute the fitess fuctio Termial criteria No 2 Yes 2 Geetic Operators (selectio) ed 1 33

34 Geetic Algorithm I GA, each idividual i a populatio is usually coded as a fixedlegth biary strig. The legth of the strig depeds o the domai of the parameters ad the required precisio. For example, if the domai of the parameter x is [-2,5] ad the precisio requiremet is six places after the decimal poit, the the domai [-2,5] should be divided ito 7,000,000 equal size rages. That implies the legth of the strig requires to be 23, for the reaso that =2 22 < <2 23 =

35 Geetic Algorithm The iitial process is quite simple. We create a populatio of idividuals, where idividual i a populatio is a biary strig with a fixed-legth, ad every bit of the biary strig is iitialized radomly. I each geeratio for which the GA is ru, each idividual i the populatio is evaluated agaist the ukow eviromet. The fitess values are associated with the values of objective fuctio. 35

36 Geetic Algorithm To perform geetic operators, oe must select idividuals i the populatio to be operated o.the selectio strategy is chiefly based o the fitess level of the idividuals actually preseted i the populatio. There are may differet selectio strategies based o fitess, such as Roulette Wheel Selectio, Elitism Strategy ad Touramet Selectio. 36

37 Geetic Algorithm Roulette Wheel Selectio: Step1. evaluate the fitess values for every idividual, s k, k = 1,2,, N. Step2. evaluate the total fitess values of all the idividuals, T. Step3. calculate ratio of fitess of each idividual. x x x x x x x x x x 1 x 2 x 2 x 3 x 4 x 5 x 6 x 7 x 7 37

38 Geetic Algorithm The crossover operator starts with two selected idividuals ad the the crossover poit is selected radomly accordig to a give probability of crossover. The secod geetic operator, mutatio, itroduces radom chages i structures i the populatio, ad it may occasioally have beeficial results: escapig from a local optimum. I biary form GA, mutatio is just to egate every bit of the strigs, i.e., chages a 1 to 0 ad vice versa, with give probability of mutatio. 38

39 Geetic Algorithm Oe-Poit Crossover Two-Poit Crossover

40 Geetic Algorithm Biary form Real-umber form

41 Global Optimizatio Toolbox [x,fval,exitflag,output,populatio,scores] = ga(@fitessfc,vars,a,b,aeq,beq,lb,ub,olco,optios) It fids a local miimum x to fitessfc, subject to the liear iequalities A x b as well as the liear equalities Aeq x = beq. fitessfc accepts iput x ad returs a scalar fuctio value evaluated at x. If the problem has m liear iequality costraits ad variables, the A is a matrix of size m-by- variables. b is a vector of legth m. 41

42 Global Optimizatio Toolbox UB ad LB defie a set of lower ad upper bouds o the desig variables, x, so that a solutio is foud i the rage LB x UB. Use empty matrices for LB ad UB if o bouds exist. optios = gaoptimset('param1',value1,'param2',value2,...) creates a structure called optios ad sets the value of param1 to value1, param2 to value2, ad so o 42

43 Global Optimizatio Toolbox fitessfc vars Aieq Bieq Aeq Beq lb ub olco optios Fitess fuctio Number of desig variables A matrix for liear iequality costraits b vector for liear iequality costraits A matrix for liear equality costraits b vector for liear equality costraits Lower boud o x Upper boud o x Noliear costrait fuctio Optios structure created usig gaoptimset or the Optimizatio Tool 43

44 Global Optimizatio Toolbox Optio Descriptio Values CreatioFc Hadle to the fuctio that creates the oliearfeasible CrossoverFractio The fractio of the populatio at the ext geeratio, ot icludig Positive scalar {0.8} elite childre, that is created by the crossover fuctio MutatioFc Hadle to the fuctio that produces EliteCout Positive iteger specifyig how may idividuals i the curret Positive iteger {2} geeratio are guarateed to survive to the ext geeratio. Not used i gamultiobj. Geeratios Positive iteger specifyig the maximum umber of iteratios Positive iteger {100} before the algorithm halts IitialPealty Iitial value of pealty parameter Positive scalar {10} IitialPopulatio Iitial populatio used to seed the geetic algorithm; ca be partial Matrix {[]} PopulatioSize Size of the populatio Positive iteger {20} PopulatioType Strig describig the data type of the populatio 'bitstrig' 'custom' {'doublevec tor'} to 'bitstrig' or 'custom'. StallGeLimit Positive iteger. The algorithm stops if there is o improvemet i Positive iteger {50} the objective fuctio for StallGeLimit cosecutive geeratios. TolCo Positive scalar. TolCo is used to determie the feasibility with Positive scalar {1e-6} respect to oliear costraits. TolFu Positive scalar. The algorithm rus util the cumulative chage i the fitess fuctio value over StallGeLimit is less tha TolFu. Positive scalar {1e-6}

45 Global Optimizatio Toolbox (Example 1) 45

46 Global Optimizatio Toolbox (Example 1) fuctio [ f ] = objectfuc_coti(x) f = -( x(1)*si(4*pi*x(1)) + x(2)*si(20*pi*x(2))); Lb = [-3 ; 4.1]; Ub = [ 12.1 ; 5.8]; optios = gaoptimset(@ga); optios = gaoptimset(optios,... 'MutatioFc',{@mutatioadaptfeasible,0.001},... 'CrossoverFractio',0.8,... 'PopIitRage', [-3 4.1; ],... 'PopulatioSize',100,... 'Geeratio',1000,... [x fitval] = ga(@objectfuc_coti,2,[],[],[],[],lb,ub,[],optios); 46

47 Global Optimizatio Toolbox (Example 2) Example y = [ ] 47

48 Global Optimizatio Toolbox (Example 2) fuctio [ f ] = objectuc_dis(x) y = [ ] for ii = 1 :10 Sx(ii) = abs(x(ii)-y(ii)); ed f = sum(sx); optios = gaoptimset(@ga); optios = gaoptimset(optios,... 'PopulatioSize',50,... 'Geeratio',3,... 'PopulatioType','bitstrig'); [x,fval,exitflag,output,populatio,scores] = ga(@objectuc_dis,10,[],[],[],[],[],[],[],optios); 48

49 Assigmet Wireless Iformatio Trasmissio System Lab. Istitute of Commuicatios Egieerig Natioal Su Yat-se Uiversity 2012/07/23 顏愷緯

50 DSI Method Sequece 50

51 DSI Method Simulatio coditio BPSK Total subcarriers = 64 Data subcarriers = 56 DSI subcarriers = 8 Oversample = 4 51

52 DSI Method GA ad CE Method Populatio size =10 Maximum iteratio =6 Elitist=1 Mutatio probability = 0.8 Crossover = 0.3 Fitess fuctio: 2 max x 0 1 PAPR= LN 2 Ex 52

53 CCDF DSI Method 10 0 DSI simlatio with GA ad CE method Origial GA CE PAPR 0 53

54 DSI Method Referece: H.-G. Ryu, J.-E. Lee, J.-S. Park, Dummy Sequece Isertio (DSI) for PAPR Reductio i the OFDM Commuicatio System, IEEE Tras. o Cosumer Electroics, Vol. 50, o 1, Feb 2004, pp