Single Variable Optimization

Transcription

1 8/4/07 Course Instructor Dr. Raymond C. Rump Oce: A 337 Phone: (95) E Mal: rcrump@utep.edu Topc 8b Sngle Varable Optmzaton EE 4386/530 Computatonal Methods n EE Outlne Mathematcal Prelmnares Sngle Varable Optmzaton Parabolc nterpolaton Newton s method Golden secton search D Optmzaton

2 8/4/07 Mathematcal Prelmnares D Optmzaton 3 Recall Dervatve Tests Frst order dervatves tell us slope and whether we have reached an etremum or not The sgn o the second order dervatves tell us whether our etremum s a mnmum or a mamum. 0 mnmum 0 0 mamum D Optmzaton 4

3 8/4/07 Sngle Varable Optmzaton: Parabolc Interpolaton D Optmzaton 5 Formulaton o the Method We t () to a polynomal and then use the rst dervatve test to nd the etremum. Step Pck three ponts that span an etremum and and and Step Ft the ponts to a polynomal a a a 0 a a a a a a a a a Step 3 se the rst dervatve test to nd etremum a0 a 3 3 3a 0 a a a a e e e a a a Step 4 Ater workng through the algebra, we get a nal epresson or the etremum e D Optmzaton 6 a a 3

4 8/4/07 Vsualzaton o the Method Appromated Etremum True Etremum e 3 D Optmzaton 7 Sngle Varable Optmzaton: Newton s Method D Optmzaton 8 4

5 8/4/07 Formulaton o the Method Recall that the Newton Raphson method was used to nd the zero o a uncton. Startng wth an ntal guess or the root, the ollowng equaton was terated untl convergence. We can easly convert ths nto an algorthm or ndng an etremum nstead o a zero. Dene an aulary uncton g() that s the rst dervatve o (). d g d The aulary uncton g() wll have a zero at an etremum o (). Ths means we can perorm the Newton Raphson method on g() to nd an etremum o (). g g D Optmzaton 9 Sngle Varable Optmzaton: Golden Secton Search D Optmzaton 0 5

6 8/4/07 Step Dene Interval We try to pck lower and upper lmts, and, so that () has only one etremum nsde the nterval. D Optmzaton Step Evaluate the Functon at the Bounds D Optmzaton 6

7 8/4/07 Step 3 Pck Two Intermedate Ponts and d d d d d R D Optmzaton 3 5 R Step 4 Evaluate the Functon at Ponts and D Optmzaton 4 7

8 8/4/07 Step 5a Determne the Poston o the Etremum < Etremum s on rght sde I >, then the etremum s on the let sde o the nterval between and. I <, then the etremum s on the rght sde o the nterval between and. D Optmzaton 5 Step 5b Adjust Ponts R D Optmzaton 6 8

9 8/4/07 Step 5c Determne the Poston o the Etremum > Etremum s on let sde I >, then the etremum s on the let sde o the nterval between and. I <, then the etremum s on the rght sde o the nterval between and. D Optmzaton 7 Step 5b Adjust Ponts R D Optmzaton 8 9

10 8/4/07 Step 5c Repeat ntl Convergence Converged : tolerance D Optmzaton 9 Step 6 Calculate Fnal Answer We estmate the nal etremum to be at the mdpont o the last nterval. e D Optmzaton 0 0

11 8/4/07 Algorthm Summary. Dene startng bounds and.. Evaluate the uncton at the two boundng ponts. 3. Pck two ntermedate ponts, and, usng Golden rato. 5 R R R 4. Evaluate uncton at the two ntermedate ponts. 5. pdate bounds by dentyng poston o the etrema. I >, etrema s on let sde. R I <, etrema s on rght sde. R 6. Repeat Step 5 untl convergence tolerance 7. Calculate nal answer e e e D Optmzaton Anmaton o the Method D Optmzaton

12 8/4/07 Dervaton o Golden Rato ( o ) From ths pcture, we dene three length parameters. 0 0 Recognzng that we want the ponts on the net teraton to le on top o ponts rom the prevous teraton, we dene two condtons to ensure ths. Condton ensures l + l covers entre span. 0 0 Condton ensures the net teraton has the same proportonal spacng as the current teraton. D Optmzaton 0 3 Dervaton o Golden Rato ( o ) 0 0 Condton ensures l + l covers entre span. Condton ensures the net teraton has the same proportonal spacng as the current teraton. Substtute Condton nto Condton to elmnate l 0. 0 Dene the Golden rato as R = l /l. R R R R 0 We solve or R usng the quadratc ormula. b b 4ac 4 5 R a We pck the postve root to keep R postve D Optmzaton 4 R

13 8/4/07 The Magc o the Golden Secton Search Iteraton Iteraton + Three o the our ponts lne up rom one teraton to the net. Ths means the uncton only has to be evaluated at one new pont each teraton. The nterval s reduced by 38.% each teraton. D Optmzaton 5 3