Lecture-7. Homework (Due 2/13/03)

Size: px

Start display at page:

Download "Lecture-7. Homework (Due 2/13/03)"

Gavin Baker
5 years ago
Views:

1 Leture-7 Ste Length Seleton Homewor Due /3/ Show equaton 3.44 he last ste n the roo o heorem 3.6. see sldes Show that >.5, the lne searh would exlude the mnmzer o a quadrat, and unt ste length may not be admssble. heorem 3.5

2 Suent ondton x x,, 4 x x,, he reduton should be roortonal to both the ste length, and dretonal dervatve. x x, x l, St lne x l Suent ondton Problem: he suent derease ondton s satsed or all small values o ste length

3 Curvature ondton x x,, Dervatve.9 or Newton and Quas - Newton he sloe o s greater than tmes the gradent..or onugate gradent Curvature ondton I the sloe s strongly negatve, that means we an redue urther along the hosen dreton I the sloe s ostve, t ndates we an not derease urther n ths dreton.

4 Wole ondtons x x,, x x,, Suent derease Curvature Batrang Lne Searh I lne searh method hooses ts ste length arorately, we an dsense wth the seond ondton Choose >, ρ,,; set ; reeat untl x ρ; end reeat ermnate wth x, or Newton and quas - Newton hs ensures that the ste length s short enough to satsy the suent derease ondton, but not too short.

5 Searhng Ste Length Usng Interolaton x x,,. Assume s the ntal guess. hen we have: Suent derease hen ths ste length satses the ondton, we termnate the searh.. Otherwse, we now [, ontans the aetable ste lengths. ] We t quadrat olynomal to three ees o normaton:, q q, q x Searhng Ste Length Usng Interolaton and nd ste length by analytally mnmzng ths olynomal I the suent derease ondton s satsed or ths we termnate the searh. 3. I not we t ub olynomal to nterolate our ees o normaton,and analytally mnmze ths olynomal to nd,,, then I neessary we an reeat ths roess wth most reent values o., and two

6 Quadrat Interolaton b a q q,, q q q d q d Cub Interolaton 3 3 b a,,, a a b b b a d b a 3

7 Algorthm 3. Lne Searh Algorthm Set, hoose >, and reeat Evaulate ; Evaulate ; hoose ; end reeat > or zoom set, and sto; set zoom,, max max ; [ >, > ],, and sto;, and sto; st Wole s ondton nd Wole s ondton Algorthm 3.3 Zoom reeat Interolate to nd a tral ste length between, ; Evaulate ; esle end reeat Evaulate ; set, and sto; - [ > ] > or lo st Wole s ondton ; h set lo set h lo h h lo lo nd Wole s ondton

8 Low HI HI Low

9 heorem 3.5 Any Desent Dreton Suose s three tmes ontnuously derentable. Consder teraton x x, where s a desent dreton, Satses Wole s ondtons, wth. I the { x onverges } to a ont x suh that x and x s d, and the searh dreton satses lm B x lm hen s admssble or all > and or all >, then {x } onverges to x suerlnearly. heorem 3.6 Quas-Newton Suose s three tmes ontnuously derentable. Consder teraton x x, where s gven by Quas-Newton dreton. Assume the sequene { x } onverges to a x ont suh that x and x s d, the { x } onverges suerlnearly the ollowng ondton holds. B x lm

10 Order Notatons C C O >, or, Gven two non-negatve nnte sequenes lm o Seth o a Proo N o B O B or, > C C O lm x B lm lm o Norm o Hessan s bounded.

11 Seth o a Proo x x x x x O x o x N x x o N x x N x o lm heorem 3.7 N Suer-lnear Show ths n Homewor heorem 3.7 Newton Suose that s twe derentable and that Hessan s Lshtze ontnuous. Consder the teraton x x where s gven by N hen:. I the startng ont x s suently lose to x, the sequene onverges to x.. he rate o onvergene s quadrat 3. he sequene o gradent norms x onverges quadratally to zero.

12 Coordnate Desent Method Cyle through n oordnate dretons n turn as a searh dreton. e, e, K e n usng eah Fx all other varables exet one, and mnmze the unton. It s an neent method, t an terate nntely wthout ever aroahng a ont, where the gradent vanshes. he gradent may beome more and more erendular to searh dretons, mang aroah to zero, but not the gradent. osθ no OK OK no no OK OK angle tan tan

Practical Newton s Method

Practical Newton s Method Practcal Newton s Method Lecture- n Newton s Method n Pure Newton s method converges radly once t s close to. It may not converge rom the remote startng ont he search drecton to be a descent drecton rue