EXCE, steepest descent, conjugate gradient & BFGS

Club Cast3M, 21th of November 2008 An amazng optmsaton problem P. Pegon & Ph. Capéran European Laboratory for Structural Assessment Jont Research Centre Ispra, Italy

An amazng optmsaton problem OUTLINE: Background: some results from photogrammetry The (VERY) smplfed optmsaton problem EXCE, steepest descent, conjugate gradent & BFGS The Gauss-Newton method (+ new operator GANE) Conclusons, further works & bblography

Background: some results from photogrammetry The structure ESECMASE The wall The camera 20cm 4096 grey levels, S/N 64 db, 1536x1024 pxels Kodak CCD KAF 1602E +-1.3mm/pxel

Background: some results from photogrammetry Intal t 0 Reference state Set of M nterpolaton functons on M reference sub-wndows Current t Current state Set of M current sub-wndows under trackng Gven one reference, C3 nterpolaton functon, W0 ( X ) Consder ts correspondng current dscrete functon ( ) wx C3 Analytc Template { P } { 1 n P + n } W0 ( T( xk ; P) ) T ( X; P ) parametrsed by jdjd { } Coordnate Transformaton P n....... Optmsaton on the Quadratc Error: k= 1.. K ( ( ) ( ( ))) 2 k 0 k P E( P) = w x ) W T x ; Wth respect to { } P n Trackng Method

Background: some results from photogrammetry Comparson wth Hedenhan measurement (K14-0.16g)

Dsplacement on targets Background: some results from photogrammetry

Dsplacement on texture Background: some results from photogrammetry

Background: some results from photogrammetry Dsplacement felds (K12-0.12g)

The (VERY) smplfed optmsaton problem In the elastc case 65 brcks E, = 1,...,65 ν = 0.3 Jonts E66 E66 E66 Kn =, Ks = e 2(1 e + ν ) Identfcaton of E???

The (VERY) smplfed optmsaton problem d s mposed d s measured + + + d ( E ) E =1 1 σ ( E + ) = 5% The reference soluton s obtaned wth and

The (VERY) smplfed optmsaton problem Fnd E * { F E } argmn ( ) = E Where T ( ) ( ) ( ) F ( E ) = f E = f E f E ( ) + + f E = d( E) d ( E ) de ( ) = d( λe) (frst) problem!!!

The (VERY) smplfed optmsaton problem Numercal dervaton of the gradents F ( Ε ) FE ( 1,..., E + δ E,..., En ) FE ( 1,..., E δ E,..., En ) E 2δ E And also (later) f ( Ε ) f ( E 1,...,, E + δe,...,, E n) f( E 1,...,, E δe,...,, E n) E 2δ E

EXCE, steepest descent, conjugate gradent & BFGS EXCE The EXCELL operator computes the mnmum of a functon F(X),the use method s the well-known MMA (Method of Movng Asymptotes) proposed by K.Svanberg. The purpose s to fnd the mnmum of a functon F(X) wth =1,N gven that : - the relatons Cj(X) < Cjmax (j>0 and j=1,m) must be satsfed - there are relatons on each unknown Xmn < X < Xmax The functons F and Cj are defned by the functon values and by ther dervatves at the startng pont X0. FE ( ), Cj( E )?????

EXCE, steepest descent, conjugate gradent & BFGS EXCE F( ( E ) = d ( E ) d + ( E + ) 1 st case 1 C ( E) = σ( E) σ ( E + ) < 0 C + + ( E ) = E E < 0, C ( E) = E E < 0 2 3 2 nd case F( ( E ) = d ( E ) d + ( E + ) + + + 2 1( E) = σ( E) σ( E ) < 0, 2( E) = ( E E ) < ( E /100) 0 C C or F( E) = σ ( E) 3 rd case but no C + + + + 4 1 ( E ) = d ( E ) d ( E ) / d ( E ) < 10 1.0 10 convncng results

EXCE, steepest descent, conjugate gradent & BFGS Descent method: general format Descent drecton wth α T E E 0 d d F < E { F E α E } d = α > 0 + argmn ( ) Lne search Robust lne search??? E + = E +αe 1 d and loop on untl convergence Meanng of convergence??? E d Here we concentrate on!!!!!!!

EXCE, steepest descent, conjugate gradent & BFGS Steepest descent E st = F E + + + + d( E) d ( E ) / d ( E ) The convergence The convergence can be very slow!s

EXCE, steepest descent, conjugate gradent & BFGS Conjugate gradent : the descent drectons are mutually orthogonal E = g + γ E g = F γ = 1 gc gc E 1 T ( g g ) g g 1 2 (wth precse lne search) + + + + d( E) d ( E ) / d ( E ) Surprsngly slow convergences

EXCE, steepest descent, conjugate gradent & BFGS BFGS: the Hessan matrx s teratvely constructed E = H F H = H +... 1 bfgs bfgs E bfgs bfgs + + + + d( E) d ( E ) / d ( E ) BFGS=Broyden-Fletcher-Goldfarb-Shanno Surprsngly slow convergences

EXCE, steepest descent, conjugate gradent & BFGS Summary: Very slow convergence Hgh level asymptotc convergence plateau Concluson: we need to do better!

The Gauss-Newton method (+ new operator GANE) Wrte the problem as a least square problem Fnd E * = x { F E } argmn ( ) Where m 1 1 1 F( E) = f E = f E = f E f E 2 2 2 ( ) ( ( )) 2 2 T ( ) ( ) ( ) = 1 + + f E = d( E) d ( E ) and use t!

The Gauss-Newton method (+ new operator GANE) The Gauss-Newton Method Same cost as computng s F / Ej ( + δ ) ( ) + ( ) δ J( E) = ( E) f E E f E J E E Usng wth { } T E = argmn F( E+ δ E ) s gven by ( ) gn δ E j JJE gn f E j T = Jf= F Use E gn as a descent drecton Warnng: no ndetermnaton f the columns of are lnearly ndependent J

The Gauss-Newton method (+ new operator GANE) Operator GA(us-)NE(wton TAB2=GANE TAB1; CHPO1 RIGI1=GANE TAB1 'MATR'; Descrpton: The functon to mnmze s 2F(X)=f(X).f(X), and J(X)=df/dX The descent drecton s H soluton of: transpose(j)j H = -transpose(j)f Contents: TAB1 : Table of type 'VECTEUR' contanng TAB1. 0 = f(x) TAB1. 1 = df/dx1. TAB1. n = df/dxn CHPO1 : -transpose(j)f RIGI1 : transpose(j)j TAB2 : TABLE of type 'VECTEUR' contanng TAB2. 1 = H1. TAB2. n = Hn

The Gauss-Newton method (+ new operator GANE) de ( ) = d ( λ E ) There s a second problem!!! (frst problem ) T ( JJ) s almost always rank 1 defcent (use of RESO ENSE ) 65 f f For nstance: = a ( a = 1 ( E66) for E = { 1} ) E E BUT 66 = 1 f GANE can be used to fnd out the a ( JJ T 65 65) a= J65 ) E 66 Assumng E = g( E ) wth g E = a J 65 wth 66 65 E gn65 / Use GANE agan for fndng and complete wth E f f f = + a E E 65 E ae = gn66 gn = 1 66

The Gauss-Newton method (+ new operator GANE) And fnally! + + + + d( E) d ( E ) / d ( E ) wth max( E E + ) 10 7

The Gauss-Newton method (+ new operator GANE) E + 66 ne +, n 1,2,4,8 66 = E = { 1} Substtutng wth (and stll startng from ) n = 1 n = 2 n = 3 n = 4 WARNING Sometmes RESO takes care tself of the ndetermnaton E + + E 7 max( ) 10 E E + E E wth

Conclusons, further works & bblography Conclusons & further works Nothng s gven for free! Cast3M demonstrates agan ts ablty to help to understand (and solve) problems! Gauss-Newton s the root for other classcal methods Damped Gauss-Newton = Levenberg-Marquardt method T T (already n GANE) ( JJ+ μie ) = Jf= F Powell s Dog Leg Method Use of nosy expermental data lm Identfcaton of non-lnear models wth more refned meshes Lne search, convergence tests

Conclusons, further works & bblography Short bblography W.H. Press, S.A. Teukolsky, W.T. Vetterlng & B.P. Flannery, Numercal Recpes : The Art of Scentfc c Computng, 1986 H. Matths & G. Strang, The Soluton of Non-Lnear Fnte Element Equatons IJNME, 14, 1613-1626, 1626, 1979 K. Madsen, H.B. Nelsen & O. Tngleff Methods for Non-Lnear Least Squares Problems Techncal Unversty of Denmark, 2004 T. Charras & J. Kchenn L optmsaton dans CAST3M