2D1240 Numerical Methods II / André Jaun, NADA, KTH NADA

Transcription

1 A NADA 2D24 Numericl Methods II / André Jun, NADA, KTH. NON-LINEAR LEAST SQUARE FITTING, OPTIMIZATION.. Remember: wht we sw during the lst lesson Itertive solution for sprse sstems (Jcobi, Guss Seidel, help Mtlb sprse) Non-liner sstems Fied point, Piccrd nd Newton itertions Non-liner interpoltion..2 Overview: wht ou will lern tod Non-liner fitting Optimiztion serching for n etremum with-/ out using derivtives Ordinr differentil equtions using Euler nd Runge-Kutt methods

2 >< >: + b sin!(:5 t ) ß :3 + b sin!(:8 t ) ß :3 + b sin!(: t ) ß :5 + b sin!(:2 t ) ß :9 + b sin!(:5 t ) ß :4 + b sin!(:8 t ) ß : + b sin!(2: t ) ß :5 + b sin!(2:4 t ) ß :3 >< >: + b sin!(:5 t ) :3 ß + b sin!(:8 t ) :3 ß + b sin!(: t ) :5 ß + b sin!(:2 t ) :9 ß + b sin!(:5 t ) :4 ß + b sin!(:8 t ) : ß + b sin!(2: t ) :5 ß + b sin!(2:4 t ) :3 ß.2 Non-liner lest squre fitting (NAM 6., H 6.6) F (t) = + b sin!(t t ) t Emple Fit the model function to eperimentl dt This ields n overdetermined non-liner sstem (4 unknowns, 8 equtions) 8 8 or which cn be solved b demnding tht residuls between the mesurements i nd the F (t model ) i be k Fk smll ß 2 i.e.k Fk, = f(c) ß 2. Guss-Newton solution obtined from c new = c old + ffic; J T Jffic = J T f(c old ) The coefficients c re obtined from the Newton method for non-liner sstems, with n increment ffic solving the overdetermined liner sstem Jffic ß f.inmtlb, the solution of the norml equtions cn gin be computed with dc=-f J.

3 Ji=ones(n,); Ji2=sin(u); Ji3=b*(t-t).*cos(u); Ji4=-w*b*cos(u); J=[Ji Ji2 Ji3 Ji4]; % Jcobin dc=-j f; c=c+dc; % Guss-Newton b w t f dc step Emple solution of the non-liner fit with Mtlb t=[ ]'; =[ ]'; subplot(,2,), h=stem(t,), title('dt') =.7; b=.7; w=pi; t=.2; c=[ b w t]'; %initil guess n=size(t,); iter=;dcnorm=.; while dcnorm>e-6 & iter< u=w*(t-t); f=+b*sin(u)-;.6 Dt Model F=+b sinω(t t) (t i ).8 F(t) dcnorm=norm(dc); iter=iter+; =c(); b=c(2); w=c(3); t=c(4); D=[iter c b w t norm(f) norm(dc) ] end tt=(:.5:3)'; Ft=+b*sin(w*(tt-t)); subplot(,2,2), plot(t,,'o',tt,ft) t i t A solution with n ccurc better thn 6 is obtined with 6 steps:

4 ( c ; c ;;b); cluclte the i =@ j =[ ]'; %dt =[ ]'; = c = c = b = Emple: fit n F (; ) = ellipsis c) 2 ( 2 + ( c) 2 b 2 = to seven dt points. Set-up the overdetermined non-liner sstem with 7 equtions for 4 = 2 c = 2( c) = 2 c b = 2( c) b 3 Non liner fit to n ellipsis 2 c=; c=8; =8; b=3; %initil guess p=[c c b]'; iter=; dp=; while norm(dp)> e-6 & iter< iter=iter+; d=-c; d=-c; 8 f=d.^2/^2+d.^2/b^2-; J=-2*[d/^2 d/b^2 d.^2/^3 d.^2/b^3]; 6 dp=-j f; p=p+dp; %Guss-Newton end 4 c=p(), c=p(2), =p(3), b=p(4) v=:2*pi/6:2*pi; 2 plot(,,'o',c+*cos(v),c+b*sin(v)), is equl, Note tht fit to circle cn be written s liner lest squre problem (lb.3).

5 f=inline('.3-ep(-(-).^2)-min(/4,m(./(-),))'); [,f,flg]=fminbnd(f,,3).3 Optimiztion in one dimension (NAM 7., H 6.4) In Mtlb use fminbnd for rel rguments nd m for integer rguments rgument. =:.:3; plot(,fevl(f,),,f,'o') =.27 f =.342 flg = A vriet of methods re combined to provide rpid convergence nd robust nswers; the require unimodl functions tht re strictl incresing in ll the direction from the minimum:»» = ( )2 min f() ; m :3 e 4 Methods tht use order (Jcobin) nd 2 even order derivtives (Hessin) converge fster... if defined! Check st nd to set the properties. optimset help ;.3 ep[ ( ) 2 ] min(/4,m[/( ),]) A discountinuous unimodl function Here is n emple of n optimiztion where the rgument is n integer nd the function is chep to evlute: n=; f=rndn(,n); [fmin,ind]=min(f) fmin = ind = 98438

6 fi = fi p 5 Golden section serch of 2 [; b] minimum Λ without computing derivtives Two evlutions re required to decide if f() the minimum is locted in the left or right intervl: 6 )>f( if ) 2 f( then Λ 2 [ ; b] else Λ 2 [; 2 ] end For efficienc, choose the proportions so tht = b fi = b 2 b b 2 i i 2 b 2 b b b in order to reuse the previoius evlutions. B smmetr ( 2 )=(b ) = fi 2 = fi ) b 2 b = fi b b fi ) fi 2 + fi = ) fi = ß :68 2 which reminds the golden + rtio 5=2 ß :68 from the ntiquit. Finll p fi = + (b )( fi) 2 = + (b )fi

7 f=ff(n); function -cos(n/3+9); return f=ep(sqrt(n)/) =; b=44; n2=89; n=+b-n2; F=Ff(n); F2=Ff(n2); disp([ n n2 b]) while b-> if F>F2, b=n2; n2=n; F2=F; n=+b-n2; F=Ff(n); else =n; n=n2; F=F2; n2=+b-n; F2=Ff(n2); end disp([ n n2 b]) end; Fm=[F F2] n n2 n Fiboncci serch cn be used when the rgument is n integer Write reltion between steps in golden serch L i+2 = L i + L i+ This ields recursion formul for the Fiboncci numbers i+2 L i+ L L i+2 L i+ L i L i+ i F 2 f; ; 2; 3; 5; 8; 3; 2; 34; 55; 89; 44; 233;:::g where consecutive integers pproimte the golden section 34=55 = :682 ß fi. This suggests more efficient lgorithm thn Mtlb s m to find the mimum of Ff(n) in the intervl i = :::2 strting from lrger Fiboncci number: Fm =

8 A = ψ 4 + 2sin A = (; ) f 2 (; ) f.4 Optimiztion in higher dimensions n (NAM 7.2, H 6.5) Efficient methods such s steepest descent, conjugte-grdients) do not require evluting the Jcobin: the re the subject of dvnced courses. Newton s method is however sufficient if the function cn differentited nlticll to clculte n etremum b solving using itertions rφ( ; 2 ;:::; n ) = new = old + d; Jd = f( old ) Emple: mimum of Φ(; ) = ( + sin )e (2 + 2) round (; ) = ( 4 ; 4 ). ψ! ( 2( + sin ))e ( ( + sin ))e (2 + 2 ) (cos @ To solve this sstem of the form f() =, clculte the Jcobin J ij i =@ j =! 2 cos J = sin + 2 cos

9 =/4; =/4; F=(+sin())*ep(-(^2+^2)); z=[ ]'; dznorm=; iter=; disp([iter dznorm F]) In mtlb the solution is then obtined with while dznorm>e-6 & iter < 3 s=+sin(); f=[-2**s; cos()-2**s]; J=[-4*-2*sin() -2**cos(); -2* -2*-3*sin()-2**cos()]; dz=-j f; z=z+dz; dznorm=norm(dz,inf); iter=iter+; =z(); =z(2); F=(+sin())*ep(-(^2+^2)); disp([iter dznorm F]) Φ(,) end.4.6 dz F iter

10 . ORDINARY DIFFERENTIAL EQUATIONS (ODEs).2 Euler nd Runge-Kutt methods (NAM 8.2, H 9.3) Problem: numericll solve the ordinr differentil eqution for (t) for t > t d dt = f(t; ); with (t ) = using pproimtions with smll steps in time t ;t ;:::such tht t i+ = t i + h. Euler s method directl follows from the forwrd difference (t i + h) (t i ) ß f(t; ); ) (t i+ ) ß i + hf(t i ; i ) Strting from the initil condition (t ; ), one step with Euler s method produces n pproimtion (t ; ) with locl error O(h 2 ); fter n steps to rech the finl time t n = t + nh, the solution (t n ; n ) hs globl error no(h 2 ) ο O(h). h

11 fend=feuler(h) function t=; tend=2; n=tend/h; T=t; Y=; =6; Emple solve (t) = sin(t) with () = 6 using Euler s method 6.4 Euler method with h=.2,.,.5 nd.25 i=:n for =+h*f; t=t+h; f=sin(t*); 6.3 t]; Y=[Y; ]; T=[T; end 6.2 plot(t,y,t,y,'o'); fend=; return (t) 6. h=.2; Y2=[]; for k=:3, Y2=[Y2 feuler(h/2^k)], end; 6 Y2 = t A Richrdson etrpoltion cn be used to cncel the leding O(h) (globl) error in nd (2) ß 5:873 + clculte the vlue = 5: :873 5: Q Peer Teching minutes to think, eplin to our neighbour (2 nd vote) Richrdson etrpoltion. Which vlue for Q should ou use here bove? ψ "! 2

12 = f(t i ; i ) k 2 = f(t i + h; i + hk ) k 3 = f(t i + h 2 ; i + h 2 k 2) k 4 = f(t i + h; i + hk 3 ) k Runge-Kutt (RK2) chieves better precision with globl error in O(h 2 ) ( i+ = i + h 2 (k + k 2 ); with Runge-Kutt (RK4) chieves high precision with globl error in O(h 4 ) 8 >< = f(t i ; i ) k 2 = f(t i + h 2 ; i + h 2 k ) k i+ = i + h 6 (k + 2k 2 + 2k 3 + k 4 ); with In Mtlb use help ode23, ode45, odeset for RK methods with vrible step size,where the step size h is continull djusted to chieve specified precision with minimum number of steps. >:

13 end plot(t,y); h=.2; YN=[]; for k=:, YN=[YN rk4(h/2^k)]; Y2=YN(end), end; % --- Alterntive using Mtlb's solvers tend=2; =6; options=odeset('reltol',e-6,'abstol',e-4]); [T,Y]=ode23(@fsin,[ tend],,options); [T,Y]=ode45(@fsin,[ tend],,options); Emple solve (t) = sin(t) with () = 6 using the RK4 method f=fsin(t,), function f=sin(t*); return 6.4 Runge Kutt RK4 method with fied h=.2,. fend=rk4(h) function t=; tend=2; n=tend/h; =6; 6.3 Y=; h2=h/2; T=t; i=:n for 6.2 k=fsin(t, ); k2=fsin(t+h2,+h2*k); k3=fsin(t+h2,+h2*k2); (t) 6.,+h *k3); k4=fsin(t+h =+h/6*(k+2*k2+2*k3+k4); 6 t=t+h; t]; Y=[Y; ]; T=[T; 5.9 fend=; return t Y2 = A Richrdson etrpoltion gin cncels the leding (globl) O(h error, ) here in 4 5: with (2) 4 = 5: :8639 5:86346 ß 2