Lecture 20 Curve fittig II Itroductio I the previous lecture we developed a method to solve the geeral liear least-squares problem. Give samples (x i, y i, the coefficiets c j of a model are foud which miimize the mea-squared error m y= f (x= c j f j (x (1 MSE= 1 i =1 j=1 f ( x i ] 2 (2 The MSE is a quadratic fuctio of the c j ad best-fit coefficiets are the solutio to a system of liear equatios. I this lecture we cosider the o-liear least-squares problem. We have a model of the form y= f (x ;c 1, c 2 (3 where the c j are geeral parameters of the fuctio f, ot ecessarily coefficiets. A example is fittig a arbitrary sie wave to data where the model is The mea-squared error y= f (x ;c 1,c 3 =c 1 si (c 2 x+c 3 MSE (c 1,, c m = 1 i=1 f (x i ;c 1 ] 2 (4 will o loger be a quadratic fuctio of the c j, ad the best-fit c j will o loger be give as the solutios of a liear system. Before we cosider this geeral case, however, let's look at a special situatio i which a o-liear model ca be liearized. Liearizatio I some cases it is possible to trasform a oliear problem ito a liear problem. For example, the model y=c 1 e c 2 x is oliear i parameter c 2. However, takig the logarithm of both sides gives us (5 l y=l c 1 +c 2 x (6 If we defie ^y=l y ad ^c1 =l c 1 the our model has the liear form ^y=^c1 +c 2 x (7 EE 221 Numerical Computig Scott Hudso 2017-11-03
Lecture 20: Curve fittig II 2/5 Fig. 1: Dashed lie: y=πe 2 x. Dots: te samples with added oise. Solid lie: fit of the model y=c 1 e c 2 x obtaied by fittig a liear model l y=^c 1 +c 2 x ad the calculatig c 1 =e^c 1. Oce we've solved for ^c 1 we ca calculate c 1 =e^c 1. Example. Noise was added to te samples of y=πe 2 x, 0 x 2. The followig code computed the fit of the liearized model. ylog = log(y; a = (mea(x.*ylog-mea(x*mea(ylog/(mea(x.^2-mea(x^2; b = mea(ylog-a*mea(x; c1 = exp(b; c2 = a; disp([c1,c2]; 3.0157453-1.2939429 Noliear least squares The geeral least-squares problem is to fid the c j that miimize MSE (c 1,, c m = 1 i=1 f (x i ;c 1 ] 2 (8 This is simply the optimizatio i dimesios problem that we dealt with i a previous lecture. We ca use ay of those techiques, such as Powell's method, to solve this problem. It is coveiet, however, to have a frot ed fuctio that forms the MSE give the data (x i, y i ad the fuctio f (x ;c 1 ad passes that to our miimizatio routie of choice. The fuctio fitleastsquares i the Appedix is a example of such a frot ed. The followig example illustrates the use of fitleastsquares.
Lecture 20: Curve fittig II 3/5 Example. Eleve data samples i the iterval 0 x 1 of the fuctio y=2 cos(6 x+0.5 were geerated. Normally distributed oise with stadard deviatio 0.1 was added to the y data. A fit of the model y=c 1 cos(c 2 x+c 3 gave c 1 =1.94, c 2 =5.91, c 3 =0.53. The fit is show i Fig. 2 ad was geerated with the followig code. rad('seed',2; y = 2*cos(6*x+0.5+rad(x,'ormal'*0.1; fuctio yf = fmod(x,c yf = c(1*cos(c(2*x+c(3; edfuctio [c,fctmi] = fitleastsquares(x,y,fmod,c0,0.01,1e-6; disp(c; 1.9846917 5.8614475 0.5453276 Fig. 2: Fit of the model y=c 1 cos(c 2 x+c 3 to 11 data poits. The lsqrsolve (Scilab ad lsqcurvefit (Matlab fuctios The Scilab fuctio lsqrsolve solves geeral least-squares problems. We create a fuctio to calculate the residuals. The followig code solves the problem i the previous example usig lsqrsolve.
Lecture 20: Curve fittig II 4/5 rad('seed',2; y = 2*cos(6*x+0.5+rad(x,'ormal'*0.1; fuctio r = residuals(c,m r = zeros(m,1; for i=1:m r(i = y(i-c(1*cos(c(2*x(i+c(3; ed edfuctio c = lsqrsolve(c0,residuals,legth(x; disp(c; I Matlab the fuctio lsqcurvefit ca be used to implemet a least-squares fit. The first step is to create a file specifyig the model fuctio i terms of the parameter vector c ad the x data. I this example the file is amed fmod.m fuctio ymod = fmod(c,x ymod = c(1*cos(c(2*x+c(3; The, i the mai program we pass the fuctio fmod as the first argumet to lsqcurvefit, alog with the iitial estimate of the parameter vector c0 ad the x ad y data. y = 2*cos(6*x+0.5+rad(size(x*0.1; c = lsqcurvefit(@fmod,c0,x,y; disp(c;
Lecture 20: Curve fittig II 5/5 Appedix Scilab code 0001 ////////////////////////////////////////////////////////////////////// 0002 // fitleastsquares.sci 0003 // 2014-11-11, Scott Hudso, for pedagogic purposes 0004 // Give data poits x(i,y(i ad a fuctio 0005 // fct(x,c where c is a vector of m parameters, fid c values that 0006 // miimize sum over i (y(i-fct(x(i,c^2 usig Powell's method. 0007 // c0 is iitial guess for parameters. cstep is iitial step size 0008 // for parameter search. 0009 ////////////////////////////////////////////////////////////////////// 0010 fuctio [c,fctmi] = fitleastsquares(xdata,ydata,fct,c0,cstep,tol 0011 Data = legth(xdata; 0012 0013 fuctio w=fmse(ctest 0014 w = 0; 0015 for i=1:data 0016 w = w+(ydata(i-fct(xdata(i,ctest^2; 0017 ed 0018 w = w/data; 0019 edfuctio 0020 0021 [c,fctmi] = optimpowell(fmse,c0,cstep,tol; 0022 edfuctio