An introduction to least-squares fitting

Transcription

1 atonale An ntroducton to least-squares fng.p. Pla hp:// Ths paper provdes a mnmall mathematcal ntroducton to least-squares fng, ntended to e of some modest value to engneerng students needng to understand or mplement smple least-squares algorthms. Often n engneerng t s necessar to ft a mathematcal model of a sstem to some measured data. A smple eample mght e relatng Ohm s law (the mathematcal model) to epermental data on currents through and voltages across a resstor. The mathematcal model s: V I The data mght e: Pont, Current, I Voltage, V (ma) (V) Tale We mght want to fnd the value of that provdes the est straght lne ft through the data (I,V ) ths would e how we could determne the resstance from the measurement. more general stll, we can wrte a mathematcal model of the form a + whch s the general equaton of a straght lne, nterceptng the as at pont a and wth slope d. In the Ohm s law d model, a and. Another eample of the general straght lne tpe can e otaned from a Wheatstone rdge ncorporatng a stran gauge, such as mght e used to measure deflecton of a eam, for eample as follows: Pont, Deflecton, Voltage, (mm) (V) Tale voltage (V) 5 5 data ponts est lnear ft voltage (V) data ponts est lnear ft current (ma) To e more general, we can call the ndependent varale (here, I), and the dependent varale (here, V),. To e deflecton (mm) The general prolem s one of how to choose the est values of the model parameters, a and, to ft the measured 3.P. Pla Page of 5

2 An ntroducton to least-squares fng data. esduals and the sum of squared dfferences Assume a set of data (,) modelled a straght lne, a +. The model and the epermental data are unlkel to agree precsel. At each pont the dfference s ( a + ) s the resdual at pont the amount whch the model and the epermental data dsagree at that pont. To choose the est ft to the data one must mnmse the, somehow (f all the were zero, the ft would e perfect). It would e possle to choose values of a and so that the sum of all the resduals would e zero: Ths s unsatsfactor: as some resduals are lkel to e postve whle others are negatve, t s possle for ndvduall large errors to cancel each other, leadng to a poor ft. To avod ths prolem, we calculate the sum of the squares of the resduals ( ) a Each squared resdual s non-negatve: now, postve and negatve resduals (errors) do not cancel. The sum of squared dfferences (D),, s a useful measure of the qualt of the ft of the data: fndng the est ft means choosng a and so that s mnmsed hence least-squares ft. Fndng the least-squares ft The D s ( ) a The and are gven: a and must e found. The partal dervatves of wth respect to a and are: a ( a ) ( ) ( a ) ( a ) The mnmum value of occurs when and, leadng to the smultaneous equatons: a + a +,,, and are all known from the data. The two unknowns, a and, can e calculated from the two smultaneous equatons. The soluton s a where, and,. These values of a and gve the least-squares ft of the data (,) to the straght lne a +. Practcal mplementaton In practce, the equatons a 3.P. Pla Page of 5

3 An ntroducton to least-squares fng suffer from roundoff errors (Press et al. 99). It s convenent to rewrte them usng the temporar parameter, t : t where s the mean value of : Then, usng t, t t the formulae for the fed parameters ecome: a t In pseudocode: t. for to - egn + + ar / for to - egn t -ar +t*t t t+t* t/ a (-*)/ Applng ths algorthm to the data n gves a.9 V,.6 kω. The D s.58 V. For the data n Tale, a 3.36 V, V.mm - and.4 V. The effect of errors on the ft The sum of squared dfferences: ( ) a s a measure of the dsagreement etween the model and the data. The can dsagree ecause: The model mght e wrong. The data ponts (, ) wll nclude some uncertant (e.g. random epermental errors). One common stuaton s that the uncertant n s ver small compared to that n, whch s tself descred a standard devaton,. In man practcal cases t s assumed that the measurement s wthn one standard devaton of the true value of the measurand wth proalt.68 (.e. 68 out of measurements would e wthn the standard devaton), that the asolute error n s less than 95% of the tme, and that t s less than 3 wth 99.7% proalt. (Ths s the socalled normal dstruton.) In ths case, one would epect the sum of squared dfferences to e gven ( ) If s around ths value, or lower, the ft can e consdered good. If s much greater than ths value, the ft s not good. Ether the model s wrong (the data do not le on a straght lne) or the uncertantes n the data are greater than epected (the measurements were naccurate). If s much smaller than epected ths ndcates that measurement errors were pessmstcall overestmated. If the measurement errors are unknown, t s possle to estmate them assumng that the model s vald: Ths would e reasonale, for eample, f one had a hgh degree of confdence n a partcular phscal model Ohm s law for a resstor would e a partcularl smple eample. ote that the standard devaton n the errors s close to the root mean square error,. Measurement errors also result n uncertant n the values of the fed parameters, a and. It s often mportant to quantf these uncertantes for 3.P. Pla Page 3 of 5

4 An ntroducton to least-squares fng eample, f represents the senstvt of a stran gauge crcut used n a load cell n a weghrdge, t would have to e known suffcentl accuratel to meet whatever standards or legslatve requrements appled for the accurac of such measurements. It s possle to estmate the uncertantes n a and usng: a + ( ) ( ) The standard errors a and can e loosel nterpreted as meanng that the value of a (or ) s accurate to wthn a ( ) wth 68% proalt, a ( ) wth 95% proalt, etc. In pseudocode: D. for to - egn -(a+*) D D+* sg D/(-) sga sqrt(sg/*(+*/(*))) sg sqrt(sg/) sg sqrt(sg) For the data n we now have a.9±.3 V,.6±.9 kω (to one standard error). The estmated standard error on s.39 V. For the data n Tale, a 3.36±.4 V, -.489±.9 V.mm -,.78 V. In man cases, the uncertant n measured data vares from pont to pont. Ths would occur, for eample, f dfferent epermental condtons were used to make measurements (e.g. dfferent equpment resoluton at dfferent range), and wll alwas occur when uncertant s epressed as a fracton of the measured value. If measurement has standard error, the equatons are modfed and we perform a χ ( ch-squared ) ft: χ t a,, t, t, t t As efore: a t and the standard errors n a and are gven + a The χ ft s just a least-squares ft to the data normalsed to the uncertant n each value. Ths allows more precsel known data ponts to e gven more nfluence n the ft, applng the weght. onlnear least-squares fts everal common nonlnear models can e fed usng a modfed verson of ths technque. For eample, a relatonshp of ths form a + ln s nonlnear n, ut lnear n ln(). It s straghtforward to taulate ln() from the, and proceed as efore. Other models mght e more complcated. For eample, the equaton ae fts man phscal phenomena, especall the deca of such thngs as the head on a glass of eer (Leke ) and s nonlnear 3.P. Pla Page 4 of 5

5 An ntroducton to least-squares fng n. It can e lnearsed takng logarthms: ln ln a + Ths plots ln() aganst ; the slope s and the ntercept s ln(a). In prncple the least-squares ft s eactl as efore, the ln() can e fed to the, ut the uncertantes n the measurement of now need to e epressed n terms of uncertantes n ln() and uncertantes n ln(a) do not relate lnearl to a. Also, some assumptons mplct n the earler analss are less lkel to e vald and oth goodness of ft measures and standard error estmates are lkel to e less accurate. In a smlar wa, power-law relatonshps of the followng form: a (also common n phscal sstems) can e lnearsed as log log a + log and parameters log(a) and can e fed to the data re-epressed as log() and log(), wth smlar provsos aout the effect of uncertantes on the valdt of measures of goodness of ft and accurac of the fed parameters. Hgher order polnomal fts and general nonlnear fts are also possle. These and other developments of the least-squares technque are descred n several references, ncludng some of those n the followng lograph. Blograph. Erlch,. Least squares fng of epermental data, PHYET MI--359, hp:// [9//3]. Leke, A. Demonstraton of the eponental deca law usng eer froth, European Journal of Phscs Vol. pp. -6, 3. Press, W.H., Teukolsk,.A., Veerlng, W.T., Flanner, B.P. umercal recpes n C: the art of scentfc computng, nd Ed., Camrdge Unverst Press, Wessten, E.W. Erc Wessten's world of mathematcs hp://mathworld.wolfram.com [9//3] 3.P. Pla Page 5 of 5