By M. O'Neill,* I. G. Sinclairf and Francis J. Smith

Transcription

1 52 Polynoal curve fttng when abscssas and ordnates are both subject to error By M. O'Nell,* I. G. Snclarf and Francs J. Sth Departents of Coputer Scence and Appled Matheatcs, School of Physcs and Appled Matheatcs, The Queen's Unversty of Belfast, Northern Ireland An teratve ethod s descrbed for the least-square curve fttng of a polynoal to a set of ponts n two densons when both the abscssas and ordnates are subject to error and when the weghts of all the readngs are known. The process converges, n general, to a polynoal gvng the exact nu of the 'weghted' perpendcular dstances onto the curve. It s shown that n practce Deng's ethod gves a soluton close to ths optu polynoal. (Frst receved Deceber 1967 and n revsed for July 1968) 1. Introducton The proble of the least square curve fttng of a functon to a set of experental data (x ;, y,), 1 < / < n, when Xj and y, are both subject to error wth weghts u> x and cj y respectvely, has been descrbed and revewed by Deng (1943). lff(x,a r ) s the functon and a r, 1 < r <, s a set of varable paraeters the proble reduces to the nsaton of the su of the squares of the weghted perpendcular dstances fro the ponts (x h y t ) to the curve, that s, the nsaton of the su: S(a r, x',) = 2 {o) xl (x, + Ax',, a,)]*} (1) wth respect to the paraeters a r and wth respect to the 'adjusted' values x\ [*;,/(*,')] les on the curve. A ethod for fndng the approxate nu of S(a r, x) when/(x, a r ) s the polynoal f{x, a r ) = 2 a,xr (2) 0 has been descrbed by Deng, but ths does not gve the exact nu of S. More recently a nuber of wrters have studed the specal case when f(x, a r ) s a straght lne. Ths work has been revewed by York (1966) who has developed a ethod for the straght lne based on the soluton of a cubc equaton and a gven approxate value of the slope of the lne. In ths paper we descrbe an teratve ethod whch n general deternes to any gven accuracy the polynoal nsng S; the polynoal can have any specfed degree so our soluton ncludes the case of the straght lne. We show also that Deng's ethod quckly gves a close approxaton to the exact soluton. We use expansons of f{x, a r ) n sets of orthogonal polynoals, followng Forsythe (1957), because ths reduces coputng te and ncreases the accuracy of the coputatons. However, the theory we descrbe can be sply altered to ft a power seres n cases when the coeffcents of certan powers are constraned (for exaple, we ght wsh to ft a quadratc of the for J\x) = a 0 + o 4 x 4 to the set of data). * Now wth Independent Coputer Servces {N. Ireland) Ltd., Belfast. t Now wth Short Brothers & Harland Ltd., Belfast. 2. Theory The proble to be solved s bascally the nsaton of the functon S(a n x' t ) wth respect to ts ndependent varables, a r and x\. A nuber of ethods are avalable for nsng a functon n any varables; these have been recently revewed by Powell (1966). Usng a lbrary progra for our I.C.T coputer based on the ethod for nsng a su of squares due to Powell (1965) we were able to obtan the polynoals for whch S s a nu, but ths and other coon ethods for nsng a functon are neffcent copared wth the ethods we are gong to descrbe because they are not able to take account of certan specal features of the functon S. The nsaton process whch we found to be ost successful s based on the well known Newton-Raphson ethod for nsng a functon of any varables. In the followng we adopt the notaton of Powell's revew as far as possble Newton-Raphson ethod We expand/(x, a r ) n the for fx, a r ) = 2 a r p r (x) (3) r=0 p r (x) s a polynoal of degree r. The orthogonal propertes of these polynoals wll be dscussed later. We now wsh to nse S{a r, x'). We consder an approxate set of varables a r0 and x' 0 close to the nu of S, then f the nu s at a r = a r0 + 8a r and x' = x' 0 + Sx' at the nu and usng Taylor's seres to frst order n 8a r and SxJ and (4)

2 We have + n non-lnear equatons n the + n unknowns 8a s and 8x); these equatons can be wrtten n atrx for by defnng the followng atrces and vectors for 0 < r, s < and 1 <, j < n:. A - Then n the for of a parttoned atrx we have L If we wrte the parttoned atrx n ths equaton as G, then the correctons 8a r and Sx- are gven by the atrx equaton The poston of the exact nu can be found by the successve use of ths equaton (provded that the teraton converges). Ths process appears practcal because at each teraton we have to nvert a atrx of order + n, and n any cases n ay be large (t s not unusual to curve ft several hundred experental ponts). The proble s ade easer, however, because Polynoal curve fttng 53 (7) 5 y = Of#y; () and, n addton, f the polynoals pax) are orthogonal over the set of ponts x' 0 wth respect to the weghts co yh that s f then 2 pa.x' 0 )p s (x' Q )<o y = 0 01) Therefore A and B are dagonal atrces. In addton the eleents of the atrces C gven by C, = - 2w yl - {[y, - Ax' 0 )]p r (x' 0 )} (12) XO contan fewer ters and are usually saller than the dagonal eleents A rr and B n. To a frst approxaton, therefore, we can approxate G~ l by the sple dagonal atrx Then r- G whch, after splfcaton, takes the for = a r = 2 "> y ypax'o)l 2 Uy[pA x 'o)] 2 - = 1 = 1 Slarly (13) (14) 2.2. The algorth We suggest the followng process, based on equatons (15) and (16), for fndng the least-square curve ft. We begn wth the experental ponts JC, as our frst approxaton to x,'; that s we let x' l0 = x,. The polynoals orthogonal over ths set of ponts are generated by the recurrence relatons (Forsythe, 1957). n whch p o (x) = 1; P(x) = (x a,); pax) = (x - «f )p,_,(x) - r/ ; r _ 2(x), (17) «r = S { ^O}/ S {«,/[/»,-1 Wo)] 2 }. Pr = S {w,p r _ (x,>,.j(x,'o)x;«)/ S {«,/[/»,- 2 Wo)] 2 }- (19) The coeffcents a r are coputed fro equaton (15). Ths gves us the polynoal whch fts the experental data when the x ; coordnates are error free. We use ths as our frst approxaton. We now terate to the correct soluton. We begn by coputng fro equaton (16) approxate values of x,'. In (16) the quanttes/(x,' o ),/'(x,'o),/"wo) are coputed usng the algorth for sung orthogonal polynoal seres and ther dervatves due to Sth (1965). In ths algorth less than 6 ultplcatons and 9 addtons or subtractons are needed to copute all three quanttes wth nal roundng error. The suaton n the nuerator n (15) s evaluated n the sae way. When the new set of x\ are calculated, a new set of orthogonal polynoals are generated by coputng sets of a r and p r usng (18) and (19), and a r coeffcents usng (15). Ths copletes the frst teraton and gves us the second approxaton to the polynoal. Ths teratve procedure s then repeated tll convergence s reached. We call ths the drect teratve ethod. The total aount of coputaton s sall and t ncreases lnearly wth the nuber of experental ponts. Each teraton conssts of approxately (17 n) ultplcatons, (16 n) addtons and subtractons and (2 n) dvsons n s the nuber of experental ponts and s the order of the polynoal. On our I.C.T coputer when n = and 2 a coplete teraton took approxately 2 seconds and cost about 6 pence. Coparable costs can be worked out for other probles fro these fgures. Often a polynoal power seres s requred rather than an orthogonal polynoal at the end of the calculaton. If we wrte 2 a r p r (x) = E c s x s ; r=0 s=0 then the coeffcents, c s, can be calculated fro the followng recurrence relatons: y k. = 0 f k > / or f k, < 1; yl\y f(x' Q )]fxx'o) < - x'o) (16) In ths last equaton ore accurate values of x\ are obtaned f the new values of a r gven n (15) are used n the calculaton off(x' lq ), /'(*/o) an d/"(*/o) (16). c s = 2 a r y s+,, r+, for 0 < s <.

3 54 O'Nell et al Convergence A queston naturally arses about whether or not ths drect teratve procedure converges, and f t does converge, s t to the nu? We found no proof that the process always converges, but a large nuber of exaples were tred, varyng the weghts and the postons of the abscssas, and convergence was found n all cases. Exaples are gven n a thess by O'Nell (1967). However, n a few extree cases, such as n an exaple we gve at the end of ths paper, convergence s slow and over 0 teratons are needed to gve accurate results. It therefore sees lkely that exaples ay be found of non-convergence or of convergence whch s neglgbly slow. If any such cases occur they ght be solved by other ethods such as Powell's ethod for the nsaton of a functon or usng the exact nverse of G n place of the approxate nverse n (13). It s easly shown that f the teraton does converge t usually converges to the polynoal nsng 5 1. Fro (14) and (16) t follows that for all r and Hence <JS\ (20) (21) for all r and /. Thus, a nu s obtaned provded that all of the second dervatves A a n d are postve, the off-dagonal eleents beng sall. It s readly shown that the frst of these second dervatves s always postve, but the second, gven by the denonator n equaton (16), s unfortunately not necessarly postve, though n practce t alost always s so. A case when ths falure ght occur and when there are three curves akng 5 statonary s llustrated scheatcally n Fg. 1 (the x and y coordnates have been scaled so that the lnes jonng {x h y t ) to the ponts on the curve at whch \JT >) = 0 are perpendcular to the curve). If the curve passes through a sharp axu and f the experental pont (*,-, yl) les below the axu then as the pont [x,',./(x,')] passes along the curve, the partal su > yl [y, - (22) passes through a axu at P 2 and two na at P y and P 3. It s possble that our process ght fnd the axu pont, but ths possblty s easly elnated (n a coputer progra) by checkng that the denonator n (16) s not negatve. If t s negatve then t follows that we have found a axu pont and that there are two na on ether sde of the axu; approxate postons of these na can be found by scannng S", on ether sde of P 2 and ore accurate values can be calculated usng Newton's ethod n (16). The lowest of the two nu ponts (n the exaple, P 3 ) then gves us our value of x'j. It ght happen that we fnd the local nu at P { ntally rather than the lower nu at P 3. Ths possblty can only be guarded aganst by scannng 5, Fg. 1. A scheatc llustraton of an exaple n whch S, the weghted su of the squares of the errors, has a local axu and two na when an experental pont s found nsde a axu or nu off(x). Another dffculty occurs f x' 0 s close to ether of the two ponts of nflecton, one of whch s between P, and P 2 and the other between P 2 and P 3. Then the denonator n (16) wll be sall and the correcton to x' 0 ght be unrealstcally large. Ths can be guarded aganst n a coputer progra by checkng that ths correcton s never larger than soe specfed value, say the larger of x, + 1 x,\ and ^ x, x,_,. If t s larger then 5, ust be scanned and the nu found as before A possble proveent In practce we have found that at least two teratons and soetes ore than 0 teratons were needed to obtan convergence n our algorth. We thought ths ght be proved by obtanng a better approxaton to the nverse C" 1 than that gven n (13). We obtaned a better nverse by notng that f X\ s an approxate nverse to the atrx G, then X 2 s a better approxaton f X 2 = (23) Puttng X t equal to the expresson n (13) our proved nverse s gven by c (24) (25)

4 Polynoal curve fttng 55 The atrces A, B and C are denned n (7). Then thus co, s kept constant when S s dfferentated wth respect to the paraeters a r. Hence for 0 < r < we (26) have and S»,b/-^)W*) = 0- (32) j These are dentcal wth the equatons we used prevously except for the addtonal ters contanng the atrces Q and Q'. In practce we found that ths gave hardly any proveent although t nvolved consderably ore coputaton. If n the calculaton of x\ usng equaton (27) we use the values a^ n the coputaton of f(x) and ts dervatves, then fro ts defnton t s easly shown that g (l) = 0. So the correcton Qg w n (27) also s zero. Slarly t can be shown that f equaton (27) s used successvely untl x,' s the nu pont of S, then n the subsequent calculaton of ct r n (26) g (2) = 0 and the extra correcton n (26) s zero. Thus no proveent s obtaned. If, however, we use the new values of a r n the calculaton of x\ n equaton (27), then g (1) wll not be zero; nor wll the correcton be zero, but t wll stll be sall. Slarly the correcton n (26) wll not be zero, but t wll be sall except n the ntal step. In practce we found that the proveent was slght and rarely reduced the nuber of teratons although t ncreased the total aount of coputng n each teraton. We conclude that ths proveent s not worth the extra coputng effort. We thnk that other approxatons to the nverse of G are unlkely to be ore effcent Deng's ethod In a ethod due to Deng (1943) an approxate soluton of consderable splcty s obtaned. The process we descrbe below s the sae n prncple as Deng's ethod but t s slghtly dfferent n practce. At the axu snce r -, = 0, we can wrte w x (28) Ths gves us + 1 lnear equatons n the + 1 varables a 0 to a f we substtute an approxate value of/'(*,) nto the expresson for to,. In practce we frst solve equatons (32) wth to, replaced by a> y (equvalent to assung that there are no errors n the x readngs), then usng the values of /'(*,-) obtaned fro ths soluton we calculate to,- and solve equatons (32) agan. The process can be repeated tll convergence s reached. In each teraton we need approxately (15 nrr) ultplcatons, (13 nrr) addtons and (2 nrr) dvsons, less than n each teraton n the prevous ethod. In addton any fewer teratons are needed to reach convergence as we wll show later An proveent on Deng's ethod Deng's ethod works well n practce but t does not gve the exact nu of S because of the two approxatons ntroduced. The frst approxaton n equaton (30) can be elnated by usng equaton (29) drectly, that s by assung W, s constant and fndng the poston of x\ by teraton. We begn wth W-, = to^,-, calculate an approxate curve ft/(x), fnd x\ fro (28), calculate/'(x/) and W h recopute a new polynoal, assung W t s constant and repeat tll convergence s reached. Ths should elnate errors due to the frst approxaton and should gve ore accurate answers, but n practce we found that t ade lttle dfference to the polynoal ft and n soe cases gave values further fro the nu than Deng's ethod. Obvously ost rof the error n Deng's ethod coes fro keepng to,- constant when dfferentatng wth respect to a r. We conclude, therefore, that ths apparent proveent s not worth the consderable extra coputng effort. Therefore, substtutng n S we have, after splfcaton, that (29) W, = to,,- (l + ^ [A*/ Deng now uses Taylor's theore to elnate /(*,') and/'(x/) fro these equatons by assung that Ax',) =Ax,) + (*/' - */)/'(*/), /'(*/') =/'(*,) (30) Ths s hs frst approxaton when/(x) s not a straght lne. Usng (28) agan, S takes the for + ^ [/'(x, In hs second approxaton Deng assues that to, does not vary quckly wth respect to the paraeters a r ; 3. Results The drect teratve ethod n 2.2 and Deng's ethod have been tred on a nuber of real exaples taken fro scentfc and ndustral data and on soe artfcal exaples chosen specfcally to test the ethods. Detals of these are gven n two theses by O'Nell (1967) and by Snclar (1967). Soe of the results obtaned by exanng these exaples have already been entoned. Our an conclusons are that the drect teratve process does gve a polynoal curveft nsng S, the su of the square of the errors, and that t usually does ths n only a few teratons and wth lttle coputer te or coputer storage. Deng's approxaton s fast and so accurate that a better result wll seldo be needed. Thus we suggest that Deng's ethod should be used to gve us our frst approxaton to f{x) n the drect teratve procedure. Usually only one teraton wll then be necessary as a check. In the rare event when the drect teratve process does not converge quckly, Atken's process for acceleratng a slowly convergng sequence has been found to be useful.

5 56 O'Nell et al. We suggest that ths procedure should be adopted f convergence s not reached after about teratons. We llustrate these an conclusons wth three exaples. The frst exaple, due to Pearson (1901), s the fttng of a straght lne to the set of data gven n Table 1, wth all of the co x and co y weghts equal to unty. Pearson obtaned, as the slope of the lne, X Table 1 Data due to Pearson y X y , York (1966) obtaned 0-546, Deng's ethod gves and the teratve approach gves Pearson's correct result, after 6 teratons. Our second exaple s due to York who ntroduced the extree set of weghts gven n Table 2 for Pearson's / Table 2 Weghts gven by York for the data n Table 1 a>x 1,000 1, Wy (Ox COy, data and obtaned a slope If the jc-coordnates are assued to be free fro error, the slope s Usng ths as our frst approxaton Deng's ethod gves a slope, and a value of S = n two teratons. The drect teratve ethod converges slowly: t obtans a result as accurate as Deng's ethod only after 40 teratons, but after 125 teratons t obtans a slope, and a value of S down to S = Usng Deng's ethod as our frst approxaton and Atken's acceleratve process begnnng after sx teratons we obtaned a slope, and the lowest value of S, S = Ths nvolved a total of 13 teratons. In our thrd exaple we ft a polynoal of degree 3 to Pearson's data wth unt weghts. Neglectng errors n the x-coordnates we fnd that/(x) = * x * 3 wth S = Usng Deng's ethod we obtan after 4 teratons that f(x)= x x 3 and that S s reduced to S = In the xed ethod of Deng's approxaton and Atken's acceleratve process we fnd after 9 teratons that f(x) converges to f(x) = x * * 3 wth S = Ths agan llustrated that Deng's ethod gves a good frst approxaton, beng correct to one or two unts n the second decal place. Acknowledgeents One of us (F.J.S.) has pleasure n thankng Mr. C. D. Kep, Professor J. Menguet and Mr. M. J. D. Powell for soe helpful suggestons. Ths research has been sponsored n part by the Physcs Branch, Offce of Naval Research, Washngton, D.C., under Contract No. F C-0012, and n part by the Mnstry of Educaton, Northern Ireland. References DEMING, W. E. (1943). Statstcal Adjustent of Data, New York: Wley. FORSYTHE, G. E. (1957). Generaton and use of orthogonal polynoals for data fttng wth a dgtal coputer, /. Soc. Indust. Appl. Math., Vol. 5, pp O'NEILL, M. (1967). Least squares fttng of a polynoal when both sets of data are subject to errors. Thess, Queen's Unversty, Belfast. PEARSON, K. (1901). On lnes and planes of closest ft to systes of ponts n space, Phl. Mag., Vol. 2, pp POWELL, M. J. D. (1965). A ethod for nsng the su of squares of non-lnear functons wthout calculatng dervatves, Coputer Journal, Vol. 7, pp POWELL, M. J. D. (1966). Nuercal Analyss: an Introducton, Ed. J. Walsh, London: Acadec Press, pp SINCLAIR, I. G. (1967). Polynoal least squares fttng when both abscssae and ordnates are subject to errors. Thess, Queen's Unversty, Belfast. SMITH, F. J. (1965). An algorth for sung orthogonal polynoal seres and ther dervatves wth applcatons to curve fttng and nterpolaton, Math. Cop., Vol. 19, pp YORK, D. (1966). Least-squares fttng of a straght lne, Can. J. ofphys., Vol. 44, pp