Order of Accuracy 1 Terminology We consider a numerical approximation of an exact value u. Te approximation depends on a small parameter, wic can be for instance te grid size or time step in a numerical metod. We denote te approximation by. Te numerical metod as order of accuracy p if tere is a number C independent of suc tat u C p, (1) at least for sufficiently small. Hence, te larger te order of accuracy, te faster te error is reduced as decreases. We say tat te convergence rate of te metod is p. Te number C typically depends on te exact solution u and possibly on oter parameters in te numerical sceme. Wat is important is tat it does not depend on. Often te error u depends smootly on. Ten tere is an error coefficient D suc tat u = D p + O ( p+1). (2) Note tat tis is not equivalent to (1) since te error may be a non-smoot function of. We will get back to tis issue in Section 4 below. For now, owever, we will assume (2) olds. Example 1 In numerical differentiation we approximate u = f (0) by te forward difference = f() f(0). After Taylor expansion we get u = f(0) + f (0) + 2 2 f (ξ) f(0) f (0) = 2 f (ξ) for some ξ [0,]. Hence, for sufficiently small, say 0 < < 1 we can write as in (1), u C, C = 1 2 max ξ [0,1] f (ξ), were C does not depend on. (But it does depend on f!) Te order of accuracy is terefore one. Moreover, if f is tree times continuously differentiable we can continue te Taylor expansion one more step to get u = 2 f (0) + 2 6 f (ξ), ξ [0,]. We tus get also (2) wit D = f (0)/2, provided f is sufficiently smoot. 1 (7)
Example 2 In piecewise linear interpolation of a function u(x) on equidistant nodes in te interval [a,b] we let be te piecewise linear interpolant wen te distance between nodes is. Te error can ten be bounded as (see Lecture notes 5) max (x) u(x) max a x b a ξ b u (ξ) 2. 8 If u(x) is two times continuously differentiable, tis is tus a second order metod. If u is tree times continuously differentiable te error also depends smootly on suc tat max x (x) u(x) = D 2 + O( 3 ) for some number D. Example 3 In te trapezoidal rule we approximate te exact integral by a sum u = b a f(x)dx, = N 1 2 f(x 0) + f(x j ) + 2 f(x N), j=1 = b a N, x j = a + j. For sufficiently smoot functions f(x) tis is a second order metod and u = D 2 + O( 3 ). 2 Determining te order of accuracy empirically We are often faced wit te problem of ow to determine te order of accuracy p given a sequence of approximations 1, 2,... Tis is can be a good ceck tat a metod is correctly implemented (if p is known) and also a way to get a feeling for te trustwortiness of an approximation (ig p means ig trustwortiness). We can eiter be in te situation tat te exact value u is known, or, more commonly, tat u is unknown. 2.1 Known u If te exact value u is known, it is straigtforward to determine te order of accuracy. Ten we can ceck te sequence log u = log D p (1 + O()) = log D p + log 1 + O() = log D + p log + O(), for 1, 2,...and fit it to a linear function of log to approximate p. A quick way to do tis is to plot u as a function of in a loglog plot in Matlab and determine te slope of te line tat appears. Te standard way to get a precise number for p is to alve te parameter and look at te ratio of te errors u and u /2, u /2 u = D p + O( p+1 ) D(/2) p + O((/2) p+1 ) = D + O() D2 p + O() = 2p + O(). Hence ( ) u log 2 = p + O(). /2 u Example 4 Te exact integral of sin(x) over [0,π] equals two. Computing wit te trapezoidal rule and plotting 2 in a loglog plot we get te result sown in Figure 1. 2 (7)
u u 10 2 2 10 3 10 4 10 2 10 1 Figure 1. Error in trapezoidal rule for f(x) = sin(x) as a function of. Te dased line is 2 as a function of wic as precisely slope two. It tus indicates te slope for a second order metod, for comparison. /2 /2 /4 log 2 /2 /4 π/5 1.933765598092805-0.049757939416650 4.024930251575880 2.008963782835339 π/10 1.983523537509455-0.012362435199260 4.006184396966857 2.002228827158397 π/20 1.995885972708715-0.003085837788350 4.001543117204195 2.000556454557076 π/40 1.998971810497066-0.000771161948770 4.000385593360853 2.000139066704584 π/80 1.999742972445836-0.000192771904301 4.000096386716427 2.000034763740606 π/160 1.999935744350136-0.000048191814813 π/320 1.999983936164949 Table 1. Table of values for te trapezoidal rule for f(x) = sin(x). Te last column is te final approximation of te order of accuracy p. 3 (7)
2.2 Unknown u Wen u is not known tere are two main approaces. Te first one is to compute a numerical reference solution wit a very small and ten proceed as in te case of a known u. Tis can be quite an expensive strategy if is costly to compute. Using te resulting p to gauge te trustwortiness of is also less relevant wen we already ave a good reference solution. Te second approac is to look at ratios of differences between computed for different. Most commonly we compare solutions were is alved successively. Wen p is large tis gives a fairly good approximation of te error u since = D p D(/2) p +O( p+1 ) = D p (1 2 p )+O( p+1 ) = ( u)(1 2 p )+O( p+1 ). Wat is more important, owever, is tat, regardless of p, tis difference decays to zero wit te same speed as te actual error u. Terefore one can do te same trick as wen u is known and consider te ratio of te differences. We get /2 /2 /4 = Dp D(/2) p + O( p+1 ) D(/2) p D(/4) p + O( p+1 ) = 1 2 p + O() 2 p 2 2p + O() = 2p + O(). (3) Hence, after computing for, /2 and /4 we can evaluate te expression above and get an estimate of p, as before ( ) /2 log 2 = p + O(). /2 /4 Example 5 Consider again Example 4. If te exact integral value was not known we would look at te values computed by te trapezoidal rule and ceck te ratios of differences as above. Te result is summarized in Table 1. 3 Asymptotic region We note tat te estimates of p in all te metods above gets better as 0 because of te O() term. (Te precise value is only given in te limit 0.) We say tat te metod is in its asymptotic region (or range) of accuracy wen is small enoug to give a good estimate of p ten te O( p+1 ) term in (2) is significantly smaller tan D p. Tis required size of can, owever, be quite different for different problems. To verify tat we are indeed in te asymptotic region, it can be valuable to make te estimate of p for several different and ceck tat we get approximately te same value. Usually one terefore computes not just for tree values of, but for a longer sequence,,/2,/4,/8,/16,... and compares te corresponding ratios, /2 /2 /4, /2 /4 /4 /8, /4 /8 /8 /16,... Similarly, if u is known one considers u for several decreasing values of wen fitting te line. Example 6 If we perform te same experiments as in Example 4 and Example 5 above, but wit f(x) = sin(31x) te constant D will be muc bigger, meaning tat te asymptotic region is sifted to smaller. Te results are sown in Figure 2 and Table 2. It is not until < π/40 10 1 tat te numbers start to look reasonable. Te general size of te error is also muc larger tan in Figure 2 because of te bigger D. 4 (7)
10 0 u u 2 10 1 10 2 10 3 10 2 10 1 Figure 2. Error in trapezoidal rule for f(x) = sin(31x). Te dased line is 2 wic indicates te slope for a second order metod. /2 /2 /4 log 2 /2 /4 π/5 1.933765598092808 1.983523537509458 14.784906442999516 3.886053209184444 π/10-0.049757939416650 0.134158680351247-0.630173999781565 π/20-0.183916619767896-0.212891487744257 7.778391902691306 2.959471924644287 π/40 0.028974867976361-0.027369601635860 4.437830912882666 2.149854700028653 π/80 0.056344469612220-0.006167337641551 4.096338487974619 2.034334932805155 π/160 0.062511807253771-0.001505573247830 π/320 0.064017380501601 Table 2. Table of values for te trapezoidal rule for f(x) = sin(31x). Te last column is te final approximation of te order of accuracy p. 5 (7)
/2 /2 /4 log 2 /2 /4 0.2 0.302842712474619 0.009289321881345 26.142135623725615 4.708305098603142 0.1 0.293553390593274 0.000355339059327 1.999999999999688 0.999999999999775 0.05 0.293198051533946 0.000177669529664 2.000000000001875 1.000000000001352 0.025 0.293020382004283 0.000088834764832 2.635450714080436 1.398049712285012 0.0125 0.292931547239451 0.000033707617584 12.589489353884787 3.654147861537719 0.00625 0.292897839621867 0.000002677441208 0.003125 0.292895162180659 Table 3. Table of values for te trapezoidal rule for f(x) = x α wit α = 1/ 2. Te last column is te final approximation of te order of accuracy p, wic fails for tis case. 4 Non-smoot error So far we ave assumed tat te error depends smootly on te parameter. Ten te error is of te form in (2). Tis is, owever not always te case. Te error can, for instance, depend discontinuously on, eventoug it is bounded as in (1). Te reason for tis can be discontinuities in te metod itself (e.g. case switces) or non-smoot functions in te problem (e.g. solutions, sources, integrands). Wen te error is non-smoot one cannot ceck convergence rates by looking at ratios of differences as in Section 2.2. Oter metods must be used. Example 7 Consider te trapezoidal rule applied to te integral 1 0 x α dx, for some value 0 < α < 1. Here te integrand is not smoot at x = α so te standard second order of accuracy of te trapezoidal rule is not guaranteed. However, since te integrand is linear away from x = α, te trapezoidal rule is exact everywere except in te node interval wic contains α. Te error tere depends crucially on te distance between α and te nearest node. More precisely, if x j α < x j+1 and x j+1 x j =, u = = xj+1 x α dx x j α + x j+1 α x j 2 α x j (α x)dx + xj+1 α (x α)dx x j+1 x j 2 = β(β 1) 2 2, were β = β() = (α x j )/, i.e. te fractional part of α/, wic is a discontinuous function of. Te metod is still second order accurate since β() 1 and (1) terefore olds wit C = 1/8. However, te results presented in Figure 3 and Table 3 clearly sows te non-smootness of te error and te failure of te ratios of te differences to predict te order of convergence. 6 (7)
10 3 u u 2 10 4 10 5 10 6 10 7 10 2 10 1 Figure 3. Error in trapezoidal rule for f(x) = x α wit α = 1/ 2. Te dased line is 2 wic indicates te slope for a second order metod. 7 (7)