Numerical techniques to solve equations

Transcription

1 Programming for Applications in Geomatics, Physical Geography and Ecosystem Science (NGEN13) Numerical techniques to solve equations Vaughan Phillips Associate Professor, Department of Physical Geography and Ecosystem Science, Lund University 10 Oct 2016, 1 to 3 pm

2 OUTLINE» Introduction» Errors and statistics» Iteration» Interpolation» Numerical integration» Summary

3 READING» Kreyszig, E., 2013: Advanced Engineering Mathematics. Wiley, Chapter 18» Optional extra reading:» Adams, D., 1979: The Hitchhiker's Guide to the Galaxy.

4 INTRODUCTION

5 Programming» Input data entering program must be carefully checked» Numerical method must be chosen carefully» Need to check the program and plan the programming Program can be tested by applying it to a simple problem with a known solution E.g. model validation

6 ERRORS AND STATISTICS

7 Errors in numerical analysis» Finite number of digits in computations Numerical analysis involves finite processes Results are approximate value of the exact result error ε = a a Approximate value True value

8 General types of errors in science (Absolute) error = Approximation - true value = X approx X real Systematic error,, is bias from the average error, < >, being non-zero, with <X approx > <X real > Random error,, is unpredictable error with an average of zero, < > = 0, such that <X approx > = <X real > Relative error, e, is defined by e = / X real

9 Errors in numerical analysis» Experimental errors are from errors in data (e.g. observations)» Truncation errors = error from finite number of computation steps» Round-off errors = error from discarding decimals might be rounded to 3 in a program» Programming errors ( bugs ) Compilers prevent runs of program until obvious errors removed Problem: errors in logic are more tricky to find» Contemplation» read through whole program» Model validation

10 Stability» Computer program ( algorithm ) = sequence of computations by well-defined rules» Algorithm is unstable if intermediate computations have round-off or truncation errors that strongly change the final result overflow or crash Stable otherwise

11 How to quantify the error of any result from a computation?

12 Estimating overall error from multiple sources of random error» General method: combine the individual random errors of quantities in the maths equation» Quantities multiplied or divided together: Relative error of product or ratio sum of relative errors Relative error of X = Y Z... is e X e Y + e Z +..., where e X, e Y and e Z are relative errors of X, Y and Z

13 Estimating overall error from multiple sources of random error» Quantity multiplied by a known constant: Absolute error of product = the constant multiplied by the absolute error error of X = a Y, where a is a constant, is X = a Y where X and Y are errors of X and Y

14 Estimating overall error from multiple sources of random error» Quantities added or subtracted: Absolute error of sum or difference = sum of absolute errors error of X = Y +Z +.. is X = Y + Z +..., where X, Y and Z are errors of X, Y and Z

15 Mean, standard deviation and variance of population of (very many) measurements A discrete probability function, f(x), is probability, P, of x having a value: P (x is x j ) = f(x j ) Mean of f(x) is μ = j x j f(x j ) = E (X) Variance of f(x) is σ 2 = E( [X - μ ] 2 ) = j (x j - μ ) 2 f(x j ) where σ is the standard deviation

16 Measurements with random error follow a`normal (or Gaussian ) probability distribution

17 Sample of measurements drawn from hypothetical population For a sample of size, n, namely x 1, x 2... x n, the sample mean is x = j=1 n n x j and sample variance is s 2 = j=1 n (x j x) 2 n 1 where s is the standard deviation of the sample

18 Sample standard deviation / n error of sample mean as estimator.. (SEM) Measurements made independently follow a normally ( N ) distributed population with mean μ and standard deviation, σ: X ~ N (μ, σ 2 ) ~ N (0, σ 2 ) sample mean, x = (X 1 + X X n )/n, drawn from population, is a random variable: x ~ N (μ, σ 2 /n) standard deviation of sample, s, approximates that of the population mean (see above formula)» standard error of the mean (SEM) equals the standard deviation of the sample-mean as an estimator of the population mean: SEM = σ 2 /n

19 Need for large enough samples» Large samples of data produce a better estimate of the population mean E.g. opinion polls are based on 100s or 1000s of people» Standard error often used for error-bars on plots when point plotted is derived from a large sample Point is estimating some population quantity» Standard deviations are also used for errorbars if we only want to denote the true spread of the data

20 Statistical models» If analytical estimates of error of the result from a computation is not possible, then: Statistical models of a large sample of instances of input variables with known errors each instance is drawn randomly from a normal (Gaussian) distribution

21 MATLAB tools: useful statistical functions» Plot a vs b with errorbars of length L and U (all four are vectors) Errorbar(a,b,L,U)» Do the mean of elements in a vector: Mean(a(:))» Their median: Median(a(:))» And their standard Deviation and variance Std(a(:)) Var(a(:))

22 ITERATION

23 Iteration» Any equation we may want to solve can be expressed as f x = 0» Only the simplest problems will have an exact value from an analytical solution» Examples: Algebraic eqs (solutions are called roots )» x 3 + x = 1 Transcendental eqs» cosh x = sec x

24 Iteration» Usually we need an iteration method» Create computational algorithm (g(x)) perform same sequence of steps many times Start Initial guess x 0 Better approximation x 1 = g(x 0 ) Even better approximation x 2 = g(x 1 )» Stop when x n+1 x n < ε (the desired error)

25 Fixed-point iteration» Transform f x = 0 algebraically into g x = x» Initial guess, x 0» Then compute iteratively x n+1 = g x n for n = 0, 1, 2,

26 Fixed-point iteration» If the sequence of values of x 0, x 1, is convergent, then the iterative process itself is convergent» If they are not convergent, then it is worth trying again with another transformation» Caution: there may be multiple solutions to f(x) = 0 and some may be unphysical» Initial guess may determine which solution is converged on

27 Newton s method» Algebraically find the derivative, f (x)» If continuous derivative, f, exists, then it is easier to find where f x intersects the x-axis (f = 0)» Initial guess, x 0» Compute iteratively: x n+1 = x n f x n /f (x n ) for n = 0, 1, 2,

28 Secant method» Same as before, except use a numerical approximation for derivative No need to differentiate algebraically f x n f x n f x n 1 x n x n 1

29 INTERPOLATION

30 Interpolation» n + 1 data as pairs of numbers (x 0, f 0 ), (x 1, f 1 ), (x 2, f 2 ), (x n, f n ),» Find a polynomial, p n x, of degree, n, or less p n x 0 = f 0 p n x 1 = f 1 p n x n = f n» Then we can interpolate to find f(x) at any intermediate value of p n x f x within the range of data, x 0 x n» Extrapolation is similar but involves going outside the range of data, and is less accurate!

31 Linear and quadratic interpolation» Linear interpolation between two points: p 1 x = f 0 + x x 0 f x 0, x 1 f x 0, x 1 = (f 1 f 0 )/(x 1 x 0 )» Quadratic interpolation between three points: p 2 x = f 0 + x x 0 f x 0, x 1 + x x 0 x x 1 f x 0, x 1, x 2 f x 0, x 1, x 2 = f x 1,x 2 f x 0,x 1 x 2 x 0

32 Spline interpolation» Problem: with large numbers of data-points, we may wish to use a higher-order polynomial to fit them But oscillations between nodes can occur» Solution: divide overall interval into n sub-intervals and use one polynomial on each subinterval All the polynomials join up Cubic splines are the most popular

33 MATLAB tools: fit a line to data in 2D» Curve-fitting toolbox fit( ) fits a line to data» Example: fit a quadratic curve to data: load census; f=fit(cdate,pop,'poly2') plot(f,cdate,pop)

34 MATLAB tools: Fit a surface to data in 3D» Load some data and fit a polynomial surface of degree 2 in x and degree 3 in y. Plot the fit and data. load franke sf = fit([x, y],z,'poly23') plot(sf,[x,y],z)

35 NUMERICAL INTEGRATION

36 Rectangular rule for numerical integration» Problem: how to evaluate a definite integral? J = a b f x dx = F b F a = area under curve of f(x) F x = f(x)» Solution: Any integral is the limit of a sum J = a b b f x dx = lim δx 0 x=a We can simply let δx be small: a b f x dx b x=a f x δx f x δx = h f x 1 + f x 2 + f x n Divided interval into n equal subintervals of length b a h =, each with mid-point x j n

37 Trapezoidal rule» Alternatively, divide area under graph into many trapezoids of equal width b a f x dx b x=a f x δx = h 1 2 f a + f x f x n f b 2 End-points of sub-intervals : x j = x 0 + jh Each trapezoid spans the subinterval between endpoints

38 Simpson s rules Simpson s rule is similar, except it uses a quadratic interpolation for each group of 3 points: a b f x dx h 3 f 0 + 4f 1 + 2f 2 + 2f 2n 2 + 4f 2n 1 + f 2n

39 SUMMARY

40 Summary» Numerical solutions are usually the only option in many engineering applications or environmental modelling problems Results are approximate, not exact Computational stability if intermediate errors (e.g. round-off) have little influence on final result» Fixed-point iteration: try various re-arrangements of equation until you get convergence

41 questions? coming next: exercises