Notions of. Antonio Mucherino. last update: August University of Rennes 1 Numerical Analysis. A.

Transcription

1 Notions of Antonio Mucherino University of Rennes 1 last update: August 2013 commom s

2 ... commom s

3 Aircraft and wind Suppose an aircraft flies from Paris to Rio and then it comes back. Suppose is constant during the whole travel and it is able to influence the speed of the aircraft. Paris - Rio, time t 1 = 5.1 hours, aircraft flying against wind Rio - Paris, time t 2 = 4.7 hours, aircraft flying with wind distance: 5700 miles... How can we find the average speed of the aircraft and the average speed of? commom s

4 How to solve this problem? Let x be the average speed of the aircraft, and let y be the average speed of : the actual aircraft speed is x y when it flies against the actual aircraft speed is x + y when it flies with the distance d for each travel can be computed as the product between the time (t 1 or t 2 ) and the actual speed We can define the following system of equations: { t1 (x y) = d t 2 (x + y) = d... commom s This is a linear system: t 1, t 2 and d are parameters (already known) x and y are variables

5 General form of a linear system with 2 equations: { a11 x + a 12 y = b 1 a 21 x + a 22 y = b 2... commom s And, in matrix form: ( a11 a 12 a 21 a 22 ) ( x y The coefficient matrix about the existence of solutions about the number of solutions (out of the scope of this course). ) = ( b1 b 2 ) ( ) a11 a 12 is able to provide information a 21 a 22

6 The solution for our example For the system { t1 x t 1 y = d t 2 x + t 2 y = d we can find a solution (x, y) analytically:... x = d t 1 + y y = d ( 1 t ) 2 2t 2 t 1 commom s

7 The solution for our example So, in our example: t 1 = 5.1 t 2 = 4.7 d = 5700 and therefore: x = miles/hours... y = 47.6 miles/hours Can we program a computer to make this work for us? commom s Note that, in this simple example, we did not consider the health rotation.

8 Back substitution... Suppose the coefficient matrix of our linear system is an upper triangular matrix: a 11 a 12 a 13 a 14 0 a 22 a 23 a a 33 a a 44 In this situation, we can compute: x 4 = b 4 a 44 x 1 x 2 x 3 x 4 = b 1 b 2 b 3 b 4 commom s

9 Back substitution... Suppose the coefficient matrix of our linear system is an upper triangular matrix: a 11 a 12 a 13 a 14 0 a 22 a 23 a a 33 a a 44 In this situation, we can compute: x 1 x 2 x 3 x 4 x 3 = b 3 a 34 x 4 a 33 = b 1 b 2 b 3 b 4 commom s

10 Back substitution... Suppose the coefficient matrix of our linear system is an upper triangular matrix: a 11 a 12 a 13 a 14 0 a 22 a 23 a a 33 a a 44 In this situation, we can compute: x 1 x 2 x 3 x 4 = x 2 = b 2 a 23 x 3 a 24 x 4 a 22 b 1 b 2 b 3 b 4 commom s

11 Back substitution... Suppose the coefficient matrix of our linear system is an upper triangular matrix: a 11 a 12 a 13 a 14 0 a 22 a 23 a a 33 a a 44 In this situation, we can compute: x 1 x 2 x 3 x 4 = x 1 = b 1 a 12 x 2 a 13 x 3 a 14 x 4 a 11 b 1 b 2 b 3 b 4 commom s

12 C function for back substitution... commom s void back(int n,double **a,double *x) { // n is the system dimension // a is the coefficient matrix // (must be upper triangular) // x is // the vector of known terms (input) // the solution (output) }; int i,j; for (i = n - 1; i >= 0; i--) { for (j = i+1; j < n; j++) { x[i] = x[i] - a[i][j]*x[j]; }; x[i] = x[i]/a[i][i]; };

13 Gaussian elimination... How to solve linear whose coefficient matrix is not in triangular form? Gaussian elimination : transform the system in an equivalent system whose coefficient matrix is in triangular form. Example: 2x +y z = 8 3x y +2z = 11 2x +y +2z = 3 = 2x +y z = y z = 1 z = 1 commom s

14 LAPACK LAPACK Algebra PACKage... free library for linear algebra (including linear ) it s a freely-available software package (library + sources) originally developed in Fortran, there are versions for C and C++ it s based on another library called BLAS, which contains for efficient matrix manipulations (sums, products,... ) commom s

15 Some references... Wikipedia page about linear, Online solution of linear, LAPACK, BLAS, commom s

16 ... commom s

17 Stationary points In many applications, stationary points of are of particular interest.... They might provide: the minimum and maximum points of the equilibrium point of a dynamic system (may be stable or not)... In order to find a stationary point, the derivative of a function must be computed, and roots of such a derivative must be identified: df(x) dx = 0 commom s

18 Given a function f : [a, b] Y, how to find its roots (zeros)? f(x) = 0 Examples: ax = b = x = b a... ax 2 + bx + c = 0 = x 1 = b b 2 4ac 2a x 2 = b + b 2 4ac 2a commom s

19 A simple Simplest for finding roots: Extract a predefined number of points from the function domain [a, b] and evaluate the function in all these points. One of these points can be a root (or be close to a root).... commom s This is not able to provide a good approximation of roots of having a more complex shape.

20 Bisection: basic idea... commom s is an iterative which defines a sequence of intervals {a k, b k } k=1,2,...,itmax converging to one function root. At the beginning, the whole function domain is considered: [a 0, b 0 ] = [a, b] At each iteration, the following two steps are performed: the average point of the current interval is computed: x k = a k + b k a k 2 the new interval is then defined as: { [ak+1, b k+1 ] = [a k, x k ] if f(a k )f(x k ) 0 [a k+1, b k+1 ] = [x k, b k ] otherwise

21 Bisection: basic idea In the bisection, intervals [a k, b k ] are reduced in size at each iteration, and they are supposed to converge to a function root.... commom s

22 Bisection: basic idea In the bisection, intervals [a k, b k ] are reduced in size at each iteration, and they are supposed to converge to a function root.... commom s

23 Bisection: basic idea In the bisection, intervals [a k, b k ] are reduced in size at each iteration, and they are supposed to converge to a function root.... commom s

24 Applicability... commom s can be applied to f : [a, b] Y : if all points in [a, b] can be evaluated if f is a continuous function Moreover, if at least one of the intervals [a k, b k ] is such that f(a k )f(b k ) < 0 then, the converges toward one of the roots contained in the interval. The can be stopped when a k b k < ε or f(a k ) f(b k ) < ε where ε is a small real number (tolerance).

25 C function for the bisection algorithm... commom s double bisection(double a,double b,double (*f)(double),double eps,int itmax) { // [a,b], function domain // double (*f)(double), pointer to a function // eps, tolerance // itmax, maximum number of iterations int it; double ca,cb,cx,fa,fb,fx; ca = a; cb = b; fa = f(a); fb = f(b); cx = (ca+cb)/2.0; fx = f(cx); it = 0; while (it <= itmax && fabs(fx) > eps && fabs(cb-ca) > eps && fabs(fb-fa) > eps) { it = it + 1; if (fa*fx < 0) { cb = cx; fb = fx; } else { ca = cx; fa = fx; }; cx = (ca+cb)/2.0; fx = f(cx); }; return cx; };

26 Pointers to A pointer to a function can make reference to any function of a predefined type: double (*f)(double)... commom s In the main function: double xcube(double x); double polynomial(double x); double bisection(double a,double b,double (*f)(double),double eps,int itmax); main() { int i,j; double a,b,root;... double *f(double);... }; f = xcube; root = bisection(a,b,f,0.001,100);... f = polynomial; root = bisection(a,b,f,0.001,100);...

27 Other s for finding roots... Newton s it is based on the computation of the tangent to the function in the current root approximation Secant similar to the Newton s, but the tangent is replaced by a secant (the function does not have to be differentiable in the whole domain) Lehmer-Schur extension of the bisection Brent s combination of different s, including the bisection, with the aim of speeding up the search commom s

28 Polynomial interpolation... commom s

29 ... The for ammonia reacting in hydrogen and nitrogen gases depends upon the hydrogen-nitrogen mole ratio, the pressure, and the temperature. For a 3-to-1 hydrogen-nitrogen mole ratio, the equilibrium constant K p for a range of pressures and temperatures is given by: 100 atm 200 atm 300 atm 400 atm 500 atm 400 C C C C C Encyclopedia of Chemical Technology, vol. 2, 2 nd edition, New York, Wiley, commom s

30 ... Suppose that values for K p related to 500 C and 300 atm are not available, and that, for same reason, we cannot perform any experiment to find them. 100 atm 200 atm 300 atm 400 atm 500 atm 400 C C C C C How can we find the needed values for the constant K p? Easiest solution: linear interpolation. commom s

31 ... Suppose that values for K p related to 500 C and 300 atm are not available, and that, for same reason, we cannot perform any experiment to find them. 100 atm 200 atm 300 atm 400 atm 500 atm 400 C C C C C How can we find the needed values for the constant K p? Easiest solution: linear interpolation. commom s

32 ... Suppose that values for K p related to 500 C and 300 atm are not available, and that, for same reason, we cannot perform any experiment to find them. 100 atm 200 atm 300 atm 400 atm 500 atm 400 C C C C C How can we find the needed values for the constant K p? Easiest solution: linear interpolation. commom s

33 ... Suppose that values for K p related to 500 C and 300 atm are not available, and that, for same reason, we cannot perform any experiment to find them. 100 atm 200 atm 300 atm 400 atm 500 atm 400 C C C C C How can we find the needed values for the constant K p? Easiest solution: linear interpolation. commom s

34 ... Suppose that values for K p related to 500 C and 300 atm are not available, and that, for same reason, we cannot perform any experiment to find them. 100 atm 200 atm 300 atm 400 atm 500 atm 400 C C C C C How can we find the needed values for the constant K p? Easiest solution: linear interpolation. commom s

35 ... Suppose that values for K p related to 500 C and 300 atm are not available, and that, for same reason, we cannot perform any experiment to find them. 100 atm 200 atm 300 atm 400 atm 500 atm 400 C C C C C How can we find the needed values for the constant K p? Easiest solution: linear interpolation. commom s

36 interpolation Let f : [a, b] Y be a function such that the pair (x 1,f(x 1 )) is known, with x 1 [a, b] the pair (x 2,f(x 2 )) is known, with x 2 [a, b] and x 2 > x 1 f(x) is not known for any x (x 1, x 2 )... commom s interpolation: assign to the interval (x 1, x 2 ) of f(x) the equation of the line between x 1 and x 2.

37 interpolation Let f : [a, b] Y be a function such that the pair (x 1,f(x 1 )) is known, with x 1 [a, b] the pair (x 2,f(x 2 )) is known, with x 2 [a, b] and x 2 > x 1 f(x) is not known for any x (x 1, x 2 )... commom s interpolation: assign to the interval (x 1, x 2 ) of f(x) the equation of the line between x 1 and x 2.

38 interpolation Let f : [a, b] Y be a function such that the pair (x 1,f(x 1 )) is known, with x 1 [a, b] the pair (x 2,f(x 2 )) is known, with x 2 [a, b] and x 2 > x 1 f(x) is not known for any x (x 1, x 2 )... commom s interpolation: assign to the interval (x 1, x 2 ) of f(x) the equation of the line between x 1 and x 2.

39 Quadratic interpolation Let f : [a, b] Y be a function such that the pair (x 1,f(x 1 )) is known, with x 1 [a, b] the pair (x 2,f(x 2 )) is known, with x 2 [a, b] and x 2 > x 1 the pair (x 3,f(x 3 )) is known, with x 3 [a, b] and x 3 > x 2 > x 1 f(x) is not known for any x [a, b] \ {x 1, x 2, x 3 }... commom s Quadratic interpolation: assign to the interval (x 1, x 3 ) of f(x) the equation of the parabola passing through x 1, x 2 and x 3.

40 Quadratic interpolation Let f : [a, b] Y be a function such that the pair (x 1,f(x 1 )) is known, with x 1 [a, b] the pair (x 2,f(x 2 )) is known, with x 2 [a, b] and x 2 > x 1 the pair (x 3,f(x 3 )) is known, with x 3 [a, b] and x 3 > x 2 > x 1 f(x) is not known for any x [a, b] \ {x 1, x 2, x 3 }... commom s Quadratic interpolation: assign to the interval (x 1, x 3 ) of f(x) the equation of the parabola passing through x 1, x 2 and x 3.

41 Quadratic interpolation Let f : [a, b] Y be a function such that the pair (x 1,f(x 1 )) is known, with x 1 [a, b] the pair (x 2,f(x 2 )) is known, with x 2 [a, b] and x 2 > x 1 the pair (x 3,f(x 3 )) is known, with x 3 [a, b] and x 3 > x 2 > x 1 f(x) is not known for any x [a, b] \ {x 1, x 2, x 3 }... commom s Quadratic interpolation: assign to the interval (x 1, x 3 ) of f(x) the equation of the parabola passing through x 1, x 2 and x 3.

42 Lagrangian interpolation... Let f : [a, b] Y be a function such that the pairs (x i,f(x i )) are known, with x i {x 1, x 2,...,x n } [a, b] f(x) is not known for any x [a, b] \ {x 1, x 2,...,x n } Lagrangian interpolation: assign to the interval (x 1, x n ) of f(x) the equation of the polynomial of degree n 1 passing through the n points x 1, x 2,... x n. commom s

43 Lagrangian interpolation A general polynomial of degree n 1 can be written as:... f(x) = a n 1 x n 1 + a n 2 x n a 2 x 2 + a 1 x + a 0 For the polynomial to pass through the n points (x i, y i ), we need to solve the following system of linear equations: y 1 = a n 1 x n a n 2 x n a 2 x1 2 + a 1x 1 + a 0 y 2 = a n 1 x n a n 2 x n a 2 x2 2 + a 1x 2 + a 0 y 3 = a n 1 x n a n 2 x n a 2 x3 2 + a 1x 3 + a 0... y n = a n 1 xn n 1 + a n 2 xn n a 2 xn 2 + a 1 x n + a 0 commom s

44 Lagrangian interpolation It can be proved that the system of linear equations has only one solution: {a 0, a 1, a 2,..., a n 1 } the polynomial of degree n 1 and having as coefficients the found a i s is such that: y i = f(x i ), i = 1, 2,..., n... commom s The general formula for Lagrangian interpolation is: f(x) = n n ( ) x xj y i i=1 j=1,j i x i x j

45 C function for interpolation... double interpol(int n,double *x,double *y,double p) { // n, number of available (x,y) // x, vector containing all x s // y, vector containing all y s // p, point where to evaluate lagrangian polynomial int i,j; double sum,prod; sum = 0.0; for (i=0; i<n; i++) { prod = 1.0; for (j=0; j<n; j++) { if (j!=i) { prod = prod * ((p-x[j])/(x[i]-x[j])); }; }; sum = sum + y[i]*prod; }; return sum; }; commom s

46 Regression Suppose that the form of f(x) is known a priori.... commom s If f(x) is linear, would the lagrangian polynomial be a good model? Solution: regression models.

47 Regression Suppose that the form of f(x) is known a priori.... commom s If f(x) is linear, would the lagrangian polynomial be a good model? No! Solution: regression models.

48 ... commom s

49 Archimedes Archimedes (287BC 212BC) was a Greek mathematician, physicist, engineer, inventor, and astronomer. He is generally considered to be the greatest mathematician of antiquity. Archimedes was able to approximate the area of a circle with polygons converging to the shape of the circle.... He was able to approximate the value of π to commom s

50 Solving the definite integral b a f(x)dx equals to finding the area under the curve y = f(x) and between x = a and x = b.... commom s We suppose: a and b are not ±, there are no points x [a, b] such that f( x) = ±.

51 Idea: approximate the area defined by f(x) with the area of the trapezoid ABCD:... commom s Area of trapezoid: 1 2 h (f(a) + f(b)). The area between y = f(x) and the segment DC corresponds to the error introduced with this approximation.

52 : divide [a, b] in n equal parts of length h and approximate each subinterval [x i, x i+1 ] with the area of the corresponding trapezoid:... commom s Trapezoidal formula: 1 n 2 h (f(x i ) + f(x i+1 )). i=1

53 : the C function... double trapez(double a,double b,int n, double (*f)(double)) { // interval [a,b] (input) // n, number of subintervals (input) // f, pointer to function (input) // returning value: approx. of the area defined by f(x) in [a,b] int i; double h,area; double ca,cb,fa,fb; h = (b - a)/n; ca = a; cb = ca + h; fa = f(ca); fb = f(cb); area = fa + fb; for (i = 1; i < n; i++) { ca = cb; fa = fb; cb = cb + h; fb = f(cb); area = area + fa + fb; }; return h*area/2.0; }; commom s

54 Other algorithms Simpson rule: instead of using trapezoids to approximate the areas, parabolas interpolating 3 consecutive points are employed. This simple modification increases the accuracy of the. Gaussian quadrature: subintervals of [a, b] do not have the same length but they are chosen so that the global accuracy increases.... W.S. Dorn, D.D. Mc Cracken, Methods with Fortran IV Case Studies, John Wiley & Sons, Inc., commom s

55 ... commom s

56 problems General form on an optimization problem:... min f(x) x A subject to a set of constraints: { x B g(x) = 0 x C h(x) 0 where f(x) is the objective function g(x) represents the equality constraints h(x) represents the inequality constraints commom s

57 Some s for optimization Deterministic s Simplex (may require some assumptions to be satisfied)... commom s Branch & Bound Branch & Prune... Heuristic s (no guarantees for optimality) Simulated Annealing Genetic Algorithms Tabu Search Variable Neighbourhood Search...