i nfoclearinghouse.com Solving Non-linear Equations with SCILAB Gilberto E. Urroz, Ph.D., P.E. Distributed by

Transcription

1 Solving Non-linear Equations with SCILAB By Gilberto E. Urroz, Ph.D., P.E. Distributed by i nfoclearinghouse.com 00 Gilberto E. Urroz All Rights Reserved

2 A "zip" file containing all of the programs in this document (and other SCILAB documents at InfoClearinghouse.com) can be downloaded at the following site: ab_docs/scilabbookfunctions.zip The author's SCILAB web page can be accessed at: Please report any errors in this document to:

3 SOLUTION TO NON-LINEAR EQUATIONS Introduction to comple numbers Eamples of basic comple number operations in SCILAB 3 Comple number calculations 5 Eamples of operations with comple numbers 6 Solution to quadratic and cubic equations 7 Quadratic equations 7 Cubic equations 9 The many roots of a real or comple number Principal values of a cubic root 3 Polynomials and solutions to polynomial equations 4 Solution of a single non-linear equation 6 Interval-halving or bisection method 7 Pseudo-code for the interval-halving or bisection method 8 SCILAB function for interval-halving or bisection 8 Eample of interval-halving (bisection) method application 9 The Newton-Raphson method 0 A SCILAB function for the Newton-Raphson method The Secant Method 4 A SCILAB function for the secant method 5 Application of secant method 5 Fied -point iteration 6 A SCILAB function for fied iteration 6 Applications of fied-point iteration 7 Solving systems of non-linear equations 8 SCILAB function for Newton-Raphson method for a system of non-linear equations 30 Illustrating the Newton-Raphson algorithm for a system of two non-linear equations 3 Solution using function newtonm 3 Secant method to solve systems of non-linear equations 33 Illustrating the secant algorithm for a system of two non-linear equations 34 SCILAB function for secant method to solve systems of non-linear equations 36 Solution using function secantm 37 Solving non-linear equations with the fsolve function 37 Solving single non-linear equations with fsolve 37 Solving a system of non-linear equations with fsolve 38 Download at InfoClearinghouse.com 00 Gilberto E. Urroz

4 Solution to non-linear equations Non-linear equations covered in this chapter include quadratic, cubic, and polynomial equations. These are known as algebraic equations. We also consider equations involving non-algebraic terms such as eponential, logarithmic, trigonometric, and hyperbolic functions. These would be referred to as transcendental equations, since those types of functions are known as transcendental functions (i.e., they transcend the realm of algebraic equations). Even the simplest of algebraic equations, namely, quadratic equations, may produce a comple solution. Therefore, we start this chapter with a quick introduction to comple numbers. The chapter continuous with numerical solutions to quadratic, cubic, polynomial, and transcendental equations. Methods presented include bisection, Newton-Raphson, and secant. The chapter concludes with the analysis of systems of non-linear equations. Introduction to comple numbers A comple number z is a number written as z = + iy, where and y are real numbers, and i is the imaginary unit defined by I = -. The comple number +iy has a real part, and an imaginary part, = Re(z), y = Im(z). We can think of a comple number as a point P(,y) in the -y plane, with the -ais referred to as the real ais, and the y-ais referred to as the imaginary ais. Thus, a comple number represented in the form +iy is said to be in its Cartesian representation. A comple number can also be represented in polar coordinates (polar representation) as where z = re iθ = r cosθ + I r sinθ r = z = ( +y ) / is the magnitude of the comple number z, and is the argument of the comple number z. θ = Arg(z) = arctan(y/) The relationship between the Cartesian and polar representation of comple numbers is given by the Euler formula: e iθ = cosθ + i sinθ Download at InfoClearinghouse.com 00 Gilberto E. Urroz

5 The comple conjugate of a comple number z = + iy = re iθ,, is z = iy = re -Iθ. The comple conjugate of z can be thought of as the reflection of z about the real (-) ais. Similarly, the negative of z, z = --iy = - re iθ, can be thought of as the reflection of z about the origin. Eamples of basic comple number operations in SCILAB The unit imaginary number, i = (-), is represented in SCILAB by the symbol %i. A sequence of the first 0 powers of I can obtained by using: -->ipow = []; for j=:0, ipow=[ipow %i^j]; ipow ipow =! i -. - i. i -. - i. i -.! A comple number, say z = 3+5i, is written as: -->z =3+5*%i z = i The functions real and imag can be used to obtain the real and imaginary parts, respectively, of a comple number, for eample: -->real(z), imag(z) The magnitude and argument of the comple number z are obtained as: -->abs(z), atan(imag(z)/real(z)) If we write the polar representation of a comple number as r ep(%i*theta), SCILAB returns the Cartesian representation. For eample, -->5*ep(0.5*%i) i Download at InfoClearinghouse.com 3 00 Gilberto E. Urroz

6 SCILAB provides the function polar to obtain the magnitude and argument of a comple number. The following eample illustrates its application: -->[r,theta] = polar(z) theta = r = The comple conjugate of a comple number is obtained by using the function conj: -->conj(z) i The negative is simply obtained by adding a minus sign to the number, i.e., -->-z i Matrices in SCILAB can have comple numbers as elements. The following commands, for eample, produce a 34 matri of comple numbers. First, we generate random matrices with integer numbers to be the real and imaginary parts of the comple numbers: -->Ms = int(0*rand(3,3)), yms = int(0*rand(3,3)) Ms =!. 5..!! !!.. 7.! yms =! !! !! ! Net, the comple-number matri is put together by using: -->zms = Ms + yms*%i zms =! i. + 4.i!! i i 6. +.i!!. +.i. + 5.i i! To obtain the magnitude of the elements of the matri just defined we simply apply the function abs to the matri name, i.e., -->abs(zms)! !! !! ! Download at InfoClearinghouse.com 4 00 Gilberto E. Urroz

7 To find the arguments of the elements we can use the function atanm applied to the term-byterm division of the imaginary part by the real part of the elements. The function atanm is a generalization of the function atan used when the argument is a matri. Here is the calculation of the arguments: -->atanm(yms./ms)! !! !! ! Comple number calculations Comple numbers can be added, subtracted, multiplied, and divided. The rules for these operations are shown below: Let z = + i y = r e i θ, z = + i y = r e i θ, and z = + i y = r e i θ, be comple numbers. In these definitions the numbers, y,,, y, and y are real numbers. Addition: z + z = ( + )+ i (y + y ) Subtraction: z - z = ( - )+ i (y - y ) Multiplication: z z = ( - y y ) + i ( y + y ) = r r e i (θ + θ ). Multiplication of a number by its conjugate results in the square of the number s magnitude, i.e.: z z = ( + i y) ( - i y) = + y = r = z Division: z z z z = z z = z z z = + y y + y i y + y + y r = r e i ( θ θ). Powers: z n = (r e i θ ) n = r n e i nθ Roots: because the argument θ of a comple number z has a periodicity of π, we can write z = r e i (θ+kπ), for k = 0,,, There are n n-th roots of z calculated as / / n z = z = r e, k = 0,,, L( n ). n n n ( θ+ kπ) i Download at InfoClearinghouse.com 5 00 Gilberto E. Urroz

8 More details on the calculation of roots of comple numbers are presented later in the chapter. Eamples of operations with comple numbers The following are operations with comple numbers using SCILAB: -->z = -5+*%i, z = 3+4*%i z = i z = i Addition, subtraction, multiplication, and division: -->z+z i -->z-z i -->z*z i -->z/z i The following sequence provides the first 5 integer powers of z: -->zpow = []; for j=:5, zpow=[zpow z^j]; zpow zpow = column to 4! i. - 0.i i i! column 5! i! The following SCILAB command attempts to find one cubic root of z: -->z^(/3) i Download at InfoClearinghouse.com 6 00 Gilberto E. Urroz

9 There are, in fact, three cubic roots for any real or comple number. These are calculated, using the formulas indicated earlier, as follows. First, we calculate the magnitude and argument of comple number z: -->r=abs(z), theta = atan(imag(z)/real(z)) r = theta = Net, we create an empty vector called cubic_roots_of_z: -->cubic_roots_of_z = []; The vector is then filled with the three roots of z as follows: -->for k = 0: --> cubic_roots_of_z = [cubic_roots_of_z r^(/3)*ep(%i*((theta+*k*%pi)/3))]; --> To see the final result, enter the name of the vector: -->cubic_roots_of_z cubic_roots_of_z = column to! i i! column 3! i! More details on the calculation of roots of real and comple numbers are presented in a subsequent section of this chapter. Solution to quadratic and cubic equations Quadratic equations, of the form a +b+c = 0, and cubic equations, of the form a 3 +b +c+d = 0, are the simplest non-linear, polynomial equations. SCILAB provides function roots to solve polynomial equations of any order. Therefore, function roots can be used to solve quadratic and cubic equations. The methods presented net for solving quadratic and cubic equations, therefore, are not strictly necessary, however, they are presented here as SCILAB programming eercises. Quadratic equations Quadratic equations are those algebraic equations with one unknown that can be reduced to the form a +b+c = 0, where a, b, and c are real numbers. Download at InfoClearinghouse.com 7 00 Gilberto E. Urroz

10 The solution to a quadratic equation can be accomplished by re-writing the equation as +(b/a)= -(c/a) and completing the square of a binomial in the left hand side. To complete the square of a binomial, add the term b /(4a ) to both sides of the equation: +(b/a)+b /(4a )= -(c/a)+ b /(4a ) The equation can now be re-written as (+b/(a)) = (b -4ac)/(4a ). Taking the square root of both sides of the equation results in Solving for results in +b/(a) = ± (b -4ac) / /(a). = [-b± (b -4ac) / ]/(a). The quantity under the square root in this result is known as the discriminant of the equation, i.e., D = b -4ac. According to the sign of the discriminant, the equation can have one or two real solutions, or two comple conjugate solutions: If D>0, the quadratic equation has two distinct real solutions:, = (-b± D / )/(a). If D=0, the quadratic equation has one real (double) solution, = = -b/(a). If D<0, the quadratic equation has two conjugate comple solutions,, = (-b± i (-D) / )/(a), where i = (-) / is the unit imaginary number. The basic algorithm (written in pseudo-code) for solving a quadratic equation, would look something like this: Enter equation coefficients a, b, c Download at InfoClearinghouse.com 8 00 Gilberto E. Urroz

11 Calculate discriminant, D = b -4ac If D > 0, Indicate that two distinct real solutions eist Calculate roots as = (-b+d / )/(a), and = (-b-d / )/(a) Display roots and Else If D = 0, Indicate that only one real solution eist Calculate root as = -b/(a) Display single root Else (default case D<0) Indicate that two conjugate comple solutions eist Calculate the real part of the comple roots as r = -b/(a) Calculate the imaginary part of the roots as i = (-D) / Display the roots as ' = +i* ' and ' = -i* ' r i r i The reader is invited to write a SCILAB function to solve the quadratic equation given the values a, b, and c. Note: Pseudo-code is a way to present an algorithm using English-like statements that are later coded into a particular computer language. The statements are general enough to be understood by any programmer and easily translated into a computer language. Cubic equations The canonical (simplest) form of the cubic equation is where a, b, c are real numbers. 3 +a +b+c = 0, Substituting the auiliary variable leads to the reduced form with and y = + a/3 y 3 +py+q = 0, p = (3b-a )/3, q = c + a 3 /7 a b/3. A discriminant for the reduced cubic equation is given by D = (p/) +(q/3) 3. An Italian mathematician, Gierolimo Cardano, proposed the following solutions to the cubic equation: = -(a/3)+u+v,3 = -(a/3)-(u+v)/ ± i 3 (u-v)/ Download at InfoClearinghouse.com 9 00 Gilberto E. Urroz

12 where q q u = 3 + D, v = 3. D Depending on the sign of the discriminant, D, the solutions to the cubic equation can be classified as follows: If D > 0, one real and two comple conjugate solutions (calculated using the formulas shown above). If D = 0, three real solutions including a double solution ( and 3 ). If D < 0, three distinct real solutions (this case is known as the irreducible case). In the irreducible case, the formulas shown above for the three roots of the cubic equation will introduce comple epressions because the calculation of u and v involves the square root of D<0. To avoid introducing such comple epressions, we can use the so-called trigonometric form of the solution: a = + 3 p 3 φ cos( ), 3 a = + 3 p φ π cos( ), 3 3 and 3 a = + 3 p φ + π cos( ), 3 3 where cos( φ) = q p 3 3. The algorithm for solving the cubic equation can be written in pseudo-code as follows: Enter coefficients for the cubic equation: a,b,c Calculate p and q Calculate the discriminant D If D > 0 Indicate that there one real and two comple conjugate solutions Calculate u and v (use subroutine for cubic equation see below) Calculate the real root, = -(a/3)+u+v Calculate the real part of roots,3, r = -(a/3)+(u+v)/ Download at InfoClearinghouse.com 0 00 Gilberto E. Urroz

13 Calculate the imaginary part of roots,3, i = 3 (u-v)/ Display roots, ' = r + i *i' and ' 3 = r - i *i' Else D = 0 Indicate that there are three real solutions with one double root Calculate = -(a/3)+u+v,,3 = -(a/3) (u+v)/ Display roots, = 3 Else (Default case, D<0) Indicate that there are three distinct real solutions Calculate cos(φ) and φ Calculate the three roots using the trigonometric form, i.e., φ = φ/3, = -a/3 + ( p /3) / cos φ φ = (φ π)/3, = -a/3 + ( p /3) / cos φ φ3 = (φ+π)/3, = -a/3 + ( p /3) / cos φ3 Display roots,, and 3 The calculation of u and v for the case D>0 requires us to calculate cubic roots, namely, q q 3 + D, 3. D I suggest you use the following user-defined SCILAB function to obtain those cubic roots: function RealCubicRoot() //This function calculates the cubic root of a real number One_Third =.0/3.0 if < 0 then RealCubicRoot = -(Abs()^One_Third) elseif == 0 then RealCubicRoot = 0 else Real CubicRoot = Abs()^One_Third end //End of the function The function RealCubicRoot calculates the real cubic root corresponding to a real number. The functions is designed so that if the real number is positive so is its cubic root, and if the number is negative so is its cubic root. With this function we ensure that we always get a real cubic root and that the calculation is not encumbered with comple number operations. The many roots of a real or comple number Consider a comple number z = r e iθ = r cos θ + i r sin θ. Because the functions sin and cos are periodic functions of period π in θ, the comple number z can be written as Download at InfoClearinghouse.com 00 Gilberto E. Urroz

14 z = r e i(θ+kπ), k = 0, ±, ±, ±3,... In other words, there are infinite ways to represent the comple number z. The most general representation being z = r e i(θ+kπ). To obtain the n-th root of this comple number we can write z /n = r /n e i[(θ/n)+kπ/n]. You can check that we need only use the values k = 0,,,... (n-), to produce all n independent roots of the number z. Eample : Consider the number z = 6 e iπ = 6 cos π + i 6 sin π = -6, which is actually a real number. Calculate the roots corresponding to z /4 : From z = 6 e iπ = 6 e i(π+kπ) = 6 e i(+k)π, it follows that r = z /4 = 6 /4 e i(+k)(π/4) = e i(+k)(π/4), k = 0,,, 3 Thus,!For k = 0, r = e i(+ 0)(π/4) = e iπ/4 = cos(π/4) + i sin(π/4) = ( + i)!for k =, r = e i(+ )(π/4) = e i3π/4 = cos(3π/4) + i sin(3π/4) = (- + i)!for k =, r 3 = e i(+ )(π/4) = e i5π/4 = cos(5π/4) + i sin(5π/4) = - ( + i)!for k = 3, r 3 = e i(+ 3)(π/4) = e i7π/4 = cos(7π/4) + i sin(7π/4) = (- + i) Using values such as k = - or k = 4 in the general epression for the 4-th root of z = -6 will produce values already accounted for in the four results found above. This eample, therefore, verifies that there are eactly n independent n-th roots of a comple number, with n = 4 for this case. Notice that the results of this eample actually correspond to the 4-th roots of a real number, z = -6. Since real numbers are special cases of comple numbers, the approach outlined above for finding the n-th root of a comple number is also applicable to a real number. The only requirement is that the number be written in the Polar form, z = r e iθ. Using SCILAB, the previous calculations are performed as follows: -->z = -6; r=abs(z); theta = -%pi; -->roots_of_z = []; -->for k = 0:3 --> roots_of_z = [roots_of_z r^(/4)*ep(%i*((theta+*k*%pi)/n))]; --> -->roots_of_z roots_of_z = Download at InfoClearinghouse.com 00 Gilberto E. Urroz

15 column to! i i! column 3 to 4! i i! Principal values of a cubic root The principal value for the n-th root of a comple (or real) number, z = r e iθ, is the value corresponding to k = 0 in the general epression: i.e., z /n = r /n e i[(θ/n)+kπ/n], z /n = r /n e iθ/n. For the case of cubic root (n = 3), if the number z is a positive real number, θ = 0, and the result is straightforward, z /3 = r /3. If the number z is a negative real number, θ = π, and the principal value is z /3 = r /3 e iπ/3 = r /3 (cos π/3 + i sin π/3). Thus, the principal value of the root of a comple number is not necessarily always a real number. The following SCILAB eercise shows the three cubic roots of a positive and a negative number. Eample - Cubic roots of a positive number: To calculate the cubic roots of z = 8, with r = 8 and θ = 0, we use: -->z = 8, r = abs(z), theta = 0 z = 8. r = 8. theta = 0. -->for k = 0: --> r^(/3)*ep(%i*(theta+*k*%pi)/3) -->end i Download at InfoClearinghouse.com 3 00 Gilberto E. Urroz

16 i Eample - Cubic roots of a negative number: To calculate the cubic roots of z = 8, with r = 9 and θ = π, we use: -->z = -8, r = abs(z), theta = %pi z = - 8. r = 8. theta = >for k = 0: --> r^(/3)*ep(%i*(theta+*k*%pi)/3) -->end i E-6i // Note: this is equivalent to i Polynomials and solutions to polynomial equations Polynomials are one of the categories of SCILAB data types. Polynomials are defined using the function poly. In Chapter 5 we indicated, for eample, that for a square matri A, the characteristic equation can be obtained by using poly(a, lam ), where lam is the independent variable defining the resulting polynomial. We also indicated that we can define a simple symbolic variable, say, s, by using s = poly(0, s ). In this section we are interested in defining a polynomial of order n, namely, P = a 0 +a +a +a a n n, we use a row vector containing the coefficients as [a 0 a a a n ], and call function poly using this vector, the independent variable that defines the polynomial, say,, and the qualifier coeff. For eample, to define a fourth-order polynomial with coefficients [ ], use: -->p4 = poly([ ],'','coeff') p4 = Download at InfoClearinghouse.com 4 00 Gilberto E. Urroz

17 A polynomial of order 0 in y is generated net. The coefficients of the polynomial are stored in vector c: -->c = [ ] c = column to 0! !! 3.! column The resulting polynomial is: -->p0 = poly(c,'y','coeff') p0 = y + 34y - 0y + 4y + 55y - y + y + 3y To find the roots of a polynomial use the function roots. For eample, the roots of the fourthorder polynomial p4 are: -->roots(p4)!.05960!! i!! i!! ! The following eample provides the roots of the polynomial p0: -->roots(p0)! i!! i!! !! i!! i!! !! !! !! i!! i! The net eample combines functions roots and poly to calculate the roots of a polynomial given its coefficients in the vector [ ]: -->roots(poly([ ],'s','coeff')) Download at InfoClearinghouse.com 5 00 Gilberto E. Urroz

18 ! i!! i!! i!! i!! ! The fundamental theorem of algebra indicates that a polynomial of integer order n has a total of n roots, among real and comple roots. It also indicates that if a comple number z is a root of a polynomial, then its comple conjugate z is also a root of the polynomial. These facts are illustrated in the eamples worked above. More information on polynomials is provided in Chapter 7. Solution of a single non-linear equation In an earlier section of this chapter we presented algorithms for the solution of quadratic and cubic equations. These two types of equations are eamples of polynomial equations of the form: a n n + a n- n- + a n- n- + + a + a + a 0 = 0, with n = corresponding to the quadratic equation, and n = 3 to the cubic equation. A linear equation would be one for which n =, and its solution is straightforward. Equations that can be cast in the form of a polynomial are referred to as algebraic equations. Equations involving more complicated terms, such as trigonometric, hyperbolic, eponential, or logarithmic functions are referred to as transcendental equations. No general algorithms for solution eist for algebraic (or polynomial) equations of order n 4, or for transcendental equations. The methods presented in this section are numerical methods that can be applied to the solution of such equations, to which we will refer, in general, as non-linear equations. In general, we will we searching for one, or more, solutions to the equation, f() = 0. We will present the methods of interval halving (or bisection method, the Newton-Raphson algorithm, the secant method, and the fied iteration method. In the interval-halving method as well as in the secant method we need to provide two initial values of to get the algorithm started. In the fied iteration and Newton-Raphson methods only one initial value is required. Because the solution is not eact, the algorithms for any of the methods presented herein will not provide the eact solution to the equation f() = 0, instead, we will stop the algorithm when the equation is satisfied within an allowed tolerance or error, ε. In mathematical terms this is epressed as f( R ) < ε. The value of for which the non-linear equation f()=0 is satisfied, i.e., = R, will be the solution, or root, to the equation within an error of ε units. Download at InfoClearinghouse.com 6 00 Gilberto E. Urroz

19 Interval-halving or bisection method This method starts by identifying two values of, a and b, with a<b, for which the corresponding function values, namely, f(a) and f(b), have opposite signs, i.e.,, f(a) f(b) < 0. Figure, below, presents two of such cases. Obviously, since the function y = f() changes sign as goes from = a to = b, somewhere within the interval (a,b), must take a value R for which f() = 0. The mid-point of the interval, namely, c = /(a+b), will be closer to R than either a or b can be. Two situations are possible, as illustrated in the figure below: () a < R < c < b; or, () a < c < R < b. In the first case, f(a) f(c) < 0, while f(b) f(c) > 0, and R is contained in the interval (a,c). In the second case, f(a) f(c) > 0, while f(b) f(c) < 0, and R is contained in the interval (c,b). We can think of c replacing b in the first case in a general interval (a,b), while c replaces a in the interval (a,b) in the second case. In other words, in the first case b takes the value of c, b"c, and in the second case, a"c. We can think of the method, therefore, as finding the center-point of an ever-decreasing interval. The mid-point value replaces the interval etreme for which the product of its function and that of the mid-point is a positive number. The process is then repeated until a value of c is found so that (3) is satisfied, i.e., until f(c) < ε. Download at InfoClearinghouse.com 7 00 Gilberto E. Urroz

20 Pseudo-code for the interval-halving or bisection method The following is one possible pseudo-code for the interval-halving method:. Function f() must be loaded through deff or getf.. Enter initial values of a and b. 3. Check that a < b, if not, send message indicating error in input data and requesting user to re-enter values of a and b. 4. When input data is correct, check that f(a) f(b) < 0. If that is not the case, inform the user that his/her initial values of a and b do not satisfy the problem conditions for solution. 5. If problem conditions are satisfied, proceed to calculate c according to c = /(a+b). 6. Check if convergence condition, f(c) < ε, is satisfied. If it is so, print the solution and stop. 7. If convergence solution is not satisfied, replace values of a or b according to the following procedure: If f(b) f(c) > 0, b"c, else a" c. 8. Repeat procedure from step 5 on. Stop if the number of iterations is too large. Send a message to user indicating that the process is not converging after a large number of iterations. SCILAB function for interval-halving or bisection The following function, half.tt, can be used to obtain a solution using interval halving. The function looks like this: function []=half(a,b,f) //interval halving routine N = 00; eps =.e-5; // define ma. no. iterations and error if (f(a)*f(b) > 0) then error('no root possible f(a)*f(b) > 0') abort; if(abs(f(a)) < eps) then error('solution at a') abort; if(abs(f(b)) < eps) then error('solution at b') abort; while (N > 0) Download at InfoClearinghouse.com 8 00 Gilberto E. Urroz

21 c = (a+b)/; if(abs(f(c)) < eps) then = c; return; if(f(a)*f(c) < 0)then b = c; else a = c; N = N - ; error('no convergence') abort; //end function Eample of interval-halving (bisection) method application Let's apply it to an eample, first define the function p() as follows: -->deff('[y]=p()',['y=^3-*^-*-']) We will produce a plot of the function to check possible solutions visually, for eample: --> = 0:0.:5; yy = p(); //(,y) data -->size() //size of to produce the -ais!. 5.! -->zeroline = zeros(,5); -->plotd([' '],[yy' zeroline']) //data for the -ais (all zeros) //plot function and -ais The figure shows a root between and 3. half, and call it with the proper arguments: The net step in the solution is to load function -->getf('half ); --> = half(,3,p) = To check the solution evaluate the function at the value of : -->p() Download at InfoClearinghouse.com 9 00 Gilberto E. Urroz

22 The function does not evaluates to zero. However, the error involved is in the order of 50-6, small enough to accept the solution provided. The Newton-Raphson method Consider the Taylor-series epansion of the function f() about a value = o : f()= f( o )+f'( o )(- o )+(f"( o )/!)(- o )+. Using only the first two terms of the epansion, a first approimation to the root of the equation f() = 0 can be obtained from Such approimation is given by, f() = 0 f( o )+f'( o )( - o ) = o - f( o )/f'( o ). The Newton-Raphson method consists in obtaining improved values of the approimate root through the recurrent application of equation (8). For eample, the second and third approimations to that root will be given by and = - f( )/f'( ), 3 = - f( )/f'( ), respectively. This iterative procedure can be generalized by writing the following equation, where i represents the iteration number: i+ = i - f( i )/f'( i ). After each iteration the program should check to see if the convergence condition, namely, is satisfied. f( i+ ) <ε, The figure below illustrates the way in which the solution is found by using the Newton- Raphson method. Notice that the equation f() = 0 f( o )+f'( o )( - o ) represents a straight line tangent to the curve y = f() at = o. This line intersects the -ais (i.e., y = f() = 0) at the point as given by = o - f( o )/f'( o ). At that point we can construct another straight line tangent to y = f() whose intersection with the -ais is the new approimation to the root of f() = 0, namely, =. Proceeding with the iteration we can see that the intersection of consecutive tangent lines with the -ais approaches the actual root relatively fast. Download at InfoClearinghouse.com 0 00 Gilberto E. Urroz

23 The Newton-Raphson method converges relatively fast for most functions regardless of the initial value chosen. The main disadvantage is that you need to know not only the function f(), but also its derivative, f'(), in order to achieve a solution. The secant method, discussed in the following section, utilizes an approimation to the derivative, thus obviating such requirement. The programming algorithm of any of these methods must include the option of stopping the program if the number of iterations grows too large. How large is large? That will depend of the particular problem solved. However, any interval-halving, fied iteration, Newton- Raphson, or secant method solution that takes more than 000 iterations to converge is either ill-posed or contains a logical error. Debugging of the program will be called for at this point by changing the initial values provided to the program, or by checking the program's logic. A SCILAB function for the Newton-Raphson method The function newton, listed below, implements the Newton-Raphson algorithm. It uses as arguments an initial value and epressions for f() and f'(). function []=newton(0,f,fp) //newton-raphson algorithm N = 00; eps =.e-5; // define ma. no. iterations and error maval = ; // define value for divergence = 0; while (N>0) n = -f()/fp(); if(abs(f(n))<eps)then =n disp(00-n); return(); if (abs(f())>maval)then disp(00-n); error('solution diverges'); abort; N = N - ; = n; Download at InfoClearinghouse.com 00 Gilberto E. Urroz

24 error('no convergence'); abort; //end function We will use the Newton-Raphson method to solve for the equation, f() = = 0. The following SCILAB commands define the function f() and its derivative, fp(), and load the function newton.tt: -->deff('[y]=f()','y=^3-*^+'); -->deff('[y]=fp()','y=3*^-'); -->getf('newton') To have an idea of the location of the roots of this polynomial we'll plot the function using the following SCILAB commands: -->= (-0.8:0.0:.0)'; y = f(); -->plot(,y,'','f()','my plot'); -->grid() We see that the function graph crosses the -ais somewhere between -.8 and -.4, between.8 and., and between.6 and.0. The following commands use the function newton with different initial values. The nature of the function is such that most initial values converge to either of the two real roots, a few diverge, but it is very difficult to make the algorithm converge to the negative root. The number listed before the variable name ans in each solution is the number of iterations required to obtain a solution. -->newton(-,f,fp) >newton(-0,f,fp) Download at InfoClearinghouse.com 00 Gilberto E. Urroz

25 -->newton(-5,f,fp) >newton(-0.,f,fp) >newton(-,f,fp) >newton(0,f,fp).. -->newton(0.5,f,fp) >newton(-0.8,f,fp) >newton(-0.,f,fp) >newton(-0.,f,fp) >newton(-00,f,fp) 0.!--error 9999 Solution diverges at line 5 of function newton called by : newton(-00,f,fp) Download at InfoClearinghouse.com 3 00 Gilberto E. Urroz

26 -->newton(-0,f,fp) >newton(00,f,fp) 0.!--error 9999 Solution diverges at line 5 of function newton called by : newton(00,f,fp) -->newton(0,f,fp) The Secant Method In the secant method, we replace the derivative f'( i ) in the Newton-Raphson method with f'( i ) (f( i ) - f( i- ))/( i - i- ). With this replacement, the Newton-Raphson algorithm becomes f ( i ) i+ i ( i f ( ) f ( ) = i i i To get the method started we need two values of, say o and, to get the first approimation to the solution, namely, f ( ) = ( f ( ) f ( ) As with the Newton-Raphson method, the iteration is stopped when f( i+ ) <ε. Figure 4, below, illustrates the way that the secant method approimates the solution of the equation f() = 0. o 0 ). ). Download at InfoClearinghouse.com 4 00 Gilberto E. Urroz

27 A SCILAB function for the secant method The function secant, listed below, uses the secant method to solve for non-linear equations. It requires two initial values and an epression for the function, f(). function []=secant(0,00,f) //newton-raphson algorithm N = 00; eps =.e-5; // define ma. no. iterations and error maval = ; // define value for divergence = 0; = 00; while (N>0) gp = (f()-f())/(-); n = -f()/gp; if(abs(f(n))<eps)then =n disp(00-n); return(); if (abs(f(n))>maval)then disp(00-n); error('solution diverges'); abort; N = N - ; = ; = n; disp(00-n); error('no convergence'); abort; //end function Application of secant method The following SCILAB commands define the function f(), load the function secant.tt: -->deff('[y]=f()','y=^3-*^+') Download at InfoClearinghouse.com 5 00 Gilberto E. Urroz

28 -->getf('secant.tt') The following commands call the function secant.tt and converge to a solution: -->secant(-0.,-9.8,f) >secant(.0,.,f) >secant(5.0,5.,f) Fied -point iteration Fied point iteration consists in re-writing the equation f()=0 in the form = g(). This equation will now represent a recursive relation by writing it as n+ =g( n ). To get the process started we use an initial value, = 0. Then, we calculate = g( 0 ), = g( ), etc. Convergence is achieved whenever f() < ε, or whenever n+ - n < ε. A SCILAB function for fied iteration The following function will perform the iterations, it will stop if there is divergence or if convergence is achieved. The function is stored in file fiedp: function []=fiedp(0,f) //fied-point iteration N = 00; eps =.e-5; // define ma. no. iterations and error maval = ; // define value for divergence = 0; while (N>0) n = f(); if(abs(n-)<eps)then =n disp(00-n); return(); Download at InfoClearinghouse.com 6 00 Gilberto E. Urroz

29 if (abs(f())>maval)then disp(00-n); error('solution diverges'); abort; N = N - ; = n; error('no convergence'); abort; //end function Applications of fied-point iteration We use fied-point iteration to solve the equation with the following recurrence equations: f() = 3 - ep() + = 0, (a) = g() = ep() - (+); (b) = g() = (ep()-)/3; and, (c) = g3() = ln(3+). The following SCILAB commands define the three functions required: -->deff('[y]=g()','y=ep()-(*+)'); -->deff('[y]=g()','y=(ep()-)/3'); -->deff('[y]=g3()','y=log(3*+)'); Net, we get the function fiedp: -->getf('fiedp') The following calls using function g diverge: -->fiedp(.0,g)!--error 9999 No convergence at line of function fiedp called by : fiedp(.0,g) -->fiedp(-.0,g)!--error 9999 No convergence at line of function fiedp called by : fiedp(-.0,g) -->fiedp(0.0,g)!--error 9999 No convergence at line of function fiedp called by : fiedp(0.0,g) The net eamples converge to one root: Download at InfoClearinghouse.com 7 00 Gilberto E. Urroz

30 -->fiedp(.0,g) >fiedp(-.0,g) >fiedp(0.0,g) These three eamples converge to a second root. -->fiedp(.0,g3) >fiedp(-.0,g3) i -->fiedp(0.0,g3) The number shown before the variable ans in these results indicates the number of iterations. The fied-point iteration method does not always converge. There is no convergence pattern for this method. Whether the method converges or not depends on the form of the function g() as well as on the initial value chosen. Solving systems of non-linear equations Consider the solution to a system of n non-linear equations in n unknowns given by f (,,, n ) = 0 f (,,, n ) = 0.. Download at InfoClearinghouse.com 8 00 Gilberto E. Urroz

31 Download at InfoClearinghouse.com 9 00 Gilberto E. Urroz. f n (,,, n ) = 0 The system can be written in a single epression using vectors, i.e., f() = 0, where the vector contains the independent variables, and the vector f contains the functions f i ():. ) ( ) ( ) ( ),...,, ( ),...,, ( ),...,, ( ) (, = = = f n n n n n n f f f f f f M M M Newton-Raphson method to solve systems of non-linear equations A Newton-Raphson method for solving the system of linear equations requires the evaluation of a determinant, known as the Jacobian of the system, which is defined as:. ] [ / / / / / / / / / ),...,, ( ),...,, ( n n j i n n n n n n n n f f f f f f f f f f f f f = = = L M O M M L L J If = 0 (a vector) represents the first guess for the solution, successive approimations to the solution are obtained from n+ = n - J - f( n ) = n - n, with n = n+ - n. Convergence criteria for the solution of a system of non-linear equation could be, for eample, that the maimum of the absolute values of the functions f i ( n ) is smaller than a certain tolerance ε, i.e.,. ) ( ma ε < n i i f Another possibility for convergence is that the magnitude of the vector f( n ) be smaller than the tolerance, i.e., f( n ) < ε. We can also use as convergence criteria the difference between consecutive values of the solution, i.e.,., ) ( ) ( ma ε < + n i n i i or, n = n+ - n < ε.

32 The main complication with using Newton-Raphson to solve a system of non-linear equations is having to define all the functions f i / j, for i,j =,,, n, included in the Jacobian. As the number of equations and unknowns, n, increases, so does the number of elements in the Jacobian, n. SCILAB function for Newton-Raphson method for a system of nonlinear equations The following SCILAB function, newtonm, calculates the solution to a system of n non-linear equations, f() = 0, given the vector of functions f and the Jacobian J, as well as an initial guess for the solution 0. function [] = newtonm(0,f,j) //Newton-Raphson method applied to a //system of linear equations f() = 0, //given the jacobian function J, with //J = del(f,f,...,fn)/del(,,...,n) // = [;;...;n], f = [f;f;...;fn] //0 is an initial guess of the solution N = 00; //define ma. number of iterations epsilon = e-0; //define tolerance maval = ; //define value for divergence = 0; //load initial guess while (N>0) JJ = J(); if(abs(det(jj))<epsilon) then error('newtonm - Jacobian is singular - try new 0'); abort; n = - inv(jj)*f(); if(abs(f(n))<epsilon)then =n; disp(000-n); return(); if (abs(f())>maval)then disp(00-n); error('solution diverges'); abort; N = N - ; = n; error('no convergence'); abort; //end function The functions f and the Jacobian J need to be defined as separate functions. To illustrate the definition of the functions consider the system of non-linear equations: f (, ) = = 0, f (, ) = - 5 = 0, Download at InfoClearinghouse.com Gilberto E. Urroz

33 whose Jacobian is f J = f f f =. We can define the function f as the following user-defined SCILAB function f: function [f] = f() //f() = 0, with = [();()] //represents a system of non-linear equations f() = ()^ + ()^ - 50; f() = ()*() -5; //end function The corresponding Jacobian is calculated using the user-defined SCILAB function jacob: function [J] = jacob() //Evaluates the Jacobian of a //system of non-linear equations J(,) = *(); J(,) = *(); J(,) = (); J(,) = (); //end function Illustrating the Newton-Raphson algorithm for a system of two nonlinear equations Before using function newtonm, we will perform some step-by-step calculations to illustrate the algorithm. We start by loading functions f and jacob: -->getf('f') -->getf('jacob') Net, we define an initial guess for the solution as: -->0 = [;] 0 =!.!!.! Let s calculate the function f() at = 0 to see how far we are from a solution: -->f(0) Download at InfoClearinghouse.com 3 00 Gilberto E. Urroz

34 ! - 45.!! - 3.! Obviously, the function f( 0 ) is far away from being zero. Thus, we proceed to calculate a better approimation by calculating the Jacobian J( 0 ): -->J0 = jacob(0) J0 =! 4..!!..! The new approimation to the solution,, is calculated as: --> = 0 - inv(j0)*f(0) =! !! ! Evaluating the functions at produces: -->f()! !! ! Still far away from convergence. Let s calculate a new approimation, : --> = - inv(jacob())*f() =! !! ! Evaluating the functions at indicates that the values of the functions are decreasing: -->f()! 0.078!! ! A new approimation and the corresponding function evaluations are: -->3 = - inv(jacob())*f() 3 =! !! ! -->f(3)!.4443!!.7893! The functions are getting even smaller suggesting convergence towards a solution. Solution using function newtonm Net, we use function newtonm to solve the problem postulated earlier. We start by loading function newtonm: Download at InfoClearinghouse.com 3 00 Gilberto E. Urroz

35 -->getf('newtonm') A call to the function using the values of 0, f, and jacob, already loaded is: -->[] = newtonm(0,f,jacob) 6. =! !! ! The result shows the number of iterations required for convergence (6) and the solution found as = and = Evaluating the functions for those solutions results in: -->f().0e-0 *!.90383!! ! The values of the functions are close enough to zero (error in the order of 0-0 ). Note: The functions f and jacob can be loaded as line functions by using deff: -->deff('[f]=f()',['f_=()^+()^-50';'f_=()*()-5';'f=[f_;f_]']) -- >deff('[j]=jacob()',['j=*()';'j=*()';'j=()';'j=()';'j=[j,j;j,j]']) Secant method to solve systems of non-linear equations We use the term secant to refer to a method for solving systems of non-linear equations through the Newton-Raphson algorithm, namely, n+ = n - J - f( n ), but approimating the Jacobian through finite differences. This approach is a generalization of the secant method for a single non-linear equation. For that reason, we refer to the method applied to a system of non-linear equations as a secant method, although the geometric origin of the term not longer applies. The secant method for a system of non-linear equations free us from having to define the n functions necessary to define the Jacobian for a system of n equations. Instead, we approimate the partial derivatives in the Jacobian with f i j f i (,, L, j +, L, n ) fi (,, L, j, L, n ), where is a small increment in the independent variables. Notice that f i / j represents element J ij in the jacobian J = (f,f,,f n )/ (,,, n ). Download at InfoClearinghouse.com Gilberto E. Urroz

36 To calculate the Jacobian we proceed by columns, i.e., column j of the Jacobian will be calculated as shown in the function jacobfd (jacobian calculated through Finite Differences) listed below: function [J] = jacobfd(f,,del) //Calculates the Jacobian of the //system of non-linear equations: //f() = 0, through finite differences. //The Jacobian is built by columns [m n] = size(); for j = :n = ; (j) = (j) + del; J(:,j) = (f()-f())/del; //end function Notice that for each column (i.e., each value of j) we define a variable which is first made equal to, and then the j-th element is incremented by del, before calculating the j-th column of the Jacobian, namely, J(:,j). This is the SCILAB implementation of the finite difference approimation for the Jacobian elements J ij = f i / j as defined earlier. Illustrating the secant algorithm for a system of two non-linear equations To illustrate the application of the secant algorithm we use again the system of two nonlinear equations defined earlier through the function f. We start by loading functions f and jacobfd: -->getf('f') -->getf('jacobfd') We choose an initial guess for the solution as 0 = [;3], and an increment in the independent variables of = 0.: -->0 = [;3] 0 =!.!! 3.! -->d = 0. d =. Variable J0 will store the Jacobian corresponding to 0 calculated through finite differences with the value of defined above: -->J0 = jacobfd(f,0,d) J0 =! 4. 6.!! 3..! Download at InfoClearinghouse.com Gilberto E. Urroz

37 A new estimate for the solution, namely,, is calculated using the Newton-Raphson algorithm: --> = 0 - inv(j0)*f(0) =! !! ! The finite-difference Jacobian corresponding to gets stored in J: -->J = jacobfd(f,,d) J =! !! ! And a new approimation for the solution ( ) is calculated as: --> = - inv(j)*f() =! !! ! The net two approimations to the solution ( 3 and 4 ) are calculated without first storing the corresponding finite-difference Jacobians: -->3 = - inv(jacobfd(f,,d))*f() 3 =! !! ! -->4 = 3 - inv(jacobfd(f,3,d))*f(3) 4 =! !! ! To check the value of the functions at = 4 we use: -->f(4)!.56733!! ! The functions are close to zero, but not yet at an acceptable error (i.e., something in the order of 0-6 ). Therefore, we try one more approimation to the solution, i.e., 5 : -->5 = 4 - inv(jacobfd(f,4,d))*f(4) 5 =! !! ! The functions are even closer to zero than before, suggesting a convergence to a solution. -->f(5)! !! ! Download at InfoClearinghouse.com Gilberto E. Urroz

38 SCILAB function for secant method to solve systems of non-linear equations To make the process of achieving a solution automatic, we propose the following SCILAB user - defined function, secantm: function [] = secantm(0,,f) ///Secant-type method applied to a //system of linear equations f() = 0, //given the jacobian function J, with //JJ approimated by (f((n)-f((n-)))/((n)-(n-)) // = [;;...;n], f = [f;f;...;fn] //0, are the initial guesses of the solution N = 00; //define ma. number of iterations epsilon = e-0; //define tolerance maval = ; //define value for divergence if abs(0-)<epsilon then error('=0 - use different values'); abort; n = 0; //load initial guesses nm = ; [n m] = size(); while (N>0) fn = f(n); fnm = f(nm); for i = :n for j = :n JJ(i,j) = (fn(i)-fnm(i))/(n(j)-nm(j)) if abs(det(jj))<epsilon then error('newtonm - Jacobian is singular - try new 0,'); abort; np = n - inv(jj)*f(n); if abs(f(np))<epsilon then =np; disp(000-n); return(); if abs(f(np))>maval then disp(00-n); error('solution diverges'); abort; N = N - ; nm = n; n = np; error('no convergence'); abort; Download at InfoClearinghouse.com Gilberto E. Urroz

39 //end function Solution using function secantm To solve the system represented by function f, we start by loading function secantm: -->getf('secantm') The following call to function secantm produces a solution after 8 iterations: --> = secantm(0,d,f) 8. =! !! ! Solving non-linear equations with the fsolve function SCILAB provides function fsolve for the solution of non-linear equations. In calling the function you can either include a derivative (for a Newton-Raphson method) or only the function f() (for a secant-type method). The fsolve function can be called by using: [ [,v [,info]]]=fsolve(0,fct [,fjac] [,tol]) where 0 is a real vector representing an initial guess for the solution; fct is an eternal (i.e function or list or string) representing the equation fct() = 0; fjac is an eternal (i.e function or list or string) representing a Jacobian or derivative; and, tol is a real scalar representing the tolerance for convergence. The default value of tol, if not provided in the function call, is tol= Termination of a fsolve function call occurs when the algorithm estimates that the relative error between and the solution is at most tol. In the left-hand side of the function call, is a real vector representing the final value of the solution; v is a real vector representing the value of the function at ; and info is an integer representing a termination indicator. Info can take any of the following values corresponding to different termination conditions: 0 : improper input parameters. : relative error between and the solution is at most tol. : number of calls to fcn reached 3 : tol is too small. No further improvement in the approimate solution is possible. 4 : iteration is not making good progress. As indicated above, this function can be used to solve for a system of linear equations, including a Jacobian matri (fjac). For a single non-linear equation, fjac is the derivative. Solving single non-linear equations with fsolve Download at InfoClearinghouse.com Gilberto E. Urroz

40 The following eamples show how to use the fsolve function for solving equations of the form f() = 0. First, we define f() and its derivative, fp(): -->deff('[y]=f()','y=^3-*^+') -->deff('[y]=fp()','y=*^-4*') Then, we perform different calls to the function fsolve: -->fsolve(.0,f) //No left-hand side, no Jacobian >fsolve(.0,f,fp) //No left-hand side, Jacobian provided >fsolve(.0,f,fp,0.00) //No left-hand side, Jacobian and tol provided > = fsolve(.0,f,fp) =. //solution name,, in left-hand side -->[,f_] = fsolve(-0,f,fp) //solution and function value in LHS f_ =.0E-6 = >[,f_,msg] = fsolve(-.0,f) //solution, function, message in LHS msg =. f_ = -.0E-6 = Solving a system of non-linear equations with fsolve We use function fsolve to solve the system of two non-linear equations defined by function f. To begin with, we load the function f, and the corresponding Jacobian jacob: -->getf('f') -->getf('jacob') Net, we define an initial value for the solution, 0, and call function fsolve using f, jacob: -->0 = [3;-] 0 = Download at InfoClearinghouse.com Gilberto E. Urroz

41 ! 3.!! -.! -->fsolve(0,f,jacob)! 5.!! 5.! A second call to the function includes a left-hand side specifying the solution,, the value of the function at the solution, f_, and information about the solution, info: -->[,f_,info] = fsolve(0,f,jacob) info =. f_ =! 0.!! 0.! =! 5.!! 5.! You can also call function fsolve without including the Jacobian: -->[,f_,info] = fsolve(0,f) info =. f_ =! 0.!! E-5! =! !! ! A second eample involves the solution of the system of 3 non-linear equations without using the Jacobian: f (,, 3 ) = f (,, 3 ) = f 3 (,, 3 ) = ( 3 ) / -.44 First, we define the system of equations as function f3: -->deff('[y]=f3()',['f_=()^-()^+(3)^-4','f_=()*()-(3)+',... -->'f_3=sqrt(()*()*(3))-.44','y=[f_;f_;f_3]']) We use the following values as first guesses for the solution: -->0 = [3,3,3] 0 = Download at InfoClearinghouse.com Gilberto E. Urroz