This page intentionally left blank

Transcription

1

2

3 This page intentionally left blank

4

5 Copyright 009, New Age International () td, ublishers ublished by New Age International () td, ublishers All rights reserved No part of this ebook may be reproduced in any form, by photostat, microfilm, xerography, or any other means, or incorporated into any information retrieval system, electronic or mechanical, without the written permission of the publisher All inquiries should be ed to ISBN () : UBISHING FR NE WRD NEW AGE INTERNATINA () IMITED, UBISHERS 485/4, Ansari Road, Daryaganj, New Delhi Visit us at wwwnewagepublisherscom

6 reface This book is based on the experience and the lecture notes of the authors while teaching Numerical Analysis for almost four decades at the Indian Institute of Technology, New Delhi This comprehensive textbook covers material for one semester course on Numerical Methods of Anna University The emphasis in the book is on the presentation of fundamentals and theoretical concepts in an intelligible and easy to understand manner The book is written as a textbook rather than as a problem/guide book The textbook offers a logical presentation of both the theory and techniques for problem solving to motivate the students for the study and application of Numerical Methods Examples and roblems in Exercises are used to explain each theoretical concept and application of these concepts in problem solving Answers for every problem and hints for difficult problems are provided to encourage the students for selflearning The authors are highly grateful to rof MK Jain, who was their teacher, colleague and co-author of their earlier books on Numerical Analysis With his approval, we have freely used the material from our book, Numerical Methods for Scientific and Engineering Computation, published by the same publishers This book is the outcome of the request of Mr Saumya Gupta, Managing Director, New Age International ublishers, for writing a good book on Numerical Methods for Anna University The authors are thankful to him for following it up until the book is complete The first author is thankful to Dr Gokaraju Gangaraju, resident of the college, rof S Raju, Director and rof Jandhyala N Murthy, rincipal, Gokaraju Rangaraju Institute of Engineering and Technology, Hyderabad for their encouragement during the preparation of the manuscript The second author is thankful to the entire management of Manav Rachna Educational Institutions, Faridabad and the Director-rincipal of Manav Rachna College of Engineering, Faridabad for providing a congenial environment during the writing of this book SRK Iyengar RK Jain

7 This page intentionally left blank

8 Contents reface (v) SUTIN F EUATINS AND EIGEN VAUE RBEMS 6 Solution of Algebraic and Transcendental Equations, Introduction, Initial Approximation for an Iterative rocedure, 4 Method of False osition, 6 4 Newton-Raphson Method, 5 General Iteration Method, 5 6 Convergence of Iteration Methods, 9 inear System of Algebraic Equations, 5 Introduction, 5 Direct Methods, 6 Gauss Elimination Method, 8 Gauss-Jordan Method, Inverse of a Matrix by Gauss-Jordan Method, 5 Iterative Methods, 4 Gauss-Jacobi Iteration Method, 4 Gauss-Seidel Iteration Method, 46 Eigen Value roblems, 5 Introduction, 5 ower Method, 5 4 Answers and Hints, 59 INTERATIN AND ARXIMATIN 6 08 Introduction, 6 Interpolation with Unevenly Spaced oints, 64 agrange Interpolation, 64 Newton s Divided Difference Interpolation, 7 Interpolation with Evenly Spaced oints, 80 Newton s Forward Difference Interpolation Formula, 89 Newton s Backward Difference Interpolation Formula, 9 4 Spline Interpolation and Cubic Splines, 99 5 Answers and Hints, 08

9 NUMERICA DIFFERENTIATIN AND INTEGRATIN Introduction, 09 Numerical Differentiation, 09 Methods Based on Finite Differences, 09 Derivatives Using Newton s Forward Difference Formula, 09 Derivatives Using Newton s Backward Difference Formula, 7 Derivatives Using Newton s Divided Difference Formula, Numerical Integration, 8 Introduction, 8 Integration Rules Based on Uniform Mesh Spacing, 9 Trapezium Rule, 9 Simpson s / Rule, 6 Simpson s /8 Rule, 44 4 Romberg Method, 47 Integration Rules Based on Non-uniform Mesh Spacing, 59 Gauss-egendre Integration Rules, 60 4 Evaluation of Double Integrals, 69 4 Evaluation of Double Integrals Using Trapezium Rule, 69 4 Evaluation of Double Integrals by Simpson s Rule, 7 4 Answers and Hints, 77 4 INITIA VAUE RBEMS FR RDINARY DIFFERENTIA EUATINS Introduction, 80 4 Single Step and Multi Step Methods, 8 4 Taylor Series Method, 84 4 Modified Euler and Heun s Methods, 9 44 Runge-Kutta Methods, System of First rder Initial Value roblems, Taylor Series Method, Runge-Kutta Fourth rder Method, Multi Step Methods and redictor-corrector Methods, 6 46 redictor Methods (Adams-Bashforth Methods), 7 46 Corrector Methods, 46 Adams-Moulton Methods, 46 Milne-Simpson Methods, 4 46 redictor-corrector Methods, 5 47 Stability of Numerical Methods, 7 48 Answers and Hints, 8

10 5 BUNDARY VAUE RBEMS IN RDINARY DIFFERENTIA EUATINS AND INITIA & BUNDARY VAUE RBEMS IN ARTIA DIFFERENTIA EUATINS Introduction, 4 5 Boundary Value roblems Governed by Second rder rdinary Differential Equations, 4 5 Classification of inear Second rder artial Differential Equations, Finite Difference Methods for aplace and oisson Equations, 5 55 Finite Difference Method for Heat Conduction Equation, Finite Difference Method for Wave Equation, 9 57 Answers and Hints, 08 Bibliography Index 5

11 This page intentionally left blank

12 Solution of Equations and Eigen Value roblems SUTIN F AGEBRAIC AND TRANSCENDENTA EUATINS Introduction A problem of great importance in science and engineering is that of determining the roots/ zeros of an equation of the form A polynomial equation of the form f(x) = 0, () f(x) = n (x) = a 0 x n + a x n + a x n + + a n x + a n = 0 () is called an algebraic equation An equation which contains polynomials, exponential functions, logarithmic functions, trigonometric functions etc is called a transcendental equation For example, x x x 5 = 0, x 4 x + = 0, x x + = 0, are algebraic (polynomial) equations, and xe x = 0, cos x xe x = 0, tan x = x are transcendental equations We assume that the function f(x) is continuous in the required interval We define the following Root/zero A number α, for which f(α) 0 is called a root of the equation f(x) = 0, or a zero of f(x) Geometrically, a root of an equation f(x) = 0 is the value of x at which the graph of the equation y = f(x) intersects the x-axis (see Fig ) f(x) x Fig Root of f(x) = 0

13 NUMERICA METHDS Simple root A number α is a simple root of f(x) = 0, if f(α) = 0 and f (α) 0 Then, we can write f(x) as f(x) = (x α) g(x), g(α) 0 () For example, since (x ) is a factor of f(x) = x + x = 0, we can write f(x) = (x )(x + x + ) = (x ) g(x), g() 0 Alternately, we find f() = 0, f (x) = x +, f () = 4 0 Hence, x = is a simple root of f(x) = x + x = 0 Multiple root A number α is a multiple root, of multiplicity m, of f(x) = 0, if Then, we can write f(x) as f(α) = 0, f (α) = 0,, f (m ) (α) = 0, and f (m) (α) 0 (4) f(x) = (x α) m g(x), g(α) 0 For example, consider the equation f(x) = x x + 4 = 0 We find f() = = 0, f (x) = x 6x, f () = = 0, f (x) = 6x 6, f () = 6 0 Hence, x = is a multiple root of multiplicity (double root) of f(x) = x x + 4 = 0 We can write f(x) = (x ) (x + ) = (x ) g(x), g() = 0 In this chapter, we shall be considering the case of simple roots only Remark A polynomial equation of degree n has exactly n roots, real or complex, simple or multiple, where as a transcendental equation may have one root, infinite number of roots or no root We shall derive methods for finding only the real roots The methods for finding the roots are classified as (i) direct methods, and (ii) iterative methods Direct methods These methods give the exact values of all the roots in a finite number of steps (disregarding the round-off errors) Therefore, for any direct method, we can give the total number of operations (additions, subtractions, divisions and multiplications) This number is called the operational count of the method For example, the roots of the quadratic equation ax + bx + c = 0, a 0, can be obtained using the method x = b ± b 4ac a For this method, we can give the count of the total number of operations There are direct methods for finding all the roots of cubic and fourth degree polynomials However, these methods are difficult to use Direct methods for finding the roots of polynomial equations of degree greater than 4 or transcendental equations are not available in literature

14 SUTIN F EUATINS AND EIGEN VAUE RBEMS Iterative methods These methods are based on the idea of successive approximations We start with one or two initial approximations to the root and obtain a sequence of approximations x 0, x,, x k,, which in the limit as k, converge to the exact root α An iterative method for finding a root of the equation f(x) = 0 can be obtained as x k + = φ(x k ), k = 0,,, (5) This method uses one initial approximation to the root x 0 The sequence of approximations is given by x = φ(x 0 ), x = φ(x ), x = φ (x ), The function φ is called an iteration function and x 0 is called an initial approximation If a method uses two initial approximations x 0, x, to the root, then we can write the method as x k + = φ(x k, x k ), k =,, (6) Convergence of iterative methods The sequence of iterates, {x k }, is said to converge to the exact root α, if lim x k = α, or lim x k α = 0 (7) k k The error of approximation at the kth iterate is defined as ε k = x k α Then, we can write (7) as lim error of approximation = lim x k k α = lim ε k = 0 k k Remark Given one or two initial approximations to the root, we require a suitable iteration function φ for a given function f(x), such that the sequence of iterates, {x k }, converge to the exact root α Further, we also require a suitable criterion to terminate the iteration Criterion to terminate iteration procedure Since, we cannot perform infinite number of iterations, we need a criterion to stop the iterations We use one or both of the following criterion: (i) The equation f(x) = 0 is satisfied to a given accuracy or f(x k ) is bounded by an error tolerance ε f (x k ) ε (8) (ii) The magnitude of the difference between two successive iterates is smaller than a given accuracy or an error bound ε x k + x k ε (9) Generally, we use the second criterion In some very special problems, we require to use both the criteria For example, if we require two decimal place accuracy, then we iterate until x k+ x k < 0005 If we require three decimal place accuracy, then we iterate until x k+ x k < As we have seen earlier, we require a suitable iteration function and suitable initial approximation(s) to start the iteration procedure In the next section, we give a method to find initial approximation(s)

15 4 NUMERICA METHDS Initial Approximation for an Iterative rocedure For polynomial equations, Descartes rule of signs gives the bound for the number of positive and negative real roots (i) We count the number of changes of signs in the coefficients of n (x) for the equation f(x) = n (x) = 0 The number of positive roots cannot exceed the number of changes of signs For example, if there are four changes in signs, then the equation may have four positive roots or two positive roots or no positive root If there are three changes in signs, then the equation may have three positive roots or definitely one positive root (For polynomial equations with real coefficients, complex roots occur in conjugate pairs) (ii) We write the equation f( x) = n ( x) = 0, and count the number of changes of signs in the coefficients of n ( x) The number of negative roots cannot exceed the number of changes of signs Again, if there are four changes in signs, then the equation may have four negative roots or two negative roots or no negative root If there are three changes in signs, then the equation may have three negative roots or definitely one negative root We use the following theorem of calculus to determine an initial approximation It is also called the intermediate value theorem Theorem If f(x) is continuous on some interval [a, b] and f(a)f(b) < 0, then the equation f(x) = 0 has at least one real root or an odd number of real roots in the interval (a, b) This result is very simple to use We set up a table of values of f (x) for various values of x Studying the changes in signs in the values of f (x), we determine the intervals in which the roots lie For example, if f () and f () are of opposite signs, then there is a root in the interval (, ) et us illustrate through the following examples Example Determine the maximum number of positive and negative roots and intervals of length one unit in which the real roots lie for the following equations (i) 8x x x + = 0 (ii) x x x 5 = 0 Solution (i) et f(x) = 8x x x + = 0 The number of changes in the signs of the coefficients (8,,, ) is Therefore, the equation has or no positive roots Now, f( x) = 8x x + x + The number of changes in signs in the coefficients ( 8,,, ) is Therefore, the equation has one negative root We have the following table of values for f(x), (Table ) Table Values of f (x), Example (i ) x 0 f(x) Since f( ) f(0) < 0, there is a root in the interval (, 0), f(0) f() < 0, there is a root in the interval (0, ), f() f() < 0, there is a root in the interval (, )

16 SUTIN F EUATINS AND EIGEN VAUE RBEMS 5 Therefore, there are three real roots and the roots lie in the intervals (, 0), (0, ), (, ) (ii) et f(x) = x x x 5 = 0 The number of changes in the signs of the coefficients (,,, 5) is Therefore, the equation has one positive root Now, f( x) = x x + x 5 The number of changes in signs in the coefficients (,,, 5) is Therefore, the equation has two negative or no negative roots We have the table of values for f (x), (Table ) Table Values of f (x ), Example (ii ) x 0 f(x) From the table, we find that there is one real positive root in the interval (, ) The equation has no negative real root Example Determine an interval of length one unit in which the negative real root, which is smallest in magnitude lies for the equation 9x + 8x 7x 70 = 0 Solution et f(x) = 9x + 8x 7x 70 = 0 Since, the smallest negative real root in magnitude is required, we form a table of values for x < 0, (Table ) Table Values of f (x ), Example x f(x) Since, f( )f( ) < 0, the negative root of smallest magnitude lies in the interval (, ) Example ocate the smallest positive root of the equations (i) xe x = cos x (ii) tan x = x Solution (i) et f(x) = xe x cos x = 0 We have f(0) =, f() = e cos = = 78 Since, f(0) f() < 0, there is a root in the interval (0, ) (ii) et f(x) = tan x x = 0 We have the following function values f(0) = 0, f(0) = 00997, f(05) = 0457, f() = 0446, f(, ) = 05, f() = 07 Since, f() f() < 0, the root lies in the interval (, ) Now, we present some iterative methods for finding a root of the given algebraic or transcendental equation We know from calculus, that in the neighborhood of a point on a curve, the curve can be approximated by a straight line For deriving numerical methods to find a root of an equation

17 6 NUMERICA METHDS f(x) = 0, we approximate the curve in a sufficiently small interval which contains the root, by a straight line That is, in the neighborhood of a root, we approximate f(x) ax + b, a 0 where a and b are arbitrary parameters to be determined by prescribing two appropriate conditions on f(x) and/or its derivatives Setting ax + b = 0, we get the next approximation to the root as x = b/a Different ways of approximating the curve by a straight line give different methods These methods are also called chord methods Method of false position (also called regula-falsi method) and Newton-Raphson method fall in this category of chord methods Method of False osition The method is also called linear interpolation method or chord method or regula-falsi method At the start of all iterations of the method, we require the interval in which the root lies et the root of the equation f(x) = 0, lie in the interval (x k, x k ), that is, f k f k < 0, where f(x k ) = f k, and f(x k ) = f k Then, (x k, f k ), (x k, f k ) are points on the curve f(x) = 0 Draw a straight line joining the points and (Figs a, b) The line is taken as an approximation of the curve in the interval [x k, x k ] The equation of the line is given by y fk x xk = f f x x k k k k The point of intersection of this line with the x-axis is taken as the next approximation to the root Setting y = 0, and solving for x, we get x = x k F HG x f k k The next approximation to the root is taken as x k+ = x k F HG x f k k x f x f k k I KJ k k I KJ f k = x k Simplifying, we can also write the approximation as x k+ = x ( f f ) ( x x ) f f f F HG k k k k k k k k x f k k x f k k I KJ f k f k (0) = x f x f f f k k k k k k, k =,, () Therefore, starting with the initial interval (x 0, x ), in which the root lies, we compute x = x f f x f f Now, if f(x 0 ) f(x ) < 0, then the root lies in the interval (x 0, x ) therwise, the root lies in the interval (x, x ) The iteration is continued using the interval in which the root lies, until the required accuracy criterion given in Eq(8) or Eq(9) is satisfied Alternate derivation of the method et the root of the equation f(x) = 0, lie in the interval (x k, x k ) Then, (x k, f k ), (x k, f k ) are points on the curve f(x) = 0 Draw the chord joining the points and (Figs a, b) We

18 SUTIN F EUATINS AND EIGEN VAUE RBEMS 7 approximate the curve in this interval by the chord, that is, f(x) ax + b The next approximation to the root is given by x = b/a Since the chord passes through the points and, we get Subtracting the two equations, we get f k = ax k + b, and f k = ax k + b f k f k = a(x k x k ), or a = f k f x x The second equation gives b = f k ax k Hence, the next approximation is given by x k+ = b f = a k ax a which is same as the method given in Eq(0) k k k k = x k f k = x k a F HG x f k k x f k k I KJ f k y y x x x x x 0 x x x 0 x x Fig a Method of false position Fig b Method of false position Remark At the start of each iteration, the required root lies in an interval, whose length is decreasing Hence, the method always converges Remark 4 The method of false position has a disadvantage If the root lies initially in the interval (x 0, x ), then one of the end points is fixed for all iterations For example, in Figa, the left end point x 0 is fixed and the right end point moves towards the required root Therefore, in actual computations, the method behaves like x k+ = x 0 f k x k f 0, k =,, () fk f0 In Figb, the right end point x is fixed and the left end point moves towards the required root Therefore, in this case, in actual computations, the method behaves like xkf xfk x k+ =, k =,, () f fk Remark 5 The computational cost of the method is one evaluation of the function f(x), for each iteration Remark 6 We would like to know why the method is also called a linear interpolation method Graphically, a linear interpolation polynomial describes a straight line or a chord The linear interpolation polynomial that fits the data (x k, f k ), (x k, f k ) is given by

19 8 NUMERICA METHDS f(x) = x x x k k x k f k + x x k x x (We shall be discussing the concept of interpolation polynomials in Chapter ) Setting f(x) = 0, we get k k f k ( x x ) f ( x x ) f k k k k x k x k = 0, or x(f k f k ) = x k f k x k f k or x = x k+ = x f x f f f k k k k This gives the next approximation as given in Eq () Example 4 ocate the intervals which contain the positive real roots of the equation x x + = 0 btain these roots correct to three decimal places, using the method of false position Solution We form the following table of values for the function f(x) k k x 0 f (x) 9 There is one positive real root in the interval (0, ) and another in the interval (, ) There is no real root for x > as f(x) > 0, for all x > First, we find the root in (0, ) We have x 0 = 0, x =, f 0 = f(x 0 ) = f(0) =, f = f(x ) = f() = x = x f f x f f Since, f(0) f(05) < 0, the root lies in the interval (0, 05) x = x f f x f f = = 05, f(x ) = f(05) = ( ) = = 0664, f(x 0 75 ) = f(0664) = 0048 Since, f(0) f(0664) < 0, the root lies in the interval (0, 0664) x 4 = x f f x f f Since, f(0) f(0487) < 0, the root lies in the interval (0, 04870) x 5 = x f f x f f ( ) = = 04870, f(x ) = f(04870) = ( ) = = 0474, f(x ) = f(0474) = Since, f(0) f(0474) < 0, the root lies in the interval (0, 0474) x 6 = x f f x f f ( ) = =

20 SUTIN F EUATINS AND EIGEN VAUE RBEMS 9 Now, x 6 x 5 = < The root has been computed correct to three decimal places The required root can be taken as x x 6 = We may also give the result as 047, even though x 6 is more accurate Note that the left end point x = 0 is fixed for all iterations Now, we compute the root in (, ) We have x 0 =, x =, f 0 = f(x 0 ) = f() =, f = f(x ) = f() = x = x f f x f f = ( ) = 5, f(x ( ) ) = f(5) = Since, f(5) f() < 0, the root lies in the interval (5, ) We use the formula given in Eq() x = x f f x f f f(x ) = f(407407) = () ( ) = = , ( ) Since, f(407407) f() < 0, the root lies in the interval (407407, ) x 4 = x f f x f f f(x 4 ) = f(4867) = () ( 04447) = = 4867, ( 04447) Since f(4867) f() < 0, the root lies in the interval (4867, ) x 5 = x f f x f f f(x 5 ) = f(556) = () ( 0897) = = 556, ( 0897) Since, f(556) f() < 0, the root lies in the interval (556, ) x 6 = xf f xf f f(x 6 ) = f(550) = () ( ) = = 550, ( ) Since, f(550) f() < 0, the root lies in the interval (550, ) x 7 = x f f x f f f(x 7 ) = f(5946) = () ( 00874) = = 5946 ( 00874) Since, f(5946) f() < 0, the root lies in the interval (5946, ) x 8 = xf f xf f f(x 8 ) = f(56) = () ( ) = = 56, ( ) Since, f(56) f() < 0, the root lies in the interval (56, )

21 0 NUMERICA METHDS x 9 = x f f x f f f(x 9 ) = f(579) = () ( 00098) = = 579, ( 00098) Since, f(579) f() < 0, the root lies in the interval (579, ) x 0 = x f f x f f () ( ) = = 5956 ( ) Now, x 0 x 9 = < The root has been computed correct to three decimal places The required root can be taken as x x 0 = 5956 Note that the right end point x = is fixed for all iterations Example 5 Find the root correct to two decimal places of the equation xe x = cos x, using the method of false position Solution Define f(x) = cos x xex = 0 There is no negative root for the equation We have f(0) =, f() = cos e = 7798 A root of the equation lies in the interval (0, ) et x 0 = 0, x = Using the method of false position, we obtain the following results x = x f f x f f () = 7798 = 0467, f(x ) = f(0467) = Since, f(0467) f() < 0, the root lies in the interval (0467, ) We use the formula given in Eq() x = x f f x f f 0467( 7798) (05986) = f(x ) = f(04467) = 0054 Since, f(04467) f() < 0, the root lies in the interval (04467, ) = 04467, x 4 = x f f x f f 04467( 7798) (0054) = = 04940, f(x 4 ) = f(04940) = Since, f(04940) f() < 0, the root lies in the interval (04940, ) x 5 = x f f x f f ( 7798) (007079) = = , f(x 5 ) = f(050995) = 0060 Since, f(050995) f() < 0, the root lies in the interval (050995, ) x 6 = x f f x f f f(x 6 ) = f(0550) = ( 7798) (006) = = 0550,

22 SUTIN F EUATINS AND EIGEN VAUE RBEMS Since, f(0550) f() < 0, the root lies in the interval (0550, ) x 7 = x f f x f f Now, x 7 x 6 = < ( 7798) (000776) = = The root has been computed correct to two decimal places The required root can be taken as x x 7 = 0569 Note that the right end point x = is fixed for all iterations 4 Newton-Raphson Method This method is also called Newton s method This method is also a chord method in which we approximate the curve near a root, by a straight line et x 0 be an initial approximation to the root of f(x) = 0 Then, (x 0, f 0 ), where f 0 = f(x 0 ), is a point on the curve Draw the tangent to the curve at, (Fig ) We approximate the curve in the neighborhood of the root by the tangent to the curve at the point The point of intersection of the tangent with the x-axis is taken as the next approximation to the root The process is repeated until the required accuracy is obtained The equation of the tangent to the curve y = f(x) at the point (x 0, f 0 ) is given by y f(x 0 ) = (x x 0 ) f (x 0 ) where f (x 0 ) is the slope of the tangent to the curve at Setting y = 0 and solving for x, we get x = x 0 f ( x 0 ), f (x f ( x ) 0 ) 0 The next approximation to the root is given by 0 x = x 0 f ( x 0 ), f (x f ( x ) 0 ) 0 We repeat the procedure The iteration method is defined as 0 x k+ = x k f ( x k ), f (x f ( x ) k ) 0 (4) k This method is called the Newton-Raphson method or simply the Newton s method The method is also called the tangent method Alternate derivation of the method et x k be an approximation to the root of the equation f(x) = 0 et x be an increment in x such that x k + x is the exact root, that is f(x k + x) 0 y x x 0 Fig Newton-Raphson method x

23 NUMERICA METHDS Expanding in Taylor s series about the point x k, we get f(x k ) + x f (x k ) + ( x )! Neglecting the second and higher powers of x, we obtain f(x k ) + x f (x k ) 0, or x = f ( x k ) f ( x ) Hence, we obtain the iteration method f (x k ) + = 0 (5) x k+ = x k + x = x k f ( x k ), f (x f ( x ) k ) 0, k = 0,,, k which is same as the method derived earlier Remark 7 Convergence of the Newton s method depends on the initial approximation to the root If the approximation is far away from the exact root, the method diverges (see Example 6) However, if a root lies in a small interval (a, b) and x 0 (a, b), then the method converges Remark 8 From Eq(4), we observe that the method may fail when f (x) is close to zero in the neighborhood of the root ater, in this section, we shall give the condition for convergence of the method Remark 9 The computational cost of the method is one evaluation of the function f(x) and one evaluation of the derivative f (x), for each iteration Example 6 Derive the Newton s method for finding /N, where N > 0 Hence, find /7, using the initial approximation as (i) 005, (ii) 05 Do the iterations converge? Solution et x = N, or x = N Define f(x) = x N Then, f (x) = x Newton s method gives x k+ = x k f ( x k ) = x f ( x ) k [( / xk ) N ] = x [ / x k + [x k Nx k ] = x k Nx k ] k (i) With N = 7, and x 0 = 005, we obtain the sequence of approximations x = x 0 Nx 0 = (005) 7(005) = x = x Nx = (00575) 7(00575) = x = x Nx = ( ) 7( ) = x 4 = x Nx = (00588) 7(00588) = Since, x 4 x = 0, the iterations converge to the root The required root is (ii) With N = 7, and x 0 = 05, we obtain the sequence of approximations x = x 0 Nx 0 = (05) 7(05) = 0085 k k

24 SUTIN F EUATINS AND EIGEN VAUE RBEMS x = x Nx = ( 0085) 7( 0805) = x = x Nx = ( ) 7( ) = x 4 = x Nx = ( 90094) 7( 90094) = 6575 We find that x k as k increases Therefore, the iterations diverge very fast This shows the importance of choosing a proper initial approximation Example 7 Derive the Newton s method for finding the qth root of a positive number N, N /q, where N > 0, q > 0 Hence, compute 7 / correct to four decimal places, assuming the initial approximation as x 0 = Solution et x = N /q, or x q = N Define f(x) = x q N Then, f (x) = qx q Newton s method gives the iteration q k x k+ = x k x N qxk xk + N ( q ) xk + N = = q q q qx qx qx k For computing 7 /, we have q = and N = 7 Hence, the method becomes x k+ = xk + 7, k = 0,,, x With x 0 =, we obtain the following results k q k q k q x = x ( ) + 7 = x 4 ( ) 0 0 = 75, x = x + 7 (75) + 7 = = 58645, x (75) x = x + 7 (58645) + 7 = = 57, x (58645) x 4 = x + 7 (57) + 7 = = 578 x (57) Now, x 4 x = = We may take x 578 as the required root correct to four decimal places Example 8 erform four iterations of the Newton s method to find the smallest positive root of the equation f(x) = x 5x + = 0 Solution We have f(0) =, f() = Since, f(0) f() < 0, the smallest positive root lies in the interval (0, ) Applying the Newton s method, we obtain x k+ = x k x k 5 x k + xk =, k = 0,,, x 5 x 5 k k

25 4 NUMERICA METHDS et x 0 = 05 We have the following results x = x0 05 ( ) = x 5 05 ( ) 5 0 = 07647, x = x ( ) = x 5 ( ) 5 x = x ( ) = x 5 ( ) 5 = 00568, = 00640, x 4 = x ( ) = = x 5 ( ) 5 Therefore, the root correct to six decimal places is x Example 9 Using Newton-Raphson method solve x log 0 x = 4 with x 0 = 0 Solution Define f(x) = x log 0 x 4 Then f (x) = log 0 x + = log log e 0 0 x Using the Newton-Raphson method, we obtain x k+ = x k x k log 0 x k 4, k = 0,,, log x With x 0 = 0, we obtain the following results 0 k (AU Apr/May 004) x = x 0 x 0 log 0 x 0 4 log x x = x x log log x 0 0 x = 0 0 log log = = log log = x = x x log log x 0 0 x log = = log We have x x = = We may take x as the root correct to four decimal places 0

26 SUTIN F EUATINS AND EIGEN VAUE RBEMS 5 5 General Iteration Method The method is also called iteration method or method of successive approximations or fixed point iteration method The first step in this method is to rewrite the given equation f(x) = 0 in an equivalent form as There are many ways of rewriting f(x) = 0 in this form x = φ(x) (6) For example, f(x) = x 5x + = 0, can be rewritten in the following forms x = x +, x = (5x ) /, x = 5 5x, etc (7) x Now, finding a root of f(x) = 0 is same as finding a number α such that α = φ(α), that is, a fixed point of φ(x) A fixed point of a function φ is a point α such that α = φ(α) This result is also called the fixed point theorem Using Eq(6), the iteration method is written as x k+ = φ(x k ), k = 0,,, (8) The function φ(x) is called the iteration function Starting with the initial approximation x 0, we compute the next approximations as x = φ(x 0 ), x = φ(x ), x = φ(x ), The stopping criterion is same as used earlier Since, there are many ways of writing f(x) = 0 as x = φ(x), it is important to know whether all or at least one of these iteration methods converges Remark 0 Convergence of an iteration method x k+ = φ(x k ), k = 0,,,, depends on the choice of the iteration function φ(x), and a suitable initial approximation x 0, to the root Consider again, the iteration methods given in Eq(7), for finding a root of the equation f(x) = x 5x + = 0 The positive root lies in the interval (0, ) (i) x k+ = x k +, k = 0,,, (9) 5 With x 0 =, we get the sequence of approximations as x = 04, x = 08, x = 009, x 4 = 0065, x 5 = 0064 The method converges and x x 5 = 0064 is taken as the required approximation to the root (ii) x k+ = (5x k ) /, k = 0,,, (0) With x 0 =, we get the sequence of approximations as x = 5874, x = 907, x = 047, x 4 = 0968, which does not converge to the root in (0, ) (iii) x k+ = 5xk, k = 0,,, () x k

27 6 NUMERICA METHDS With x 0 =, we get the sequence of approximations as x = 0, x =, x = 80, x 4 = 84, which does not converge to the root in (0, ) Now, we derive the condition that the iteration function φ(x) should satisfy in order that the method converges Condition of convergence The iteration method for finding a root of f(x) = 0, is written as et α be the exact root That is, x k+ = φ(x k ), k = 0,,, () α = φ(α) () We define the error of approximation at the kth iterate as ε k = x k α, k = 0,,, Subtracting () from (), we obtain x k+ α = φ(x k ) φ(α) or ε k+ = φ (t k ) ε k, x k < t k < α Setting k = k, we get ε k = φ (t k ) ε k, x k < t k < α Hence, ε k+ = φ (t k )φ (t k ) ε k Using (4) recursively, we get = (x k α)φ (t k ) (using the mean value theorem) (4) ε k+ = φ (t k )φ (t k ) φ (t 0 ) ε 0 The initial error ε 0 is known and is a constant We have et φ (t k ) c, k = 0,,, ε k+ = φ (t k ) φ (t k ) φ (t 0 ) ε 0 Then, ε k+ c k+ ε 0 (5) For convergence, we require that ε k+ 0 as k This result is possible, if and only if c < Therefore, the iteration method () converges, if and only if φ (x k ) c <, k = 0,,, or φ (x) c <, for all x in the interval (a, b) (6) We can test this condition using x 0, the initial approximation, before the computations are done et us now check whether the methods (9), (0), () converge to a root in (0, ) of the equation f(x) = x 5x + = 0 (i) We have φ(x) = x + x x, φ (x) =, and φ (x) = the method converges to a root in (0, ) for all x in 0 < x < Hence,

28 SUTIN F EUATINS AND EIGEN VAUE RBEMS 7 (ii) We have φ(x) = (5x ) /, φ (x) = 5 5 ( x ) / Now φ (x) <, when x is close to and φ (x) > in the other part of the interval Convergence is not guaranteed 5x (iii) We have φ(x) =, φ (x) = Again, φ (x) <, when x is close to x x / 5x / ( ) and φ (x) > in the other part of the interval Convergence is not guaranteed Remark Sometimes, it may not be possible to find a suitable iteration function φ(x) by manipulating the given function f(x) Then, we may use the following procedure Write f(x) = 0 as x = x + α f(x) = φ(x), where α is a constant to be determined et x 0 be an initial approximation contained in the interval in which the root lies For convergence, we require φ (x 0 ) = + α f (x 0 ) < (7) Simplifying, we find the interval in which α lies We choose a value for α from this interval and compute the approximations A judicious choice of a value in this interval may give faster convergence Example 0 Find the smallest positive root of the equation x x 0 = 0, using the general iteration method Solution We have f(x) = x x 0, f(0) = 0, f() = 0, f() = 8 0 = 4, f() = 7 0 = 4 Since, f() f() < 0, the smallest positive root lies in the interval (, ) Write x = x + 0, and x = (x + 0) / = φ(x) We define the iteration method as x k+ = (x k + 0) / We obtain φ (x) = ( x + 0) / We find φ (x) < for all x in the interval (, ) Hence, the iteration converges et x 0 = 5 We obtain the following results x = (5) / = 08, x = (08) / = 097, x = (097) / = 090, x 4 = (090) / = 089 Since, x 4 x = = 0000, we take the required root as x 089 Example Find the smallest negative root in magnitude of the equation x 4 + x + x + 4 = 0, using the method of successive approximations Solution We have f(x) = x 4 + x + x + 4 = 0, f(0) = 4, f( ) = + 4 = 6 Since, f( ) f(0) < 0, the smallest negative root in magnitude lies in the interval (, 0)

29 8 NUMERICA METHDS Write the given equation as x(x + x 4 + ) + 4 = 0, and x = x + x + The iteration method is written as x k+ = 4 x + x + k k = φ(x) We obtain φ (x) = 49 ( x + x) ( x + x + ) We find φ (x) < for all x in the interval (, 0) Hence, the iteration converges et x 0 = 05 We obtain the following results 4 x = = 090, ( 05 ) + ( 05 ) + x = 4 ( 0 9) + ( 0 9) + = 0, x = 4 ( 0 ) + ( 0 ) + The required approximation to the root is x 0 = 0 Example The equation f(x) = x + 4x + 4x + = 0 has a root in the interval (, 0) Determine an iteration function φ(x), such that the sequence of iterations obtained from x k+ = φ(x k ), x 0 = 05, k = 0,,, converges to the root Solution We illustrate the method given in Remark 0 We write the given equation as x = x + α (x + 4x + 4x + ) = φ(x) where α is a constant to be determined such that φ (x) = + α f (x) = + α (9x + 8x + 4) < for all x (, 0) This condition is also to be satisfied at the initial approximation Setting x 0 = 05, we get φ (x 0 ) = + α f (x 0 ) = + 9α < 4 or < + 9 α 8 < or 4 9 < α < 0 Hence, α takes negative values The interval for α depends on the initial approximation x 0 et us choose the value α = 05 We obtain the iteration method as x k+ = x k 05 (x k + 4x k + 4x k + )

30 SUTIN F EUATINS AND EIGEN VAUE RBEMS 9 = 05 (x k + 4x k + x k + ) = φ(x k ) Starting with x 0 = 05, we obtain the following results x = φ(x 0 ) = 05 (x 0 + 4x 0 + x 0 + ) = 05 [( 05) + 4( 05) + ( 05) + ] = 05 x = φ(x ) = 05(x + 4x + x + ) = 05[( 05) + 4( 05) + ( 05) + ] = 0706 x = φ(x ) = 05(x + 4x + x + ) = 05[ ( 0706) + 4( 0706) + ( 0706) + ] = 07 x 4 = φ(x ) = 05(x + 4x + x + ) = 05[( 07) + 4( 07) + ( 07) + ] = 045 x 5 = φ(x 4 ) = 05(x 4 + 4x 4 + x 4 + ) = 05[( 045) + 4( 045) + ( 045) + ] = 06 Since x 5 x 4 = = < 00005, the result is correct to three decimal places We can take the approximation as x x 5 = 06 The exact root is x = / We can verify that φ (x j ) < for all j 6 Convergence of the Iteration Methods We now study the rate at which the iteration methods converge to the exact root, if the initial approximation is sufficiently close to the desired root Define the error of approximation at the kth iterate as ε k = x k α, k = 0,,, Definition An iterative method is said to be of order p or has the rate of convergence p, if p is the largest positive real number for which there exists a finite constant C 0, such that ε k+ C ε k p (8) The constant C, which is independent of k, is called the asymptotic error constant and it depends on the derivatives of f(x) at x = α et us now obtain the orders of the methods that were derived earlier Method of false position We have noted earlier (see Remark 4) that if the root lies initially in the interval (x 0, x ), then one of the end points is fixed for all iterations If the left end point x 0 is fixed and the right end point moves towards the required root, the method behaves like (see Figa) x k+ = x 0 f k x k f 0 fk f0 Substituting x k = ε k + α, x k+ = ε k+ + α, x 0 = ε 0 + α, we expand each term in Taylor s series and simplify using the fact that f(α) = 0 We obtain the error equation as f ( α) ε k+ = Cε 0 ε k, where C = f ( α)

31 0 NUMERICA METHDS Since ε 0 is finite and fixed, the error equation becomes ε k+ = C* ε k, where C* = Cε 0 (9) Hence, the method of false position has order or has linear rate of convergence Method of successive approximations or fixed point iteration method We have x k+ = φ(x k ), and α = φ(α) Subtracting, we get x k+ α = φ(x k ) φ(α) = φ(α + x k α) φ(α) = [φ(α) + (x k α) φ (α) + ] φ(α) or ε k+ = ε k φ (α) + (ε k ) Therefore, ε k+ = C ε k, x k < t k < α, and C = φ (α) (0) Hence, the fixed point iteration method has order or has linear rate of convergence Newton-Raphson method The method is given by x k+ = x k f ( x k), f (x k ) 0 f ( x ) Substituting x k = ε k + α, x k+ = ε k+ + α, we obtain ε k+ + α = ε k + α f ( εk + α) f ( ε + α) k k Expand the terms in Taylor s series Using the fact that f(α) = 0, and canceling f (α), we obtain ε k+ = ε k = ε k ε = ε k ε = ε k ε εkf ( α) + εk f ( α) + f ( α) + ε f ( α) f ( α) + f ( α) ε k f α + + ( ) f ( α) ε k k k f ( α) + f ( α) ε f α + ( ) f ( α) ε k k k k + + f ( α) α ε f α k + f = ( ) ε k + ( ) f ( α) Neglecting the terms containing ε k and higher powers of ε k, we get ε k+ = Cε k, where C = f ( α), f ( α)

32 SUTIN F EUATINS AND EIGEN VAUE RBEMS and ε k+ = C ε k () Therefore, Newton s method is of order or has quadratic rate of convergence Remark What is the importance of defining the order or rate of convergence of a method? Suppose that we are using Newton s method for computing a root of f(x) = 0 et us assume that at a particular stage of iteration, the error in magnitude in computing the root is 0 = 0 We observe from (), that in the next iteration, the error behaves like C(0) = C(0 ) That is, we may possibly get an accuracy of two decimal places Because of the quadratic convergence of the method, we may possibly get an accuracy of four decimal places in the next iteration However, it also depends on the value of C From this discussion, we conclude that both fixed point iteration and regula-falsi methods converge slowly as they have only linear rate of convergence Further, Newton s method converges at least twice as fast as the fixed point iteration and regula-falsi methods Remark When does the Newton-Raphson method fail? (i) The method may fail when the initial approximation x 0 is far away from the exact root α (see Example 6) However, if the root lies in a small interval (a, b) and x 0 (a, b), then the method converges (ii) From Eq(), we note that if f (α) 0, and f (x) is finite then C and the method may fail That is, in this case, the graph of y = f(x) is almost parallel to x-axis at the root α Remark 4 et us have a re-look at the error equation We have defined the error of approximation at the kth iterate as ε k = x k α, k = 0,,, From x k+ = φ(x k ), k = 0,,, and α = φ(α), we obtain (see Eq(4)) x k+ α = φ(x k ) φ(α) = φ(α + ε k ) φ(α) = φα ( ) + φ ( α) εk + φ ( α) εk + φ(α) or ε k+ = a ε k + a ε k + () where a = φ (α), a = (/)φ (α), etc The exact root satisfies the equation α = φ(α) If a 0 that is, φ (α) 0, then the method is of order or has linear convergence For the general iteration method, which is of first order, we have derived that the condition of convergence is φ (x) < for all x in the interval (a, b) in which the root lies Note that in this method, φ (x) 0 for all x in the neighborhood of the root α If a = φ (α) = 0, and a = (/)φ (α) 0, then from Eq (), the method is of order or has quadratic convergence et us verify this result for the Newton-Raphson method For the Newton-Raphson method x k+ = x k f ( x k), we have φ(x) = x f ( x ) f ( x ) f ( x) Then, φ (x) = [ f ( x )] f ( x ) f ( x ) = [ f ( x)] k f( x) f ( x) [ f ( x)]

33 NUMERICA METHDS and φ (α) = f ( α) f ( α) = 0 [ f ( α)] since f(α) = 0 and f (α) 0 (α is a simple root) When, x k α, f(x k ) 0, we have φ (x k ) <, k =,, and 0 as n Now, φ (x) = [ f ( x)] [f (x) {f (x) f (x) + f(x) f (x)} f(x) {f (x)} ] f ( α) and φ (α) = f ( α) 0 Therefore, a 0 and the second order convergence of the Newton s method is verified REVIEW UESTINS Define a (i) root, (ii) simple root and (iii) multiple root of an algebraic equation f(x) = 0 Solution (i) A number α, such that f(α) 0 is called a root of f(x) = 0 (ii) et α be a root of f(x) = 0 If f(α) 0 and f (α) 0, then α is said to be a simple root Then, we can write f(x) as (iii) et α be a root of f(x) = 0 If f(x) = (x α) g(x), g(α) 0 f(α) = 0, f (α) = 0,, f (m ) (α) = 0, and f (m) (α) 0, then, α is said to be a multiple root of multiplicity m Then, we can write f(x) as f(x) = (x α) m g(x), g(α) 0 State the intermediate value theorem Solution If f(x) is continuous on some interval [a, b] and f (a)f (b) < 0, then the equation f(x) = 0 has at least one real root or an odd number of real roots in the interval (a, b) How can we find an initial approximation to the root of f (x) = 0? Solution Using intermediate value theorem, we find an interval (a, b) which contains the root of the equation f(x) = 0 This implies that f (a)f(b) < 0 Any point in this interval (including the end points) can be taken as an initial approximation to the root of f(x) = 0 4 What is the Descartes rule of signs? Solution et f (x) = 0 be a polynomial equation n (x) = 0 We count the number of changes of signs in the coefficients of f (x) = n (x) = 0 The number of positive roots cannot exceed the number of changes of signs in the coefficients of n (x) Now, we write the equation f( x) = n ( x) = 0, and count the number of changes of signs in the coefficients of n ( x) The number of negative roots cannot exceed the number of changes of signs in the coefficients of this equation 5 Define convergence of an iterative method Solution Using any iteration method, we obtain a sequence of iterates (approximations to the root of f(x) = 0), x, x,, x k, If

34 SUTIN F EUATINS AND EIGEN VAUE RBEMS lim x k = α, or lim x k α = 0 k k where α is the exact root, then the method is said to be convergent 6 What are the criteria used to terminate an iterative procedure? Solution etε be the prescribed error tolerance We terminate the iterations when either of the following criteria is satisfied (i) f(x k ) ε (ii) x k+ x k ε Sometimes, we may use both the criteria 7 Define the fixed point iteration method to obtain a root of f(x) = 0 When does the method converge? Solution et a root of f(x) = 0 lie in the interval (a, b) et x 0 be an initial approximation to the root We write f(x) = 0 in an equivalent form as x = φ(x), and define the fixed point iteration method as x k+ = φ(x k ), k = 0,,, Starting with x 0, we obtain a sequence of approximations x, x,, x k, such that in the limit as k, x k α The method converges when φ (x) <, for all x in the interval (a, b) We normally check this condition at x 0 8 Write the method of false position to obtain a root of f(x) = 0 What is the computational cost of the method? Solution et a root of f(x) = 0 lie in the interval (a, b) et x 0, x be two initial approximations to the root in this interval The method of false position is defined by x k+ = x k f k x k f k, k =,, fk fk The computational cost of the method is one evaluation of f(x) per iteration 9 What is the disadvantage of the method of false position? Solution If the root lies initially in the interval (x 0, x ), then one of the end points is fixed for all iterations For example, in Figa, the left end point x 0 is fixed and the right end point moves towards the required root Therefore, in actual computations, the method behaves like x k+ = x 0 f k x k f 0 fk f0 In Figb, the right end point x is fixed and the left end point moves towards the required root Therefore, in this case, in actual computations, the method behaves like x k+ = x k f x f k f fk 0 Write the Newton-Raphson method to obtain a root of f(x) = 0 What is the computational cost of the method? Solution et a root of f(x) = 0 lie in the interval (a, b) et x 0 be an initial approximation to the root in this interval The Newton-Raphson method to find this root is defined by

35 4 NUMERICA METHDS x k+ = x k f ( x k), f (x f ( x ) k ) 0, k = 0,,,, k The computational cost of the method is one evaluation of f(x) and one evaluation of the derivative f (x) per iteration Define the order (rate) of convergence of an iterative method for finding the root of an equation f(x) = 0 Solution et α be the exact root of f(x) = 0 Define the error of approximation at the kth iterate as ε k = x k α, k = 0,,, An iterative method is said to be of order p or has the rate of convergence p, if p is the largest positive real number for which there exists a finite constant C 0, such that ε k+ C ε k p The constant C, which is independent of k, is called the asymptotic error constant and it depends on the derivatives of f(x) at x = α What is the rate of convergence of the following methods: (i) Method of false position, (ii) Newton-Raphson method, (iii) Fixed point iteration method? Solution (i) ne (ii) Two (iii) ne EXERCISE In the following problems, find the root as specified using the regula-falsi method (method of false position) Find the positive root of x = x + 5 (Do only four iterations) (AU Nov/Dec 006) Find an approximate root of x log 0 x = 0 Solve the equation x tan x =, starting with a = 5 and b =, correct to three decimal places 4 Find the root of xe x =, correct to two decimal places 5 Find the smallest positive root of x e x = 0, correct to three decimal places 6 Find the smallest positive root of x 4 x 0 = 0, correct to three decimal places In the following problems, find the root as specified using the Newton-Raphson method 7 Find the smallest positive root of x 4 x = 0, correct to three decimal places 8 Find the root between 0 and of x = 6x 4, correct to two decimal places 9 Find the real root of the equation x = cos x + (AU Nov/Dec 006) 0 Find a root of x log 0 x = 0, correct to three decimal places (AU Nov/Dec 004) Find the root of x = sin x, near 9, correct to three decimal places (i) Write an iteration formula for finding N where N is a real number (AU Nov/Dec 006, AU Nov/Dec 00) (ii) Hence, evaluate 4, correct to three decimal places

36 SUTIN F EUATINS AND EIGEN VAUE RBEMS 5 (i) Write an iteration formula for finding the value of /N, where N is a real number (ii) Hence, evaluate /6, correct to four decimal places 4 Find the root of the equation sin x = + x, which lies in the interval (, ), correct to three decimal places 5 Find the approximate root of xe x =, correct to three decimal places In the following problems, find the root as specified using the iteration method/method of successive approximations/fixed point iteration method 6 Find the smallest positive root of x 5x + = 0, correct to four decimal places 7 Find the smallest positive root of x 5 64x + 0 = 0, correct to four decimal places 8 Find the smallest negative root in magnitude of x x + = 0, correct to four decimal places 9 Find the smallest positive root of x = e x, correct to two decimal places 0 Find the real root of the equation cos x = x (AU Nov/Dec 006) The equation x + ax + b = 0, has two real roots α and β Show that the iteration method (i) x k+ = (ax k + b)/x k, is convergent near x = α, if α > β, (ii) x k+ = b/(x k + a), is convergent near x = α, if α < β INEAR SYSTEM F AGEBRAIC EUATINS Introduction Consider a system of n linear algebraic equations in n unknowns a x + a x + + a n x n = b a x + a x + + a n x n = b a n x + a n x + + a nn x n = b n where a ij, i =,,, n, j =,,, n, are the known coefficients, b i, i =,,, n, are the known right hand side values and x i, i =,,, n are the unknowns to be determined In matrix notation we write the system as Ax = b () where A = N M a a an a a an an an ann, x = M x x x n, and b = M b b b n The matrix [A b], obtained by appending the column b to the matrix A is called the augmented matrix That is

37 6 NUMERICA METHDS [A b] = We define the following M a a an b a a an b a a a b n n nn n (i) The system of equations () is consistent (has at least one solution), if rank (A) = rank [A b] = r If r = n, then the system has unique solution If r < n, then the system has (n r) parameter family of infinite number of solutions (ii) The system of equations () is inconsistent (has no solution) if rank (A) rank [A b] We assume that the given system is consistent The methods of solution of the linear algebraic system of equations () may be classified as direct and iterative methods (a) Direct methods produce the exact solution after a finite number of steps (disregarding the round-off errors) In these methods, we can determine the total number of operations (additions, subtractions, divisions and multiplications) This number is called the operational count of the method (b) Iterative methods are based on the idea of successive approximations We start with an initial approximation to the solution vector x = x 0, and obtain a sequence of approximate vectors x 0, x,, x k,, which in the limit as k, converge to the exact solution vector x Now, we derive some direct methods Direct Methods If the system of equations has some special forms, then the solution is obtained directly We consider two such special forms (a) et A be a diagonal matrix, A = D That is, we consider the system of equations Dx = b as a x = b a x = b (4) a n, n x n = b n a nn x n = b n This system is called a diagonal system of equations Solving directly, we obtain x i = b i, a a ii 0, i =,,, n (5) ii (b) et A be an upper triangular matrix, A = U That is, we consider the system of equations Ux = b as