University of Alberta 2016

Transcription

1 University of Alberta 2016 Equation Solving and Newton s Method Railroad track problem English Units Version: Somewhere in Alberta, there is a one mile stretch of rail with fixed ends over a flat bed. Someone welds in 1 foot of additional rail and the rail buckles upward into a circular arc. How high above the original bed is the lengthened rail? Metric Version: Somewhere in Alberta, there is a one kilometer stretch of rail over a (nearly) flat bed. Someone welds in 1 meter of additional rail and the rail buckles upward into a circular arc with its ends in their original positions. How high above the original bed is the lengthened rail? In[1]:= a = 2640; ϵ = 1 / 2 / a {6 ϵ // N, 6 ϵ / (1 + ϵ) // N} R = Sqrt (1 + ϵ)^3 6 ϵ // N X = Sqrt[R^2-1] // N d = a (R - X) Out[2]= Out[3]= { , } Out[4]= Out[5]= Out[6]= Introduction to equation solving for one equation One of the most important problems in mathematics and applied mathematics is solving (systems of) equations, which might be linear equations, nonlinear equations, differential equations, and so on. The simplest case should be familiar: solving a single equation such as a x=b or x^2+b x+c=0, where a, b, and c are supposed to represent given real numbers and the problem is to solve for the variable x. In elementary mathematics, we learn how to solve some of these equations. Indeed many problems reduce to solving the two types just mentioned---a single linear equation or a single quadratic equation. At least, these are the most common algebraic equations that are solvable by hand. As you might know, cubic and quartic polynomial equatipons can be solved by formulas discovered during the Italian renaissance (circa 1530). But, mathematicians have proved that there is no formula for finding roots of quintic polynomials or polynomials of higher degree.

2 2 UAlberta.nb By now you would have been introduced to transcendental functions (trig functions, exponential functions, logarithms, and perhaps some special functions). Most equations that involve such functions are not solvable with pencil and paper. For example, there are no formulas for solving equations such as tan(x)=2 x, exp(x)=x+2, or log(x)= a x+1. Thus, even for one equation in one real unknown, the best we can do is to approximate solutions. Let s have some fun and see what Mathematica can do when asked to solve equations. As in several modern computer algebra systems, Mathematica can solve all the equations for which formulas exist. Let s try ax=b. [To run the next cell click on the innermost square bracket along the right-hand side of the page and then press shift+return] In[7]:= Solve[a x b, x] Out[7]= x b 2640 No surprise here. Mathematica returns the correct answer. But, note that it does not tell us that there is no solution when a=0. It seems that there is still room for some education in mathematics. How about quadratics? In[8]:= Solve[a x^2 + b x + c 0, x] Out[8]= x -b - b c 5280, x -b + b c 5280 Not bad. Clearly, Mathematica knows the quadratic formula. Cubics? Let s try a special type of cubic: In[9]:= Solve[a x^3 + b x + c 0, x] Out[9]= x b 55 1/3-990 c + 55 b c 2 1/ c + 55 b c 2 1/3 55 2/3, x 1 + i 3 b /3-990 c + 55 b c 2 1/3-1 - i c + 55 b c 2 1/ /3, 1 - i 3 b x /3-990 c b c 2 1/3 1 + i c + 55 b c 2 1/ /3 The formulas are complicated, especially by the appearance of the complex number i. As mentioned

3 UAlberta.nb 3 The formulas are complicated, especially by the appearance of the complex number i. As mentioned before, cubics and quartics are solvable by formulas---usually called Cardano s formula; quintics are not: In[10]:= Solve[a x^5 + b x + c 0, x] Out[10]= x Root c + b # #1 5 &, 1, x Root c + b # #1 5 &, 2, x Root c + b # #1 5 &, 3, x Root c + b # #1 5 &, 4, x Root c + b # #1 5 &, 5 Mathematica says the roots are the roots! Even when formulas for roots exist (which is a wonderful theoretical fact), these formulas are not always practical for determining the nature of the roots. For example, Cardano s formula for solving cubics does not always give the real root in a form where it is easy to tell if it is positive or negative, greater than 7, and so on. Of course, computations with complicated formulas is practical on a computer at least when the computation is only needed once. When the computation must be done many times, the time it takes the computer to finish can become a major consideration. Putting these issues aside for the moment, the exact roots returned from Cardano s formula are evaluated numerically in the next code block. There is one real and two complex roots. Note that the computer must extract square and cube roots to obtain numerical values. How does the computer accomplish this task? [The Mathematica command Q//N asks the computer to evaluate the quantity Q on the left to a numerical value] In[11]:= ans = Solve[ x^3 - x + 1 0, x] ans // N Out[11]= x / /3 2 -, 3 2/3 x 1 + i / i /3 2 2/ /3, x 1 - i / i /3 2 2/ /3 Out[12]= {{x }, {x i}, {x i}} To extract a real cube root of some real number a, the problem is to find the real root of the cubic equation x^3=a. For definiteness, let s take the number 88. What is its cube root? The answer is obviously some number between 4 and 5. One way to do this is to draw a graph to see where the root resides. This could be done roughly by hand with pencil and paper and some graph paper. Of course, Mathematica can make a beautiful graph:

4 4 UAlberta.nb In[13]:= Plot[x^3-88, {x, 4, 5}, Frame True] Out[13]= By cubing numbers when no computer is available, or by graphing over a smaller interval, the search can continue. One way to proceed is to check the number halfway between 4 and 5 to see if the value of the function f(x)=x^3-88 is positive or negative. In fact, f(4.5)>0. Using this fact, the root must be between 4 and 4.5; or in other words, the root must lie in the interval (4,4.5). The process can be repeated to better approximate the root. We could take the current approximation of the root to be 4.5. The next approximation is obtained by taking the number halfway between 4 and 4.5 (which is(4+4.5)/2=4.25) and proceeding in the obvious way. The next approximation is So far, the sequence of computed approximations are 4.5, 4.25, and What is the next approximation? It is not too difficult to write a computer program to continue what is called the bisection method for approximating the root of an equation. This algorithm is guaranteed to converge to a root---maybe not the desired root; and at each step, the error of the approximation is no larger than half of the length of the interval where the root resides. This algorithm is viable; but the method is slow to converge. You can play with this process by hand as in the next few Mathematica cells or you could try to write a program to do the computations for you. In[14]:= ( ) / 2 Out[14]= 4.25 In[15]:= (4.25)^3-88 Out[15]= In[16]:= ( ) / 2 Out[16]= In[17]:= 88^(1 / 3) // N Out[17]= In[18]:= 4.375^3-88 Out[18]=

5 UAlberta.nb 5 In[19]:= ( ) / 2 Out[19]= Is there a faster algorithm to compute this root? As you might expect there are many possible ways to proceed. The most famous (and usually the best) method is one that you might have learned when studying first semester calculus: Newton s method. Newton s idea for inventing this algorithm was probably from a geometric interpretation. But, a simple way to derive the basic formula is to consider a Taylor approximation (which Newton surely understood before Taylor s or Maclaurin s name was attached). In any case, suppose we desire to find a zero of the function f; that is, we wish to solve the equation f(x)=0. Let us suppose a zero x=a exists so that f(a)=0. For some x near a, Taylor s formula is f(a)=f(x)+f (x) (a-x)+f (ξ) (a-x)^2, where ξ is some number between x and a. Assume x is so close to a that the term f (ξ) (x-a)^2 can be neglected in the equation. This gives the approximation f(a)=f(x)+ f (x)(a-x). Because f(a)=0 by assumption, solving for a gives an approximation of a with respect to the point x: a=x-f(x)/f (x). In other words, Newtons approximation of the desired and unknown root a is x-f(x)/f (x). Clearly the process can be repeated. Starting from some approximation x 0, the (n+1)th iterate is x n+1 = x n - f(x n )/f (x n ). How does this work in practice? To test out a new method, a good idea is to choose a problem where the answer is known. Let s approximate the cube root of 125. For the function f(x)=x^3-125, and starting value x 0 =4, the iterates are obtained easily in Mathematica using the function F(x):=x+f(x)/f (x): [Mathematica functions use square brackets f[x], not f(x) and the defintion of a function requires specifying the variable by the underscore mark. In the next formula the variable is x and it is designated by x_. The second command used to define the function F uses f [x] on the right hand side. Mathematica understands f to mean the derivative of the function f already defined. In particular, Mathematica knows how to compute derivatives.] In[20]:= f[x_] := x^3-125; F[x_] := x - f[x] f'[x]; In[22]:= {F[4], F[F[4]], F[F[F[4]]]} // N Out[22]= { , , } We are not taking advantage of the computer yet. Too much handwork. Computers are really good at iteration! One way to take advantage of this fact is embodied in the next code block, which uses a doloop. [Do-loops are not always the best way to program interations in Mathematica, but they are easy to understand. An empty list is defined by iterates={}. Every time a new iterate is computed it is appended as a new entry in the existing list. Simply typing the name of the variable, in this case iterates, causes

6 6 UAlberta.nb as a new entry in the existing list. Simply typing the name of the variable, in this case iterates, causes Mathematica to print out the list.] In[23]:= f[x_] := x^3-125; F[x_] := x - f[x] f'[x]; x = 4; iterates = {}; Do[{y = F[x], AppendTo[iterates, y // N], x = y}, {n, 1, 4}] iterates Out[28]= { , , , 5.} Note that Newton s method converges very rapidly to the exact solution. For the original problem, the same code can be used to approximate the cube root of 88: In[29]:= f[x_] := x^3-88; F[x_] := x - f[x] f'[x]; x = 4; iterates = {}; Do[{y = F[x], AppendTo[iterates, y // N], x = y}, {n, 1, 4}] iterates Out[34]= {4.5, , , } Compare the convergence rate to the bisection method. Do you see the difference: Newton s method is much faster. After just two iterates the approximation is correct to 2 decimal points and after three iterates it is correct to 5 decimal points. Newton s method is applicable whenever f is a differentiable function such that the derivative of f does not vanish at the desired root. Of course, it works for transcendental equations as well. For example, to approximate a solution of the equation tan(x)= 2 x, we proceed as follows: In[35]:= f[x_] := Tan[x] - 2 x; F[x_] := x - f[x] f'[x]; x = 0.2; iterates = {x}; Do[{y = F[x], AppendTo[iterates, y // N], x = y}, {n, 1, 5}] iterates Out[40]= 0.2, , , , 0., 0. The iterations are converging to 0, which is indeed a zero of the function f. How about finding x such that of tan(x)=x? From trigonometry x=0 is again a solution. In[41]:= f[x_] := Tan[x] - x; F[x_] := x - f[x] f'[x]; x = 0.2; iterates = {x}; Do[{y = F[x], AppendTo[iterates, y // N], x = y}, {n, 1, 5}] iterates Out[46]= {0.2, , , , , } The iterates seem to be converging to zero, as they should, but the convergence is slow. What is wrong with applying Newton s method in this case? A perhaps more interesting problem is to find the zero of f that lies between 0 and π/2. The problem is

7 UAlberta.nb 7 A perhaps more interesting problem is to find the zero of f that lies between 0 and π/2. The problem is to start close enough to this root. In[47]:= f[x_] := Tan[x] - 2 x; F[x_] := x - f[x] f'[x]; x = 1.5; iterates = {x}; Do[{y = F[x], AppendTo[iterates, y // N], x = y}, {n, 1, 7}] iterates Out[52]= {1.5, , , , , , , } We can check the values of f on the computed iterates to see that they are indeed converging to zero. In[53]:= residuals = Table[f[iterates[[i]]], {i, 1, 7}] Out[53]= { , , , , , , } We can also make a plot to see the convergence of the iterates. In[54]:= ListPlot[iterates] Out[54]= The residuals are plotted next using a few fancy additional commands. In[55]:= ListPlot[residuals, Frame True, PlotRange {{0, 8}, {-1, 12}}, PlotStyle {Red}] Out[55]= Approximate the solution of the equation cos x=x that lies in the interval [0,π/2]. Are you convinced that Newton s method is useful?

8 8 UAlberta.nb Are you convinced that Newton s method is useful? The fast convergence of Newton s method can be explained using calculus without too much difficulty. This is done in the course notes that accompany this Notebook. Equation solving for more than one equation Many real world applications involve solving a system of equations for more than one unknown. This subject is often called solving simultaneous equations. The most important special case is solving simultaneous linear equations, for example, the system of three equations in three unknowns x+y+z=1 2 x+y+z=2 x+2y+z=3 Mathematica can solve this system exactly. In[56]:= Clear[x, y, z] Solve[{x + y + z 1, 2 x + y + z == 2, x + 2 y + z == 3}, {x, y, z}] Out[57]= {{x 1, y 2, z -2}} Can you solve the system by hand? As you might know, there is a beautiful and useful algorithm for solving a system of linear equations called Gaussian elimination. Its analysis, usefulness, and limitations is one of the main problems that leads to a vast and important subject called numerical linear algebra. This subject will be considered later in your education. This section is about solving nonlinear systems of equations. But, as we will see, sometimes solving linear systems is part of the algorithm for solving nonlinear systems. We will avoid too much complication by solving only a few equations in a few unknowns and using black box linear solvers Via satellite (for example) elevations are mapped over a region of the earth s surface and a function is fit to this data to produce an approximation to the three-dimensional terrain. (How do you suppose the fitting is done?) The result is the function given in the next cell, where a=0.01 and b=0.01. A 3D plot of the function is also provided.

9 UAlberta.nb 9 In[58]:= F[x_, y_, a_, b_] := (a x)^3-3 a x b^2 y^2-0.1 a^4 x^4 + b^4 y^ Plot3D[F[x, y, 0.01, 0.01], {x, -1000, 1000}, {y, -1000, 1000}] Out[59]= The next cell is commented out using (* and *), a useful way to insert comments in active cells. The commands were used to export the 3-d figure for inclusion as an encapsulated postscript file in the course notes. You don t need to worry about understanding this procedure until you want to write your own documents! In[60]:= (* SetDirectory["~/Box Sync/Alberta2016"] Export["mountainterrain.eps",Out[6]]*) Let s continue work on the applied problem concerning the mapped terrain. An observer resides at the point on the terrain with coordinates (-550, 10, ), where all distances are measured in meters. The first coordinate measures the east-west direction with the positive direction pointing east. Likewise, the second coordinate measures north-south with the positive direction pointing north. Here are some problems: (1) Determine the curve of points that are 700 meters away via laser shots in three dimensions from this observation point and lie farther north. A much harder problem is to determine the curve of points that is 700 meters away when distance is measured along the terrain. This would make a nice future project. (2) Is there a point on the terrain equidistant via laser shots from the first observation point, the second observation point (-900, 990, ), and third observation point (-300, 250, )? If so, give the coordinates of such a point. (3) How many solutions of (2) exist within the mapped terrain? (4) What happens for (2) in case the third observation point is (1000, 0, 1501)? (5) What is the distance from observation point 1 to observation point 2 along the terrain? Questions (1), (3), (4) and (5) are left as exercises. Exercise (5) requires some new ideas that are not presented in this Notebook. The second question boils down to solving a system of two equations. They state the equality of distances from the observation points to the desired location with unknown coordinates (x,y,z). Well almost. In such problems the distance involves a square root. Usually the best idea is to measure the

10 10 UAlberta.nb almost. In such problems the distance involves a square root. Usually the best idea is to measure the square of the distance to avoid computing the square root. In this problem there are three unknown x, y, and z. But, the desired point is on the terrain. So, only two equations are needed: the difference of the distances from the unknown point to observation points 1 and 2 set to zero and likewise for observation points 1 and 3. To use Newton s method, the left-hand sides of the two equations are defined as functions g and h so that the problem is converted to finding a zero of the function (x,y)->(g(x,y),h(x,y)). The functions g and h are defined in the next cell. In[61]:= g[x_, y_] := (x + 550)^2 + (y - 10)^2 + (F[x, y, 0.01, 0.01] )^2 - (x + 900)^2 + (y - 990)^2 + (F[x, y, 0.01, 0.01] )^2 h[x_, y_] := (x + 550)^2 + (y - 10)^2 + (F[x, y, 0.01, 0.01] )^2 - (x + 300)^2 + (y - 250)^2 + (F[x, y, 0.01, 0.01] - F[-300, 250, 0.01, 0.01])^2 We have not yet discussed how to apply Newton s method when there is more than one equation. To do this requires the notion of a partial derivative. You should have already learned to use partial derivatives in your calculus class, but if not, don t worry. The idea is very simple: Consider a function f that depends on the two variables x and y. Its partial derivative with respect to x is simply the derivative of the function x->f(x,y) with y held fixed. Likewise the partial derivative of f with respect to y is the derivative of the function y -> f(x,y) with x held fixed. One notation for the partial derivative is to use subscripts corresponding to the name of the variable with respect to which the derivative is taken. So, f x denotes the partial derivative of the function f with respect to x and f y its partial derivative with respect to y. The values of these functions at some point (x,y) are of course denoted by f x (x,y) and f y (x,y), respectively. For example, let f(x,y)=x^2+ 2 y^3. For this function f x (x,y)=2 x and f y (x,y)=6 y. Can you imagine how to define partial derivatives for a function of three or more variables? What is f w (2,1,-1,1) in case f(x,y,z,w)=x y+e^z -zw+w^2? Newton s method in several variables applies when we seek a simultaneous zero for n functions of n variables. In the case at hand we have two functions of two variables. Just like for one variable functions where f(a)=f(x)+f (x) (a-x) gives the linear approximation for f(a), for functions of two variables the linear approximation (of g(a,b) with respect to a nearby point (x,y) is g (a, b) = g (x, y) + g x (x, y) (a - x) + g y (x,y)(b-y) and likewise for h we have h (a, b) = h (x, y) + h x (x, y) (a - x) + h y (x,y)(b-y). As before, suppose that the desired simultaneous zero is (a,b) so that g(a,b)=0 and h(a,b)=0. Then Newton s approximation for this unknown point with respect to some nearby point (x,y) is obtained by solving the two equations g (x, y) + g x (x, y) (a - x) + g y (x,y)(b-y)=0 h (x, y) + h x (x, y) (a - x) + h y (x,y)(b-y)=0 for a and b. For the one variable case, the corresponding equation f(x)+ f (x)(a-x)=0 is solved for a to obtain the formula for Newton iterations. Likewise, an explicit solution can be obtained in the multivariate case because the simultaneous equations are linear equations for the unknowns a and b. As remarked

11 UAlberta.nb 11 because the simultaneous equations are linear equations for the unknowns a and b. As remarked earlier, methods for solving linear equations are important. As an exercise, you should solve these equations by hand. The notebook s author has solved the equations by hand at least once in his life! So, to show you the answer, he will use the computer to display the answer. [ Note: Subscripts do not denote partial derivatives in Mathematica, but they can be used as parts of function names as in the next cell.] In[63]:= Clear[g, h, a, b] eqs = { g [x, y] + g x [x, y] (a - x) + g y [x, y] (b - y) 0, h [x, y] + h x [x, y] (a - x) + h y [x, y] (b - y) == 0} vars = {a, b} Solve[eqs, vars] Out[64]= {g[x, y] + (a - x) g x [x, y] + (b - y) g y [x, y] 0, h[x, y] + (a - x) h x [x, y] + (b - y) h y [x, y] 0} Out[65]= {a, b} Out[66]= a - h[x, y] g y[x, y] - x g y [x, y] h x [x, y] - g[x, y] h y [x, y] + x g x [x, y] h y [x, y], g y [x, y] h x [x, y] - g x [x, y] h y [x, y] b - -h[x, y] g x[x, y] + g[x, y] h x [x, y] - y g y [x, y] h x [x, y] + y g x [x, y] h y [x, y] g y [x, y] h x [x, y] - g x [x, y] h y [x, y] As you can see the explicit solution is complicated. It gets worse as the number of variables is increased. In practice, when approximations are sought, the explicit solution is almost never used. Applied mathematicians have developed fast, efficient and stable methods for approximating solutions of linear equations. The best idea is to use these methods when employing Newton s method. The usual algorithm is set up as follows. (1) Define a new variables X=x-a and Y=y-b. (2) Rearrange the linear equations to the form g x (x, y) X + g y (x, y) Y = g(x, y), h x (x, y) X + h y (x, y) Y = h(x, y). (3) Pick a starting value x 0, y 0 ) and suppose n-1 iterations have already been computed. The nth iteration is performed by solving the linear system g x (x n-1, y n-1 ) X + g y (x n-1, y n-1 ) Y = g(x n-1, y n-1 ), h x (x n-1, y n-1 ) X + h y (x n-1, y n-1 ) Y = h(x n-1, y n-1 ), for X and Y. Note that the linear system (once the functions are evaluated) is in the standard form a 11 X + a 12 Y= b 1, a 21 X + a 22 Y= b 2.

12 12 UAlberta.nb Using the computed values X and Y, the updated iterate is (x^n, y^n) = x n-1 -X, y n-1 -Y). Here a and b are replaced by the last approximations of these values. The algorithm is not complicated in principle. In practice, the main difficulty is computing the partial derivatives. This can sometimes be done by hand. One alternative is to use the computer to perform this task. In some situations the equations to be solved are so complicated that it is not feasible to compute derivatives. When faced with this problem, alternatives to Newton s method are used. The next few cells in this Mathematica Notebook implement Newton s method using a black box to solve the linear equations at each step of the iteration. You will have to eventually fill in this gap by learning the algorithms used by the programmers to approximate solutions of systems of linear equations on a computer. As usual, the code is first tested on an example where the answer is known. Don t forget this important step when you write your own programs. The code presented here is meant to be easy to understand. Much more efficient code can be written in the Mathematica programming language. Of course, there is a built in Mathematica function (called FindRoot) that can be used to approximate the desired solution using one line of code. But, don t forget that someone had to write a program using Newton s method (or one of its variants) to make this user friendly black box command. We are not interested in the answer; instead, the idea is to learn methods that are used to make these black boxes. The next cells are used to define a test problem and to implement Newton s method. [The command D[g[x,y],x] computes the partial derivative of g(x,y) with respect to x. Semicolons tell Mathematica to not print out the result of the assignment or computation. The variable sol is assigned to the output of the Mathematica command black box linear equation solver LinearSolve, where the coefficients of the linear equations and their right-hand sides are specified. The output is a list called sol, denoted by curly brackets in Mathematica, with two elements. The notation sol[[1]] specifies the first element of the list. ] Here we go... In[67]:= Clear[g, h, x0, y0] g[x_, y_] := x^2 + y^2-1 h[x_, y_] := x - y a11[x_, y_] = D[g[x, y], x]; a12[x_, y_] = D[g[x, y], y]; a21[x_, y_] = D[h[x, y], x]; a22[x_, y_] = D[h[x, y], y];

13 UAlberta.nb 13 In[74]:= x0 = 0.5; y0 = 0.5; list = {{x0, y0}}; Do[{ sol = LinearSolve[{{a11[x0, y0], a12[x0, y0]}, {a21[x0, y0], a22[x0, y0]}}, {g[x0, y0], h[x0, y0]}], x0 = x0 - sol[[1]], y0 = y0 - sol[[2]], AppendTo[list, {x0, y0}]}, {n, 1, 7}] list Out[78]= {{0.5, 0.5}, {0.75, 0.75}, { , }, { , }, { , }, { , }, { , }, { , }} In[79]:= The code produces the correct result, which should be one of the two solutions (1/Sqrt[2],1/Sqrt[2]) or (-1/Sqrt[2],-1/Sqrt[2]). Both solutions can be found via Newton s method by judicious choices of starting values. What about the order of convergence? The Mathematica command Norm computes the distance between points for us. exact = {1 / Sqrt[2], 1 / Sqrt[2]}; Table Norm[list[[i + 1]] - exact] Norm[list[[i]] - exact], {i, 1, Length[list] - 1} Power::infy: Infiniteexpression 1 0. encountered.. Out[80]= , , , , , 0., ComplexInfinity The sequence of ratios seems to be converging to zero at least over the first 4 elements. After that the approximations are so good that the denominator is almost zero and the ratios are not reliable. There is a lesson here in numerical analysis: noise sets in when errors are close to machine precision. Let s see about 2nd order convergence discussed in the course notes. In[81]:= Table Norm[list[[i + 1]] - exact] Norm[list[[i]] - exact]^2, {i, 1, Length[list] - 1} Power::infy: Infiniteexpression 1 0. encountered.. Out[81]= , , , , , 0., ComplexInfinity The convergence seems to be toward a number close to 0.5 over the first 4 iterations until the suspected noise sets in. This would indicate 2nd order convergence as predicted by the theory in the class notes. Of course, this interpretation is not rigorous. Maybe the convergence is to zero before the noise sets in and this particular example has higher-order convergence. How could we tell? The answer is we can NEVER tell by numerical experiments alone. The results of the experiments might provide evidence for some conclusion as they do here. To be sure of the convergence order requires a mathematical proof. Two comments: (1) We could compute in multiple precision (that is, by carrying a lot of digits after the decimal point in the calculations) so that the onset of noise is delayed in the interation process). (2) We could check convergence rates from multiple starting values.

14 14 UAlberta.nb Continuing with the applied problem, we can go after the point equidistant from the three observation points. Before we do, there is something important to consider: Does a solution exist? As a general rule, we should not try to approximate the solution of a problem unless we know that a solution exists. Also, in most physical problems---but not this one---there should be no more than one solution. For this reason, applied mathematicians put a lot of effort into showing that unique solutions exist for their model equations. This is the ideal situation. In real life, proving that a unique solution exists is often beyond present understanding. When this occurs, numerical experiments can be used to explore for solutions. When reliable algorithms are used, numerical experiments can provide useful information. For example, when approximate solutions are found, there is strong evidence to believe that solutions actually exist. Often the importance of the problem determines whether numerical evidence is sufficient. To be sure, a mathematical proof is required. In[82]:= Clear[F, g, h, x0, y0] F[x_, y_, a_, b_] := (a x)^3-3 a x b^2 y^2-0.1 a^4 x^4 + b^4 y^ g[x_, y_] := (x + 550)^2 + (y - 10)^2 + (F[x, y, 0.01, 0.01] )^2 - (x + 900)^2 + (y - 990)^2 + (F[x, y, 0.01, 0.01] )^2 h[x_, y_] := (x + 550)^2 + (y - 10)^2 + (F[x, y, 0.01, 0.01] )^2 - (x + 300)^2 + (y - 250)^2 + (F[x, y, 0.01, 0.01] - F[-300, 250, 0.01, 0.01])^2 a11[x_, y_] = D[g[x, y], x]; a12[x_, y_] = D[g[x, y], y]; a21[x_, y_] = D[h[x, y], x]; a22[x_, y_] = D[h[x, y], y]; In[90]:= x0 = 0.0; y0 = 0.0; list = {{x0, y0}}; Do[{ sol = LinearSolve[{{a11[x0, y0], a12[x0, y0]}, {a21[x0, y0], a22[x0, y0]}}, {g[x0, y0], h[x0, y0]}], x0 = x0 - sol[[1]], y0 = y0 - sol[[2]], AppendTo[list, {x0, y0}]}, {n, 1, 5}] list Out[94]= {{0., 0.}, { , }, { , }, { , }, { , }, { , }} Newton iterations seem to converge rapidly, which strongly suggests that the location with coordinates ( , , ) is equidistant from the three observation sites. Also, this application of the algorithm does not seem to be very sensitive to the choice of the starting value. This is not always the case. You should try some other starting values to see that the method converges to the same point. Using the graphical power of Mathematica, additional evidence for the correctness of the result is provided by the graphs shown in the next cell where g, h, and the function whose value is zero are plotted together.

15 UAlberta.nb 15 In[95]:= Plot3D[{g[x, y], 0, h[x, y]}, {x, -797, -794}, {y, 590, 597}, PlotRange All] Out[95]= As mentioned, Mathematica has a built in black box root finder. Just for fun, let s note that it produces exactly the same result. In[96]:= FindRoot[{g[x, y] 0, h[x, y] 0}, {x, 0.0}, {y, 0.0}] Out[96]= {x , y } The Mathematica command FindRoot is a much more sophisticated program than our simple implementation of Newton s method. The documentation for FindRoot says: If you specify only one starting value of x, RootFind searches for a solution using Newton methods. At least this should make you feel that the method you are learning in this course is the method of choice for state-of-the-art commercial software. Diffusion and the Heat Equation We would like to approximate solutions of the heat equation u t =κ u xx, defined on a finite spatial interval with boundary conditions and initial data. As discussed in the course notes, the basic idea is to discretize space and then time-step forward using Euler s method. In doing this, the boundary data must be taken into account. In the literature of numerical analysis, the idea just described is called the method of lines. You should know that Euler s method is the simplest possible choice for timestepping. Many other methods for approximating solutions of ordinary differential equations are known. For a simple example, consider an insulated rod where no heat flows in or out along the rod or through its ends. The rod is heated to some temperature profile, the heat source is removed and the rod is left to cool. What will happen? Our physical intuition is strong. Since no heat leaves the system, the amount of heat---whatever that is---must remain constant. Also, becasue heat diffuses from higher to lower temperatures, after a long time has passed all points on the rod will have the same temperature. Maybe we can prove these facts are predicted by Fourier s model using pencil and paper. Always remember that mathematical models are not exact representations of reality. But, to be a useful model, we certainly want our basic physical intuition to agree with predictions derived from the model.

16 16 UAlberta.nb Let s see if numerical experiments agree with our intuition. Assume our rod is one unit long. By choosing coordinates, we may as well make life simple by assuming the rod is coordinatized by the unit interval [0,1]. We can and should try a range of initial data. Because we might wish to compare our numerical experiments with Fourier series solutions, let s take the initial temperature to be given by the function f given by f(x)=2+cos π x defined for x in [0,1]. A reasonable model for no heat flow at the ends of the rod is provided by zero Neumann boundary conditions. Again for simplicity, the initial data is chosen to agree with these boundary conditions; indeed, the derivative of f vanishes at the end points of the interval. As a basic rule of numerical computation, code should be tested on examples where the exact answer is known. There are other rules! Don t start with a discretization that is too fine. It is better to start with a coarse discretization, then make refinements to see if the numerical algorithm produces results that converge to some numerical approximation as the discretization is refined. There are lots of different ways to implement the basic finite difference algorithm that has been discussed. Remember that the spatial second derivative is approximated by a second-order accurate approximation. Using Euler s method for time-stepping reduces the accuracy of the numerical method because it uses a first-order approximation to the time derivative. Undetered, let s try thes approximations to see what happens. One more thing before we do: To implement the zero Neumann boundary conditions let s try to make the temperature constant near the end points so that its spatial derivative would vanish. The discrete approximation of the temperature is called U. It is a vector of some finite length n+1. The value U[1] corresponds to the temperature at x=0 and U[n+1] is the temperature at x=1. In the algorithm, the set of values U[i], for i=2,3,4,..,n, is updated at each time step. When U[1] is needed in a calculation it is replaced by U[2] to respect the zero Neumann boundary condition, and for the same reason, U[n+1] is replaced by U[n]. After each timestep the new U[1] is replaced by U[2] and U[n+1] by U[n]. In the next code block n=15. The time step is taken to be rather small Δt= so that the flow of heat is not too fast to see in an animation. At least that was one of the reasons for this choice. There is another reason why Δt should not be chosen too large: the numerical computations (as discussed in the course notes) become unstable; that is, the the approximation vector U will not follow the true solution and large values of its components might appear. The exact bound for Δt is given by the CFL condition discussed in the course notes. Each step of the following code should be read and understood. In esssence a vector function of a vector variable is iterated at each time step and the values of the vector U at each step are stored in the list called Solution. The variable L (set to L=1 in the program) is the length of the rod. It could be changed

17 UAlberta.nb 17 In[97]:= L = 1; n = 15; Δx = 1.0 L / n; Δt = ; f[x_] = 2 + Cos[ π / L x]; (* Initial Data *) (* f[x_]=2+cos[ π/l x]+3 Cos[2 π /L x];*) (* f[x_]:=piecewise[{{2.0,x<0.2},{0.0,x>=0.2}}]*) Solution = {}; κ = 1; CFL = κ Δt Δx^2; Print "The CFL condtion requires κ Δt/Δx^2<1. Here this number is ", κ Δt Δx^2 tmax = 1500; (* Number of timesteps *) xvec = Table i - 1 Δx, {i, 1, n + 1} ; (* Discretized positions on rod *) U = Table[f[xvec[[i]]], {i, 1, n + 1}]; (* Initial discrete temperature distribution *) (* Position,Temperature initial data*) Udata = Table[{xvec[[i]], U[[i]]}, {i, 1, n + 1}]; AppendTo[Solution, Udata]; (* Udata as first element of Solution list *) Do (* Update U[[2]] as first element of a list called Utemp*) Utemp = U[[2]] + κ Δt Δx^2 ( U[[2]] - 2 U[[2]] + U[[3]]), Utable = Table U[[i]] + κ Δt Δx^2 U[[i - 1]] - 2 U[[i]] + U[[i + 1]], {i, 3, n - 1}, Utemp = Flatten[Append[Utemp, Utable]], Utemp = Append Utemp, U[[n]] + κ Δt Δx^2 (U[[n - 1]] - 2 U[[n]] + U[[n]]), Utemp = Append[Utemp, U[[n]]], (* U[[n+1]] set to U[[n]] for Neumann BC *) U = Prepend[Utemp, U[[2]]], (* U[[1]] set to U[[2]] *) (* Updated discrete temperature distribution *) Udata = Table[{xvec[[i]], U[[i]]}, {i, 1, n + 1}], (* Udata saved as next element of Solution list *) AppendTo[Solution, Udata], {j, 1, tmax} The CFL condtion requires κ Δt/Δx^2<1. Here this number is The next cell might execute slowly depending on the speed of your computer. It plots and animates the changing temperature distribution.

18 18 UAlberta.nb In[112]:= ListAnimate[Table[ListPlot[Solution[[j]], Frame True, Axes False, PlotRange {{0, L}, {0, 3}}], {j, 1, tmax}]] Out[112]= The animation of the changing approximate temperature along the rod seems to agree with intuition. At least the temperature distribution along the rod seems to converge to a constant function whose value is about 2. The number, 2, must have something to do with the amount of heat in the rod. The average temperature is, of course, the integral of the temperature over the interval representing the rod divided by the length of the interval. This quantity remains constant over time. Can you prove this fact? When you want to show that a changing quantity is actually a constant, calculus sometimes comes to the rescue. In fact, if the derivative of a function is the zero function, then the original function is constant. Check that the average value of the intial data is 2. Just for fun, what if we added another term, say 3 Cos[ 2 π / L x], to the initial data? By the above reasoning, the temperature of the rod should again settle down to a constant temperature of 2 units. Does it? Using slight modifications to our code, which you may wish to translate to another programming language, you should try to see what happens when the temperature at one or both ends is held constant. This type of boundary condition is called a Dirichlet boundary condition. Additional Interesting Exercise: Rod heated periodically at one end and insulated at the other. Imagine heat flow in and out of one of the ends of an insulated rod according to some preassigned function of time. For example, consider a periodic function of time that specifies a changing temperature at one end of the rod and suppose the other end of the rod is insulated. We woud expect the temperature in the rod to fluctuate periodically. How does the temperature at the insulated end fluctuate? Perform some numerical experiments and try to make some general statement about how the input

19 UAlberta.nb 19 Perform some numerical experiments and try to make some general statement about how the input temperature affects the (output) temperature at the other end of the rod. Can you prove your conjecture(s) about the temperature fluctuation at the insulated end using Fourier series or by other means? The most basic quesiton you might ask: Is the output temperature periodic? An interesting question that you might ask: Can the maximum temperature at the insulated end of the rod ever exceed the maximum input temperature? The following code is set up to begin working on the exercise. The idea is that you can make changes in the code to perform other numerical experiments. In[113]:= L = 1.0; n = 15; dc = 100; (* Data is saved every dc steps *) Δx = 1.0 L / n; Δt = 0.001; g[x_] = 0.0; (* Initial data *) f[t_] = Sin[2 t]; (* Temperature at the left end of the rod x=0 *) Solution = {}; κ = 1.0; CFL = κ Δt Δx^2; Print "The CFL condtion requires κ Δt/Δx^2<1. Here this number is ", κ Δt Δx^2 tmax = ; (* Number of timesteps *) Print["The final time is ", Δt tmax] xvec = Table i - 1 Δx, {i, 1, n + 1} ; U = Table[g[xvec[[i]]], {i, 1, n + 1}]; Udata = Table[{xvec[[i]], U[[i]]}, {i, 1, n + 1}]; AppendTo[Solution, Udata]; Do U1 = f Δt j - 1, Utemp = U1 + κ Δt Δx^2 ( U1-2 U1 + U[[2]]), Utable = Table U[[i]] + κ Δt Δx^2 U[[i - 1]] - 2 U[[i]] + U[[i + 1]], {i, 2, n}, Utemp = Flatten[AppendTo[Utemp, Utable]], U = AppendTo Utemp, U[[n + 1]] + κ Δt Δx^2 (U[[n - 1]] - 2 U[[n]] + U[[n]]), Udata = Table[{xvec[[i]], U[[i]]}, {i, 1, n + 1}], If[Mod[j, dc] 0, AppendTo[Solution, Udata]], {j, 1, tmax} (* The input function f (blue) and the temperature at the right end (green) are plotted. The time interval for the plot is specified in PlotRange.*) ListPlot Table j - 1 dc Δt, f j - 1 dc Δt, {j, 1, Length[Solution]}, Table j - 1 dc Δt, Solution[[j, n + 1, 2]], {j, 1, Length[Solution]}, Frame True, PlotRange {{90, 100}, {-1.5, 1.5}}, PlotStyle {Blue, Green}

20 20 UAlberta.nb The CFL condtion requires κ Δt/Δx^2<1. Here this number is The final time is Out[131]= In[132]:= (* The computed points in this plot are connected with smooth curves. *) ListPlot Table j - 1 dc Δt, f j - 1 dc Δt, {j, 1, Length[Solution]}, Table j - 1 dc Δt, Solution[[j, n + 1, 2]], {j, 1, Length[Solution]}, Frame True, Joined True, PlotRange {{0, 1}, {-1.5, 1.5}}, PlotStyle {Blue, Green} Out[132]= Additional Interesting Exercise: Diffusion Coefficient for Molecules in Liquid Put some sugar at the bottom of a jar of water and don t stir the mixture. How long does it take for water drawn from the top of the jar to taste sweet? In reality, diffusion is a very slow process. This exercise is about some diffusion measurements. Sugar molecules are large; they diffuse very slowly. So we will consider some chemical element that diffuses faster---think copper sulfate, for example. The data discussed here is not accurate to experimental standards, but the sizes of the numbers involved are physically realistic. To be more precise, imagine a chemical element that dissolves in water and a mixture of 50 grams per liter of this element poured into the bottom of a round cylindrical flask. Pure water is added on top of the fluid mixture in a very careful manner so that no stirring takes place. The pure water has a depth of 10 centimeters above the surface Σ of the mixture. The flask has a small hole located exactly 2 centimeters above Σ that allows small samples to be extrated and tested for the chemical concentrations every 24 hours for 10 days. Here are the results given in the form (days, grams/liter of the chemical element)

21 UAlberta.nb 21 flask. Pure water is added on top of the fluid mixture in a very careful manner so that no stirring takes place. The pure water has a depth of 10 centimeters above the surface Σ of the mixture. The flask has a small hole located exactly 2 centimeters above Σ that allows small samples to be extrated and tested for the chemical concentrations every 24 hours for 10 days. Here are the results given in the form (days, grams/liter of the chemical element) In[133]:= data = {{1, 0.0}, {2, 0.0}, {3, 0.096}, {4, 0.594}, {5, 1.484}, {6, 2.598}, {7, 3.806}, {8, 5.031}, {9, 6.23}, {10, 7.382}} Out[133]= {{1, 0.}, {2, 0.}, {3, 0.096}, {4, 0.594}, {5, 1.484}, {6, 2.598}, {7, 3.806}, {8, 5.031}, {9, 6.23}, {10, 7.382}} The problem: What is the diffusion coefficient for the dissolved chemical element? Remember our diffusion model is u t =κ u xx, where is the concentration that can be taken in units of g/l. The diffusion coefficient is κ. It has units of area/time and is usually expressed in the units cm^2/sec. The answer to our problem should be given in these units. Hints: Set up boundary conditions for the diffusion equation and think about how the data would be obtained by solving your model system were the diffusion coefficient known. Then try to find the choice of diffusion coeffiient that produces concentrations that best fit the data. Ans: Approximately ^-6 cm^2/sec.