Homework 2. Matthew Jin. April 10, 2014

Transcription

1 Homework Matthew Jin April 10, 014 1a) The relative error is given by ŷ y y, where ŷ represents the observed output value, and y represents the theoretical output value. In this case, the observed output value is 0, as shown in the following code: format long a=sqrt(1+10^(-16)); b=a-1 % b represents the actual function output Output: b = 0 Therefore, ŷ y y = 0 y y = y y = 1. So the relative error of a naive implementation of f(x) is 1. 1b) The first four terms of the Taylor series for f about zero may be determined by writing out the first 5 terms of the binomial expansion of (1 + x ) 1/, and then subtracting 1: (1 + x ) 1/ = 1 + ( ) n x + 1 ( ) n x 4 + ( ) n x ( ) n x 8, where n = 1/. 4 Note that although combinatorials do not make sense for fractional n, the formula for combinatorics still applies as a shorthand way of writing the coefficients of the binomial expansion. After plugging in n appropriately, the expression comes out to 1

2 f(x) = 1 + x! x4 8 + x6 16 5x = x! x4 8 + x6 16 5x8 18 1c) For this part, focus on which of the operations could result in catastrophic cancellation, which would in turn result in a magnified relative error. Consider separately the operation 1 + x ; in the computer, 1 + x is evaluated as 1 + x (1 + ɛ 1 ) = 1 + x + ɛ x 1 + x + ɛ 1, because x is very small. Overall, the machine would evaluate the expression 1 + x 1 and output (excluding the other, less significant errors present among the entire operation) 1 + x + ɛ 1 1. The relative error of the machine would overall be, letting the first term of the Taylor series of f(x) centered at 0 to be the theoretical value of 1 + x, 1+x +ɛ x +1 x = ɛ1 x ɛ mach x. To find the x c for which the relative error is about 10 1 for all x > x c, evaluate 10 1 = x, which yields an x c of about x c is on the order of 10. 1d) One would need the first three terms of the expression. One allows the absolute error to be dominated by the first omitted Taylor series term and lets x to be the approximation of the theoretical answer. The absolute error of the 4th Taylor series term is the one that most nearly results in a 1 digit accuracy: ɛ relative = , which indeed implies roughly 1 digits of.0141 accuracy, if one uses a three-term Taylor series approximation. 1e) Letting x c =.0141, one implements the following MATLAB code: format long x=.0141; %this value represents the calculated x_c a=(x^)/-(x^4)/8+(x^6)/16 % a represents the three-term taylor % approximation of the function at x_c

3 b=sqrt(1+x^)-1 % b represents the naive implementation s %approximation of the function at x_c reldiff=abs((a-b))/b %reldiff represents the relative %difference between the taylor approximation and the naive implementation Output: a = e-05 b = e-05 reldiff = e-1 One can see that indeed, the two outputs from the two methods differ relatively only on the order of a) Using the function goodnewton developed by Dan Cianci, I run the following script: f Df=@(x) 3*x^; goodnewton(f,df,1i,1*10^(-8)) Output: x0 = i x0 = 3

4 i x0 = i x0 = i x0 = i ans = i So one can see that the Newton iteration converges on The third root is 1 3. The three roots appear as follows: The angle between the three roots is π 3. To calculate this, let the root be re iθ. Then to determine the roots of f(z) = z 3 1, evaluate z 3 = 1. For some r and θ, r 3 e 3iθ = 1. In order for this to be true, 3iθ must be equal to some integer multiple of π, and r 3 must be equal to 1, becuase re iθ = r(cos(θ) + isin(θ)). 4

5 Thus, r=1 and θɛ[0, π 3, 4π 3 ], for three solutions: 0, π 3, 4π 3. Note that the angle between consecutive roots is π 3. This is because θ is equal to n π 3, for integer n. As one increments n by one, the angle between the roots increases by π 3. Finally, one knows that there are only three unique roots because the increments of π 3 for θ larger than 4π 3 will just cause the roots to cycle back to the ones already noted. b) The problem requires one to find all z of the form re iθ such that z 5 = r 5 e 5iθ =. As in part (a), this implies that r = 1 5 and θɛ[0, π 5, 4π 5, 6π 5, 8π 5 ]. Overall, the solutions are 1 5 e 0i, 1 5 e i π 5, 1 5 e i 4π 5, 1 5 e i 6π 5, and 1 5 e i 8π 5. c) The following code was used to determine in which of the three basins each point in the complex plane lies: a=-:.01:; [X,Y]=meshgrid(a,a); %set up the grid of values over the range and increment of a f=@(x) x.^3-1; Df=@(x) 3*x.^; tol=.01; Z=goodnewton(f,Df,X+Y*1i,tol); %call goodnewton, a modified version of Dan Cianci s % goodnewton, to output the value on which newton s iteration converges Z=sign(angle(Z)); %if the angle is pi/3, return "1", %if angle is 0, return "0" %if angle is -pi/3, return "-1" surf(x,y,z) imagesc(a,a,z); The code for goodnewton is as follows: function [ r ] = goodnewton( f,df,x0,tol ) %goodnewton uses Newton iteration to find a root of f with initial %guess x0 to the specified tolerance tol % % %Inputs: 5

6 %f,df are the function and derivative of the function respectively %x0 the initial starting point of the iteration %tol specifies the relative error of the first iterate and the last %iterate % % %Outputs: %r- the computed root % %Cianci 4/1/014, edited by Matthew Jin 4/9/014 x1 = x0 - f(x0)./df(x0); i=0; %implement the iteration 100 times for k=1:100 x0=x1; x1 = x0 - f(x0)./df(x0); end r=x1; end The resulting plot appears as follows: If one decreases the range of the meshgrid parameter, one can zoom in on smaller portions of the plot: 6

7 Decreasing the range of the meshgrid parameter by 10x and then by 100x, one can see that the shape of the original graph is still retained, indeed confirming that this figure is a fractal: 7

8 3a) κ = f (x)x f(x), so for the function sin 1 (x), κ(x) = x 1 x sin 1 (x), which eval uated at x= yields κ( ) = sin 1 ( ) b) Applying the same formula for κ as above to ln(x), one finds κ(x) = x 1 x ln(x) = 1 ln(x). This expression is large when ln(x) is small, which occurs in the neighborhood of x=1. 3c) The roots of the function f(x) = x x + c are given by the quadratic formula: r = ± 4 4c = 1 ± 1 c. Again applying the formula for κ to r, one finds κ(c) = ±c 1 (1 c) 1 1± 1 c c = (1± 1 c) = c 1 c (. If c = 1 c±(1 c)) , then κ( ) = 16 ( = ±( )) (10 8 ± This yields κ( ) = or The roots are a distance of 10 8 apart. 3d) A backwards stable algorithm will evaluate sin(10 10 ) as sin(10 10 ) = sin(10 10 (1 + ɛ)) = sin( ɛ) = sin(10 10 )cos(10 10 ɛ) + cos(10 10 )sin(10 10 ɛ) Let ɛ = Then sin(10 10 ) = sin(10 10 )cos(10 6 ) + cos(10 10 )sin(10 6 ). To further simplify this problem, consider the Taylor series expansions of cos(x) and sin(x). For small x, sin(x) x. On the other hand, for small x, cos(x) 1 x. Therefore, sin(10 6 ) 10 6, and cos(10 6 ) 1. 8

9 From here, one can see that sin(10 10 ) = sin(10 10 ) + cos(10 10 )sin(10 6 ). The latter term clearly indicates the absolute error; to find the relative error, we evaluate cos(1010 )sin(10 6 ) sin(10 10 ) The backwards-stable algorithm would therefore probably be accurate to about 6 digits when computing sin(10 10 ). 4) First, use Taylor s theorem to expand f(x + h) and f(x h) around x. f(x + h) = f(x) + f (x)(x + h x) + f (x) (x + h x) + f (x) 3! (x + h x) 3 + f (q) (x + h x) 4 = f(x) + f (x)(h) + f (x) (h ) + f (x) 3! (h 3 ) + f (q) (h 4 ), for some q between x and x+h. Similarly, substituting in x-h instead of x+h to the Taylor series, one gets f(x h) = f(x) + f (x)( h) + f (x) ( h) + f (x) 3! ( h) 3 + f (p) ( h) 4 for some p between x and x-h Thus, f(x+h) f(x)+f(x h) h evaluates to (after cancelling out the odd powered terms, due to them having equal magnitude and opposite sign), f(x)+f (x)(h )+ f f (x)+( f (q) + f (p) )(h ). Notice that the latter term represents the difference between f (x) and the computer s execution of the finite difference formula it is the error. Also, notice that f (q) + f (p) is a constant. Therefore, it immediately follows that the error for the finite difference formula for f (x) is O(h ). To use this formula to approximate the second derivative of sin(5x) at x=1, I created the following function: (q) (h 4 )+ f (p) (h 4 ) f(x) h = % This function accepts an input of h, and implements the finite difference % formula for f (x) for sin(5x) at x=1. It then computes the relative error of % this approximation against the theoretical value. % result=hp4(h) % Matthew Jin April 08, 014 function result=hp4(h) approx=(sin(5+5*h)-*sin(5)+sin(5-5*h))./(h.^); %implement the finite difference formula fo result=abs((approx-(-5*sin(5))))/(-5*sin(5)); %compute the relative error end I then implemented this function in a separate test script as follows: 9

10 format long x=10.^[-16:.05:0]; hp4(x),x); %apply function hp4 to each element in array x loglog(x,myanswer) The output graph of epsilon with respect to h on a logarithmic scale for epsilon on h appears as follows: This graph is reasonable. The error due to rounding is approximately going to be on the order of O( ɛ mach h ). This is because the computational errors in the numerator of the finite difference approximation may in the worst-case scenario constructively add machine error, which would then be divided by h. Furthermore, as proved just above, the error caused by the formula for finite difference approximation is on the order of O(h ). To find an approximation for the h at which error is minimal, one might thus set h = ( ɛ mach h ). This implies that h 10 4 will result in the minimal error, which is indeed what we see on the graph, after allowing for the effect of constant multipliers on the relative significance of the error due to the formula vs. rounding error the bottom of the V occurs in the neighborhood of One can see that for h larger than approximately 10 4, the Taylor series error takes over, with a slope on the log-scale graph of approximately (in that region, the relative error increases approximately 10 orders of magnitude over an increase of approximately 5 orders of magnitude of h). This corroborates the result proven just above, in which the formulaic error was proven to be O(h ). For h smaller than roughly 10 4, the rounding error takes over, with the error increasing with a slope of roughly as h shrinks (again, the error increases about 10 orders of magnitude over a decrease of about 5 orders of magnitude of h). This corroborates the idea that the rounding error occurs on the order O( ɛ mach h ). For h smaller than one can see that the graph becomes somewhat unpredictable, yet still appears 10

11 to maintain the slope of -. 5a) This MATLAB code would ideally run 10 times, and then terminate. In reality, it is an infinite loop. This is because decimals fractions, such as 0.1, cannot be expressed precisely in the floating point system. The computer therefore introduces some rounding error with each addition of 0.1. By the 10th addition, the error is large enough that the value that the computer should compute to be 1 is not, in fact, precisely equal to 1. This can be shown by running through the loop ten times in debug mode and then checking the difference between the output that should be 1 and 1. Below is the slightly modified code: format long x=0; while x~=1 keyboard x=x+.1 end Here is the output and the check: \begin{verbatim}

12 K>> x-1 ans = e-16 Clearly, the computer perceives some difference between what is displayed as 1 and 1, due to the error compounded with each addition of 0.1. To make this code go a precise number of times, one might use integer values rather than decimals in the code. Because integers are elements of the floating point system, the rounding error would be avoided. format long 1

13 x=0; while x~=10 x=x+1 end b) The relative error can be computed by evaluating Plugging this directly into MATLAB, one finds a relative error ɛ Because log (19 1 ) is about 130.9, it immediately follows that one would need about 130 digits of binary to precisely handle integers of this size. I evaluate 19 1 because it is slightly larger than c) Matrix multiplication of n-by-n matrices involves n multiplications and n-1 additions for each of the product matrix elements (for a total of n-1 operations per element), due to the nature of the dot product. In the product matrix there are n elements, so to perform the entire matrix multiplication n (n 1) operations must be done. By the same argument, to multiply an n-by-n matrix with a column vector of length n, one must perform n(n-1) operations (n elements in the product, n-1 operations to compute each element). To compute the product ABx, one can either compute (AB)x or A(Bx). Computing (AB)x would require n (n 1) + n(n 1) operations, whereas computing A(Bx) would require n(n 1) operations. For any n > 1, A(Bx) requires fewer operations, thus reducing the chance for error and reducing the amount of time the program might take to perform the multiplication. Therefore, it is the better way of computing ABx. Attribution: I worked with Michael Downs and Matt Marcus to reach some of the solutions in this homework. I also adapted Dan Cianci s Newton s method code for problem. 13