Program 1 Foundations of Computational Math 1 Fall 2018

Transcription

1 Program 1 Foundations of Computational Math 1 Fall 2018 Due date: 11:59PM on Friday, 28 September 2018 Written Exercises Problem 1 Consider the summation σ = n ξ i using the following binary fan-in tree algorithm described below for n = 8 but which clearly generalizes easily to n =2 k : or equivalently σ = {[(ξ 0 + ξ 1 )+(ξ 2 + ξ 3 )] + [(ξ 4 + ξ 5 )+(ξ 6 + ξ 7 )]} σ (0) j = ξ j, 0 j 3 σ (1) 0 = ξ 0 + ξ 1, σ (1) 1 = ξ 2 + ξ 3, σ (1) 2 = ξ 4 + ξ 5, σ (1) 3 = ξ 6 + ξ 7 σ (2) 0 = σ (1) 0 + σ (1) 1, σ (2) 1 = σ (1) 2 + σ (1) 3 σ = σ (3) 0 = σ (2) 0 + σ (2) 1 In general, there will be k =logn levels and σ = σ (k) 0. Level i has 2 k i values of σ (i) j, each of which corresponds to a sum σ (i) j = ξ 2j ξ 2j+2 i 1. The algorithm is easily adaptable to n that are not powers of Derive an expression for the absolute forward error of the method for n =2 k Derive an expression for the absolute backward error of the method for n =2 k Bound the errors and discuss stability relative to the simple sequential summation algorithm given by, for n = 8 but easily generalizable to any n, or equivalently σ = (((((((ξ 1 + ξ 2 )+ξ 3 )+ξ 4 )+ξ 5 )+ξ 6 )+ξ 7 )+ξ 8 ) σ = ξ 1 σ σ + ξ i, i =2,...,8 1

2 Problem 2 Most recent machines use a binary base,β = 2, but the number of bits, t, may vary in a floating point system. Suppose you wanted to implement your own routines to determine the number of bits and therefore machine epsilon. The following algorithm is an attempt to determine t experimentally. x =1.5, u =1.0,t =0,α =1.0 while x>α u = u/2 x = α + u t = t +1 end 2.1. Assume that the floating point system has β = 2 and uses a hidden bit normalization. Does this algorithm find t? Does the method of rounding affect your answer? 2.2. Apply the algorithm to a machine that uses β = 2 in single precision and double precision using your code developed for the first part of the problem. Do your observations from the output of the code agree with the IEEE floating point standard for single and double precision floating point numbers information and with the information returned by the epsilon(x) function you implemented for the first part of the problem? 2

3 Programming Exercise Problem 3 An approach to organizing sequential approaches to summation of n numbers ξ i,0 i n 1 that easily facilitates the computation of a running error bound and the constant of unit roundoff u in the a priori bound based on the length of the longest path in the computational graph of the summation approach, i.e., n 1 in the standard recursive summation. We concentrate here on the running bound and summation. A somewhat imprecise form is Algorithm 1 (General Select-Sum-Insert Algorithm): E =0;u =unitroundoff; initialize the set L = {l i } with ξ i,0 i n 1 satisfying constraint 1 i =0 while the termination criteria are not met: if a pair (l, ˆl) satisfying constraint 2 exists then extract l and ˆl from L T i = l + ˆl update running error bound parameter E using T i insert T i into L satisfying constraint 3 else process L and update constraints based on failure to find pair (l, ˆl) end if i i +1 end while perform termination computations; return σ =Σ n 1 i=0 ξ i and running error bound End Algorithm 1 This description may be adapted for particular algorithms of course and the specific constraints involved may influence the choice and implmentation of the data structure for L and its manipulation during extraction, insertion and reorganization of the elements to preserve properties relevant to the constraints. These details are crucial to the efficiency of the implementation. For example, L may be organized as an ordered list to make it easy to find the smallest or largest elements in magnitude. The pair might be constrained to contain a positive and a negative number and L may be organized to facilitate finding such a pair quickly or determining that all remaining elements of the list have the same sign. The processing that results when a pair cannot be found might involve updating the constraints in order to indicate another phase of the algorithm has begun or to force a termination computation. 3

4 The termination criteria might be a simple single criterion such as L = 1 or L = 2, i.e., no pair or a single pair of elements remain. Algorithm 1 can be reduced in complexity if the contraints are simplified. Algorithm 2 is a simple version: Algorithm 2 ( Simple Select-Sum-Insert Algorithm): E =0;u =unitroundoff; initialize the set L = {l i } with ξ i,0 i n 1 i =0 while L 2 extract l and ˆl from L T i = l + ˆl E E + T i insert T i into L i i +1 end while extract (l, ˆl), the last two elements of L; returnσ = l + ˆl and ɛ =(E + σ ) u End Algorithm 2 Note that this is not the accumulation based algorithm that has been analyzed for stability in class: σ σ + ξ for i =0,...,n 1. In Algorithm 2, each pair consists of arbitrary elements of the current version of L. The accumulation based algorithm requires constraining that pair as seen in Algorithm 3: Algorithm 3 (Sequential Accumulation Algorithm): E =0;u =unitroundoff; initialize the set L = {l i } with ξ i,0 i n 1 T 0 =0;insertT 0 into L so that it is easily retrievable. i =0 while L 2 extract T i 1 and l from L where l is a different element than T i 1 T i = T i 1 + l E E + T i insert T i into L so that it is easily retrievable. i i +1 end while extract (T n 1,l), the last two elements of L; returnσ = T n 1 + l and ɛ =(E + σ ) u End Algorithm 3 The sum is accumulated by always taking T i 1 as one of the elements in the pair defining T i and l is assumed to be an element of L that is distinct from T i 1 (although they may 4

5 have the same value). This implies that L must be organized so that after computing T i it is inserted in a privileged position and is easily retrieved. This requirement is not as onerous as it sounds. In fact, it is easily done by either keeping T i separate from all other elements of L (as is done in simple code). More generally, however, it can be done by recognizing this as an example of stack-based code. That is, L is organized as a stack a data structure that only allows elements to removed in the reverse order of their insertion. A stack may be implemented in many ways depending on the situation. Algorithm 3 can be modified to accumulate the sum with terms in increasing magnitude order. Algorithm 4 shows this without expressing implementation details of L. Algorithm 4.0 (Sequential Accumulation Increasing Magnitude Order Algorithm): E =0;u =unitroundoff; initialize the set L = {l i } with ξ i,0 i n 1 T 0 =0;insertT 0 into L so that it is easily retrievable. i =0 while L 2 extract (T i 1,l)fromL where l ˆl for any ˆl Lwhere l and all ˆl are different elements than T i 1 T i = l + T i 1 E E + T i insert T i into L so that it is easily retrievable. i i +1 end while extract (T n 1,l), the last two elements of L; returnσ = T n 1 + l and ɛ =(E + σ ) u End Algorithm 4.0 There is, of course, a corresponding verision that uses decreasing magnitude order that will be referred to here as Algorithm 4.1 (Sequential Accumulation Decreasing Magnitude Order Algorithm). Clearly, the details are crucial to the efficient implementation of this form of summation. For any version of summation there must be exactly n 1 T i values produced. The particular ξ i contributing to this summation is dependent on the particular algorithm and implementation. These partial sums yield an error bound that can be used as a running error bound, i.e., one that depends on the specific data and order of operations to improve on the a priori error bound. For each addition we have a computed T i that satisfies ˆT i = l r + l s 1+δ i, δ i <u (1) 5

6 which can be combined to give the error in the computed version of σ =Σ n 1 i=0 ξ i of the form ˆσ = σ + n δ i ˆTi = σ + E n E n u n ˆT i (see, e.g., Higham s text). This clearly motivates the running bound and the magnitude ordering to reduce the error compared to the a priori bound that arises from the loose bound E n (n 1)u n ξ i + O(u 2 ) ˆT i n x j + O(u). j=1 This motivates an increasing magnitude order of the Select-Sum-Insert algorithm, given as Algorithm 5. It removes the stack-based accumulation requirement from Algorithms 3 and 4. Once T i is computed it is inserted into L and is treated the same as any of the other current elements. L is implmented in such a way that the two elements with the two smallest magnitudes can be retrieved. Clearly, this has implications for the insertion procedure. Algorithm 5.0(Select-Sum-Insert Increasing Magnitude Order Algorithm): E =0;u =unitroundoff; initialize the set L = {l i } with ξ i,0 i n 1 i =0 while L 2 extract (l, ˆl) froml where l ˆl l for any l L and with l, ˆl and all l distinct elements T i = l + ˆl E E + T i insert T i into L i i +1 end while extract (l, ˆl), the last two elements of L; returnσ = l + ˆl and ɛ =(E + σ ) u End Algorithm 5.0 6

7 As with Algorithm 4.0 there is a decreasing order version of Algorithm 5.0 that will be referred to here as Algorithm 5.1 (Select-Sum-Insert Dencreasing Magnitude Order Algorithm). For this programming problem you will implement the set of summation approaches listed below and systematically empirically explore their numerical behavior and the relationships between the a priori bound and running bound when applied to various sets of ξ i values. You must describe the details of your data structures and operations and their efficiency with respect to operations and space. This should be done relative to the constraints imposed by the particular algorithm. The implementations must be of the form presented above with the appropriate modifications for each algorithm and efficiency. You must implement a single and double precision version of these algorithms. You should do so in a way that makes it simple to choose the precision before compiling. In your experiments you may know the true sum σ true analytically. If not use the double precision version of the best behaved code as an approximation to σ true. You experiments will concentrate on the behavior of the single precision code. Codes to be Implemented: 3.1. Implement Algorithm 3 Sequential Accumulation Algorithm Implement Algorithm 4.0 and Implement Algorithms 5.0 and Implement the binary fan-in summation by expressing it as some version of Algorithm 1 that is appropriately adapted to be efficient call it Algorithm 6. Your algorithm should work for any n, i.e., not just powers of Implement the binary fan-in summation that assumes the ξ i have been ordered by increasing magnitude call it Algorithm 7.0 and a coresponding version that uses decreasing magnitude order call it Algorithm Implement a summation that adds all of the negative elements together to get S, adds all of the negative elements together to get S + and then computes σ = S +S + call it Algorithm 8. The two intermediate sums can be done using the accumulation summation or the binary fan-in but the algorithm must be put into the form above. Be sure to describe how you accomplish this in sufficient detail Implement a version of Algorithm 8.0 where the negative elements are initially ordered in terms of increasing magnitude and the positive elements are similarly initially ordered. The compute S and S + using the ideas of Algorithm 5 call it Algorithm 9.0. Note that this can be put into a single version of the Select- Sum-Insdert algorithm. It does not have to be and should not be implmented as two separate calls to Algorithm 5.0. Implement a corresponding decreasing magnitude order version call it Algorithm

8 Note it should be possible to implement the increasing and decreasing magnitude order codes as a single code with simple decisions to impose the order. Comments on Experiments: True error, running error and a priori error should be discussed relative to the properties of the data sets including magnitude profiles, signs, and the number of elements, e.g., n vs unit roundoff u. Include a small number of very large lists, e.g., n =10 8 so nu 1 or larger. You can design your own sets of numbers to probe the behavior of the different algorithms but include the sums and tests described below. Some Test Sums: S 1 = N ( 1) i 1 i, p > 0 p S 2 = N [( i i sin ) ] π i S 3 = N ( i i sin ) i S 4 = N a i where {a i } = {1, ω 1 M,...,ω N 1 M}, ω i for 1 i N 2 are random integers from [ 3, 3], N 2 w N 1 = w n, i.e., n=1 N 1 w i =0 and M>10 6 is a big number compared to 1. Finally, S 5 = N a i where the {a i } are as in S 4 except the ω i for 1 i N 2 are random values from the se { 3, 2.9,, 0.1, 0, 0.1,, 2.9, 3}. 8

9 3.8. Compute S 1 with p multiple values in the interval [0.3, 2]. Investigate the behavior of the true error, the a priori bound and the running bound at last step with respect to p. For the binary fan-in method, also include the behavior of the better priori bound derived in the earlier exercise. It is suggested to display the errors and bounds in one or more multicurve graphs or tables Compute S 2, S 3, S 4 and S 5, investigate the true error, running error bound, and a priori error bound as a function of the step k for the various methods ForS 2 and S 3, do you observe significantly different behaviors of some algorithms for these two sums? If so, explain those difference based on your knowledge of the series being summed Sums S 4 and S 5 are 1 in exact arithmetic (and 1 or very close in double precision) but they involve a wide range of numbers that indicate a significant condition number, i.e., ill-conditioning in single precision. However, as we have seen in class, different algorithms will induce different backward errors and therefore may or may not illustrate the loss of digits predicted by the worst case conditioning bound. Based on your observations of errors and bounds, explain the behavior of the algorithms for particular choices of weights {ω i } and value of M and why some get good results and the others do not. 9

10 Submission of Results Expected results comprise: A document describing your solutions as prescribed in the notes on writing up a programming solution posted on the class website. The source code, makefiles, and instructions on how to compile and execute your code including the Math Department machine used, if applicable. Code documentation should be included in each routine. All text files that do not contain code or makefiles must be PDF files. Do not send Microsoft word files of any type. These results should be submitted by 11:59 PM on the due date. Submission of results is to be done via FSU Dropbox at Drop the files off for Zhifeng Deng using his MyFSU zd16d@my.fsu.edu. (Note: do not use zdeng@math.fsu.edu to drop off the files) You should login to FSU Dropbox using your MyFSU login. If for some reason you cannot use FSU Dropbox please the files to Zhifeng at the address above. 10