# Pascal s Triangle on a Budget. Accuracy, Precision and Efficiency in Sparse Grids

Save this PDF as:

Size: px
Start display at page:

Download "Pascal s Triangle on a Budget. Accuracy, Precision and Efficiency in Sparse Grids"

## Transcription

1 Covering : Accuracy, Precision and Efficiency in Sparse Grids jburkardt/presentations/auburn 2009.pdf... John Interdisciplinary Center for Applied Mathematics & Information Technology Department Virginia Tech... Computational Mathematics Seminar Mathematics Department Auburn University February 2009

2 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

3 INTRODUCTION - Modern Computations Modern computations require high dimensional integration: Financial mathematics: 30D or 360D; Evolutionary biology (integration over evolutionary trees); Stochastic differential equations: 10D, 20D, 50D; Integrals derived from physics (Ising, quantum field theory). Atomic properties by integration over electron configurations weighted by probabilities

4 INTRODUCTION - High Dimensional Integration is Hard These integrals are too complex to evaluate symbolically. But even numerical approximation of high dimensional integrals can be intractable: The region may have very high dimension; The integrand may have endpoint singularities; The integrand may be very expensive to evaluate; There may be a need for extreme accuracy.

5 INTRODUCTION - Accuracy, Precision, Efficiency In this talk, we consider the problem of constructing interpolatory quadrature rules for high dimensional regions. For smooth integrands, rule precision implies accuracy. But the natural method of creating precise rules, using products, incurs a cost that is exponential in the spatial dimension. We show that this explosion is not a necessary feature of interpolation, and we investigate efficient methods of achieving precision, and hence accuracy, for smooth integrands in high dimensional spaces.

6 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

7 QUADRATURE: Approximation of Integrals Integrals are numerically approximated by quadrature rules. In 1D, this is a mature (dead?) area. Ω f (x) dx N w i f (x i ) i=1 Interpolatory rules: Newton-Cotes, Chebyshev points; Semi-interpolatory rules: Gauss rules; Sampling rules: Monte Carlo and Quasi-Monte Carlo; Transform rules: tanh, tanh-sinh, erf rules.

8 QUADRATURE: Families of Rules Most quadrature rules are available in any order N. Generally, increasing N produces a more accurate result (more about this in a minute!) Under that assumption, a cautious calculation uses a sequence of increasing values of N. An efficient calculation chooses the sequence of N in such a way that previous function values can be reused. This is called nesting.

10 QUADRATURE: Precision If a quadrature rule is exact when applied to any polynomials of degree P or less, the rule has precision P. The precision of common quadrature families can be given in terms of the order N: Interpolatory rules: P = N. (assuming symmetry); Gauss rules P = 2 * N - 1 ; Monte Carlo and Quasi-Monte Carlo rules, P = 0 ; transform rules : tanh, tanh-sinh, erf rules P = 1. High precision is a property of interpolatory and Gauss rules.

11 QUADRATURE: Precision Can Mean Accuracy Using a rule with P = N on a smooth function, low order terms get integrated, leaving an error that is O( 1 N N+1 ), (Take the typical spacing between abscissas to be h = 1 N.) However, the integrands encountered in high dimensional problems are typically smooth, and suitable for precision rules. Keep in mind that precision: is not necessary - after all, Monte Carlo rules work. is not a guarantee - Newton Cotes rules are unstable; can be harmful - f(x) = step or piecewise or singular!

12 QUADRATURE: Accuracy is the Goal Accuracy simply measures the error in the result. A rule is accurate for a given class of integrands if we can show that the error (or expectation of the error) goes to zero as h 0 or N. A rule can be accurate without being precise. The 1 N accuracy of the Monte Carlo rule depends on the Law of Large Numbers. The (exponential) accuracy of tanh, tanh-sinh and erf rules is essentially anecdotal.

13 QUADRATURE: Efficiency is the Number of Abscissas Efficiency measures the amount of work expended for the result. For quadrature, we measure our work in terms of the number of function evaluations, which in turn is the number of abscissas. Since it is common to use a sequence of rules, it is important, for efficiency, to take advantage of nestedness, that is, to choose a family of rule for which the function values at one level can be reused on the next.

14 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

15 PRODUCT RULES: Formed from 1D Rules Let Q i be the i-th member of a family of nested quadrature rules, with order N i and precision P i. We can construct a corresponding family of 2D product rules as Q i Q i, with order N 2 i and precision P i. This rule is based on interpolating data on the product grid; the analysis of precision and accuracy is similar to the 1D case. Everything extends to the general M-dimensional case... except that the order growth Ni M is unacceptable!

16 PRODUCT RULES: 17x17 Clenshaw-Curtis

17 PRODUCT RULES: Do We Get Our Money s Worth? Suppose we square a rule which is precise for monomials 1 through x 4. We get a product rule precise for any monomial in x and y with individual degrees no greater than 4. The number of monomials we will be able to integrate exactly matches the number of abscissas the rule requires. Our expense, function evaluations at the abscissa, seems to buy us a corresponding great deal of monomial exactness. But for interpolatory quadrature, many of the monomial results we buy are actually nearly worthless!.

18 PRODUCT RULES: Pascal s Precision Triangle Precision for only part of a row of Pascal s triangle is not useful!. If we can t integrate x 5 or y 5 exactly, errors in those monomials determine our accuracy x y 2 x 2 xy y 2 3 x 3 x 2 y xy 2 y 3 4 x 4 x 3 y x 2 y 2 xy 3 y 4 5 x 4 y x 3 y 2 x 2 y 3 xy 4 6 x 4 y 2 x 3 y 3 x 2 y 4 7 x 4 y 3 x 3 y 4 8 x 4 y 4

19 PRODUCT RULES: It Gets Worse in Higher Dimensions Consider products of a 10 point rule with precision up to x 9. We only need to get to row 9 of Pascal s precision triangle. The monomials up to that row can be computed as a multinomial coefficient. Compare the number of abscissas to monomials! Dim Abscissas Monomials Wasted Percentage 1D % 2D % 3D 1, % 4D 10, ,285 92% 5D 100,000 2,002 97,998 97% 6D 1,000,000 5, ,995 99% In 5D, there are only 2,002 items to search for. Can t we find a quadrature rule of roughly that order?

20 PRODUCT RULES: What Do We Want? Product rules can t take us where we want to go. But where is that, exactly? We want rules with faster convergence than Monte Carlo rules. We are hoping to buy faster convergence using polynomial precision (smooth integrand f(x)) We d like a family of rules, of increasing precision, so we can control our work and estimate our accuracy. But we can t afford rules for which, at a fixed precision P, the number of abscissas N grows exponentially with spatial dimension M.

21 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

22 SMOLYAK QUADRATURE Sergey Smolyak (1963) suggested sparse grids: an algebraic combination of low order product grids; Pascal s precision rows achieved with far fewer points; Smooth f (x) + precision accuracy + efficiency.

23 SMOLYAK QUADRATURE: Construction We have an indexed family of 1D quadrature rules Q i. We form rules for dimension M, indexed by level q starting at M. Here i = i i M. ( ) A(q, M) = ( 1) q i M 1 (Q i 1 Q i M ) q i q M+1 i q Thus, the rule A(q, M) is a weighted sum of product rules.

24 SMOLYAK QUADRATURE: A sum of rules/a rule of sums The Smolyak construction rule can be interpreted to say: Compute the integral estimate for each rule, then compute the algebraic sum of these estimates. but it can also be interpreted as: Combine the component rules into a single quadrature rule, the new abscissas are the set of the component abscissas; the new weights are the component weights multiplied by the sparse grid coefficient.

25 SMOLYAK QUADRATURE: Efficiency from Nesting Under the second interpretation, we can see that in cases where an abscissa is duplicated in the component rules, the combined rule can use a single copy of the abscissa, with the sum of the weights associated with the duplicates. Duplication is a property inherited from the 1D rules. Duplication is useful when computing a single sparse grid rule, but also when computing a sequence of sparse grids of increasing level. In some cases, all the values from the previous level can be reused.

26 SMOLYAK QUADRATURE: Using Clenshaw-Curtis A common choice is 1D Clenshaw-Curtis rules. We can make a nested family by choosing successive orders of 1, 3, 5, 9, 17,... We wrote Q i to indicate the 1D quadrature rules indexed by a level running 0, 1, 2, 3, and so on. We will use a plain Q i to mean the 1D quadrature rules of order 1, 3, 5, 9 and so on. We will find it helpful to count abscissas.

27 SMOLYAK QUADRATURE: Using Clenshaw-Curtis Theorem The Clenshaw-Curtis Smolyak formula of level L is precise for all polynomials of degree 2 L + 1 or less. Thus, although our construction of sparse grids seems complicated, we still know the level of precision we can expect at each level.

28 SMOLYAK QUADRATURE: Precision Level 1D abscissas 5D abscissas 10D abscissas Precision Recall 5D product rule required 100,000 abscissas to integrate 2,002 entries in Pascal s precision triangle (precision 9).

29 SMOLYAK QUADRATURE: Asymptotic Accuracy Let N be the number of points used in the rule A(q, M). let I be the integral of f (x), f (x) : [ 1, 1] M R D α continuous if α i r for all i; Then the error satisfies: I A(q, M) = O(N r (log N) (M 1)(r+1) ) This behavior is near optimal; no family of rules could do better than O(N r ) for this general class of integrands.

30 SMOLYAK QUADRATURE: Efficiency The space ( of M-dimensional ) polynomials of degree P or less has P + M dimension MP M P!. For large M, Clenshaw-Curtis Smolyak uses N (2M)P P! points. Thus, if we are seeking exact integration of polynomials, the Clenshaw-Curtis Smolyak rule uses an optimal number of points (to within a factor 2 P that is independent of M).

31 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

32 COVERING PASCAL S TRIANGLE A family of precise interpolatory rules must cover successive rows of Pascal s precision triangle in a regular way. In higher dimensions, the triangle is a tetrahedron or a simplex. The product rule does this by overkill. Smolyak s construction covers the rows, but does so much more economically, using lower order product rules.

33 COVERING PASCAL S TRIANGLE Let s watch how this works for a family of 2D rules. I ve had to turn Pascal s triangle sideways, to an XY grid. If we count from 0, then box (I,J) represents x i y j. Thus a row of Pascal s triangle is now a diagonal of this plot. The important thing to notice is the maximum diagonal that is completely covered. This is the precision of the rule. We will see levels 0 through 4 and expect precisions 1 through 11 by 2 s.

34 COVERING PASCAL S TRIANGLE: 2D Level 0 Q 1 Q 1

35 COVERING PASCAL S TRIANGLE: 2D Level 1 Q 3 Q 1 + Q 1 Q 3 - old

36 COVERING PASCAL S TRIANGLE: 2D Level 2 Q 5 Q 1 + Q 3 Q 3 + Q 1 Q 5 - old.

37 COVERING PASCAL S TRIANGLE: 2D Level 3 Q 9 Q 1 + Q 5 Q 3 + Q 3 Q 5 + Q 1 Q 9 - old;

38 COVERING PASCAL S TRIANGLE: 2D Level 4 Q 17 Q 1 + Q 9 Q 3 + Q 5 Q 5 + Q 3 Q 9 + Q 1 Q 17 - old;

39 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

40 HOW GRIDS COMBINE We said that the Smolyak construction combines low order product rules, and that the result can be regarded as a single rule. Let s look at the construction of the Smolyak grid of level 4 in 2D. Our construction will involve 1D rules of orders 1, 3, 5, 9 and 17, and product rules formed of these factors. Because of nesting, every product rule we form will be a subset of the 17x17 full product grid.

41 HOW GRIDS COMBINE: 2D Order 17 Product Rule A 17x17 product grid (289 points).

42 HOW GRIDS COMBINE: 2D Level4 Smolyak Grid An equivalent sparse grid (65 points).

43 HOW GRIDS COMBINE A(6, 2) = i 6 ( ( 1) 6 i i = Q 1 Q 4 (Q 1 Q 9 ) Q 2 Q 3 (Q 3 Q 5 ) Q 3 Q 2 (Q 5 Q 3 ) Q 4 Q 1 (Q 9 Q 1 ) + Q 1 Q 5 (Q 1 Q 17 ) + Q 2 Q 4 (Q 3 Q 9 ) + Q 3 Q 3 (Q 5 Q 5 ) + Q 4 Q 2 (Q 9 Q 3 ) + Q 5 Q 1 (Q 17 Q 1 ) ) (Q i 1 Q i 2 )

44 HOW GRIDS COMBINE: 2D Level4 17x1 component

45 HOW GRIDS COMBINE: 2D Level4 9x3 component

46 HOW GRIDS COMBINE: 2D Level4 5x5 component

47 HOW GRIDS COMBINE: 2D Level4 3x9 component

48 HOW GRIDS COMBINE: 2D Level4 1x17 component

49 HOW GRIDS COMBINE It s easy to get confused, and think that our Smolyak grid might have the same precision P=17 as the 17x17 product rule. But the level 4 rule has precision P = 2*4+1 = 9. So in terms of efficiency, we should count the 65 points in the Smolyak grid against the 81 points in a product rule of precision 9. This is not a huge savings; however, the advantage improves with increasing precision P or spatial dimension M.

50 HOW GRIDS COMBINE: 3D Level5 Smolyak Grid 3D sparse grid, level 5, precision 11 uses 441 abscissas; 3D product grid of precision 11 uses 1,331 abscissas.

51 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

52 SPARSE GRIDS IN ACTION Let s take a problem that s reasonable but not trivial. We ll work in a space with dimension M = 6. We ll try to integrate the Genz Product Peak: where C i and Z i are prescribed. 1 f (X ) = M i=1 (C i 2 + (X i Z i ) 2 )

53 SPARSE GRIDS IN ACTION: 6D Smolyak Level Order Estimate Error

54 SPARSE GRIDS IN ACTION:6D Smolyak

55 SPARSE GRIDS IN ACTION: 6D Gauss-Legendre 1D Order 6D Order Estimate Error

56 SPARSE GRIDS IN ACTION: 6D Monte Carlo log2 (N) N Estimate Error

57 SPARSE GRIDS IN ACTION: 6D Smolyak/GL/MC

58 SPARSE GRIDS IN ACTION: 10D Smolyak/GL/MC

59 SPARSE GRIDS IN ACTION: Thoughts The graphs suggests that the accuracy behavior of the sparse grid rule is similar to the Gauss-Legendre rule, at least for this kind of integrand. For 6 dimensions, the sparse grid rule is roughly 3 times as efficient as Gauss-Legendre, ( 4,865 abscissas versus 15,625 abscissas ). Moving from 6 to 10 dimensions, the efficiency advantage is 60: (170,000 abscissas versus 9,700,000 abscissas). The Gauss-Legendre product rule is beginning the explosive growth in abscissa count.

60 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

61 A Few Words of Wisdom When good results are obtained in integrating a high-dimensional function, we should conclude first of all that an especially tractable integrand was tried and not that a generally successful method has been found. A secondary conclusion is that we might have made a very good choice in selecting an integration method to exploit whatever features of f made it tractable. Art Owen, Stanford University.

62 A Few Words of Wisdom Art Owen s words apply here. A sparse grid approach is the right choice when the function to be integrated is known to be polynomial, or to have bounded derivatives up to the order of the rule we are applying. In those cases, a sparse grid can extract extra information from the function values, to provide an answer that is exact for polynomials, and highly accurate for other smooth functions. In order to ruin everything, however, we can simply suppose that f(x) is a step function!

63 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

64 SOFTWARE PRODUCTS Smolyak s definition of sparse grids is almost magical; but it can take the novice a while to master the tricks. So it s important to bottle some of that magic in accessible tools!

65 SOFTWARE PRODUCTS: Rule Generation The simplest family of sparse quadrature rules is based on a single 1D rule, and each spatial dimension is treated the same. The family of rules is indexed by L, the level, which starts at 0 with the 1 point rule. The number of points N depends on L, the spatial dimension D, and the nesting of the underlying rule. For a given 1D rule (say laguerre), the routines available are: sparse grid laguerre size returns the number of points sparse grid laguerre index shows which 1D rule each abscissa component comes from sparse grid laguerre returns the weights and abscissas

66 SOFTWARE PRODUCTS: Rule Generation N = sparse_grid_laguerre_size ( D, L ); W = new double[n]; X = new double[d*n]; sparse_grid_laguerre ( D, L, N, W, X ); sum = 0; for ( p = 0; p < N; p++ ) { sum = sum + W[p] * f ( X+p*D ); }

67 SOFTWARE PRODUCTS: File Format A file format for quadrature rules means that software programs can communicate; Results can be precomputed. File data can easily be checked, corrected, ed, or otherwise exploited. The basic format uses 3 files: R file, 2 lines, D columns, the corners of the region W file, N lines, 1 column, the weight for each abscissa X file, N lines, D columns, the abscissas

68 SOFTWARE PRODUCTS: File Format The columns are simply numbers separated by blanks. A single file could have been used, but it would have internal structure. To determine D and N, a program reads the X file and counts the number of words on a line, and the number of lines. No particular ordering for the abscissas is assumed, but each line of the W and X files must correspond. I have used this format for a 3x3 Clenshaw Curtis product rule and a sparse grid rule for integration in 100D!

69 SOFTWARE PRODUCTS: File Format R file W file X file

70 SOFTWARE PRODUCTS: File Format Another advantage of exporting quadrature rules to a file is that it is possible to precompute a desired family of rules and store them. These files can be read in by a program written in another computer language; they can be mailed to a researcher who does not want to deal with the initial rule generation step.

71 SOFTWARE PRODUCTS: Precision Testing Once we have quadrature rules stored in files, we can easily run degree of precision tests. An executable program asks the user for the quadrature file names, and M, the maximum polynomial degree to check. The program determines the spatial dimension D implicitly from the files, as well as N, the number of points. It then generates every appropriate monomial, applies the quadrature rule, and reports the error.

72 SOFTWARE PRODUCTS: Precision Checking 23 October :04: AM NINT_EXACTNESS C++ version Investigate the polynomial exactness of a quadrature rule by integrating all monomials of a given degree over the [0,1] hypercube. NINT_EXACTNESS: User input: Quadrature rule X file = "ccgl_d2_o006_x.txt". Quadrature rule W file = "ccgl_d2_o006_w.txt". Quadrature rule R file = "ccgl_d2_o006_r.txt". Maximum total degree to check = 4 Spatial dimension = 2 Number of points = 6

73 SOFTWARE PRODUCTS: Precision Checking Error Degree Exponents

74 Accuracy, Precision and Efficiency in Sparse Grids 1 Introduction 2 Quadrature Rules in 1D 3 Product Rules for Higher Dimensions 4 Smolyak Quadrature 5 Covering Pascal s Triangle 6 How Grids Combine 7 Sparse Grids in Action 8 A Few Words of Wisdom 9 Software Products 10 Conclusion

75 CONCLUSION: A few observations Sparse grids are based on combinations of product rules. The combinations seek specific precision levels. For integrands with bounded derivatives, precision produces accuracy. By discarding some of the unneeded precision of product rules, sparse grids have a higher efficiency. Abstract probability integrals, stochastic collocation and polynomial chaos expansions are examples of settings in which sparse grids may be useful.

76 CONCLUSION: References Volker Barthelmann, Erich Novak, Klaus Ritter, High Dimensional Polynomial Interpolation on Sparse Grids, Advances in Computational Mathematics, Volume 12, Number 4, March 2000, pages John, Max Gunzburger, Clayton Webster, Reduced Order Modeling of Some Nonlinear Stochastic Partial Differential Equations, International Journal of Numerical Analysis and Modeling, Volume 4, Number 3-4, 2007, pages Thomas Gerstner, Michael Griebel, Numerical Integration Using Sparse Grids, Numerical Algorithms, Volume 18, Number 3-4, January 1998, pages Sergey Smolyak, Quadrature and Interpolation Formulas for Tensor Products of Certain Classes of Functions, Doklady Akademii Nauk SSSR, Volume 4, 1963, pages

### Slow Exponential Growth for Clenshaw Curtis Sparse Grids

Slow Exponential Growth for Clenshaw Curtis Sparse Grids John Burkardt Interdisciplinary Center for Applied Mathematics & Information Technology Department Virginia Tech... http://people.sc.fsu.edu/ burkardt/presentations/sgmga

### Numerical Integration (Quadrature) Another application for our interpolation tools!

Numerical Integration (Quadrature) Another application for our interpolation tools! Integration: Area under a curve Curve = data or function Integrating data Finite number of data points spacing specified

### Orthogonal Polynomials, Quadratures & Sparse-Grid Methods for Probability Integrals

1/33 Orthogonal Polynomials, Quadratures & Sparse-Grid Methods for Probability Integrals Dr. Abebe Geletu May, 2010 Technische Universität Ilmenau, Institut für Automatisierungs- und Systemtechnik Fachgebiet

### STOCHASTIC SAMPLING METHODS

STOCHASTIC SAMPLING METHODS APPROXIMATING QUANTITIES OF INTEREST USING SAMPLING METHODS Recall that quantities of interest often require the evaluation of stochastic integrals of functions of the solutions

### Multi-Element Probabilistic Collocation Method in High Dimensions

Multi-Element Probabilistic Collocation Method in High Dimensions Jasmine Foo and George Em Karniadakis Division of Applied Mathematics, Brown University, Providence, RI 02912 USA Abstract We combine multi-element

### Linear Algebra and Eigenproblems

Appendix A A Linear Algebra and Eigenproblems A working knowledge of linear algebra is key to understanding many of the issues raised in this work. In particular, many of the discussions of the details

### An Empirical Chaos Expansion Method for Uncertainty Quantification

An Empirical Chaos Expansion Method for Uncertainty Quantification Melvin Leok and Gautam Wilkins Abstract. Uncertainty quantification seeks to provide a quantitative means to understand complex systems

### Scientific Computing: An Introductory Survey

Scientific Computing: An Introductory Survey Chapter 7 Interpolation Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted

### Ross Program 2017 Application Problems

Ross Program 2017 Application Problems This document is part of the application to the Ross Mathematics Program, and is posted at http://u.osu.edu/rossmath/. The Admission Committee will start reading

### So we will instead use the Jacobian method for inferring the PDF of functionally related random variables; see Bertsekas & Tsitsiklis Sec. 4.1.

2011 Page 1 Simulating Gaussian Random Variables Monday, February 14, 2011 2:00 PM Readings: Kloeden and Platen Sec. 1.3 Why does the Box Muller method work? How was it derived? The basic idea involves

### Comparison of Clenshaw-Curtis and Gauss Quadrature

WDS'11 Proceedings of Contributed Papers, Part I, 67 71, 211. ISB 978-8-7378-184-2 MATFYZPRESS Comparison of Clenshaw-Curtis and Gauss Quadrature M. ovelinková Charles University, Faculty of Mathematics

### Factoring. there exists some 1 i < j l such that x i x j (mod p). (1) p gcd(x i x j, n).

18.310 lecture notes April 22, 2015 Factoring Lecturer: Michel Goemans We ve seen that it s possible to efficiently check whether an integer n is prime or not. What about factoring a number? If this could

### Outline. 1 Interpolation. 2 Polynomial Interpolation. 3 Piecewise Polynomial Interpolation

Outline Interpolation 1 Interpolation 2 3 Michael T. Heath Scientific Computing 2 / 56 Interpolation Motivation Choosing Interpolant Existence and Uniqueness Basic interpolation problem: for given data

### Quasi-Newton Methods

Newton s Method Pros and Cons Quasi-Newton Methods MA 348 Kurt Bryan Newton s method has some very nice properties: It s extremely fast, at least once it gets near the minimum, and with the simple modifications

### A note on multivariate Gauss-Hermite quadrature

A note on multivariate Gauss-Hermite quadrature Peter Jäckel 1th May 5 1 Introduction Gaussian quadratures are an ingenious way to approximate the integral of an unknown function f(x) over a specified

### CS261: A Second Course in Algorithms Lecture #11: Online Learning and the Multiplicative Weights Algorithm

CS61: A Second Course in Algorithms Lecture #11: Online Learning and the Multiplicative Weights Algorithm Tim Roughgarden February 9, 016 1 Online Algorithms This lecture begins the third module of the

### Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques

Institut für Numerische Mathematik und Optimierung Karhunen-Loève Approximation of Random Fields Using Hierarchical Matrix Techniques Oliver Ernst Computational Methods with Applications Harrachov, CR,

### Simple Examples on Rectangular Domains

84 Chapter 5 Simple Examples on Rectangular Domains In this chapter we consider simple elliptic boundary value problems in rectangular domains in R 2 or R 3 ; our prototype example is the Poisson equation

### There are four irrational roots with approximate values of

Power of the Quadratic Formula 1 y = (x ) - 8(x ) + 4 a = 1, b = -8, c = 4 Key 1. Consider the equation y = x 4 8x + 4. It may be a surprise, but we can use the quadratic formula to find the x-intercepts

### Nonlinear stochastic Galerkin and collocation methods: application to a ferromagnetic cylinder rotating at high speed

Nonlinear stochastic Galerkin and collocation methods: application to a ferromagnetic cylinder rotating at high speed Eveline Rosseel Herbert De Gersem Stefan Vandewalle Report TW 541, July 29 Katholieke

### Data-Driven Global Optimization

Data-Driven Global Optimization Fani Boukouvala & Chris Kieslich Christodoulos A Floudas Memorial Symposium Princeton University May 6 th 2017 A story of global optimization & more September 17 th 2014

### MATH 310, REVIEW SHEET 2

MATH 310, REVIEW SHEET 2 These notes are a very short summary of the key topics in the book (and follow the book pretty closely). You should be familiar with everything on here, but it s not comprehensive,

### In manycomputationaleconomicapplications, one must compute thede nite n

Chapter 6 Numerical Integration In manycomputationaleconomicapplications, one must compute thede nite n integral of a real-valued function f de ned on some interval I of

### Multistage Methods I: Runge-Kutta Methods

Multistage Methods I: Runge-Kutta Methods Varun Shankar January, 0 Introduction Previously, we saw that explicit multistep methods (AB methods) have shrinking stability regions as their orders are increased.

### Making the grade: Part II

1997 2009, Millennium Mathematics Project, University of Cambridge. Permission is granted to print and copy this page on paper for non commercial use. For other uses, including electronic redistribution,

### 4. Determinants.

4. Determinants 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 2 2 determinant 4.1. Determinants; Cofactor Expansion Determinants of 2 2 and 3 3 Matrices 3 3 determinant 4.1.

### Solving Boundary Value Problems (with Gaussians)

What is a boundary value problem? Solving Boundary Value Problems (with Gaussians) Definition A differential equation with constraints on the boundary Michael McCourt Division Argonne National Laboratory

### TAYLOR POLYNOMIALS DARYL DEFORD

TAYLOR POLYNOMIALS DARYL DEFORD 1. Introduction We have seen in class that Taylor polynomials provide us with a valuable tool for approximating many different types of functions. However, in order to really

### Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms

Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds

### CHAPTER 10: Numerical Methods for DAEs

CHAPTER 10: Numerical Methods for DAEs Numerical approaches for the solution of DAEs divide roughly into two classes: 1. direct discretization 2. reformulation (index reduction) plus discretization Direct

### Scenario generation for stochastic optimization problems via the sparse grid method

Scenario generation for stochastic optimization problems via the sparse grid method Michael Chen Sanjay Mehrotra Dávid Papp March 7, 0 Abstract We study the use of sparse grids methods for the scenario

### Homework. Basic properties of real numbers. Adding, subtracting, multiplying and dividing real numbers. Solve one step inequalities with integers.

Morgan County School District Re-3 A.P. Calculus August What is the language of algebra? Graphing real numbers. Comparing and ordering real numbers. Finding absolute value. September How do you solve one

### Quick Introduction to Nonnegative Matrix Factorization

Quick Introduction to Nonnegative Matrix Factorization Norm Matloff University of California at Davis 1 The Goal Given an u v matrix A with nonnegative elements, we wish to find nonnegative, rank-k matrices

### Two Issues in Using Mixtures of Polynomials for Inference in Hybrid Bayesian Networks

Accepted for publication in: International Journal of Approximate Reasoning, 2012, Two Issues in Using Mixtures of Polynomials for Inference in Hybrid Bayesian

### High Performance Nonlinear Solvers

What is a nonlinear system? High Performance Nonlinear Solvers Michael McCourt Division Argonne National Laboratory IIT Meshfree Seminar September 19, 2011 Every nonlinear system of equations can be described

### Cosets and Lagrange s theorem

Cosets and Lagrange s theorem These are notes on cosets and Lagrange s theorem some of which may already have been lecturer. There are some questions for you included in the text. You should write the

### Numerical Methods for Inverse Kinematics

Numerical Methods for Inverse Kinematics Niels Joubert, UC Berkeley, CS184 2008-11-25 Inverse Kinematics is used to pose models by specifying endpoints of segments rather than individual joint angles.

### A primer on matrices

A primer on matrices Stephen Boyd August 4, 2007 These notes describe the notation of matrices, the mechanics of matrix manipulation, and how to use matrices to formulate and solve sets of simultaneous

### CS 6820 Fall 2014 Lectures, October 3-20, 2014

Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given

### Review for Exam 2 Ben Wang and Mark Styczynski

Review for Exam Ben Wang and Mark Styczynski This is a rough approximation of what we went over in the review session. This is actually more detailed in portions than what we went over. Also, please note

### LECTURE 16 GAUSS QUADRATURE In general for Newton-Cotes (equispaced interpolation points/ data points/ integration points/ nodes).

CE 025 - Lecture 6 LECTURE 6 GAUSS QUADRATURE In general for ewton-cotes (equispaced interpolation points/ data points/ integration points/ nodes). x E x S fx dx hw' o f o + w' f + + w' f + E 84 f 0 f

### Bindel, Fall 2016 Matrix Computations (CS 6210) Notes for

1 Logistics Notes for 2016-08-26 1. Our enrollment is at 50, and there are still a few students who want to get in. We only have 50 seats in the room, and I cannot increase the cap further. So if you are

### Chapter 3-1 Polynomials

Chapter 3 notes: Chapter 3-1 Polynomials Obj: SWBAT identify, evaluate, add, and subtract polynomials A monomial is a number, a variable, or a product of numbers and variables with whole number exponents

### Abstract Vector Spaces and Concrete Examples

LECTURE 18 Abstract Vector Spaces and Concrete Examples Our discussion of linear algebra so far has been devoted to discussing the relations between systems of linear equations, matrices, and vectors.

### Eigenvalues and Eigenvectors

LECTURE 3 Eigenvalues and Eigenvectors Definition 3.. Let A be an n n matrix. The eigenvalue-eigenvector problem for A is the problem of finding numbers λ and vectors v R 3 such that Av = λv. If λ, v are

### A description of a math circle set of activities around polynomials, especially interpolation.

A description of a math circle set of activities around polynomials, especially interpolation. Bob Sachs Department of Mathematical Sciences George Mason University Fairfax, Virginia 22030 rsachs@gmu.edu

### ( )( b + c) = ab + ac, but it can also be ( )( a) = ba + ca. Let s use the distributive property on a couple of

Factoring Review for Algebra II The saddest thing about not doing well in Algebra II is that almost any math teacher can tell you going into it what s going to trip you up. One of the first things they

### Matrix Assembly in FEA

Matrix Assembly in FEA 1 In Chapter 2, we spoke about how the global matrix equations are assembled in the finite element method. We now want to revisit that discussion and add some details. For example,

### Introduction to the Tensor Train Decomposition and Its Applications in Machine Learning

Introduction to the Tensor Train Decomposition and Its Applications in Machine Learning Anton Rodomanov Higher School of Economics, Russia Bayesian methods research group (http://bayesgroup.ru) 14 March

### DIFFERENTIAL EQUATIONS

DIFFERENTIAL EQUATIONS Basic Concepts Paul Dawkins Table of Contents Preface... Basic Concepts... 1 Introduction... 1 Definitions... Direction Fields... 8 Final Thoughts...19 007 Paul Dawkins i http://tutorial.math.lamar.edu/terms.aspx

Dimension Adaptive Tensor Product Quadrature Thomas Gerstner and Michael Griebel Department for Applied Mathematics University of Bonn {gerstner,griebel}@iam.uni-bonn.de Abstract We consider the numerical

### means is a subset of. So we say A B for sets A and B if x A we have x B holds. BY CONTRAST, a S means that a is a member of S.

1 Notation For those unfamiliar, we have := means equal by definition, N := {0, 1,... } or {1, 2,... } depending on context. (i.e. N is the set or collection of counting numbers.) In addition, means for

### Don t forget to think of a matrix as a map (a.k.a. A Linear Algebra Primer)

Don t forget to think of a matrix as a map (aka A Linear Algebra Primer) Thomas Yu February 5, 2007 1 Matrix Recall that a matrix is an array of numbers: a 11 a 1n A a m1 a mn In this note, we focus on

### An Overly Simplified and Brief Review of Differential Equation Solution Methods. 1. Some Common Exact Solution Methods for Differential Equations

An Overly Simplified and Brief Review of Differential Equation Solution Methods We will be dealing with initial or boundary value problems. A typical initial value problem has the form y y 0 y(0) 1 A typical

### MATH 320, WEEK 6: Linear Systems, Gaussian Elimination, Coefficient Matrices

MATH 320, WEEK 6: Linear Systems, Gaussian Elimination, Coefficient Matrices We will now switch gears and focus on a branch of mathematics known as linear algebra. There are a few notes worth making before

### MCR3U Unit 7 Lesson Notes

7.1 Arithmetic Sequences Sequence: An ordered list of numbers identified by a pattern or rule that may stop at some number or continue indefinitely. Ex. 1, 2, 4, 8,... Ex. 3, 7, 11, 15 Term (of a sequence):

### Introduction - Motivation. Many phenomena (physical, chemical, biological, etc.) are model by differential equations. f f(x + h) f(x) (x) = lim

Introduction - Motivation Many phenomena (physical, chemical, biological, etc.) are model by differential equations. Recall the definition of the derivative of f(x) f f(x + h) f(x) (x) = lim. h 0 h Its

### Linear Programming and Marginal Analysis

337 22 Linear Programming and Marginal Analysis This chapter provides a basic overview of linear programming, and discusses its relationship to the maximization and minimization techniques used for the

### Review of matrices. Let m, n IN. A rectangle of numbers written like A =

Review of matrices Let m, n IN. A rectangle of numbers written like a 11 a 12... a 1n a 21 a 22... a 2n A =...... a m1 a m2... a mn where each a ij IR is called a matrix with m rows and n columns or an

### Name Period Date. Use mathematical reasoning to create polynomial expressions that generalize patterns. Practice polynomial arithmetic.

Name Period Date POLYNOMIALS Student Packet 4: Polynomial Arithmetic Applications POLY4.1 Hundred Chart Patterns Gather empirical data to form conjectures about number patterns. Write algebraic expressions.

### GAP CLOSING. Algebraic Expressions. Intermediate / Senior Facilitator s Guide

GAP CLOSING Algebraic Expressions Intermediate / Senior Facilitator s Guide Topic 6 Algebraic Expressions Diagnostic...5 Administer the diagnostic...5 Using diagnostic results to personalize interventions...5

### Chapter 7 - Exponents and Exponential Functions

Chapter 7 - Exponents and Exponential Functions 7-1: Multiplication Properties of Exponents 7-2: Division Properties of Exponents 7-3: Rational Exponents 7-4: Scientific Notation 7-5: Exponential Functions

### Journal of Computational Physics

Journal of Computational Physics 229 (2010) 1536 1557 Contents lists available at ScienceDirect Journal of Computational Physics journal homepage: www.elsevier.com/locate/jcp Multi-element probabilistic

### Review: complex numbers

October 5/6, 01.5 extra problems page 1 Review: complex numbers Number system The complex number system consists of a + bi where a and b are real numbers, with various arithmetic operations. The real numbers

### Spatial Data Analysis in Archaeology Anthropology 589b. Kriging Artifact Density Surfaces in ArcGIS

Spatial Data Analysis in Archaeology Anthropology 589b Fraser D. Neiman University of Virginia 2.19.07 Spring 2007 Kriging Artifact Density Surfaces in ArcGIS 1. The ingredients. -A data file -- in.dbf

### THEIR GRAPHS, AND THEIR

St. Michael Albertville High School Teacher: Kim Benson Algebra 2 (Master) August 2015 CEQs: WHAT RELATIONSHIP S EXIST BETWEEN VARIOUS FUNCTIONS, THEIR GRAPHS, AND THEIR SOLUTION(S)? HOW DO WE SIMPLIFY

### Lecture 31: Reductions

Lecture 31: Page 1 of 10 Aims: To discuss the idea of a reduction and reducibility; To discuss the idea of polynomial-time reductions, and to see what we can learn from them; To see what we can learn from

### Unit 2, Ongoing Activity, Little Black Book of Algebra II Properties

Unit 2, Ongoing Activity, Little Black Book of Algebra II Properties Little Black Book of Algebra II Properties Unit 2 - Polynomial Equations & Inequalities 2.1 Laws of Exponents - record the rules for

### arxiv: v1 [physics.comp-ph] 22 Jul 2010

Gaussian integration with rescaling of abscissas and weights arxiv:007.38v [physics.comp-ph] 22 Jul 200 A. Odrzywolek M. Smoluchowski Institute of Physics, Jagiellonian University, Cracov, Poland Abstract

### Notes on multivariable calculus

Notes on multivariable calculus Jonathan Wise February 2, 2010 1 Review of trigonometry Trigonometry is essentially the study of the relationship between polar coordinates and Cartesian coordinates in

### Solving Equations Quick Reference

Solving Equations Quick Reference Integer Rules Addition: If the signs are the same, add the numbers and keep the sign. If the signs are different, subtract the numbers and keep the sign of the number

### MA554 Assessment 1 Cosets and Lagrange s theorem

MA554 Assessment 1 Cosets and Lagrange s theorem These are notes on cosets and Lagrange s theorem; they go over some material from the lectures again, and they have some new material it is all examinable,

### Multilevel accelerated quadrature for elliptic PDEs with random diffusion. Helmut Harbrecht Mathematisches Institut Universität Basel Switzerland

Multilevel accelerated quadrature for elliptic PDEs with random diffusion Mathematisches Institut Universität Basel Switzerland Overview Computation of the Karhunen-Loéve expansion Elliptic PDE with uniformly

### Progress in high-dimensional numerical integration and its application to stochastic optimization

Progress in high-dimensional numerical integration and its application to stochastic optimization W. Römisch Humboldt-University Berlin Department of Mathematics www.math.hu-berlin.de/~romisch Short course,

### Natural Evolution Strategies for Direct Search

Tobias Glasmachers Natural Evolution Strategies for Direct Search 1 Natural Evolution Strategies for Direct Search PGMO-COPI 2014 Recent Advances on Continuous Randomized black-box optimization Thursday

### 6.842 Randomness and Computation Lecture 5

6.842 Randomness and Computation 2012-02-22 Lecture 5 Lecturer: Ronitt Rubinfeld Scribe: Michael Forbes 1 Overview Today we will define the notion of a pairwise independent hash function, and discuss its

### TENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY

TENSOR APPROXIMATION TOOLS FREE OF THE CURSE OF DIMENSIONALITY Eugene Tyrtyshnikov Institute of Numerical Mathematics Russian Academy of Sciences (joint work with Ivan Oseledets) WHAT ARE TENSORS? Tensors

### PYP Mathematics Continuum

Counting from One By Imaging to Advanced Counting (-) Y E+ 0- K N O W L E D G E Read,Write, and Count Whole numbers up to 00, forwards and backwards in s, s, 5 s, and 0 s Recall How many tens in a two-digit

### Lecture 3: Special Matrices

Lecture 3: Special Matrices Feedback of assignment1 Random matrices The magic matrix commend magic() doesn t give us random matrix. Random matrix means we will get different matrices each time when we

### Problem Set # 1 Solution, 18.06

Problem Set # 1 Solution, 1.06 For grading: Each problem worths 10 points, and there is points of extra credit in problem. The total maximum is 100. 1. (10pts) In Lecture 1, Prof. Strang drew the cone

### Lecture 3: Reductions and Completeness

CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic

### Section Exponential Functions

121 Section 4.1 - Exponential Functions Exponential functions are extremely important in both economics and science. It allows us to discuss the growth of money in a money market account as well as the

### The Not-Formula Book for C2 Everything you need to know for Core 2 that won t be in the formula book Examination Board: AQA

Not The Not-Formula Book for C Everything you need to know for Core that won t be in the formula book Examination Board: AQA Brief This document is intended as an aid for revision. Although it includes

### Polynomial one or more monomials added or subtracted. (i.e. : 5x or 6xy-3 or 6xy - 5x + 3 or

Polynomials Necessary Terms for Success Welcome back! We will now tackle the world of polynomials. Before we get started with performing operations or using polynomials for applications, we will need some

### Review from Bootcamp: Linear Algebra

Review from Bootcamp: Linear Algebra D. Alex Hughes October 27, 2014 1 Properties of Estimators 2 Linear Algebra Addition and Subtraction Transpose Multiplication Cross Product Trace 3 Special Matrices

### Note-Taking Guides. How to use these documents for success

1 Note-Taking Guides How to use these documents for success Print all the pages for the module. Open the first lesson on the computer. Fill in the guide as you read. Do the practice problems on notebook

### Course Description for Real Analysis, Math 156

Course Description for Real Analysis, Math 156 In this class, we will study elliptic PDE, Fourier analysis, and dispersive PDE. Here is a quick summary of the topics will study study. They re described

### Notes on vectors and matrices

Notes on vectors and matrices EE103 Winter Quarter 2001-02 L Vandenberghe 1 Terminology and notation Matrices, vectors, and scalars A matrix is a rectangular array of numbers (also called scalars), written

### Hypothesis testing and the Gamma function

Math 10A November 29, 2016 Announcements Please send me email to sign up for the next two (last two?) breakfasts: This Thursday (December 1) at 9AM. Next Monday (December 5), also at 9AM. Pop-in lunch

### Functions: Polynomial, Rational, Exponential

Functions: Polynomial, Rational, Exponential MATH 151 Calculus for Management J. Robert Buchanan Department of Mathematics Spring 2014 Objectives In this lesson we will learn to: identify polynomial expressions,

### Numerical Methods I: Numerical Integration/Quadrature

1/20 Numerical Methods I: Numerical Integration/Quadrature Georg Stadler Courant Institute, NYU stadler@cims.nyu.edu November 30, 2017 umerical integration 2/20 We want to approximate the definite integral

### INTERSECTION THEORY CLASS 2

INTERSECTION THEORY CLASS 2 RAVI VAKIL CONTENTS 1. Last day 1 2. Zeros and poles 2 3. The Chow group 4 4. Proper pushforwards 4 The webpage http://math.stanford.edu/ vakil/245/ is up, and has last day

### Vector, Matrix, and Tensor Derivatives

Vector, Matrix, and Tensor Derivatives Erik Learned-Miller The purpose of this document is to help you learn to take derivatives of vectors, matrices, and higher order tensors (arrays with three dimensions

### Orthogonal iteration revisited

Week 10 11: Friday, Oct 30 and Monday, Nov 2 Orthogonal iteration revisited Last time, we described a generalization of the power methods to compute invariant subspaces. That is, starting from some initial

CMPSCI611: NP Completeness Lecture 17 Essential facts about NP-completeness: Any NP-complete problem can be solved by a simple, but exponentially slow algorithm. We don t have polynomial-time solutions

### Honors Advanced Algebra Unit 2 Polynomial Operations September 14, 2016 Task 7: What s Your Identity?

Honors Advanced Algebra Name Unit Polynomial Operations September 14, 016 Task 7: What s Your Identity? MGSE9 1.A.APR.4 Prove polynomial identities and use them to describe numerical relationships. MGSE9

### Exact and Approximate Numbers:

Eact and Approimate Numbers: The numbers that arise in technical applications are better described as eact numbers because there is not the sort of uncertainty in their values that was described above.