Introduction to Scientific Computing

Transcription

1 Introduction to Scientific Computing Benson Muite benson 5 March 218 [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 1 / 29

2 Course Aims General introduction to numerical and computational mathematics Review programming methods Learn about some numerical algorithms Understand how these methods are used in real world situations [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 2 / 29

3 Course Overview Lectures Monday J. Livii Benson Muite (benson dot muite at ut dot ee) Practical Monday J. Livii Benson Muite (benson dot muite at ut dot ee) Homework typically due once a week until project. Expected to start this in the labs. Exam/final project presentation will be scheduled at end of course. Grading: Homework 5%, Exam 3%, Active participation 1%, Tests 1% Course Texts: Solomon Numerical Algorithms: Methods for Computer Vision, Machine Learning, and Graphics Pitt-Francis and Whiteley Guide to scientific computing in C++ [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 3 / 29

4 Lecture Topics 1) 12 February: Graphing, differentiation and integration dimension 2) 19 February: programming, recursion, arithmetic operations 3) 26 February: Linear algebra 4) 5 March: Floating point numbers, errors and ordinary differential equations 5) 12 March: Image analysis using statistics 6) 19 March: Image analysis using differential equations - Lecture by Gul Wali Shah 7) 26 March: Case study: Application of machine learning to analyse literary corpora 8) 2 April: Case study: DNA simulation using molecular dynamics [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 4 / 29

5 Lab Topics 1) 12 February: Mathematical functions, differentiation and integration, plotting, reading: https: //doi.org/1.18/ ) 19 February: Sequences, Series summation, convergence and divergence, Fibonacci numbers and Collatz conjecture or similar experimental mathematics 3) 26 February: Monte Carlo integration, parallel computing introduction, Matrix Multiply, LU decomposition 4) 5 March: Finite difference method, error analysis, interval analysis, image segmentation by matrix differences Reading [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 5 / 29

6 Lab Topics 5) 12 March: Eigenvalue computations, singular value decomposition, use of matrix operations in statistics - Eigenfaces 6) 19 March: fixed point iteration, image segmentation - solve partial differential equation from finite differences discretization with implicit timestepping (Mumford-Shah model), compare iterative and direct solvers 7) 26 March: Optimization algorithms, deep learning Reading 8) 2 April: Molecular dynamics simulation using Gromacs [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 6 / 29

7 Reading 1) 12 February: Solomon chapter 1 and 2) 19 February: Solomon chapters 2 and 3 3) 26 February: Monte Carlo integration, parallel computing introduction Solomon chapters 4, 5 4) 5 March: Solomon chapters 14 and 15, interval analysis Reading Differential Equations and Exact Solutions in the Moving Sofa Problem [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 7 / 29

8 Reading 5) 12 March: Matrix Multiply, Eigenvalue computations, singular value decomposition, use of matrix operations in statistics Solomon chapters 6 and 7 6) 19 March: Solomon chapters 11, 13 and 16 7) 26 March: Word count, clustering algorithms, optimization algorithms, deep learning Reading Solomon chapters 8, 9 and 12 8) 2 April: Molecular dynamics simulation [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 8 / 29

9 Some Nice Pictures and Videos ISC 216 Visualization showcase Paraview VisIt computer-codes/visit/gallery Amit Chourasia amit/web/home National Center for Supercomputing Applications Visualization Group University of Stuttgart http: // Brian Leu, Albert Liu, Parth Sheth brianleu/ Michael Quell UCSG6bca26nybgK39n5vnE6w [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 9 / 29

10 Monte Carlo Method: A Probabilistic Way to Calculate Integrals f = 1 b b a a f (x) dx Hence given f, then b a f (x)dx = (b a) f Doing the same in 2 dimensions and estimating the error using the standard deviation f (x, y) da A(R) f ± A(R) R Approximate f by random sampling f 2 ( f ) 2 N 1, f N i=1 f (x i, y i ) N and f 2 N i=1 (f (x i, y i )) 2 N [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 1 / 29

11 Monte Carlo Method: Python Program A program to approximate an i n t e g r a l using a Monte Carlo method This could be made faster by using vectorization, however i t is kept as simple as possible for c l a r i t y and ease of translation into other languages import math import numpy import time numpoints=496 # number of random sample p o i n t s I2d =. # i n i t i a l i z e value I2dsquare =. # i n i t i a l i z e to allow for calculation of variance for n in range ( numpoints ) : x=numpy. random. uniform ( ) y=4. numpy. random. uniform ( ) I2d=I2d+x x+2. y y I2dsquare=I2dsquare +( x x+2. y y) 2 # we scale the i n t e g r a l by the t o t a l area and d i v i d e by the number of # p o i n t s used I2d=I2d / numpoints I2dsquare=I2dsquare / numpoints EstimError=4 numpy. sqrt ( ( I2dsquare I2d 2)/ numpoints ) # estimated e r r o r I2d=I2d 4 p r i n t ( Value : %f %I2d ) print ( Error estimate : %f %EstimError ) Listing 1: A Python program to calculate the volume below z = x 2 + 2y 2, with (x, y) (, 1) (, 4). [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 11 / 29

12 Sample Results of Monte Carlo Program N Value Error Estimate / / / / [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 12 / 29

13 References on Monte Carlo Method https: //en.wikipedia.org/wiki/monte_carlo_method Monte-Carlo method. G.A. Mikhailov (originator), Encyclopedia of Mathematics. php?title=monte-carlo_method&oldid=15336 [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 13 / 29

14 Minimization and Image Alignment One area where minimization is important is in image alignment This may be useful for generating image panoramas Aligning objects [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 14 / 29

15 Minimization and Image Alignment If we have two pictures, and a region of overlap, want to find the transformation for the region of overlap Can minimize RX Y + A where R is a transformation matrix, X and Y contain pixel values for the image in the region of overlap and A is a translation. Choice of norm can affect speed and complexity of computation, in addition to quality of result [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 15 / 29

16 Timestepping for Ordinary Differential Equations: Single Step methods Some of these can be derived from quadrature formulae because du = F(u) u(t = ) = u dt is equivalent to t u(t) = u + F (u(τ))dτ For a single time step δt F (u(τ))dτ F (u())δt gives forward Euler. δt δt δt F (u(τ))dτ F (u(δt))δt gives backward Euler. F (u(τ))dτ.5 [F(u()) + F(u(δt))] δt gives Crank Nicolson. F (u(τ))dτ F (.5u() +.5u(δt))δt gives Implicit Midpoint Rule [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 16 / 29

17 Example from social sciences Abrams-Strogatz model for language death dx = yp yx xp xy dt p xy = csx a p yx = c(1 s)(1 x) a y = 1 x x fraction of population speakers of language x, y fraction of population speakers of language x, p xy probability of switching from language x to y, p yx probability of switching from language y to x, c time scaling constant, a influence of population size on probability of switching language (claim a 1.3 for many groups), s - relative status of language [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 17 / 29

18 Abrams-Strogatz model Demo - Forward Euler method [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 18 / 29

19 Fourth order Runge Kutta Consider du = F(u) u(t = ) = u dt. A popular explicit fourth order Runge-Kutta method is q 1 = f (t k, y k ) q 2 = f (t k + h/2, y k + hq 1 /2) q 3 = f (t k + h/2, y k + hq 2 /2) q 4 = f (t k + h, y k + hq 3 ) y k+1 = y k + h(q 1 + 2q 2 + 2q 3 + q 4 )/6 [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 19 / 29

20 Generating Runge Kutta Methods Butcher Tableau Order conditions [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 2 / 29

21 Generating Runge Kutta Methods Consider 1 1 φ(x)dx φ() + dφ dx x + d2 φ x 2 x= dx 2 x= 2 dx φ() + dφ 1 dx x= 2 + d2 φ 1 dx 2 x= 6 Approximate derivative by finite difference dφ dx φ(θ) φ() θ [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 21 / 29

22 Generating Runge Kutta Methods 1 1 [ ] φ(θ) φ() φ(x)dx φ() + xdx θ ( 1 1 ) φ() + 1 φ(θ) θ (, 1] 2θ 2θ hence δt y(δt) = y() + f (τ, y(τ))dτ ( y() ) (δt)f (, y() 2θ + 1 (δt)f (θ, y() + θ(δt)f (, y()))) 2θ Can generalize this, but first let us calculate the error [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 22 / 29

23 Generating Runge Kutta Methods hence Y 1 = y() + θδtf (, y()) f (θδt, Y 1 ) f (, y() + d f (, y())θδt ( dt f f (, y()) + t + f ) y θδt y t y(δt) = y() + ( 1 1 ) 2θ + 1 2θ δt [f (, y()) + δtf (, y()) ( f t + f y y t ) ] θδt y() + δtf (, y()) + δt2 d 2 dt f (, y()) + O(δt3 ) thus get a second order method [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 23 / 29

24 Generating Runge Kutta Methods To get a higher order method, consider the Butcher tableau so that Y 1 = y() c 1 a 21 c 2 a 31 a 32 b 1 b 2 b 3 Y 2 = y() + δta 21 f (c 1 δt, Y 1 ) Y 3 = y() + δta 31 f (c 2 δt, Y 1 ) + +δta 32 f (c 2 δt, Y 2 ) y(δt) = y() + δt [b 1 Y 1 + b 2 Y 2 + b 3 Y 3 ] Use Taylor expansions to generate order conditions [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 24 / 29

25 Generating Runge Kutta Methods Resulting equations for order conditions b 1 + b 2 + b 3 = 1 b 2 c 2 + b 3 c 3 = 1 2 b 2 c b 3c 2 3 = 1 3 b 3 a 32 c 2 = 1 6 i 1 c i = j=1 a ij [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 25 / 29

26 Generating Runge Kutta Methods Some example solutions Nystrom method RK32 RK [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 26 / 29

27 Generating Runge-Kutta Methods Can systematically generate order conditions using rooted trees and theory of B-series. See Butcher (28), Iserles (29) and Nodepy Can also use similar procedure to generate implicit Runge-Kutta methods for example [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 27 / 29

28 Lorenz equations [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 28 / 29

29 References Butcher An algebraic theory of integration methods Math. Comp. 26 (1972), /S #sthash. DAsEU3y.dpuf Butcher Runge-Kutta methods Scholarpedia (27) Runge-Kutta_methods Krasny Numerical Methods Lecture notes math.lsa.umich.edu/ krasny/math471.html [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 29 / 29

30 References Kouya, Performance evaluation of multiple precision matrix multiplications using parallelized Strassen and Winograd algorithms arxiv: v1 Nodepy Parallel Spectral Numerical Methods Spectral_Numerical_Methods Trefethen Finite Difference and Spectral Methods for Ordinary and Partial Differential Equations pdetext.html [Public Domain, and Poincare 1911 Solvay.jpg], via Wikimedia Commons 3 / 29