Paul Heckbert Computer Science Department Carnegie Mellon University 26 Sept. 2 5-859B - Introduction to Scientific Computing
aerospace: simulate subsonic & supersonic air flow around full aircraft, no wind tunnel divide space into millions of tetrahedra or parallelepipeds, solve sparse linear or nonlinear PDE nuclear: simulate nuclear eplosion! especially important because of nuclear test bans weather prediction: entire planet, 2 weeks into future astrophysics: galactic collisions automotive: simulate car crash biology: simulate protein folding drug design 26 Sept. 2 5-859B - Introduction to Scientific Computing 2
replace infinite process with finite process e.g. integrals to sums replace general matrices with simple matrices e.g. diagonal replace comple functions with simple ones e.g. polynomials replace nonlinear problems with linear problems replace differential equations with algebraic equations e.g. linear systems replace high-order systems with low-order systems replace infinite-dimensional spaces with finite-dim. ones e.g. all real functions on [,] with samples on n-element grid 26 Sept. 2 5-859B - Introduction to Scientific Computing 3
error type eample: car crash simulation modeling empirical measurements previous computations truncation or discretization rounding approimate car geometry incorrect tire friction coeff. error in initial speed of car numerical solution to dif.eq. used floats, not doubles Each step introduces some error, but magnitudes may differ greatly. Look for the largest source of error the weak link in the chain. 26 Sept. 2 5-859B - Introduction to Scientific Computing 4
(absolute error) = (approimate value) (true value) (absolute error) (approimate value) (relative error) = (true value) = (true value) Fundamental difficulty with measuring error: For many problems we cannot compute the eact answer, we can only approimate it! Often, the best we can do is estimate the error! 26 Sept. 2 5-859B - Introduction to Scientific Computing 5
main() { float f =./3.; printf("%.2f\n", f); // print to 2 digits }.333333343267448 we get 7 significant digits; the rest is junk! When reporting results, only show the significant digits! 26 Sept. 2 5-859B - Introduction to Scientific Computing 6
very widely used standard for floating point C float is 4 bytes: 24 bit mantissa, 8 bit eponent about 7 significant digits smallest pos. no:.3e-38, largest: 3.4e+38 C double is 8 bytes: 53 bit mantissa, bit eponent about 6 significant digits smallest pos.: 2.3e-38, largest:.7e+38 special values Inf - infinity (e.g. /) NaN - not a number, undefined (e.g. /) 26 Sept. 2 5-859B - Introduction to Scientific Computing 7
#include <math.h> main() { int i; float f; double d; } for (i=; i<55; i++) { f =. + pow(.5, i); d =. + pow(.5, i); printf("%2d %.9f %.8f\n", i, f, d); } 26 Sept. 2 5-859B - Introduction to Scientific Computing 8
i float double +2^(-i) 2. 2..5.5 2.25.25 3.25.25 4.625.625 2.477.476837582 22.238.23848579 23.9.9292896 24..59646448 5..9 5..4 52..2 53.. 26 Sept. 2 5-859B - Introduction to Scientific Computing 9
some problems are harder to solve accurately than others The condition number is a measure of how sensitive a problem is to changes in its input ) relative change in output f( ) f( ) / f( ) f ( ) Cond = = = relative change in input f( ) [ ] ) [ ]/ where f ( ) represents the eact solution to problem with input Cond< or so Cond>> problem is well-conditioned problem is ill-conditioned 26 Sept. 2 5-859B - Introduction to Scientific Computing
well conditioned ill-conditioned taking a step on level ground step near cliff tan() near =45, say cos() not near =9 tan() near =9 (because f infinite) cos() near =9 (because f zero) 26 Sept. 2 5-859B - Introduction to Scientific Computing
Solve A=b for A is n n matri and b are n-vectors (column matrices) Later we ll look at overdetermined and underdetermined systems, where the matri is not square (#equations not equal to #unknowns) 26 Sept. 2 5-859B - Introduction to Scientific Computing 2
For a square, n n matri: rank is the ma. no. of linearly independent rows or columns full rank = rank is n rank-deficient = rank is less than n singular matri = determinant zero = no inverse = linearly dependent = rank-deficient = (A= for some nonzero ) 26 Sept. 2 5-859B - Introduction to Scientific Computing 3
26 Sept. 2 5-859B - Introduction to Scientific Computing 4 Rank 2 Rank Rank 4 3 2. 2 2 6 3 2
26 Sept. 2 5-859B - Introduction to Scientific Computing 5 intersection of 2 lines 2 5 2 2 5 2 = = + = y y y y +2y=5 -y=2 a y y a y = = + = 6 3 3 2 3 6 3 2 y +2y=3-3-6y=a a Rank matri means lines are parallel. For most a, lines non-coincident, so no solution. For a=-9, lines coincident, one-dimensional subspace of solutions.
Gaussian Elimination on square matri A A 2 4 2 2 4 2 = 4 9 3 = 2 = 2 3 7 4 LU computes an LU decomposition L is unit lower triangular ( s on diagonal) U is upper triangular 26 Sept. 2 5-859B - Introduction to Scientific Computing 6
G.E. can be done on any square matri if A singular then diagonal of U will contain zero(s) usually partial pivoting is used (swapping rows during elimination) to reduce errors G.E. is an eample of an eplicit method for solving linear systems solve for solution in one sweep Other, more efficient algorithms can be used for specialized matri types, as we ll see later 26 Sept. 2 5-859B - Introduction to Scientific Computing 7
to solve A=b: decompose A into LU -- cost 2n 3 /3 flops solve Ly=b for y by forw. substitution -- cost n 2 flops solve U=y for by back substitution -- cost n 2 flops slower alternative: compute A - -- cost 2n 3 flops multiply =A - b -- cost 2n 2 flops this costs about 3 times as much as LU lesson: if you see A - in a formula, read it as solve a system, not invert a matri 26 Sept. 2 5-859B - Introduction to Scientific Computing 8
Symmetric Positive Definite an important matri class symmetric: A=A T positive definite: T A> for Ø all λ i > if A is spd, LU decomposition can be written A=LL T, where L is lower triangular (not unit) this is the Cholesky factorization no pivoting required -- cost n 3 /3 flops 26 Sept. 2 5-859B - Introduction to Scientific Computing 9
A method for solving n n linear systems What is its cost? 26 Sept. 2 5-859B - Introduction to Scientific Computing 2
2 = n i= n 2 2 = i i= i -norm, Manhattan norm = ma -norm i i 2-norm, Euclidean norm norms differ by at most a constant factor, for fied n n n 2 2 26 Sept. 2 5-859B - Introduction to Scientific Computing 2
matri norm defined in terms of vector norm: A = ma A geometric meaning: the maimum stretch resulting from application of this transformation eact result depends on whether -, 2-, or -norm is used 26 Sept. 2 5-859B - Introduction to Scientific Computing 22
A measure of how close a matri is to singular cond( A) = κ ( A) = A A maimum stretch = = maimum shrink ma i min i λ λ i i cond(i) = cond(singular matri) = 26 Sept. 2 5-859B - Introduction to Scientific Computing 23