Contents Basics of Numerical Analysis

Transcription

1 Contents 1 Basics of Numerical Analysis Introduction Finite-PrecisionArithmetic ApproximationofExpressions Optimal (Minimax) and Almost Optimal Approximations Rational (Padé) Approximation Summation of Series by Using Padé Approximations (Wynn s ɛ-algorithm) Approximation of the Evolution Operator for a HamiltonianSystem Power and Asymptotic Expansion, Asymptotic Analysis Power Expansion Asymptotic Expansion Asymptotic Analysis of Integrals by Integration by Parts Asymptotic Analysis of Integrals by the Laplace Method Stationary-Phase Approximation DifferentialEquationswithLargeParameters SummationofFiniteandInfiniteSeries Tests of Convergence SummationofSeriesinFloating-PointArithmetic Acceleration of Convergence AlternatingSeries Levin stransformations PoissonSummation BorelSummation AbelSummation Problems IntegraloftheGaussDistribution Airy Functions Bessel Functions ix

2 x Contents AlternatingSeries Coulomb Scattering Amplitude and Borel Resummation.. 52 References Solving Non-linear Equations Scalar Equations Bisection The Family of Newton s Methods and the Newton Raphson Method The Secant Method and Its Relatives Müller smethod VectorEquations Newton Raphson s Method Broyden s (Secant) Method Convergence Acceleration Polynomial Equations of a Single Variable LocatingtheRegionsContainingZeros Descartes RuleandtheSturmMethod Newton ssumsandinvièto sformulas Eliminating Multiple Zeros of the Polynomial ConditioningoftheComputationofZeros General Hints for the Computation of Zeros Bernoulli s Method Horner s Linear Method Bairstow s (Horner s Quadratic) Method Laguerre s Method Maehly Newton Raphson s Method Algebraic Equations of Several Variables Problems Wien s Law and Lambert s Function Heisenberg s Model in the Mean-Field Approximation Energy Levels of Simple One-Dimensional Quantum Systems Propane Combustion in Air Fluid Flow Through Systems of Pipes AutomatedAssemblyofStructures References Matrix Methods BasicOperations Matrix Multiplication Computing the Determinant Systems of Linear Equations Ax = b AnalysisofErrors GaussElimination Systems with Banded Matrices...115

3 Contents xi Toeplitz Systems Vandermonde Systems Condition Estimates for Matrix Inversion Sparse Matrices Linear Least-Square Problem and Orthogonalization The QR Decomposition Singular Value Decomposition (SVD) The Minimal Solution of the Least-Squares Problem MatrixEigenvalueProblems Non-symmetricProblems SymmetricProblems Generalized Eigenvalue Problems ConvertingaMatrixtoItsJordanForm Eigenvalue Problems for Sparse Matrices Random Matrices General Random Matrices Gaussian Orthogonal or Unitary Ensemble Cyclic Orthogonal and Unitary Ensemble Problems Percolation in a Random-Lattice Model Electric Circuits of Linear Elements Systems of Oscillators Image Compression by Singular Value Decomposition Eigenstates of Particles in the Anharmonic Potential Anderson Localization Spectra of Random Symmetric Matrices References Transformations of Functions and Signals FourierTransformation FourierSeries Continuous Fourier Expansion Discrete Fourier Expansion Aliasing Leakage FastDiscreteFourierTransformation(FFT) Multiplication of Polynomials by Using the FFT Power Spectral Density Transformations with Orthogonal Polynomials Legendre Polynomials Chebyshev Polynomials LaplaceTransformation Use of Laplace Transformation with Differential Equations...182

4 xii Contents 4.5 Hilbert Transformation Analytic Signal Kramers KronigRelations Numerical Computation of the Continuous Hilbert Transform DiscreteHilbertTransformation Wavelet Transformation NumericalComputationoftheWaveletTransform DiscreteWaveletTransform Problems Fourier Spectrum of Signals Fourier Analysis of the Doppler Effect UseofLaplaceTransformationandItsInverse UseoftheWaveletTransformation References Statistical Analysis and Modeling of Data BasicDataAnalysis Probability Distributions MomentsofDistributions Uncertainties of Moments of Distributions RobustStatistics Hunting for Outliers M-EstimatesofLocation M-Estimates of Scale StatisticalTests Computing the Confidence Interval for the Sample Mean Comparing the Means of Two Samples with Equal Variances Comparing the Means of Two Samples with Different Variances Determining the Confidence Interval for the Sample Variance Comparing Two Sample Variances Comparing Histogrammed Data to a Known Distribution ComparingTwoSetsofHistogrammedData Comparing Non-histogrammed Data to a Continuous Distribution Correlation Linear Correlation Non-parametricCorrelation Linear and Non-linear Regression Linear Regression Regression with Orthogonal Polynomials Linear Regression (Fitting a Straight Line)...230

5 Contents xiii Linear Regression (Fitting a Straight Line) with Errors in BothCoordinates Fitting a Constant Generalized Linear Regression by Using SVD Robust Methods for One-Dimensional Regression Non-linear Regression Multiple Linear Regression TheBasicMethod Principal-Component Multiple Regression Principal-Component Analysis Principal Components by Diagonalizing the Covariance Matrix Standardization of Data for PCA Principal Components from the SVD of the Data Matrix Improvements of PCA: Non-linearity, Robustness Cluster Analysis HierarchicalClustering Partitioning Methods: k-means GaussianMixtureClusteringandtheEMAlgorithm Spectral Methods Linear Discriminant Analysis BinaryClassification Logistic Discriminant Analysis Assignment to Multiple Classes Canonical Correlation Analysis Factor Analysis Determining the Factors and Weights from the Covariance Matrix Standardization of Data and Robust Factor Analysis Problems Multiple Regression Nutritional Value of Food Discrimination of Radar Signals from Ionospheric Reflections Canonical Correlation Analysis of Objects in the CDFS Area References Modeling and Analysis of Time Series Random Variables BasicDefinitions Generation of Random Numbers Random Processes BasicDefinitions Stable Distributions and Random Walks Central Limit Theorem...283

6 xiv Contents StableDistributions Generalized Central Limit Theorem Discrete-Time Random Walks Continuous-Time Random Walks Markov Chains Discrete-TimeorClassicalMarkovChains Continuous-Time Markov Chains Noise Types of Noise Generation of Noise TimeCorrelationandAuto-Correlation Sample Correlations of Signals RepresentationofTimeCorrelations FastComputationofDiscreteSampleCorrelations Auto-Regression Analysis of Discrete-Time Signals Auto-Regression (AR) Model Use of AR Models Estimate of the Fourier Spectrum Independent Component Analysis Estimate of the Separation Matrix and the FastICA Algorithm TheFastICAAlgorithm StabilizationoftheFastICAAlgorithm Problems LogisticMap Diffusion and Chaos in the Standard Map Phase Transitions in the Two-Dimensional Ising Model Independent Component Analysis References Initial-Value Problems for ODE EvolutionEquations Explicit Euler s Methods Explicit Methods of the Runge Kutta Type Errors of Explicit Methods Discretization and Round-Off Errors Consistency, Convergence, Stability Richardson Extrapolation Embedded Methods AutomaticStep-SizeControl Stability of One-Step Methods Extrapolation Methods Multi-Step Methods Predictor Corrector Methods Stability of Multi-Step Methods Backward Differentiation Methods...356

7 Contents xv 7.8 Conservative Second-Order Equations Runge Kutta Nyström Methods Multi-Step Methods Implicit Single-Step Methods SolutionbyNewton siteration Rosenbrock Linearization StiffProblems Implicit Multi-Step Methods Geometric Integration PreservationofInvariants PreservationoftheSymplecticStructure Reversibility and Symmetry ModifiedHamiltoniansandEquationsofMotion Lie-Series Integration Taylor Expansion of the Trajectory Problems Time Dependence of Filament Temperature Oblique Projectile Motion with Drag Force and Wind Influence of Fossil Fuels on Atmospheric CO 2 Content Synchronization of Globally Coupled Oscillators ExcitationofMuscleFibers Restricted Three-Body Problem (Arenstorf Orbits) LorenzSystem Sine Pendulum Charged Particles in Electric and Magnetic Fields ChaoticScattering Hydrogen Burning in the pp I Chain Oregonator Kepler sproblem NorthernLights Galactic Dynamics References Boundary-Value Problems for ODE Difference Methods for Scalar Boundary-Value Problems Consistency, Stability, and Convergence Non-linear Scalar Boundary-Value Problems Difference Methods for Systems of Boundary-Value Problems Linear Systems Schemes of Higher Orders Shooting Methods Second-Order Linear Equations Systems of Linear Second-Order Equations Non-linear Second-Order Equations Systems of Non-linear Equations...419

8 xvi Contents Multiple (Parallel) Shooting Asymptotic Discretization Schemes Discretization Collocation Methods Scalar Linear Second-Order Boundary-Value Problems Scalar Linear Boundary-Value Problems of Higher Orders Scalar Non-linear Boundary-Value Problems of Higher Orders Systems of Boundary-Value Problems Weighted-Residual Methods Boundary-Value Problems with Eigenvalues Difference Methods Shooting Methods with Prüfer Transformation PruessMethod Singular Sturm Liouville Problems Eigenvalue-Dependent Boundary Conditions Isospectral Problems Problems Gelfand Bratu Equation MeaslesEpidemic Diffusion-ReactionKineticsinaCatalyticPellet Deflection of a Beam with Inhomogeneous Elastic Modulus A Boundary-Layer Problem Small Oscillations of an Inhomogeneous String One-Dimensional Schrödinger Equation AFourth-OrderEigenvalueProblem References Difference Methods for One-Dimensional PDE DiscretizationoftheDifferentialEquation Discretization of Initial and Boundary Conditions Consistency Implicit Schemes Stability and Convergence Initial-Value Problems Initial-Boundary-Value Problems Energy Estimates and Theorems on Maxima EnergyEstimates Theorems on Maxima Higher-Order Schemes Hyperbolic Equations Explicit Schemes Implicit Schemes WaveEquation...490

9 Contents xvii 9.9 Non-linear Equations and Equations of Mixed Type Dispersion and Dissipation Systems of Hyperbolic and Parabolic PDE Conservation Laws and High-Resolution Schemes High-Resolution Schemes Linear Problem v t + cv x = Non-linear Conservation Laws of the Form v t +[F(v)] x = Problems DiffusionEquation Initial-Boundary Value Problem for v t + cv x = Dirichlet Problem for a System of Non-linear Hyperbolic PDE Second-Order and Fourth-Order Wave Equations BurgersEquation The Shock-Tube Problem Korteweg de Vries Equation Non-stationary Schrödinger Equation Non-stationary Cubic Schrödinger Equation References Difference Methods for PDE in Several Dimensions Parabolic and Hyperbolic PDE ParabolicEquations Explicit Scheme Crank Nicolson Scheme Alternating Direction Implicit Schemes Three Space Dimensions Hyperbolic Equations Explicit Schemes Schemes for Equations in the Form of Conservation Laws Implicit and ADI Schemes Elliptic PDE Dirichlet Boundary Conditions Neumann Boundary Conditions Mixed Boundary Conditions Relaxation Methods Conjugate Gradient Methods High-Resolution Schemes PhysicallyMotivatedDiscretizations Two-Dimensional Diffusion Equation in Polar Coordinates Two-Dimensional Poisson Equation in Polar Coordinates...544

10 xviii Contents 10.5 Boundary Element Method Finite-Element Method One Space Dimension Two Space Dimensions Mimetic Discretizations Multi-Grid and Mesh-Free Methods A Mesh-Free Method Based on Radial Basis Functions Problems Two-Dimensional Diffusion Equation Non-linear Diffusion Equation Two-Dimensional Poisson Equation High-Resolution Schemes for the Advection Equation Two-Dimensional Diffusion Equation in Polar Coordinates Two-Dimensional Poisson Equation in Polar Coordinates Finite-ElementMethod Boundary Element Method for the Two-Dimensional LaplaceEquation References Spectral Methods for PDE Spectral Representation of Spatial Derivatives Fourier Spectral Derivatives Legendre Spectral Derivatives Chebyshev Spectral Derivatives Computing the Chebyshev Spectral Derivative by FourierTransformation Galerkin Methods Fourier Galerkin Legendre Galerkin Chebyshev Galerkin Two Space Dimensions Non-stationary Problems Tau Methods Stationary Problems Non-stationary Problems Collocation Methods Stationary Problems Non-stationary Problems Spectral Elements: Collocation with B-Splines Non-linear Equations Time Integration Semi-Infinite and Infinite Definition Domains Complex Geometries...607

11 Contents xix 11.9 Problems Galerkin Methods for the Helmholtz Equation Galerkin Methods for the Advection Equation GalerkinMethodfortheDiffusionEquation Galerkin Method for the Poisson Equation: Poiseuille Law Legendre Tau Method for the Poisson Equation Collocation Methods for the Diffusion Equation I Collocation Methods for the Diffusion Equation II BurgersEquation References Appendix A Mathematical Tools A.1 AsymptoticNotation A.2 The Norms in Spaces L p ( ) and L p w( ), 1 p A.3 DiscreteVectorNorms A.4 MatrixandOperatorNorms A.5 Eigenvalues of Tridiagonal Matrices A.6 Singular Values of X and Eigenvalues of X T X and XX T A.7 The Square Root of a Matrix References Appendix B Standard Numerical Data Types B.1 RealNumbersinFloating-PointArithmetic B.1.1 Combining Types with Different Precisions B.2 IntegerNumbers B.3 (Almost)ArbitraryPrecision References Appendix C Generation of Pseudorandom Numbers C.1 Uniform Generators: From Integers to Reals C.2 TransformationsBetweenDistributions C.2.1 DiscreteDistribution C.2.2 Continuous Distribution C.3 Random Number Generators and Tests of Their Reliability C.3.1 Linear Generators C.3.2 Non-linear Generators C.3.3 Using and Testing Generators References Appendix D Convergence Theorems for Iterative Methods D.1 General Theorems D.2 Theorems for the Newton Raphson Method References...654

12 xx Contents Appendix E Numerical Integration E.1 Gauss Quadrature E.1.1 Gauss Kronrod Quadrature E.1.2 Quadrature in Two Dimensions E.2 Integration of Rapidly Oscillating Functions E.2.1 AsymptoticMethod E.2.2 Filon smethod E.3 Integration of Singular Functions References Appendix F Fixed Points and Stability F.1 Linear Stability F.2 Spurious Fixed Points F.3 Non-linear Stability References Appendix G Construction of Symplectic Integrators References Appendix H Transforming PDE to Systems of ODE: Two Warnings H.1 DiffusionEquation H.2 AdvectionEquation References Appendix I Numerical Libraries, Auxiliary Tools, and Languages I.1 ImportantNumericalLibraries I.2 BasicsofProgramCompilation I.3 UsingLibrariesinC/C++andFortran I.3.1 Solving Systems of Equations Ax = b by Using the GSLLibrary I.3.2 Solving the System Ax = b in C/C++ Language and I.3.3 FortranLibraries Solving the System Ax = b in Fortran95 by Using a Fortran77 Library I.4 Auxiliary Tools I.5 Choosing the Programming Language References Appendix J Measuring Program Execution Times on Linux Systems References Index...703

13