Numerical Analysis for Statisticians

Transcription

1 Kenneth Lange Numerical Analysis for Statisticians Springer

2 Contents Preface v 1 Recurrence Relations Introduction Binomial CoefRcients Number of Partitions of a Set Horner's Method Sample Means and Variances Expected Family Size Poisson-Binomial Distribution A Multinomial Test Statistic An Unstable Recurrence Quick Sort Problems 9 References 10 2 Power Series Expansions Introduction Expansion of P(s) n Application to Moments Expansion of e p W Moments to Cumulants and Vice Versa Compound Poisson Distributions Evaluation of Hermite Polynomials Standard Normal Distribution Function 15

3 x Contents 2.5 Incomplete Gamma Function Incomplete Beta Function Connections to Other Distributions Chi-Square and Standard Normal Poisson Binomial and Negative Binomial F and Student's t Monotonie Transformations Problems 21 References 24 3 Continued Fraction Expansions Introduction Wallis's Algorithm Equivalence Transformations Gauss's Expansion of Hypergeometric Functions Expansion of the Incomplete Gamma Function Problems 33 References 35 4 Asymptotic Expansions Introduction Order Relations Finite Taylor Expansions Expansions via Integration by Parts Exponential Integral Incomplete Gamma Function Laplace Transforms General Definition of an Asymptotic Expansion Laplace's Method Moments of an Order Statistic Stirling's Formula Posterior Expectations Validation of Laplace's Method Problems 49 References 51 5 Solution of Nonlinear Equations Introduction Bisection Computation of Quantiles by Bisection Shortest Xüonfidence Interval Functional Iteration Fractional Linear Transformations Extinction Probabilities by Functional Iteration.. 59

4 Contents xi 5.4 Newton's Method Division Without Dividing Extinction Probabilities by Newton's Method Problems 65 References 67 6 Vector and Matrix Norms Introduction Elementary Properties of Vector Norms Elementary Properties of Matrix Norms Iterative Solution of Linear Equations Jacobi's Method Pan and Reif's Iteration Scheme Equilibrium Distribution of a Markov Chain Condition Number of a Matrix Problems 77 References 78 7 Linear Regression and Matrix Inversion Introduction Motivation from Linear Regression Motivation from Multivariate Analysis Definition of the Sweep Operator Properties of the Sweep Operator Applications of Sweeping Gram-Schmidt Orthogonalization Woodbury's Formula Problems 87 References 90 8 Eigenvalues and Eigenvectors Introduction Jacobi's Method The Rayleigh Quotient Problems 100 References Splines Introduction Definition and Basic Properties Applications to Differentiation and Integration Application to Nonparametric Regression Problems 112 References 114

5 xii Contents 10 The EM Algorithm Introduction General Definition of the EM Algorithm Ascent Property of the EM Algorithm Technical Note Allele Frequency Estimation Transmission Tomography Problems 125 References Newton's Method and Scoring Introduction Newton's Method Scoring Generalized Linear Models The Gauss-Newton Algorithm Quasi-Newton Methods Problems 138 References Variations on the EM Theme Introduction Iterative Proportional Fitting EM Gradient Algorithm Application to the Dirichlet Distribution Bayesian EM Accelerated EM EM Algorithms Without Missing Data Quadratic Lower Bound Principle Elliptically Symmetrie Densities and L p Regression Transmission Tomography Revisited Problems 153 References Convergence of Optimization Algorithms Introduction Calculus Preliminaries Local Convergence Global Convergence Problems 170 References Constrained Optimization Introduction 177

6 Contents xiii 14.2 Necessary and Sufficient Conditions for a Minimum Quadratic Programming with Equality Constraints An Adaptive Barrier Method Standard Errors Problems 188 References Concrete Hubert Spaces Introduction Definitions and Basic Properties Fourier Series Orthogonal Polynomials Problems 204 References Quadrature Methods Introduction Euler-Maclaurin Sum Formula Romberg's Algorithm Adaptive Quadrature Taming Bad Integrands Gaussian Quadrature Problems 217 References The Fourier Transform Introduction Basic Properties Examples Further Theory Edgeworth Expansions Problems. 233 References The Finite Fourier Transform Introduction Basic Properties Derivation of the Fast Fourier Transform Approximation of Fourier Series Coefficients Convolution Time Series Problems 247 References 250

7 xiv Contents 19 Wavelets Introduction Haar's Wavelets Histogram Estimators Daubechies' Wavelets Multiresolution Analysis Image Compression and the Fast Wavelet Transform Problems 265 References Generating Random Deviates Introduction The Inverse Method Normal Random Deviates Acceptance-Rejection Method Ratio Method Deviates by Definition Multivariate Deviates Problems 281 References Independent Monte Carlo Introduction Importance Sampling Stratifled Sampling Antithetic Variates Control Variates Rao-Blackwellization Exact Tests of Independence in Contingency Tables Problems 295 References Bootstrap Calculations Introduction Range of Applications Balanced Bootstrap Simulations Antithetic Bootstrap Simulations Importance Resampling Problems 310 References Finite-State Markov Chams Introduction Discrete-Time Markov Chams Hidden Markov Chains 318

8 Contents xv 23.4 Continuous-Time Markov Chains Calculation of Matrix Exponentials Problems 325 References Markov Chain Monte Carlo Introduction The Hastings-Metropolis Algorithm Gibbs Sampling Other Examples of Hastings-Metropolis Sampling Some Practical Advice Convergence of the Independence Sampler Simulated Annealing Problems 340 References 342 Index 345