1196 Index. hidden Markov model

Transcription

1 Index Abstract Base Class (ABC) 24, 33, 34, 87, 114, 703, 874 Accelerated convergence of series 177, Accuracy 8 12 achievable in minimization 493, 497, 503 achievable in root finding 448 contrasted with fidelity 1037, 1046 CPU different from memory 230 vs. stability 907, 931, 932, 1035, 1050 Adams-Bashford-Moulton method 943 Adams stopping criterion 467 Adaptive integration 901, , 928, 930, 935, 946, 995 Monte Carlo PI stepsize control 915 predictive stepsize control 939 see also Adaptive quadrature Adaptive quadrature 155, 167, and singularities 195 termination criterion 194 Addition multiple precision 1186 theorem, elliptic integrals 310 ADI (alternating direction implicit) method 1052, 1053, 1065, 1066, 1185 Adjoint operator 1071 Advanced topics (explanation) 6 Advective equation 1032 Affine scaling 543 Agglomerative clustering AGM (arithmetic geometric mean) 1185 Airy function 254, 283, 289, 291 routine for 290 Aitken s delta squared process 212, 214 interpolation algorithm 118 Algorithms, less-numerical Aliasing 606, 685 see also Fourier transform Alignment of strings by DP All-poles or all-zeros models 681, 682 see also Maximum entropy method (MEM); Periodogram Alternating-direction implicit method (ADI) 1052, 1053, 1065, 1066, 1185 Alternating series 211, 216 Alternative extended Simpson s rule 160 AMD (approximate minimum degree) 544, 548 Amoeba 503 see also Simplex, method of Nelder and Mead Amplification factor 1033, 1035, 1038, 1045, 1046 Amplitude error 1036 Analog-to-digital converter 1018, 1166 Analyticity 246 Analyze/factorize/operate package 76, 1030 Anderson-Darling statistic 739 Andrew s sine 821 Angle between vectors 1120, 1121 coordinates on n-sphere 1128 exterior, of polygons 1122 Annealing, method of simulated 487, 488, assessment 554, 555 for continuous variables 550, schedule 551, 552 thermodynamic analogy 550 traveling salesman problem 551, 552 ANSI ANSI/ISO C++ standard 5 Antonov-Saleev variant of Sobol sequence , 408, 409 Apple Mac OS X 5 Approximate inverse of matrix 63 Approximation of functions 110 by Chebyshev polynomials 234, 625 by rational functions by wavelets 711, 712, 989 Padé approximant 212, see also Fitting Area polygon 1126 sphere in n-dimensions 1128 triangle 1111 Arithmetic arbitrary precision 1160, floating point 1163 IEEE standard 1164, 1165 rounding 1164, bit 341 Arithmetic coding 755, 1160,

2 1196 Index Arithmetic-geometric mean (AGM) method 1185 Array assign function 27 centered subarray of 115 classes for resize function 27 size function 27 three-dimensional 36 unit-offset 36 zero-offset 36 Artificial viscosity 1037, 1042 Ascending transformation, elliptic integrals 310 ASCII character set 1168, 1175, 1181 assign 27 Associated Legendre polynomials 971 recurrence relation for 294 relation to Legendre polynomials 293 Association, measures of 721, 741, Asymptotic series 210, 216 exponential integral 216, 269 Attenuation factors 698 Autocorrelation in linear prediction use of FFT 648, 649 Wiener-Khinchin theorem 602, 682 Autoregressive model (AR) see Maximum entropy method (MEM) Average deviation of distribution 723 Averaging kernel, in Backus-Gilbert method 1014 B-spline 148 Backsubstitution 47, 49, 53, 56, 103 complex equations 55 direct for computing A 1 B 53 in band-diagonal matrix 60 relaxation solution of boundary value problems 966 Backtracking 522 in quasi-newton methods Backus-Gilbert method Backward deflation 464, 465 Bader-Deuflhard method 940 Bahl-Cocke-Jelinek-Raviv algorithm forward-backward algorithm 867 Bairstow s method 466, 471 Balancing 592, 594 Band-diagonal matrix 56, backsubstitution 60 LU decomposition 59 multiply by vector 58 storage 58 Band-pass filter 667, 670 wavelets 701 Bandwidth limited function 605 Bank accounts, checksum for 1174 Bar codes, checksum for 1174 Barrier method 541 Bartels-Golub update 535 Bartlett window 657 Barycentric coordinates 1114, 1116 Barycentric rational interpolation 113, 127, 128 Base class 23 Base of representation 8, 1164 Basin of convergence 461, 463 Basis functions in general linear least squares 788 Baum-Welch re-estimation hidden Markov model relation to expectation-maximization 866 Bayes theorem 774, 777, 825 Bayesian approach to inverse problems 1005, 1022 contrasted with frequentist 774 estimation of parameters by MCMC 774, lack of goodness-of-fit methods 779, 1010 normalizing constant 779 odds ratio 757, 779 parameter estimation 777, 778 prior 757, 775, 777, 1005 views on straight line fitting 787 vs. historic maximum entropy method 1022 Bayesian algorithms hidden Markov model 868 Viterbi decoding 868 Bayesian networks 840, 841 node parents 841 nodes 840 posterior probabilities 841 prior probabilities 841 Bayesian re-estimation hidden Markov model Belief networks 840 forward-backward algorithm 867 Bellman-Dijkstra-Viterbi algorithm 556, 850, 853 Berlekamp-Massey decoding algorithm 852 Bernoulli number 164 Bessel functions asymptotic form 274, 279, 284 complex 254 continued fraction 283, 284, 287, 288 fractional order 274, Miller s algorithm 221, 278 modified modified, fractional order modified, normalization formula 282, 288 modified, routines for 280 normalization formula 221 recurrence relation 219, 274, 275, 278, 281, reflection formulas 286 reflection formulas, modified functions 289 routines for 276, 286 routines for modified functions 289 series for 210, 274 series for K 288 series for Y 284, 285 spherical 283, 291, 292 turning point 283

3 Index 1197 Wronskian 283, 284, 287 Best-fit parameters 773, 781, 785, see also Fitting Beta function 256, 258, 259 incomplete see Incomplete beta function Beta probability distribution 333, 334 deviates 371 gamma as limiting case 333 Betting , 760, 761 fair bet 755, 756, 758, 760, 761 proportional 758, 760 Bezier curve 148 BFGS algorithm see Broyden-Fletcher-Goldfarb-Shanno algorithm Bias of exponent 8 removal in linear prediction 145, 678, 679 Biconjugacy 88 Biconjugate gradient method elliptic partial differential equations 1030 for sparse system 88, 716 preconditioning 89, 1030 Bicubic interpolation Bicubic spline 135 Big-endian 9 Biharmonic equation 153 Bilinear interpolation 133, 134 Binary block code 851 Binomial coefficients 256, 258 recurrences for 258 Binomial probability function 258, 338, 339 deviates from moments of 735 Poisson as limiting case 338 Binormal distribution 746, 813 Biorthogonality 88 Bisection 115, 460 compared to minimum bracketing 492 root finding 445, , 454, 492, 584 Bispectrum 604 Bit 8, , 760, 761 phantom 9 pop count 16 reversal in fast Fourier transform (FFT) 610, 638 Bit-parallel random comparison 374 Bit-twiddling hacks 16 Bitwise logical functions 1170 test if integer a power of 2 16, 611 trick for next power of 2 16, 361 Black-Scholes formula 329 BLAST (software) 562 BLAT (software) 562 Block-by-block method 994 Bluetooth 1168 Bode s rule 158 Boltzmann probability distribution 550 Boltzmann s constant 550 Bolyai-Gerwien theorem 1127 Bookie, information theory view of 758 Bool 25 Bootstrap method 809, 810 Bordering method for Toeplitz matrix 96 Borwein and Borwein method for 1185 Boundary 196, 528, 955 Boundary conditions for differential equations 900 for spheroidal harmonics 972, 973 in multigrid method 1072 initial value problems 900 partial differential equations 620, 1025, two-point boundary value problems 900, Boundary value problems 1026 see also Differential equations; Elliptic partial differential equations; Two-point boundary value problems Bounds checking 35 in vector by at 35 Box test if point inside 1100 tree of, as data structure 1101 Box-Muller algorithm for normal deviate 364 Bracketing of function minimum 445, , 503 of roots 443, , 454, 455, 464, 465, 470, 492 Branch cut, for hypergeometric function Break iteration 15 Brenner s FFT implementation 611, 628 Brent s method minimization 489, , 785 minimization, using derivative 489, 499, 500 root finding 443, 449, , 459, 786 Broyden-Fletcher-Goldfarb-Shanno algorithm 490, Broyden s method 474, singular Jacobian 486 Bubble sort 420 Bugs, how to report 5 Bulirsch-Stoer algorithm for rational function interpolation 125 for second order equations 929 method 252, 318, 900, 901, 909, , 942 method, dense output 927 method, implementation 927 method, stepsize control , 929 Burg s LP algorithm 677 Burn-in 826, Butterfly 360, 361, 610 Byte 8 C (programming language) 1 FILE and LINE macros 30 idioms 16 syntax C++

4 1198 Index ANSI/ISO standard 5 C family syntax const statement 31, 32 contiguous storage for vector 27 control structures 14, 15 error class 30 inline directive 29 NR not a textbook on 2 operator associativity 12 operator precedence 12 overloading 28 scope, temporary 20, 21 standard library 10, 24 templates 17, 22, 26, 33, 34, 419, 421 throw 30 try and catch 30 types 25 types used in NR 4 user-defined conversions 31 valarray class 25 vector class 24 virtual function 33 why used in NR 1 C# (programming language) 1, 12 Calendar algorithms 2, 3, 6, 7 Calibration 778 Cardinal functions Cards, sorting a hand of 420, 422 Carlson s elliptic integrals Carpe diem 830 catch 30 Cauchy principal value integrals 178 Cauchy probability distribution 322, 323 deviates from 367 see also Lorentzian probability distribution Cauchy problem for partial differential equations 1024 Cavender-Felsenstein model 873 Cayley s representation of exp. iht/ 1049 CCITT (Comité Consultatif International Télégraphique et Téléphonique) 1171, 1180 CCITT CDF see Cumulative Distribution Function Center of mass 399, 400, 1113, 1127 Central limit theorem 777 Central tendency, measures of 721 Centroid see Center of mass Change of variable in integration , 995 in Monte Carlo integration 401 in probability distribution 362 Char 25 Character-based clustering methods 869 Characteristic polynomial digital filter 670 eigensystems 563, 583, 665 linear prediction 676 matrix with a specified 469 of recurrence relation 221 of tridiagonal system 665 Characteristics of partial differential equations Chebyshev acceleration in successive over-relaxation (SOR) 1064 Chebyshev approximation 95, 156, Clenshaw-Curtis quadrature 241 Clenshaw s recurrence formula 236 coefficients for 234 contrasted with Padé approximation 245 derivative of approximated function 232, 240, 241 economization of series even function 237 fast cosine transform and 625 for error function 264 gamma functions 285 integral of approximated function 240, 241 odd function 237 polynomial fits derived from 241, 243, 248 rational function Remes exchange algorithm for filter 669 Chebyshev polynomials 183, 187, basis functions for spectral methods 1085 continuous orthonormality 233 discrete orthonormality 233 explicit formulas for 233 formula for x k in terms of 233 Check digit (decimal) 1173 Checksum 1160, cyclic redundancy (CRC) Chemical reaction networks Chi-by-eye 774 Chi-square fitting see Fitting; Least-squares fitting Chi-square probability function 330, 331, 732, 778, 779, 1003 as boundary of confidence region 812 deviates from 371 Chi-square test and confidence limit estimation 812 chi-by-eye 774 chi-square-gamma test 735 degrees of freedom 732, 733 for binned data for contingency table for inverse problems 1003 for straight line fitting 781 for straight line fitting, errors in both coordinates 785 for two binned data sets 732 goodness-of-fit 780 how much 2 is significant 816 least-squares fitting and likelihood ratio test 735 modified Neyman 735 nonlinear models 799 small numbers of counts 734, 735 for two binned data sets 735 unequal size samples 733 Chirp signal 672 Cholesky decomposition , 525, 568

5 Index 1199 and covariance structure 378, 379 decorrelating random variables 379 multivariate Gaussian distribution 847, 848 operation count 100 pivoting 101 solution of normal equations 543, 790, 791 sparse decomposition 544, 548 Circle inscribed or circumscribed 1112 largest empty 1147 random point on 1131 Circulant 700 Circumscribed circle (circumcircle) 1112 CLAPACK 567 Class abstract base (ABC) 24, 33, 34, 87, 114, 703, 874 base class 23 derived 23 error class 30 inheritance 23, 24 is-a relationship 23 matrix partial abstraction via 24 prerequisite relationship 23 public vs. private 17 pure virtual 34 suffix _I, _O, _IO 26, 32 templated 22, 33, 34 vector see also Object Class library 2 Classification kernel methods 889, 892 support vector machine Clenshaw-Curtis quadrature 156, 241, 624, 625 Clenshaw s recurrence formula 219, 222, 223 for Chebyshev polynomials 236 stability 223 Clock, program timing routine 355 Clocking errors 1172 CLP (linear programming package) 536 Clustering agglomerative hierarchical k-means neighbor-joining (NJ) 873, cn function 316 Coarse-grid correction 1068 Coarse-to-fine operator 1068 Codes binary block codes 851 codeword 851 correcting bit errors 855 error-correcting Golay code 852 Hamming code 852 Hamming distance 851, 1168 hard-decision decoding 853 linear codes 851 minimal trellis 853 perfect code 852 Reed-Solomon 852, 855 soft-decision decoding 853 syndrome decoding 852 trellis 853, 856 turbo codes 855 Viterbi decoding 854 Coding arithmetic 755, checksums compression 754, 756 decoding a Huffman-encoded message 1178 Huffman 713, run-length 1180 variable length code 1176 Ziv-Lempel 1176 see also Arithmetic coding; Huffman coding Coefficients binomial 258 for Gaussian quadrature 179, 180 for Gaussian quadrature, nonclassical weight function , 995 for quadrature formulas , 995 Column operations on matrix 43, 45 Column totals 743, 759 Combinatorial minimization see Annealing Comité Consultatif International Télégraphique et Téléphonique (CCITT) 1171, 1180 Communications protocol 1168 Comparison function for rejection method 366 Compiler check on via constructors 36 tested 5 Complementary error function see Error function Complete elliptic integral see Elliptic integrals Complex arithmetic 225, 226 access vector as if complex 613, 620 avoidance of in path integration 253 Complex type 25 cubic equations 228, 229 linear equations 55 quadratic equations 227 Complex error function 302 Complex plane fractal structure for Newton s rule 462 path integration for function evaluation , 318 poles in 124, 210, 252, 256, 670, 682, 922 Complex systems of linear equations 55 Compression of data 713, 715, 1160, Computational geometry floating point arithmetic in 1098 Computer graphics 1097 Computer vision 1097 Concordant pair for Kendall s tau 751 Condition number 69, 89, 791, 793 Conditional entropy Confidence level 810, 811, Confidence limits

6 1200 Index and chi-square 811 bootstrap method 809, 810 by Monte Carlo simulation confidence region, confidence interval 810, 811 from singular value decomposition (SVD) 816, 817 on estimated model parameters Confluent hypergeometric function 254, 287 Conjugate directions 509, 511, 512, 516 Conjugate gradient method and wavelets 716 biconjugate 88 compared to variable metric method 521 elliptic partial differential equations 1030 for minimization 489, , 1011, 1020 for sparse system 87 92, 716 minimum residual method 89 preconditioner 89, 90 Conservative differential equations 928, 930 const correctness 26, 31, 32 protects container, not contents 31, 32 to protect data 32 Constellation in Viterbi decoding 855 Constrained linear inversion method 1006 Constrained linear optimization see Linear programming Constrained optimization 487 Constraints deterministic linear 526, 530 Constructor 18, 27 Container, STL 421 Contingency coefficient C 743, 744 Contingency table , 753, 758, 759 statistics based on chi-square statistics based on entropy Continue statement 15 Continued fraction and recurrence relation 222 Bessel functions 283, 284, 288 convergence criterion 208 equivalence transformation 208 evaluation evaluation along with normalization condition 288 even and odd parts 208, 260, 267, 298, 301 exponential integral 267 Fresnel integral 298 incomplete beta function 270 incomplete gamma function 260 Lentz s method 207, 260 modified Lentz s method 208 Pincherle s theorem 222 ratio of Bessel functions 287 rational function approximation 207, 260 recurrence for evaluating 207, 208 sine and cosine integrals 301 Steed s method 207 tangent function 206 typography for 206 Continuous variable (statistics) 741 Control structures and scope 21 Convergence accelerated, for series 177, basin of 461, 463 criteria for 448, 493, 503, 598, 599, 802, 969 eigenvalues accelerated by shifting 585 exponential , 180, 238, 239, golden ratio 449, 500 hyperlinear (series) 211 linear 448, 495 linear (series) 211 logarithmic (series) 211 Markov model 858 of algorithm for 1185 of golden section search 494, 495 of Levenberg-Marquardt method 802 of QL method 584, 585 of Ridders method 452 quadratic 64, 452, 459, 511, 512, 522, 1185 rate 448, 454, 457, 459 recurrence relation 222 series vs. continued fraction 206 spectral radius and 1061, 1066 Conversions, user-defined 31 Convex hull 1097, 1132, 1146 Convex sets, use in inverse problems Convolution and polynomial interpolation 129 denoted by asterisk 602 finite impulse response (FIR) 642, 643 multiple precision arithmetic 1188 multiplication as 1188 necessity for optimal filtering 645 of functions 602, 616, 617, 631 of large data sets 646, 647 overlap-add method 647 overlap-save method 646 relation to wavelet transform 700, 701 theorem 602, 641, 656 theorem, discrete 642, 643 treatment of end effects 643 use of FFT wraparound problem 643 Cooley-Tukey FFT algorithm 616 Cornwell-Evans algorithm 1021 Corporate promotion ladder 427 Corrected distance transformation 873 Corrected two-pass algorithm 724 Correction, in multigrid method 1067 Correlation coefficient (linear) Correlation function 602, 617 and Fourier transforms 602, 617 autocorrelation 602, 649, theorem 602, 648 three-point 604 treatment of end effects 648

7 Index 1201 using FFT 648, 649 Wiener-Khinchin theorem 602, 682 Correlation, statistical 721, 741 among parameters in a fit 782, 793 Kendall s tau 749, linear correlation coefficient , 783 linear related to least-squares fitting 745, 783 nonparametric or rank statistical Spearman rank-order coefficient sum squared difference of ranks 749 uncertainty coefficient 761 Coset leader 852 Cosine function, recurrence 219 Cosine integral 297, continued fraction 301 routine for 301 series 301 Cosine transform see Fast Fourier transform (FFT); Fourier transform Coulomb wave function 254, 283 Counts, small numbers of 734, 735 Courant condition 1034, 1036, , 1042 multidimensional 1051 Courant-Friedrichs-Lewy stability criterion see Courant condition Covariance a priori 824 from singular value decomposition (SVD) 817 in general linear least squares 790, 791, 794 in nonlinear models 802 in straight line fitting 782 matrix, and normal equations 790 matrix, Cholesky decomposition 101 matrix, is inverse of Hessian matrix 802 matrix, of errors 1003, 1015 matrix, when it is meaningful 812, 813 relation to chi-square CR method see Cyclic reduction (CR) Cramer s V 743, 744 Crank-Nicolson method 1045, 1049, 1051, 1052 Cray, Seymour 1163 CRC (cyclic redundancy check) CRC Critical (Nyquist) sampling 605, 607, 653 Cross (denotes matrix outer product) 78 Crosstabulation analysis 742 see also Contingency table Crout s algorithm 49, 59 Cubic equations , 461 Cubic spline interpolation see also Spline Cumulative Distribution Function (cdf) 435 Curse of dimensionality 556, 891 Curvature matrix see Hessian matrix Curve interpolation 147 Cycle, in multigrid method 1069 Cyclic Jacobi method 573 Cyclic reduction (CR) 224, 1054, 1057, 1058 Cyclic redundancy check (CRC) Cyclic tridiagonal systems 79, 80 D.C. (direct current) 602 Danielson-Lanczos lemma 609, 610, 638 Data continuous vs. binned 731 entropy , 1176 essay on 720 fitting fraudulent 780 glitches in 777 iid (independent and identically distributed) 809 linearly separable 884 missing data points modeling smoothing 721, statistical tests unevenly or irregularly sampled , 685, 690, 771 use of CRCs in manipulating 1169 windowing see also Statistical tests Data compression 713, 715, 1160 arithmetic coding cosine transform 625 Huffman coding 713, linear predictive coding (LPC) lossless 1175 Data Encryption Standard (DES) Data type 8 DAUB4 700, 702, 706, 707, 711, 715 DAUB6 702 DAUB Daubechies wavelet coefficients , 704, , 715 Davidon-Fletcher-Powell algorithm 490, 521, 522 Dawson s integral 302, 304, 717 approximation for 303 routine for 303 DE rule 174 implementation 175 infinite range 176 Decoding Berlekamp-Massey algorithm for Reed-Solomon code 852 directed graph 556, 850 hard-decision 853 hard-decision vs. soft-decision 855 maximum likelihood 854 Reed-Solomon codes 855 soft-decision decoding 853 syndrome decoding 852 Turbo codes 855 Viterbi algorithm 854 Viterbi, compared to hidden Markov model 867, 868 Decomposition see Cholesky decomposition; LU decomposition; QR decomposition; Singular value decomposition (SVD)

8 1202 Index Deconvolution , 650 see also Convolution; Fast Fourier transform (FFT); Fourier transform Decorrelating random variables 379 Defect, in multigrid method 1067 Deferred approach to the limit see Richardson s deferred approach to the limit Deflation of matrix 585 of polynomials , 471 Degeneracy kernel 992 linear algebraic equations 73, 793 minimization principle 1002 Degrees of freedom 732, 733, 778, 779, Delaunay triangulation 1097, applications of incremental constructions 1134 interpolation using 1141 largest minimum angle property 1134 minimum spanning tree 1147 not minimum weight 1134 Delone, B.N. see Delaunay Triangulation Dense output, for differential equations 904, 915, 927 Dependencies, program 4 Dependency graph or matrix 949 Derivatives approximation by sinc expansion 178 computation via Chebyshev approximation 232, 240, 241 computation via Savitzky-Golay filters 232, 769 matrix of first partial see Jacobian determinant matrix of second partial see Hessian matrix numerical computation , 480, 769, 936, 960, 978 of polynomial 202 use in optimization Derived class 23 DES see Data Encryption Standard Descending transformation, elliptic integrals 310 Descent direction 478, 484, 522 Descriptive statistics see also Statistical tests Design matrix 768, 788, 1002 Design of experiments 410 Detailed balance equation Determinant 39, 54, 55 Devex 535 Deviates, random see Random deviates DFP algorithm see Davidon-Fletcher-Powell algorithm Diagonal dominance 57, 802, 987, 1060 Diagonal rational function 125 Diehard test, for random numbers 345 Difference equations, finite see Finite difference equations (FDEs) Difference operator 212 Differential equations accuracy vs. stability 907, 931 Adams-Bashforth-Moulton schemes 943 adaptive stepsize control 901, , , 929, 939, 941, 943, 944, 946 algebraically difficult sets 970 backward Euler s method 932 Bader-Deuflhard method for stiff 940 boundary conditions 900, 955, 962, 977 Bulirsch-Stoer method 252, 318, 900, 901, 909, , 942 Bulirsch-Stoer method for conservative equations 928, 930 comparison of methods 900, 901, 942, 946, 957 conservative 928, 930 dense output 904, 915, 927 discreteness effects eigenvalue problem 958, 973, embedded Runge-Kutta method 911, 936 equivalence of multistep and multivalue methods 945 Euler s method 900, 907, 931 forward Euler s method 931 free boundary problem 958, 983 global vs. local error 914 high-order implicit methods 934 implicit differencing 932, 933, 944 initial value problems 900 integrating to an unknown point 916 internal boundary conditions 983, 984 internal singular points 983, 984 interpolation on right-hand sides 115 Kaps-Rentrop method for stiff 934 local extrapolation 911 modified midpoint method 922, 923 multistep methods 900, multivalue methods Nordsieck method 944 order of method 907, 922 path integration for function evaluation , 318 predictor-corrector methods 900, 909, 934, r.h.s. independent of x 932, 934 reduction to first-order sets 899, 956 relaxation method 957, relaxation method, example of 971, Rosenbrock methods for stiff Runge-Kutta method 900, , 934, 942, 1096 Runge-Kutta method, high-order , 912 scaling stepsize to required accuracy 913, 914 second order 928, 930 semi-implicit differencing 934 semi-implicit Euler method 934, 940 semi-implicit extrapolation method 934, 935, 940, 941 semi-implicit midpoint rule 940 shooting method 956,

9 Index 1203 shooting method, example 971, similarity to Volterra integral equations 993 singular points 921, 962, 983, 984 solving with sinc expansions 178 step doubling 910 stepsize control 901, , 924, 929, 938, 941, 944, 946 stepsize, danger of too small 920 stiff 901, stiff methods compared 941 stochastic simulation Stoermer s rule 928 see also Partial differential equations; Two-point boundary value problems Differentiation matrix 1091 routine for 1092 Diffusion equation 1024, , 1059 Crank-Nicolson method 1045, 1049, 1051, 1052 forward time centered space (FTCS) 1044, 1046, 1059 implicit differencing 1045 multidimensional 1051, 1052 Digamma function 267 Digital filtering see Filter Dihedral angle 1116 Dihedral group D Dimensionality, curse of 556, 891 Dimensions (units) 801 Diminishing increment sort 422 Dingbats, Zapf 1162 Dirac delta function 700, 987 Direct method see Periodogram Direct methods for linear algebraic equations 40 Direct product see Outer product of matrices Directed graph Markov model 856 stages and states 556, 850 transition matrix 856 transition probability 856 trellis 856 Viterbi decoding 850 Direction numbers, Sobol s sequence 404 Direction of largest decrease 512 Direction set methods for minimization 489, Dirichlet boundary conditions 1026, 1045, 1055, 1061, 1063 Discordant pair for Kendall s tau 751 Discrete convolution theorem 642, 643 Discrete Fourier transform (DFT) approximation to continuous transform 607, 608 see also Fast Fourier transform (FFT) Discrete optimization 536, 549 Discrete prolate spheroidal sequence (dpss) Discretization error 173 Discriminant 227, 572 Dispersion 1036 DISPO see Savitzky-Golay filters Dissipation, numerical 1035 Distance matrix 869 Distributions, statistical see Statistical distributions Divergent series 210, 211, 216 Divide-and-conquer method 589 Division complex 226 integer vs. floating 8 multiple precision 1190 of polynomials 204, 464, 471 dn function 316 DNA sequence , 869, 884 Do-while iteration 15 Dogleg step methods 486 Domain of integration 196 Dominant solution of recurrence relation 220 Dormand-Prince parameters 912, 920 Dot denotes matrix multiplication 37 denotes row or column sums 759 Doub 25 Double exponential error distribution 820 Double root 443 Doubling rate 756 Downhill simplex method see Simplex, method of Nelder and Mead DP see Dynamic programming dpss (discrete prolate spheroidal sequence) Dual problem 538, 886 Dual viewpoint, in multigrid method 1077 Duality gap 538 Duplication theorem, elliptic integrals 311 DWT (discrete wavelet transform) see Wavelet transform Dynamic programming Bellman-Dijkstra-Viterbi algorithm 556, 850 directed graph 556, 850 e-folding rate 756 Eardley s equivalence class method 440 Economization of power series Eigensystems and integral equations 987, 992 balancing matrix 592, 594 bounds on eigenvalues 64 calculation of few eigenvectors or eigenvalues 568, 598 canned routines 567 characteristic polynomial 563, 583 completeness 564, 565 defective 564, 591, 598, 599 deflation 585 degenerate eigenvalues 563, 565 divide-and-conquer method 589 eigenvalues 563 elimination method 567, 594 factorization method 567 fast Givens reduction 578

10 1204 Index generalized eigenproblem 568, 569 Givens reduction Givens transformation 587 Hermitian matrix 590 Hessenberg matrix 567, 585, , 598 Householder transformation 567, , 587, 590, 594 ill-conditioned eigenvalues 591, 592 implicit shifts invariance under similarity transform 566 inverse iteration 568, 584, 589, Jacobi transformation 567, , 578, 590, 599 left eigenvalues 565 list of tasks 568 Markov model transition matrix 858, 859 MRRR algorithm 589, 599 multiple eigenvalues 599 nonlinear 568, 569 nonsymmetric matrix operation count of balancing 592 operation count of Givens reduction 578 operation count of Householder reduction 582 operation count of inverse iteration 598, 599 operation count of Jacobi method 573, 574 operation count of QL method 585, 588 operation count of QR method for Hessenberg matrices 596 operation count of reduction to Hessenberg form 594 orthogonality 564 polynomial roots and 469 QL method , 590 QR method 67, 567, 571, QR method for Hessenberg matrices 596 real symmetric matrix 188, 576, 577, 582, 992 reduction to Hessenberg form 594, 595 relation to singular value decomposition (SVD) 569, 570 right eigenvalues 565 shifting eigenvalues 563, 585, 596 special matrices 568 termination criterion 598, 599 tridiagonal matrix 567, 576, 577, , 598 Eigenvalue and eigenvector, defined 563 Eigenvalue problem for differential equations 958, 973, Eigenvalues and polynomial root finding 469 EISPACK 567 Electromagnetic potential 631 Elimination see Gaussian elimination Ellipse in confidence limit estimation 811, 814, 815 Elliptic integrals , 1185 addition theorem 310 Carlson s forms and algorithms Cauchy principal value 311 duplication theorem 311 Legendre 309, 314, 315 routines for symmetric form 309, 310 Weierstrass 310 Elliptic partial differential equations 1024 alternating-direction implicit method (ADI) 1065, 1066, 1185 analyze/factorize/operate package 1030 biconjugate gradient method 1030 boundary conditions 1026 comparison of rapid methods 1058 conjugate gradient method 1030 cyclic reduction 1054, 1057, 1058 Fourier analysis and cyclic reduction (FACR) Gauss-Seidel method 1060, 1061, 1068, 1078 Jacobi s method 1060, 1061, 1068 matrix methods 1028, 1030 multigrid method 1030, rapid (Fourier) method 1029, relaxation method 1028, spectral methods 1096 successive over-relaxation (SOR) , 1070 EM algorithm see Expectation-maximization algorithm Embedded networks 1168 Embedded Runge-Kutta method 911, 936 Encryption 358 Entropy , 1006, 1176 chain rule 759 conditional of data 1017 relative 756 EOM (end of message) 1178, 1181 epsilon () algorithm 212 Equality constraints 526, 528 Equations cubic , 461 differential see Differential equations normal (fitting) 768, , 1007 quadratic 10, see also Differential equations; Partial differential equations; Root finding Equilibrium, physical 825 Equivalence classes 419, Equivalence transformation 208 Ergodic Markov model 858 Ergodic property 825 Error 8 12 checksums for preventing 1172 clocking 1172 discretization 173 double exponential distribution 820 in multigrid method 1067 interpolation 113 local truncation 1077, 1078 Lorentzian distribution 820

11 Index 1205 nonnormal 779, 812, relative truncation 1077 roundoff 10, 11, 229, 1163, 1164 series, advantage of an even 165, 923 systematic vs. statistical 778 trimming 173 truncation 11, 173, 229, 500, 910, 911, 1163 varieties of, in PDEs see also Roundoff error Error-correcting codes Berlekamp-Massey decoding algorithm 852 binary block code 851 codeword 851 correcting bit errors 855 coset leader 852 Golay code 852 Hamming code 852 Hamming distance 851, 1168 hard-decision decoding 853 linear codes 851 minimal trellis 853 parity-check matrix 851 perfect code 852 Reed-Solomon 852, 855 soft-decision decoding 853 syndrome 852 syndrome decoding 852 trellis 853, 856 turbo codes 855 Viterbi decoding 854, 855 Error ellipse, how to draw 817, 847 Error function 259, , 718 approximation via sampling theorem 718, 719 Chebyshev approximation 264 complex 302 Fisher s z-transformation 747 inverse 264 relation to Dawson s integral 302 relation to Fresnel integrals 298 relation to incomplete gamma function 264 routine for 264 significance of correlation 746 sum squared difference of ranks 750 Error handling in programs 2, 30, 31, 35 Estimation of parameters see Fitting; Maximum likelihood estimate Estimation of power spectrum Euclid 1098 Euler equation (fluid flow) 1037 Euler-Maclaurin summation formula 164, 167 Euler s constant 267, 269, 300 Euler s method for differential equations 900, 907, 931 Euler s transformation 211, 212 Evaluation of functions see Function Even and odd parts, of continued fraction 208, 260, 267 Even parity 1168 Exception handling in programs 2, 30, 31, 35 Expectation-maximization algorithm expectation step (E-step) 843 for hidden Markov model 866 maximization step (M-step) 843 relation to Baum-Welch re-estimation 866 Explicit differencing 1032 Exponent in floating point format 8, 1164 Exponential convergence , 180, 238, 239, Exponential integral asymptotic expansion 269 continued fraction 267 recurrence relation 219 related to incomplete gamma function 267 relation to cosine integral 301 routine for Ei.x/ 269 routine for E n.x/ 268 series 267 Exponential probability distribution 326, 327, 686 deviate from 362 relation to Poisson process 369, 829 Extended midpoint rule 157, 161, 167 Extended precision, use in iterative improvement 62 Extended Simpson s rule 160, 994, 997 three-eighths rule 995 Extended trapezoidal rule 157, 159, 162, 167, 993 roundoff error 165 Extirpolation (so-called) 690, 691 Extrapolation Bulirsch-Stoer method 922, 924 by linear prediction differential equations 900 local 911 maximum entropy method as type of 683 polynomial 922, 924, 943 rational function 922 relation to interpolation 110 Romberg integration 166 see also Interpolation Extremization see Minimization F -distribution probability function 332, 333 deviates 371 F-test for differences of variances 728, 730 FACR see Fourier analysis and cyclic reduction (FACR) Facsimile standard 1180 Factorial evaluation of 210 relation to gamma function 256 representability 257 routine for 257 routine for log 258 False position 449, 452, 454 Family tree 440 FAS (full approximation storage algorithm)

12 1206 Index Fast Fourier transform (FFT) , 640, 1160 alternative algorithms 615, 616 applications approximation to continuous transform 608 bare routine for 611 bit reversal 610, 638 butterfly 360, 361, 610 Clenshaw-Curtis quadrature 241 convolution 616, 631, , 1189 convolution of large data sets 646, 647 Cooley-Tukey algorithm 616 correlation 648, 649 cosine transform 241, , 1056 cosine transform, second form 625, 1057 Danielson-Lanczos lemma 609, 610, 638 data sets not a power of data smoothing 766, 767 data windowing decimation-in-frequency algorithm 616 decimation-in-time algorithm 615 decomposition into blocks 614 differentiation matrix using 1092 discrete autocorrelation 649 discrete convolution theorem 642, 643 discrete correlation theorem 648 double frequency 690 endpoint corrections 694 external storage 637, 638 figures of merit for data windows 658 filtering FIR filter 668, 669 for quadrature 156 for spherical harmonic transforms 296 four-step framework 615 Fourier integrals Fourier integrals,infinite range 699 history 609 IIR filter image processing 1010, 1012 integrals using 156 inverse of sine transform 623 large data sets 637, 638 leakage 655, 656 Lomb periodogram and 689 memory-local algorithm 638 multidimensional multiple precision arithmetic 1185 multiple precision multiplication 1189 number-theoretic transforms 616 of real data in 2D and 3D of real functions , of single real function of two real functions simultaneously 617, 618 operation count 609, 610 optimal (Wiener) filtering , 673, 674 order of storage in 611 parallel 614 partial differential equations 1029, periodicity of 608 periodogram , 681, 683 power spectrum estimation related algorithms 615, 616 Sande-Tukey algorithm 616 sine transform , 1055 Singleton s algorithm 637, 638 six-step framework 615 spectral methods 1086 treatment of end effects in convolution 643 treatment of end effects in correlation 648 Tukey s trick for frequency doubling 690 two real functions simultaneously 617 use in smoothing data 766, 767 virtual memory machine 638 Winograd algorithms 616 zoom transforms 615 see also Discrete Fourier transform (DFT); Fourier transform; Spectral density Fast Legendre transform 295, 297 Fast multipole methods 140, 1150 FASTA (software) 562 Faure sequence 404 Fax (facsimile) Group 3 standard 1180 Feasible vector 526, 538 basis vector 528 Fermi-Dirac integral 178 FFT see Fast Fourier transform (FFT) Field, in data record 428 Figure-of-merit function 773 FILE (ANSI C macro) 30 Fill-in, sparse linear equations 59, 76, 535, 544 Filon s method 698 Filter acausal 668 bilinear transformation method 670, 672 by fast Fourier transform (FFT) 637, 649, causal 668, 767, 770 characteristic polynomial 670 data smoothing 766 digital DISPO 767 finite impulse response (FIR) 642, 643, 668, 669 homogeneous modes of 670 infinite impulse response (IIR) , 681 Kalman 824 linear low-pass for smoothing 766 nonrecursive 668 optimal (Wiener) 645, , 673, 674, 767 quadrature mirror 701, 708 realizable 668, 670, 671 recursive , 681 Remes exchange algorithm 669 Savitzky-Golay 232, stability of 670, 671 time domain Fine-to-coarse operator 1068

13 Index 1207 Finite difference equations (FDEs) 964, 970, 981 accuracy of 1085 alternating-direction implicit method (ADI) 1052, 1053, 1065, 1066 art, not science 1035 Cayley s form for unitary operator 1049 Courant condition 1034, 1036, 1038, 1042 Courant condition (multidimensional) 1051 Crank-Nicolson method 1045, 1049, 1051, 1052 eigenmodes of 1033, 1034 explicit vs. implicit schemes 1033 forward Euler 1032 forward time centered space (FTCS) 1032, 1044, 1049, 1059 implicit scheme 1045 in relaxation methods 964 Lax method , 1042 Lax method (multidimensional) 1050, 1051 mesh drifting instability 1040 numerical derivatives 229 partial differential equations 1027 relation to spectral methods 1093 staggered leapfrog method 1038, 1039 two-step Lax-Wendroff method 1040 upwind differencing 1037, 1042 see also Partial differential equations Finite element methods 132, 1030 Finite impulse response (FIR) 642, 643 FIR (finite impulse response) filter 668, 669 First-class objects 397 Fisher discriminant algorithm 892 Fisher s z-transformation 746 Fitting basis functions 788 by Chebyshev approximation 234 by rational Chebyshev approximation chi-square confidence levels from singular value decomposition (SVD) 816, 817 confidence levels related to chi-square values confidence limits on fitted parameters covariance matrix not always meaningful 774, 812 degeneracy of parameters 797 exponential, an 797 freezing parameters in 791, 824 Gaussians, a sum of 805 general linear least squares how much 2 is significant 816 K S test, caution regarding 740 Kalman filter 824 kriging least squares Legendre polynomials 797 Levenberg-Marquardt method , 1022 linear regression Markov chain Monte Carlo maximum likelihood estimation 777, 818 Monte Carlo simulation 740, 779, multidimensional 798, nonlinear models nonlinear models, advanced methods 806 nonlinear problems that are linear 797 nonnormal errors 781, 812, of sharp spectral features 682 polynomial 94, 129, 241, 243, 768, 788, 797 robust methods standard (probable) errors on fitted parameters 781, 782, 786, 787, 790, 794, 795, straight line , straight line, errors in both coordinates see also Error; Least-squares fitting; Maximum likelihood estimate; Robust estimation Five-point difference star 1071 Fixed point format 8 Fletcher-Powell algorithm see Davidon-Fletcher-Powell algorithm Fletcher-Reeves algorithm 489, Floating point format 8 11, care in numerical derivatives 229, 230 in computational geometry 1098 enabling exceptions 35, 575 history 1163 IEEE 9, 10, 34, 1164 little- vs. big-endian 9 NaN 34, 35 Flux-conservative initial value problems FMG (full multigrid method) 1067, for iteration 14 Formats of numbers 8 11, Fortran 1 INTENT attribute 26 Forward-backward algorithm as a sum-product algorithm 867 Bahl-Cocke-Jelinek-Raviv algorithm 867 belief propagation 867 compared to Viterbi decoding 867 hidden Markov model 861, 862, renormalization 862 Forward deflation 464, 465 Forward difference operator 212 Forward Euler differencing 1032 Forward Time Centered Space see FTCS Four-step framework, for FFT 615 Fourier analysis and cyclic reduction (FACR) 1054, 1058 Fourier and spectral applications 600, Fourier integrals attenuation factors 698 endpoint corrections 694 tail integration by parts 699 use of fast Fourier transform (FFT)

14 1208 Index Fourier series as basis functions for spectral methods 1085 Fourier transform 110, aliasing 606, 685 approximation of Dawson s integral 303 autocorrelation 602 basis functions compared 621 contrasted with wavelet transform 699, 700, 711 convolution 602, 616, 617, 631, , 1189 correlation 602, 617, 648, 649 cosine transform 241, , 1056 cosine transform, second form 625, 1057 critical sampling 605, 653, 655 decomposition into blocks 614 definition 600 discrete Fourier transform (DFT) 233, 236, Gaussian function 717, 718 image processing 1010, 1012 infinite range 699 inverse of discrete Fourier transform 608 method for partial differential equations missing data 685 missing data, fast algorithm Nyquist frequency 605, 607, 632, 653, 655, 685 optimal (Wiener) filtering , 673, 674 Parseval s theorem 602, 603, 608, 654 power spectral density (PSD) 602, 603 power spectrum estimation by FFT power spectrum estimation by maximum entropy method properties of 601 sampling rate 605 sampling theorem 605, 653, 655, scalings of 601 significance of a peak in 686 sine transform , 1055 symmetries of 601 uneven sampling, fast algorithm unevenly sampled data wavelets and 707, 708 Wiener-Khinchin theorem 602, 674, 682 see also Fast Fourier transform (FFT); Spectral density Fractal region 462 Fractional step methods 1052 Fredholm alternative 987 Fredholm equations 986 eigenvalue problems 987, 992 error estimate in solution 991 first kind 986 Fredholm alternative 987 homogeneous vs. inhomogeneous 987 homogeneous, second kind 991 ill-conditioned 987 infinite range 995 inverse problems 987, kernel 986 nonlinear 988 Nystrom method , 995 product Nystrom method 995 second kind subtraction of singularity 996 symmetric kernel 992 with singularities with singularities, worked example 999, 1000 see also Inverse problems Frequency domain 600 Frequency spectrum see Fast Fourier transform (FFT) Frequentist, contrasted with Bayesian 774 Fresnel integrals asymptotic form 298 continued fraction 298 routine for 299 series 298 Friday the 13th 7 FSAL (first-same-as-last) 913 FTCS (forward time centered space) 1032, 1044, 1049 stability of 1033, 1044, 1060 Full approximation storage (FAS) algorithm Full conditional distribution 827 Full moon 7 Full multigrid method (FMG) 1067, Full Newton methods, nonlinear least squares 806 Full pivoting 43 Full weighting 1071 Function Airy 254, 283, 289, 291 approximation 110, associated Legendre polynomial 293, 971 autocorrelation of 602 bandwidth limited 605 Bessel 219, 254, beta 258, 259 branch cuts of chi-square probability 1003 complex 251 confluent hypergeometric 254, 287 convolution of 602, 617 correlation of 602, 617 Coulomb wave 254, 283 cumulative distribution (cdf) Dawson s integral 302, 304, 717 digamma 267 elliptic integrals , 1185 error , 298, 302, 718, 746, 750 error function 259 evaluation evaluation by path integration , 318 exponential integral 219, , 301 factorial 256, 257 Fermi-Dirac integral 178

15 Index 1209 Fresnel integral functor 21 23, 444, 459, 905 gamma 256, 257 hypergeometric 252, incomplete beta incomplete gamma , 732, 779 inverse cumulative distribution inverse hyperbolic 227, 310 inverse incomplete gamma 263 inverse of x log.x/ , 335 inverse trigonometric 310 Jacobian elliptic 309, 316, 317 Kolmogorov-Smirnov probability 737, 763 Legendre polynomial 219, 293, 797 log factorial 258 logarithm 310 minimization modified Bessel modified Bessel, fractional order object 21 path integration to evaluate pathological 111, 445 probability representations of 600 routine for plotting a 444 sine and cosine integrals 297, sn, dn, cn 316, 317 spherical Bessel 283 spherical harmonics spheroidal harmonic statistical templated 17, 22, 26 utility 17 virtual 33 Weber 254 Function object see Functor Functional iteration, for implicit equations 943 Functor 21 23, 202, 204, 237, 240, 444, 459, 660, 905, 936, 940 FWHM (full width at half maximum) 659 g++ 5 Gambling , 760, 761 Gamma function 256, 257 and area of sphere 1129 complex 257 incomplete see Incomplete gamma function Gamma probability distribution 331, 332 as limiting case of beta 333 deviates from 369 relation to Poisson process 829 sum rule for deviates 370 Gauss-Chebyshev integration 180, 183, 187, 625 Gauss-Hermite integration 183, 995 abscissas and weights 185 normalization 185 Gauss-Jacobi integration 183 abscissas and weights 186 Gauss-Jordan elimination 41 46, 75 operation count 47, 54 solution of normal equations 790 storage requirements 44 Gauss-Kronrod quadrature 192, 195 Gauss-Laguerre integration 183, 995 Gauss-Legendre integration 183, 193 see also Gaussian integration Gauss-Lobatto quadrature 191, 192, 195, 241, 624, 1089 Gauss-Markov estimation 144 Gauss-Radau quadrature 191 Gauss-Seidel method (relaxation) , 1068 nonlinear 1078 Gauss transformation 310 Gaussian Hardy s theorem on Fourier transforms 717 multivariate 378, 379, 842, 843, 847, 848, 1006, 1129, 1130 see also Gaussian (normal) distribution Gaussian (normal) distribution 341, 776, 778, 1004 central limit theorem 777 Cholesky decomposition of 847, 848 deviates from 364, 365, 368, 686 kurtosis of 723, 724 multivariate 813, 842 semi-invariants of 725 sum of 12 uniform 377 tails compared to Poisson 778 two-dimensional (binormal) 746 variance of skewness of 723 see also Normal (Gaussian) distribution Gaussian elimination 46 48, 65, 71 fill-in 59, 76, 535 in reduction to Hessenberg form 594 integral equations 993 operation count 47 relaxation solution of boundary value problems 966, 984 Gaussian integration 159, , 238, 296, 995, 997, and orthogonal polynomials 181, 1087 calculation of abscissas and weights discrete orthogonality relation 1087 error estimate in solution 991 exponential convergence of 180, 1089 extensions of , 1089 for integral equations 988, 990 from known recurrence relation 188, 189 Golub-Welsch algorithm for weights and abscissas 188 for incomplete beta function 271 for incomplete gamma function 260, 262 nonclassical weight function , 995 preassigned nodes 191 weight function log x 190, 191 weight functions , 995 Gaussian mixture model Gaussian process regression 144, Gear s method (stiff ODEs) 934, 941

16 1210 Index Geiger counter 340 Gene sequencing alignment algorithms hidden Markov model 866 Generalized eigenvalue problems 568, 569 Generalized minimum residual method (GMRES) 89 Genetic algorithms 840 Geometric series 211, 214 Geophysics, use of Backus-Gilbert method 1016 Gerchberg-Saxton algorithm 1012 Ghostscript 1161 Gibbs sampler 827, 828 recommended for discrete distributions 828 Gilbert and Sullivan 920 Gillespie method 947 Givens reduction , 587 fast 578 operation count 578 Glassman, A.J. 229 Global optimization 487, 488, , 774 continuous variables difficulty of 803 Globally convergent minimization root finding 474, , 959, 960, 963 GLPK (linear programming package) 536 GMRES (generalized minimum residual method) 89 GNU C++ compiler 5 GNU Scientific Library 3 Gnuplot 1162 Godunov s method 1043 Golden mean (golden ratio) 11, 449, 494, 500 Golden section search 443, 489, 496 Goldman-Tucker theorem 539 Golub-Welsch algorithm, for Gaussian quadrature 188 Goodness-of-fit 773, 779, 782, 783, 787, 813 no good Bayesian methods 779, 1010 Gram-Schmidt orthogonalization 105, 564, 565, 589, 598 SVD as alternative to 74 Graphics, function plotting 444, Gravitational potential 631 Gray code 405, 1160, Greenbaum, A. 90 Gregorian calendar 6 Grid square 132 Gridding Group, dihedral 1174 Guard digits 1164 Half weighting 1071 Halley s method 263, 264, 271, 335, 463 Halton s quasi-random sequence 404 Hamming code 852 Hamming distance 873 error-correcting codes 851, 1168 Hamming window 658 Hamming s motto 443 Hann window 657 Hard-decision decoding 853 error correction 855 Harmonic analysis see Fourier transform Harwell-Boeing format 83 Hash collision strategy 387, 390 examples 396 function 352, key 387 memory multimap memory table of whole array Heap (data structure) 426, 434, 952, 1178 Heapsort 420, , 434 Helmholtz equation 1057 Hermite interpolation 916 Hermite polynomials 183, 185 Hermitian matrix 564, 590 Hertz (unit of frequency) 600 Hessenberg matrix 105, 567, 585, , 598 QR algorithm 596 see also Matrix Hessian matrix 483, 510, 517, 521, 522, , 1011, 1020, 1021 is inverse of covariance matrix 802 second derivatives in 800, 801 Hidden Markov model backward estimate 861 Baum-Welch re-estimation Bayesian nature of 868 Bayesian posterior probability 860, 861, 864 Bayesian re-estimation compared to Viterbi algorithm 867, 868 convergence of Baum-Welch re-estimation 866 expectation-maximization algorithm 866 forward-backward algorithm 861, 862, forward estimate 860 gene sequencing 866 hidden state 859 know intermediate states 864 missing data 864 observations 859 re-estimation of symbol probability matrix 865 re-estimation of transition probabilities 865 renormalization 862 speech recognition 866 symbols 859 trellis decoding 864 variations 864 Hierarchical clustering Hierarchically band-diagonal matrix 716 High-order not same as high-accuracy 112, 156, 238, 489, 500, 908, 911, 943