Piet M.T. Broersen Automatic Autocorrelation and Spectral Analysis With 104 Figures Sprin ger
1 Introduction 1 1.1 Time Series Problems 1 2 Basic Concepts 11 2.1 Random Variables 11 2.2 Normal Distribution 14 2.3 Conditional Densities 17 2.4 Functions of Random Variables 18 2.5 Linear Regression 20 2.6 General Estimation Theory 23 2.7 Exercises 26 3 Periodogram and Lagged Product Autocorrelation 29 3.1 Stochastic Processes 29 3.2 Autocorrelation Function 31 3.3 Spectral Density Function 33 3.4 Estimation of Mean and Variance 38 3.5 Autocorrelation Estimation 40 3.6 Periodogram Estimation 49 3.7 Summary of Nonparametric Methods 55 3.8 Exercises 56 4 ARMA Theory 59 4.1 Time Series Models 59 4.2 White Noise 60 4.3 Moving Average Processes 61 4.3.1 MA(1) Process with Zero Outside the Unit Circle 63 4.4 Autoregressive Processes 63 4.4.1 AR(1) Processes 64 4.4.2 AR(1) Processes with a Pole Outside the Unit Circle 68 4.4.3 AR(2) Processes 69 4.4.4 AR( p) Processes 72 4.5 ARMA( p,q) Processes 74 4.6 Harmonie Processes with Poles on the Unit Circle 78
x 4.7 Spectra of Time Series Models 80 4.7.1 Some Examples 82 4.8 Exercises 86 Relations for Time Series Models 89 5.1 Time Series Estimation 89 5.2 Yule-W alker Relations and the Levinson-Durbin Recursion 89 5.3 Additional AR Representations 95 5.4 Additional AR Relations 96 5.4.1 The Relation between the Variances of x and e n for an AR( p) Process 96 5.4.2 Parameters from Reflection Coefficients 96 5.4.3 Reflection Coefficients from Parameters 97 5.4.4 Autocorrelations from Reflection Coefficients 97 5.4.5 Autocorrelations from Parameters 98 5.5 Relation for MA Parameters 98 5.6 Accuracy Measures for Time Series Models 99 5.6.1 Prediction Error 99 5.6.2 Model Error 102 5.6.3 Power Gain 103 5.6.4 Spectral Distortion 104 5.6.5 More Relative Measures 104 5.6.6 Absolute and Squared Measures 105 5.6.7 Cepstrum as a Measure for Autocorrelation Functions 107 5.7 ME and the Triangulär Bias 108 5.8 Computational Rules for the ME 111 5.9 Exercises 113 Estimation of Time Series Models 117 6.1 Historical Remarks About Spectral Estimation 117 6.2 Are Time Series Models Generally Applicable? 120 6.3 Maximum Likelihood Estimation 121 6.3.1 AR ML Estimation 121 6.3.2 MA ML Estimation 122 6.3.3 ARMA ML Estimation 123 6.4 AR Estimation Methods 124 6.4.1 Yule-Walker Method 124 6.4.2 Forward Least-squares Method 125 6.4.3 Forward and Backward Least-squares Method 125 6.4.4 Burg's Method 126 6.4.5 Asymptotic AR Theory 129 6.4.6 Finite-sample Practice for Burg Estimates of White Noise 130 6.4.7 Finite-sample Practice for Burg Estimates of an AR(2) Process 133 6.4.8 Model Error (ME) of Burg Estimates of an AR(2) Process 134 6.5 MA Estimation Methods 135
xi 6.6 ARMA Estimation Methods 140 6.6.1 ARMA( p,q) Estimation, First-stage 141 6.6.2 ARMA( p,q) Estimation, First-stage Long AR 142 6.6.3 ARMA( p,q) Estimation, First-stage Long MA 143 6.6.4 ARMA( p,q) Estimation, First-stage Long COV 143 6.6.5 ARMA(/?,<?) Estimation, First-stage Long Rinv 144 6.6.6 ARMA( p,q) Estimation, Second-stage 144 6.6.7 ARMA( p,q) Estimation, Simulations 146 6.7 Covariance Matrix of ARMA Parameters 155 6.7.1 The Covariance Matrix of Estimated AR Parameters 155 6.7.2 The Covariance Matrix of Estimated MA Parameters 157 6.7.3 The Covariance Matrix of Estimated ARMA Parameters 157 6.8 Estimated Autocovariance and Spectrum 160 6.8.1 Estimators for the Mean and the Variance 160 6.8.2 Estimation of the Autocorrelation Function 160 6.8.3 The Residual Variance 162 6.8.4 The Power Spectral Density 162 6.9 Exercises 164 7 AR Order Selection 167 7.1 Overview of Order Selection 167 7.2 Order Selection in Linear Regression 169 7.3 Asymptotic Order-selection Criteria 176 7.4 Relations for Order-selection Criteria 180 7.5 Finite-sample Order-selection Criteria 183 7.6 Kullback-Leibler Discrepancy 187 7.7 The Penalty Factor 192 7.8 Finite-sample AR Criterion CIC 200 7.9 Order-selection Simulations 203 7.10 Subset Selection 208 7.11 Exercises 208 8 MA and ARMA Order Selection 209 8.1 Introduction 209 8.2 Intermediate AR Orders for MA and ARMA Estimation 210 8.3 Reduction of the Number of ARMA Candidate Models 213 8.4 Order Selection for MA Estimation 216 8.5 Order Selection for ARMA Estimation 218 8.6 Exercises 221 9 ARMASA Toolbox with Applications 223 9.1 Introduction 223 9.2 Selection of the Model Type 223 9.3 The Language of Random Data 226 9.4 Reduced-statistics Order Selection 227 9.5 Accuracy of Reduced-statistics Estimation 230 9.6 ARMASA Applied to Harmonie Processes 233
xii 9.7 ARMASA Applied to Simulated Random Data 235 9.8 ARMASA Applied to Real-life Data 236 9.8.1 Turbulence Data 236 9.8.2 Radar Data 243 9.8.3 Satellite Data 244 9.8.4 Lung Noise Data 245 9.8.5 River Data 246 9.9 Exercises 248 ARMASA Toolbox 250 10 Advanced Topics in Time Series Estimation 251 10.1 Accuracy of Lagged Product Autocovariance Estimates 251 10.2 Generation of Data 262 10.3 Subband Spectral Analysis 264 10.4 Missing Data 268 10.5 Irregulär Data 276 10.5.1 Multishift, Slotted, Nearest-neighbour Resampling 282 10.5.2 ARMAsel for Irregulär Data 283 10.5.3 Performance of ARMAsel for Irregulär Data 284 10.6 Exercises 286 Bibliography 287 Index 295