Index. Regression Models for Time Series Analysis. Benjamin Kedem, Konstantinos Fokianos Copyright John Wiley & Sons, Inc. ISBN.

Transcription

1 Regression Models for Time Series Analysis. Benjamin Kedem, Konstantinos Fokianos Copyright John Wiley & Sons, Inc. ISBN Index Adaptive rejection sampling, 233 Adjacent categories logits, 98 Airline passenger study, seasonal time series, Akaike s information criterion (AIC) applications: binary time series, 73-74, 79 categorical time series, 115, 117 generalized linear model (GLM), 23, 25, 35 mixture transition distribution (MTD) model, 293 Alternative approachesimodels: autoregressive conditionally heteroscedastic (ARCH) models, discrete autoregressive moving average (DARMA) models, hidden Markov models, integer autoregressive and moving average models, longitudinal data, mixed models for, mixture transition distribution (MTD) model, 19&194 sinusoidal regression model, variable mixture models, Alternative modeling, categorical time series, 125 Anscombe residuals, 26 ARCH( 1 ) model, Asymptotic distribution: categorical time series, , 112, 115 generalized linear model (GLM), 16-24, 27 quasi-partial likelihood, 28 Asymptotic relative efficiency (ARE), binary time series, 6465 Asymptotic theory: categorical time series, count time series, 156 generalized linear model (GLM), 1&20 Autocorrelation estimation, Autocorrelation function (acf): binary time series, 60-62, 75 categorical time series, 94-96, 10&101, 118 count time series, 143, 145, , 150, 152 hidden Markov models, 196 integer moving average (MMA) models, 186 stationary processes, Autocovariance function: integer autoregressive models, 184 stationary processes, 286, 289, 292 Autoregressive moving average (ARMA): count time series, 141 Gaussian, linear state space models, 2 17 prediction and,

2 ~~ 328 INDEX Autoregressive moving average (ARMA) (continued) process, 294 Autoregressive process, binary time series, 53 Bayesian forecasting, 2 15 Bayesian inference, 232 Bayesian information criterion (BIC) applications: binary time series, 73-75,79 categorical time series, 115, 117 generalized linear model (GLM), 23,25, 35-36,39,41 mixture transition distribution (MTD) model, 293 Bayesian mixture models, 237 Bayesian spatial prediction: auxiliary Gaussian process, 258,260 likelihood, model parameters, prior and posterior, transformations, normalizing, 265 Z, Prediction, Bayes theorem, 82,223, 225, 230, , 262 Bernoulli data, 10 Bernoulli random variables, integer autoregressive models, Best linear unbiased predictor (BLUP), 252 Binary time series: characteristics of, generally, 9-10 goodness of fit, inference, for logistic regression, link functions, partial likelihood estimation, real data examples, regression models, state space model, 228 Binomial data, 10 Binomial thinning, 178, 188 Bivariate exponential autoregressive moving average models, 190 Box-Cox power transformation: Bayesian spatial prediction, 265, 267 generalized linear model (GLM), 7 Box-Pierce portmanteau statistic, 27 Branching process: with immigration, 173, size-dependent, 154 BTG algorithm: defined, kriging, comparison with, 274 seasonal time series, spatial rainfall prediction, time series prediction, BTG program, 265 Buffon s needle experiment, 23 1 Bum-in period, 233 Canonical links/parameters: count time series, Poisson model, 142, 148, 156 generalized linear model (GLM), 5, 8 partial likelihood inference, 13 Categorical time series: alternative modeling, 125 asymptotic theory, characteristics of, data examples, goodness of fit, link functions, longitudinal data, 125 modeling, partial likelihood estimation, spectral analysis, 125 Central limit theorem, 3, 18, Chi-square: approximation, 24 asymptotic, 207 distribution,68, , 113, 117, Neyman-modified statistic, 113 random variable, 2 I, 23 Cholesky square root, 130, 134 Complementary log-log (cloglog), 10, 54, Composite hypothesis, 20 Computation, generalized linear model example, 33 Computer software: btg program, GMTD, 192 MTD, 192 SAS, 33 S-PLUS, 33,203,223 state space models, 223 Conditional covariance matrices:

3 INDEX 329 binary time series, 60 categorical time series, 105, 130 Conditional density: count time series, doubly truncated Poisson, 141 Gaussian, generalized linear model (GLM), 9 Conditional distribution: Gaussian, 225 INAR(2), 184 simulation-based state space models, 235 variable mixture models, Conditional expectation: categorical time series, 114 count time series, 140 Conditional Gaussian marginals, 8 Conditional inference, 1,4, 12, 206 Conditional information matrix: binary time series, 58 categorical time series, 10&107, 109 generalized linear model (GLM), 14-15, 17 Conditional likelihood, 3, 108 Conditional linear autoregressive process (CLAR(I)), Conditional maximum likelihood estimates, integer autoregressive models, 184 Conditional mean, count time series, 142 Conditional probability: binary time series, 5 1 success, 49 Conditional variance: count time series, 157 working, 31 Confidence bands, 123 Conjugate analysis, nonlinear and non- Gaussian state space models, Continuation ratio model, 98 Contraction mapping (CM): algorithm, method, Contraction parameter, 203 Correlation function, 286 Count time series: asymptotic theory, 156 characteristics of, generally, 139 data examples, doubly truncated Poisson model, goodness of fit, 159 hypothesis testing, 158 inference, link functions, modeling, over-dispersion, 3&3 1 partial likelihood estimation, Poisson model, , prediction intervals, 157 Zeger-Qaqish model, , Covariance function: in spatial prediction, 25 1 stationary processes 286 Covariance matrix: generalized linear model (GLM), 20 linear Gaussian state space models, 219, 222 mixed models, 207 Covariate(s), generally: in generalized linear model (GLM), 1 matrix, categorical time series, 107 process, defined, 5 random time dependent, 3,90 vector, see Covariate vector Covariate vector: binary time series, 50, 56 count time series, 153 in generalized linear model (GLM), 6,28 Covariogram, Cox s proportional hazards, 54 Cramtr-Wold device, 130, 132, 136 Cross-correlation, categorical time series, 96, 101 Cross-validation studies, ,273, 275, ,280 Cumulative conditional information matrix, 12 Cumulative distribution function (cdf): binary time series, 5 I, 54 categorical time series, generalized linear model (GLM), 9 Cumulative odds, categorical time series, Data examples, see specific fypes of regression models Degrees of freedom: binary time series, 68, 71

4 INDEX Degrees of freedom (continued) categorical time series, 110, , 117 count time series, generalized linear model (GLM), 35 mixed models, 207 Delta method, 59, 108 Densityidensities: conditional, 3,9, 14C141, probability, 7, 56, ,265,286, 289 Dependence: binary time series, 53 Markov, 2-3 nonhomogeneous Markovian, 7 in partial likelihood, 2, 10 Deviance: binary time series, 65 count time series, , 164 generalized linear model (GLM), partial likelihood inference, 14 Deviance residuals, Diagnostics: categorical time series, 115 generalized linear model (GLM), Discrete autoregressive moving average (DARMA), 125, Dispersion parameter: defined, 6 estimation of, 14 DNA sequence data analysis: categorical time series example, 1llsl19 mixture transition distribution model, example of, 194 Double chain Markov model, 191 Doubly truncated Poisson model, count time series, 141, Dynamic generalized linear models (DGLM): characteristics of, conjugate analysis, defined, 227 formulation of, 228 linear Bayes estimation, posterior mode estimation, 23@-231 EM algorithm, Empirical likelihood, 4 Ergodic binary time series, 5 1 Estimation, generally: autocorrelation, GARMA models, 7 linear Gaussian state space models, Existence, maximum partial likehood estimators, 17 Exponential autoregressive moving average (EARMA), 188 Exponential autoregressive process of order 1 (EAR(I), 188 Exponential correlation, 251, 267,271, 275 Exponential dispersion model, 7 Exponential families, nonlinear and non- Gaussian state space models, Exponential moving average (EMA( I)), 188 Extensions: autoregressive conditionally heterscedastic (ARCH) models, 20&201 integer autogressive and moving average models, Fast Fourier transform (FFT), 202,204 Filtered data, Fi I tering: Kalman, Monte Carlo method, 227 nonlinear and non-gaussian state space models, ,23 1 Finite dimensional distribution, 286 First-order Markov chains, 90 First-order Taylor expansion, 27 Fisher scoring, 12, 14-15, 155 Fixed effects, in mixed models, 205 FORTRAN 77,265 Fourier frequencies, 202, 204 Full likelihood, binary time series, Galton-Watson process, 173, 175 Gamma, generally: defined, 5 distribution, 188 Gaussian ARMA, 184 Gaussian distribution, 227,23 1 Gaussian process, 286 Gaussian random fields,

5 INDEX 33 1 Gaussian-sum filter, 227 Gaussian white noise, 27-28, 36 Generalized ARCH (GARCH) models, 2ock201 Generalized Autoregressive Moving Average (GARMA), 6 Generalized estimating equations (GEE) method, Generalized linear models (GLM): applications of, generally, 1 asympotic theory, data examples, defined, 5 diagnostics, hypotheses testing, 2&23 partial likelihood, 14, quasi-partial likelihood, random component, 5-6 systematic component, 5-8 time series and, Generalized Linear Autoregressive Moving Average (GLARMA), 6 Geometric autoregressive moving average models, 190 Gibbs sampling algorithm, ,237 Goodness of fit: binary time series, 65-69, 7 I categorical time series, I5 count time series, 159 Herglotz, G., 289 Hidden Markov models, 125, Hyperparameters: linear Gaussian state space models, 22 1 nonlinear and non-gaussian state space models, 225 simulation-based state space models, 235 Hypotheses testing: categorical time series, count time series, 158 generalized linear model (GLM), Importance, sequence sampling (SIS), INARb), ,188 Independence, generally: binary time series, 52 of irrelevant alternatives, 93 Inference: Bayesian, 232 conditional, 1, 12,206 count time series, likelihood, 240 logistic regression, binary time series, Markov Chain Monte Carlo for state space models, mixture, 206 nonparametric, 4 partial likelihood, see Partial likelihood Infinite moving average, 293 INMA(q), , 188 Integer autoregressive models, regression analysis of, 185 Integer autoregressive models of order I (INAR( I)): characteristics of, estimation for, Integer autoregressive models of order p (INAR(2)), Integer autoregressive moving average (INARMA), 188 Integer moving average (TNMA) models, Integrated GARCH (IGARCH) models, 201 Interpolation, kriging, 257, 259 Intervals: categorical time series, 93 prediction, 20 Inverse Gamma distribution, 234 Inverse link function, categorical time series, 92 Isotropic correlation function, 25 1 Iterative reweighted least squares, Kalman, R. E., 214 Kalman filter/filtering: defined 2 14 in linear Gaussian state space model estimation, see Kalman filtering, linear Gaussian state space model estimation in space-time data, 241 simulation-based state space models, 236 Kalman filtering, linear Gaussian state space model estimation: characteristics of,

6 332 INDEX Kalman filtering, linear Gaussian state space model estimation (continued) structural model, Kalman prediction, 2 18 Kalman smoothing, linear Gaussian state space model estimation, Kriging: BTG algorithm compared with, 274 trans-gaussian, 258,260, variance, 256 Large sample theory, categorical time series, Left truncated Poisson distribution, 141 Linear Bayes estimation, nonlinear and non- Gaussian state space models, Linear Gaussian state space models: characteristics of, 2 15 estimation, generally, estimation, by Kalman filtering and smoothing, examples of, Linear mixed-effects model, Linear predictor, 6 Linear regression: classical, 5, 8, 265 ordinary, 1 Link functions: binary time series, 5&56 categorical time series, count time series, generalized linear model (GLM), 6, 8 Log-likelihood: estimation, mixture transition distribution (MTD) model, 192 linear Gaussian state space models, 22 1 Log-linear model: count time series, , 146, 153, 156 generalized linear model (GLM), 7 Log-partial likelihood: binary time series, 7 1 categorical time series, 102, 105 count time series, 155 generalized linear model (GLM), 10, 12, 19,21-22,40 ratio test, 40 Logistic autoregression model, binary time series, 53-54,6&62 Logistic model, 9 Logistic regression: binary time series, generalized linear model (GLM), 9,39, 41,53 Logi t( s): adjacent, 98 defined, 53 multinomial, 92-96, Longitudinal datdstudies: categorical time series, 125 generalized estimating equations (GEE), 31 linear mixed models for, simulation-based state space models, 24 1 Marginal partial likelihood, 4 Markov assumption, categorical time series, 108 Markov chain(s): branching processes, 175 binary time series, 73 categorical time series, 89-90, 110, 125 generalized linear model (GLM), 41 hidden, homogeneous, 194 mixture transition distribution (MTD) model, Markov Chain Monte Carlo (MCMC) techniques: applications, generally, 221, 225 defined, 232 Gibbs sampling algorithm, , 237 inference, for state space models, Metropolis-Hastings algorithm, 233 Monte Carlo simulation techniques, Markov dependence, 2 Markov process, 7, 182,225 Markov s inequality, 132, 135 Martingale(s): binary time series, 57, categorical time series, 114 Central Limit Theorem, 67, 13&13 1 difference, 176 generalized linear model (GLM), 3, 18, 26,29

7 INDEX 333 Matern correlation, 25 I, 267,271, , 278 Maximum distribution function, categorical time series, 98 Maximum likelihood estimation (MLE): asymptotic properties of, 3 autoregressive conditionally heterscedastic (ARCH) models, 200 hidden Markov models, 197 WAR( 1) process, linear Gaussian state space models, 223 mixed models, 206 simulation-based state space models, 240 Maximum partial likelihood estimator (MPLE): binary time series, 57, 59, categorical time series, 105, 107, 110, 114, 118, 130, count time series, 156 generalized linear model (GLM), 4, 12, 16-19,24,29 Mean response model, categorical time series, 98 Mean square error (MSE), 72 Mean square prediction error (MSE), 274 Metropolis-Hastings algorithm, 233 Minimal distribution function, categorical time series, 98 Minimum eigenvalue, categorical time series, 131 Mixing distribution, 206 Mixture inference, 206 Mixture models, see Bayesian mixture models; Variable mixture models Mixture transition distribution (MTD) model: characteristics of, data examples, estimation in, 192 Model(s), generally: adequacy, binary time series, 68 building, selection, 25 Modeling: categorical time series, count time series, Modifications, autoregressive moving average models, Monte Carlo, generally: algorithm, Bayesian spatial prediction, 264 Markov Chain techniques, 22 1 Moving average: discrete autoregressive models, generalized autoregressive (GARMA), 6 generalized linear autoregressive (GLARM), 6 generalized linear autoregressive moving average (GLARMA), 6 integer models, Multi-matrix mixture transition distribution (MTDg) model, 191, 192 Multinomial distribution, integer autoregressive models, 182, 184 Multinomial logit, categorical time series: defined, inference for, nominal time series model, with periodic component, 9&96 Multinomial thinning, 178, 183 Multiplicative error model, count time series, 153 Multiplicative model, count time series, 153 Multivariate, generally: functions, 102 generalized linear model, 92, 130 normal distribution, 219,251,287 state space models, 24 1 NASA, Apollo space program, 214 Natural parameter, in generalized linear model (GLM), 5-7 Neural network prediction models, Newton-Raphson procedure, 12, Nominal categorical time series, 93-96, 1 16 Non-Gaussian fields, 256 Non-Gaussian processes, 29CL29 1 Nonhomogeneous Markovian dependence,7 Nonlinear filters, 249 Nonlinear least squares, quasi-partial likelihood, 30 Nonlinear transformation, 258 Non-Markovian processes, 4 Nonnegative definite, 89 Nonparametric prediction, binary time series, 5 1 Nonstationary binary time series, 52

8 334 INDEX Nonstationary response, 49 Normal distribution: binary time series, 61 categorical time series, 115 generalized linear model (GLM), 8, 18, 29 multivariate, 219,251, 287 simulation-based state space models, 236 stationary processes, Null hypothesis, 21-23,25, 71 Observation equation, 213,228 Observed information matrix, 58 Old Faithful eruptions: binary time series example, mixture transition distribution model, example of, One-step prediction errorshnnovations, 22 1 Ordinal categorical time series, 93, , Oscillation: binary time series, 50 count time series, 163 Over-dispersion: in count data analysis, 3&3 1 count time series, 158 quasi-partial score, 29 Parametric hazard models, 54 Parsimonious modeling, 90 Partial likelihood: binary time series, 5659, 61 characteristics of, 1-3 count time series, Poisson model, defined, 3 4 estimation, see Partial likelihood estimation inference, quasi-, ratio test, see Partial likelihood ratio test variable mixture models, 198 Partial likelihood estimation: binary time series, 5C59 categorical time series, count time series, 15&156 generalized linear model (GLM), 33 Partial likelihood ratio test: categorical time series, 110, count time series, 158 Partial score: categorical time series, , 110, 130 count time series, 158 generalized linear model, 1 I, 14,28 Pearson goodness of fit statistic, 66, 110, , 159, 164 Pearson residuals: autocorrelation of, 2C27 binary time series, 80 categorical time series, 115, 118, count time series, , generalized linear model (GLM), 25&27, 30,35,41 defined, 25 quasi-partial likelihood, 30 Periodic component: binary time series, 6&61 categorical time series, count time series, , 148 Periodogram: binary time series, categorical time series, count time series, 166 sinusoidal regression model, Permutation sampling, 221 Poisson distribution: integer autoregressive models, 180, 186 generalized linear model (GLM), 5,9,28 integer moving average (TNMA) models, 186 Poisson-gamma, 28 Poisson GARMA, 6 Poisson hidden Markov model, Poisson INAR( I), 182 Poisson model, count time series: characteristics of, doubly truncated , partial likelihood estimation, Poisson regression, 19-20,26,30-3 I, 34, 141 Positive Poisson distribution, 141 Posterior distributions, Bayesian spatial predictions, Posterior mode estimation, nonlinear and non-gaussian state space models, 23&23 1

9 INDEX 335 Power divergence: categorical time series, , 118, 123 generalized linear model (GLM), 40,42 Power transformations, Bayesian spatial prediction, 265 Prediction: Bayesian spatial, 250, BTG algorithm, applications of, implications of, generally, intervals, see Prediction intervals kriging algorithm, 252, nonlinear and non-gaussian state space models, 227,231 ordinary kriging, 252, stationary random fields, elements of, Prediction intervals: binary time series, 59, 77 categorical time series, 108 count time series, 157 generalized linear model (GLM), 20 Predictive density: Bayesian spatial prediction, 260,264, 273,275,28 1 BTG algorithm, simulation-based state space models, 240 Prior distributions, Bayesian spatial predictions, Prior weight, 6 Probability density function (pdf) binary time series, 56 generalized linear model (GLM), 8 Monte Carlo simulation techniques, Probability distribution, 265, 286, 289 Probit regression: binary time series, 55, 73 generalized linear model (GLM), 10,41 Proportional odds, categorical time series, 97,99-100, 121, 123 Pseudo-likelihoods, 4 Pulse function. I63 Quasi-partial likelihood: characteristics of, generalized estimating equations (GEE), nonlinear least squares example, 30 over-dispersion in count data, Quasi-partial score, 29,3 1 Quasi-score, 157 Rainfall prediction, binary time series example, Random walk method, 234 Rational quadratic correlation, 252, 275 Raw residuals: categorical time series, 1 I5 count time series, 165 generalized linear model (GLM), 25 Real-valued stationary time series, 294 Rejection sampling algorithm, 233 Residual matrix, 58 Residuals, see specific types of residuals analysis, binary time series, categorical time series, 114 count time series, 159 generalized linear model (GLM), Response classification: binary time series, categorical time series, Response residuals, 25 Response series, 4-5 Rice s formula. 290 Sample autocorrelation (sample ACF), Sampling: adaptive rejection, 233 asymptotic theory, permutation, 22 1 rejection algorithm, 233 sequential importance (SIS), sequential Monte Carlo methods, Scaled deviance: categorical time series, 110, 1 15 generalized linear model (GLM), Score statistic, Seasonal time series, BTG algorithm, Second-order generalized estimating equations (GEE2), 33 Second-order stationary, 287 Self-exciting threshold autoregressive (SETAR) model,

10 INDEX Sequential importance sampling (SIS), Sequential Monte Carlo sampling methods, Simulation-based state space models: characteristics of, likelihood inference, 240 longitudinal data, 241 Markov Chain Monte Carlo (MCMC), Monte Carlo, generally, sequential Monte Carlo sampling methods, Sinusoidal component: binary time series, 62 categorical time series, 99 count time series, 143, 148 Sinusoidal regression models, Size-dependent branching, 154 Sleep prediction: binary time series example, categorical time series example, Slow convergence, simulation-based state space models, 236 Slutsky s theorem, 136 Smoothed data, 34 Smoothing, generally: density, simulation-based state space models, 235 Kalman, nonlinear and non-gaussian state space models, ,23 1 Soccer forecasting: categorical time series example, mixture transition distribution model, example of, Spatial data analysis, 4 Spatial MTD model, 191 Spatial rainfall, prediction with BTG algorithm, Spectral coherence, 34 Spectral density, stationary processes, 292, 294 Spectral distribution function, 289,291 Spectrum, zero-crossing rate and, 29&29 1 Spherical correlation, 25 I, 275 S-PLUS, 33,203,223 Square integrable zero-mean martingale, 133 Squared Pearson residuals, 115, 118 State space models: characteristics of, 213 defined, 2 13 historical perspective, Kalman filtering, space-time data, 241 linear, 214 linear Gaussian, non-gaussian, nonlinear, simulation-based methods, State space representation, linear state space models, Stationarity, Stationary AR( 1) process, Stationary AR(p) process, Stationary binary time series, Stationary Gaussian random field, 274 Stationary Gaussian time series, 295 Stationary in the wide sense, 287 Stationary processes, elements of: complex-valuated, stationarity, Stock price prediction, binary time series example, Strict stationarity, 287 Structural time series: Kalman filtering, 222 linear state space models, 2 16 Systematic component, in generalized linear model (GLM), 5-8 System equation, 213 Taylor expansion, 27,68, , 137 Testing data, binary time series, 68 Test statistics, 23, 30, 110 Thinning operation, integer autoregressive models, Time series: prediction, BTG algorithm, seasonal, BTG algorithm, Tourist arrivals, count time series example, , Training data, binary time series, 68 Trans-Gaussian kriging, 258,260, Transition, generally: density, 240 probabilities, 5, , 194

11 INDEX 337 Two step response model, categorical time series, 98 Unconditional covariance matrix, categorical time series, 130 Unconditional information matrix: categorical time series, 106, 131 count time series, 155 generalized linear model (GLM), 12, 17 Unemployed women, Bayesian spatial prediction, Uniqueness, maximum partial likelihood estimators, Univariate zero-mean martingale, 130 Variable length Markov chains (VLMC), 125 Variable mixture models: overview, partial likelihood inference, 198 threshold models, Variance components, in mixed models, 207 Variance function, 8 Vector INAR(I), Volterra-type expansion models, Wald statistic, 21, 23, 110, 158 Wald test, 23, 165 Wavelet methods, 124 Weakly stationary, 287 Weight, 6 Weighted least squares: estimation, 15 generalized linear model (GLM), integer autoregressive and moving average models, 176, 185 White noise, 27-28, 75,214 Wide sense stationary, 290 Working covariance matrix, Working residuals: binary time series, 72 count time series, 159 generalized linear model (GLM), 25-26, 35 Working variance, quasi-partial likelihood, 28 Yule-Walker equation, 19 I, 293 Z,, Prediction of, Zeger-Qaqish model, count time series, , Zero-crossing rate, Zero-mean martingale, 68, 114 Zero mean square integrable martingale, 3, 130, 136