Multivariate time series estimation via projections and matrix equations

Size: px

Start display at page:

Download "Multivariate time series estimation via projections and matrix equations"

Corey Jones
5 years ago
Views:

1 Multivariate time series estimation via projections and matrix equations Federico Poloni 1 Giacomo Sbrana 2 1 U Pisa, Italy, Dept of Computer Science 2 NEOMA Business School, Rouen, France Structured Numerical Linear and Multilinear Algebra Analysis, Algorithms and Applications September 2014 Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

2 Moving average models Moving average models [Lütkepohl 06 book, Tsay 14 book] {u t R n } Gaussian independent N(0, I) y t = A 0 u t + A 1 u t 1 y t = A 0 u t + A 1 u t 1 + A 2 u t A q u t q MA(1) MA(q) (A i real square matrices, y t, u t vectors) Example: Yearly inflation data looks a lot like a y t Our problem: estimation Given output y t for t = 1, 2,..., T, find the A i s Scalar models popular for macroeconomics; multivariate ones less used because of estimation difficulties Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

3 Transfer function formalism To A 0 u t + A 1 u t 1 + A 2 u t A q u t q we associate G(λ) = A 0 + A 1 λ + A 2 λ A q λ q G(λ) assumed nonsingular (det(a 0 ) 0), and all eigvals outside unit circle Definition G (λ) := A 0 + A 1 λ 1 + A 2 λ A q λ q λ behaves like a complex number on the unit circle G(λ)G (λ) = M q λ q + + M 1 λ 1 + M 0 + M 1 λ + + M q λ q is palindromic (M 0 = M 0 and M i = M i for each i) Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

4 Autocovariance generating function G(λ)G (λ) = M q λ q + + M 1 λ 1 + M 0 + M 1 λ + + M q λ q Theorem [Box, Jenkins, 76 book] [ ] M i = E y t yt i for each i = q,..., q 1 Estimate M i := 1 T 1 Ti=1 y t y t i 2 Factorize M(λ) = G(λ) G (λ) Point 2. is spectral factorization [Wiener, Kailath, Kucera, Ran et al, Ephremidze et al; many more for the scalar case] Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

5 Spectral factorization and matrix equations M 1 λ 1 + M 0 + M 1 λ = (A 1 λ + A 0 )(A 1 λ + A 0 ) Several approaches to solve it: Y = A 0 A 0 solves M 1Y 1 M 1 + Y = M 0 [Ran et al., Meini, CH Guo... ] Z = A 0 A 1 solves M 1 + M 0 Z + M 1 Z 2 [Bini et al, Higham and Kim, Meerbergen and Tisseur... ] G(λ) left spectral divisor of M(λ) [Gohberg Lancaster Rodman]: linearization (palindromic?)+ Schur form Extreme accuracy not necessary here larger error from approximations M i already Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

6 What can go wrong 2 Factorize M(λ) = G(λ) G (λ) Factorization M(λ) = G(λ) G (λ) exists iff M(z) 0 for each z C, z = 1 M i are only approximate, so this property may fail Possible solutions... 1 Apply a correction to M(z) 2 Robust factorization algorithm do something in case of error (something like a robust chol ) 3 Get better M i s Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

7 Possible solutions Apply a correction to M(z) [Brüll, P, Sbrana, Schröder, preprint] regularization approach: perturb polynomial to move eigenvalues Other approaches for similar structures around [Alam et al, Benner and Voigt, Grivet-Talocia, Guglielmi-Overton... ] Robust factorization algorithm Not much usable around! [Kucera, 91] Trying to modify some (Riccati-type eigensolvers, symbolic Cholesky factorization [Janashia, Lagvilava, Ephremidze 11]) Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

8 Get better M i s Other (slower) estimation methods typically get better M i s, such as Maximum Likelihood (ML) Our idea: get the M i s from projection + scalar ML Algorithm 1 Choose weight row vector w, build scalar process z t = {wy t } t=1,...,t 2 Use scalar ML to get g(λ) 3 g(λ)g (λ) = m(λ) = wm(λ)w 4 Repeat with many different w s to reconstruct M(λ) Convergence properties to correct solution when T (technical) [P, Sbrana, submitted] Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

9 The algorithm in a picture Observed data y t ML-estimate Parameters A i of G(λ) compute Autocovariances [ ] M i E y t yt k aggregate compute Scalars z t = w T y t ML-estimate Parameters a i of g(λ) compute Autocovariances m k E [z t z t k ] We take the blue road rather than the red Many (easier) scalar problems rather than a (difficult) multivariate one Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

10 A 2 2 example [ ] w = 1 0 : [ ] ([ ] [ ] [ ] ) [ ] wm(λ)w T a b = 1 0 λ 1 e f a c λ c d f g b d 0 = aλ 1 + e + aλ [ ] w = 0 1 : [ ] ([ ] [ ] [ ] ) [ ] wm(λ)w T a b = 0 1 λ 1 e f a c λ c d f g b d 1 w = [ ] 1 1 : = dλ 1 + g + dλ m(λ) = (a + b + c + d)λ 1 + (e + 2f + g) + (a + b + c + d)λ If b = c, enough to get the remaining entries Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

11 Symmetric case For a symmetric MA(1), w = e i for all i and w = e j + e k for all j k enough to get M(λ) Aλ 1 + B + Aλ, with A = A, B = B can be simultaneously diagonalized (congruence) easier to find spectral factorization This is enough to solve some easier models: Exponentially weighted moving average (random-walk-plus-noise) MA(1) with symmetric matrices Multiple trends MA(2) with symmetric matrices and M 1 = 4M 2 Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

12 Nonsymmetric case Idea Use w(λ), not constant w! [ ] [ ] y 1 λ t,1 = y y t,1 + y t 1,2 t,2 [ ] w = 1 λ [ ] ([ ] [ ] [ ] ) [ ] w(λ)m(λ)w a b = 1 λ λ 1 e f a c λ c d f g b d λ 1 = cλ 2 + (a + d + f )λ 1 + e + 2b + g + (a + d + f )λ + cλ 2 MA(q + 1): more difficult to estimate, but still scalar Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

13 Least squares and more weights Each aggregation vector w(λ) gives us several equations Combine them using least squares! But... joint covariance? W = Same-aggregation values (in red) easy to compute (independence) Black ones are hard set them to zero Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

14 Some results LongAR Quicker but less accurate method META Our method (Moment Estimation Through Aggregation) CML Conditional Maximum Likelihood. Scales badly, basically impossible to beat Z LongAR Z META Z CML Frobenius norm relative errors, 300 trials, 2 2 problem, excluding 48 non-passive samples. eigvals(theta)=[ , ] Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

15 Some more results Symmetric model (EWMA), against out-of-the-box ML on the MA(1) representation (larger parameter space, difficult to enforce symmetry) d = 2, easy d = 2, harder d = 3, easy d = 3, harder T Θ META Θ ML Σ u META Σ u ML Time META Time ML Frobenius norm parameter errors 10 3, average on 500 trials + real-life inflation test by Bankitalia Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

16 Parallelizability for i = 1:nWeights # Estimate g i (λ) with weight w i end # Put together estimates to form M(λ) (fast) # Spectral factorization (fast) while (convergence) #compute likelihood(g(λ)) #compute G i+1 (λ) from G i (λ) using BFGS-like search end function likelihood(g) for t = 1:T # update state[t] from state[t-1], compute likelihood end end Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

17 Why isn t this... Why isn t this model reduction/system identification? Similar to tangential interpolation [Antoulas, Beattie, Gugercin 10 inbook] but... No access to input u t No possibility to get G(λ) exactly: G(λ)Q equally good We work on a squared version: not clear how to relate g(λ) and G(λ) in our setting Why isn t this compressed sensing? Similar to phase retrieval [Candès, Strohmer, Voroninski 11], but it is a matrix version, uncertainty by an orthogonal matrix no sparsity/rank constraint, we want full reconstruction with a full set of measurements Why isn t this Kalman filtering? KF assumes model known, our goal is estimating it here Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

18 Conclusions and issues Interesting idea to explore, new application area Works well in some cases (including symmetric problems) Still missing ideas: fit G(λ) directly without squaring, find optimal weights and equations in LS Tricky to develop with the current software ecosystem Open problem: better regularizing spectral factorization algorithm: We know well how to solve Riccati-like equations, now we want to solve the ones that have no solution! Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

19 Conclusions and issues Interesting idea to explore, new application area Works well in some cases (including symmetric problems) Still missing ideas: fit G(λ) directly without squaring, find optimal weights and equations in LS Tricky to develop with the current software ecosystem Open problem: better regularizing spectral factorization algorithm: We know well how to solve Riccati-like equations, now we want to solve the ones that have no solution! Thanks for your attention Poloni, Sbrana (Pisa, Rouen) Matrix eqns estimators SLA / 18

Perturbing Palindromic Matrix Equations to Make Them Solvable

Perturbing Palindromic Matrix Equations to Make Them Solvable Federico Poloni 1 Joint work with Tobias Brüll 2, Giacomo Sbrana 3, Christian Schröder 4 1 U Pisa, Dept of Computer Science 2 Computer Simulation