Computational Time Series Analysis (IPAM, Part I, stationary methods) Illia Horenko

Transcription

1 Computational Time Series Analysis (IPAM, Part I, stationary methods) Illia Horenko Research Group Computational Time Series Analysis Institute of Mathematics Freie Universität Berlin Germany and Institute of Computational Science Universita della Swizzera Italiana, Lugano Switzerland

2 Motivation (I): Data Inverse problem: data-based identification of model parameters other anticyclonic cyclonic time series of vectors (continous, finite dim.) time series of regimes (discrete, finite dim.) time series of functions (continuous, infin. dim.) Challenges: VERY LARGE (~Tb size), non-markovian (memory), non-stationary (trends), multidimensional, ext. influence

3 Motivation (II): Models Inverse problem: data-based identification of model parameters Dynamical (f.e., deterministic, stochastic,...) Geometrical (f.e, essential manifolds,...) Models Mixed (f.e., MTV, PIP/POP,...) Challenges: existence and uniqueness, robustness, conditioning

4 Plan for Today (stationary case) eagle eye perspective on stochastic processes from the viewpoint of deterministic dynamical systems geometric model inference: EOF/SSA multivariate dynamical model inference: VARX handling the ill-posed problem motivation for tomorrow : examples where stationarity assumption does not work

5 Plan for Today (stationary case) eagle eye perspective on stochastic processes from the viewpoint of deterministic dynamical systems geometric model inference: EOF/SSA multivariate dynamical model inference: VARX handling the ill-posed problem motivation for tomorrow : examples where stationarity assumption does not work

6 Classification of Stochastic Process Discrete State Space, Discrete Time: Markov Chain Discrete State Space, Continuous Time: Markov Process Stochastic Processes Continuous State Space, Discrete Time: Autoregressive Process Continuous State Space, Continuous Time: Stochastic Differential Equation

7 Discrete Markov Process Realizations of the process: Markov-Property: Example:

8 Discrete Markov Process Realizations of the process: Markov-Property: State Probabilities:

9 Discrete Markov Process Realizations of the process: Markov-Property: State Probabilities:

10 Discrete Markov Process Realizations of the process: Markov-Property: State Probabilities: This Equation is Deterministic

11 Continuous Markov Process

12 Continuous Markov Process

13 Continuous Markov Process

14 Continuous Markov Process infenitisimal generator

15 Continuous Markov Process infenitisimal generator This Equation is Deterministic: ODE

16 Continuous State Space Discrete State Space, Discrete Time: Markov Chain Discrete State Space, Continuous Time: Markov Process Stochastic Processes Continuous State Space, Discrete Time: AR(1) Continuous State Space, Continuous Time: Stochastic Differential Equation

17 Deterministic Markov Process in R Realizations of the process: Process: Flow operator: Properties:

18 Deterministic Markov Process in R Realizations of the process: Process: Flow operator: Properties: Example: ODE with

19 Deterministic Markov Process in R Realizations of the process: Process: Flow operator: Properties: Liouville Theorem: let be the flow ( ), given as a solution of If is defined as then the following equation is fulfilled:

20 Deterministic Markov Process in R Realizations of the process: Process: Flow operator: Properties: Liouville Theorem: let be the flow ( ), given as a solution of If Excercise 1: is think defined of the as proof (hint: use the partial integration ), think about the Hilbert space H and appropriate boundary conditions then the following equation is fulfilled:

21 Stochastic Markov Process in R Realizations of the process: Process: (follows from Ito s lemma) This Equation is Deterministic

22 Stochastic Markov Process in R Infenitisimal Generator: Markov Process Dynamics: This Equation is Deterministic: PDE

23 Numerics of Stochastic Processes Discrete State Space, Discrete Time: Markov Chain Discrete State Space, Continuous Time: Markov Process Continuous State Space, Discrete Time: Autoregressive Process Continuous State Space, Continuous Time: Stochastic Differential Equation

24 Numerics of Stochastic Processes Discrete State Space, Discrete Time: Markov Chain Discrete State Space, Continuous Time: Markov Process Deterministic Numerics of ODEs and PDEs! Continuous State Space, Discrete Time: Autoregressive Process Continuous State Space, Continuous Time: Stochastic Differential Equation

25 Conclusions 1) Numerical Methods from ODEs and (multidimensional) PDEs like Runge-Kutta-Methods, FEM and (adaptive) Rothe particle methods are applicable to stochastic processes H./Weiser, JCC 24(15), ) Monte-Carlo-Sampling of resulting p.d.f. s Stochastic Numerics = ODE/PDE numerics + Rand. Numb. Generator 3) Concepts from the Theory of Dynamical Systems are Applicable (Whitney and Takens theorems, model reduction by identification of attractors)

26 Plan for Today (stationary case) eagle eye perspective on stochastic processes from the viewpoint of deterministic dynamical systems geometric model inference: EOF/SSA multivariate dynamical model inference: VARX handling the ill-posed problem motivation for tomorrow : examples where stationarity assumption does not work

27 EOF/PCA/SSA/...: Geometrical methods For a given time series we look for a minimum of the reconstruction error µ T TT (X-µ) t T (X-µ) t x t

28 EOF/PCA/SSA/...: Geometrical methods For a given time series we look for a minimum of the reconstruction error µ T TT (X-µ) t T (X-µ) t x t

29 EOF/PCA/SSA/...: Geometrical methods For a given time series we look for a minimum of the reconstruction error µ T TT (X-µ) t T (X-µ) t x t

30 EOF/PCA/SSA/...: Geometrical methods For a given time series we look for a minimum of the reconstruction error µ T TT (X-µ) t T (X-µ) t x t Excercise 2: think of the proof (hint: use the Lagrange multipliers ), think of the estimate of projection error

31 Plan for Today (stationary case) eagle eye perspective on stochastic processes from the viewpoint of deterministic dynamical systems geometric model inference: EOF/SSA multivariate dynamical model inference: VARX handling the ill-posed problem motivation for tomorrow : examples where stationarity assumption does not work

32 Predicting Stochastic Processes Discrete State Space, Discrete Time: Markov Chain Discrete State Space, Continuous Time: Markov Process Deterministic Numerics of ODEs and PDEs! Continuous State Space, Discrete Time: Autoregressive Process Continuous State Space, Continuous Time: Stochastic Differential Equation

33 Predicting the Dynamics: VAR(p)

34 Estimating VAR(p) : least squares

35 Estimating VAR(p) : least squares

36 Estimating VAR(p) : least squares

37 Estimating VAR(p) : least squares Excercise 3: think of the proof (hint: use the matrix function derivatives from the Matrix Coockbook: www2.imm.dtu.dk/pubdb/views/edoc_download.../imm3274.pdf))

38 Plan for Today (stationary case) eagle eye perspective on stochastic processes from the viewpoint of deterministic dynamical systems geometric model inference: EOF/SSA multivariate dynamical model inference: VARX handling the ill-posed problem motivation for tomorrow : examples where stationarity assumption does not work

39 A. N. Tikhonov ( Estimating VAR(p) : regularization

40 A. N. Tikhonov ( Estimating VAR(p) : regularization

41 Plan for Today (stationary case) eagle eye perspective on stochastic processes from the viewpoint of deterministic dynamical systems geometric model inference: EOF/SSA multivariate dynamical model inference: VARX handling the ill-posed problem motivation for tomorrow : examples where stationarity assumption does not work

42 Lorenz96: order zero atmosph. model fast Wilks, Quart.J.Royal Met.Soc., 2005 slow stationary model (time-independent parameters) VARX (standard stationary stochastic Model) Lorenz, Proc. Of. Sem. On Predict., 1996 Majda/Timoffev/V.-Eijnden, PNAS, 1999 Orell, JAS, 2003 Wilks, Quart.J.Royal Met.Soc., 2005 Crommelin/V.-Eijnden, Journal of Atmos. Sci., 2008

43 Lorenz96: order zero atmosph. model fast Wilks, Quart.J.Royal Met.Soc., 2005 slow + =??? + F(t) non-stationary model (time-dependent parameters) VARX (standard stationary stochastic Model) Lorenz, Proc. Of. Sem. On Predict., 1996 Majda/Timoffev/V.-Eijnden, PNAS, 1999 Orell, JAS, 2003 Wilks, Quart.J.Royal Met.Soc., 2005 Crommelin/V.-Eijnden, Journal of Atmos. Sci., 2008

44 Lorenz96: order zero atmosph. model fast Wilks, Quart.J.Royal Met.Soc., 2005 slow + =??? + F(t) non-stationary model (time-dependent parameters) Predictors of Lorenz 96-Model VARX (standard stationary stochastic Model) Lorenz, Proc. Of. Sem. On Predict., 1996 Orell, JAS, 2003 Wilks, Quart.J.Royal Met.Soc., 2005 Crommelin/V.-Eijnden, Journal of Atmos. Sci., 2008

45 Tomorrow: Non-stationarity in TSA Direct problem: Inverse problem: Example: