Phase-Space Reconstruction Gerrit Ansmann
Reprise: The Need for Non-Linear Methods. Lorenz oscillator x = 1(y x), y = x(28 z) y, z = xy 8z 3 Autoregressive process measured with non-linearity: y t =.8y t 1 + ε t x t = (1 + y t ) 2 2
Reprise: The Need for Non-Linear Methods. Lorenz oscillator x = 1(y x), y = x(28 z) y, z = xy 8z 3 Autoregressive process measured with non-linearity: y t =.8y t 1 + ε t x t = (1 + y t ) 2 2 3 x t 1 1 x t 2 1 2 5 1 15 2 25 3 t 5 1 15 2 25 3 t 2
Reprise: The Need for Non-Linear Methods. Lorenz oscillator x = 1(y x), y = x(28 z) y, z = xy 8z 3 Skewness: v.4, Kurtosis: γ.71 Autoregressive process measured with non-linearity: y t =.8y t 1 + ε t x t = (1 + y t ) 2 Skewness: v 2.6 Kurtosis: γ 9.7 PDF (a.u.) PDF (a.u.) -2-1 1 2 1 2 3 x t x t 2
Reprise: The Need for Non-Linear Methods. Lorenz oscillator x = 1(y x), y = x(28 z) y, z = xy 8z 3 Autoregressive process measured with non-linearity: y t =.8y t 1 + ε t x t = (1 + y t ) 2 spectral density (a.u) spectral density (a.u) 5 1 15 2 25 3 period length 5 1 15 2 25 3 period length 2
Reprise: The Need for Non-Linear Methods. Lorenz oscillator x = 1(y x), y = x(28 z) y, z = xy 8z 3 Autoregressive process measured with non-linearity: y t =.8y t 1 + ε t x t = (1 + y t ) 2 1 1 R(x,τ) R(x,τ) 1 1 2 3 4 5 τ 1 1 2 3 4 5 τ 2
Reprise: The Need for Non-Linear Methods When faced with chaotic time series, linear methods fail to detect the dynamics / structure do not tell much about the dynamics cannot distinguish chaos from noise Structure can be seen in attractors. 3
Reminder: Attractors States of the dynamics for t. Type of dynamics can be deduced from topology of attractor: point fixed-point dynamics limit cycle periodic dynamics torus quasiperiodic dynamics strange attractor chaos Attractor reflects further central properties of dynamics. 4
Attractor of the Lorenz System adapted from commons.wikimedia.org/wiki/file:lorenz_system_r28_s1_b2-6666.png 5
The Need for Phase-Space Reconstruction Directly observing the phase space/attractor requires access to all dynamical variables. But: Often, only one dynamical variable is available. The dimension of phase space is unknown. 6
The Need for Phase-Space Reconstruction Directly observing the phase space/attractor requires access to all dynamical variables. But: Often, only one dynamical variable is available. The dimension of phase space is unknown. Can we obtain from one time series a set that preserves important properties of the attractor? 6
The Need for Phase-Space Reconstruction Directly observing the phase space/attractor requires access to all dynamical variables. But: Often, only one dynamical variable is available. The dimension of phase space is unknown. Can we obtain from one time series a set that preserves important properties of the attractor? Yes, but it s a bit of work. 6
Good and Bad Reconstructions actual attractor perfect reconstruction perfect reconstruction preserves structure, practically bad preserves structure, practically bad 2D: structure not preserved 3D: structure may be preserved (phase as real number) structure not preserved (forth and back on the same line) structure not preserved 7
Reconstructions in the Language of Topology original attractor an n-manifold A R d measurement and reconstruction a map: Φ A R m structure-preserving reconstruction topology-preserving map an embedding 8
Reconstructions in the Language of Topology original attractor an n-manifold A R d measurement and reconstruction a map: Φ A R m structure-preserving reconstruction topology-preserving map an embedding. embedding. A map Φ A R m is called an embedding, if:. Φ has full rank. Φ is a diffeomorphism: Φ is differentiable Φ 1 exists and is differentiable 8
Whitney s Embedding Theorems. Strong Whitney embedding theorem..for m = 2n, there exists a Φ A R m that is an embedding. Problem: We do not know Φ. 9
Whitney s Embedding Theorems. Strong Whitney embedding theorem..for m = 2n, there exists a Φ A R m that is an embedding.. Weak Whitney embedding theorem. For m 2n + 1, almost every continuously differentiable (C 1 ) map. Φ A R m is an embedding. Problems: Often, we do not have m independent observables. (Redundant observables are one of the reasons for almost every.) We do not know n. 9
Delay Embedding: Idea Time series: x 1, x 2,, x N x t, x t, x t, are not fully redundant. Approximate: x t = x t +1 x t x t = x t +2 2x t +1 + x t x t, x t +1, are not fully redundant. inverse Taylor expansion. 1
Takens Delay-Embedding Theorem.. Takens theorem. Let A = {y, y 1,, y N } with the index indicating time.. y t is the state of the system at time t. 11
Takens Delay-Embedding Theorem.. Takens theorem. Let A = {y, y 1,, y N } with the index indicating time. Let h A R be the measurement function.. y t is the state of the system at time t. h(y ), h(y 1),, h(y N ) is the time series. 11
Takens Delay-Embedding Theorem.. Takens theorem. Let A = {y, y 1,, y N } with the index indicating time. Let h A R be the measurement function. If m 2n + 1,. Φ hτ(y t ) = (h(y t ), h(y t τ),, h(y t (m 1)τ)) is an embedding for almost all dynamics, τ, and h. y t is the state of the system at time t. h(y ), h(y 1),, h(y N ) is the time series. m is called embedding dimension. τ is called embedding delay. 11
Takens Phase-Space Reconstruction Once more, for application: Univariate time series x: x 1, x 2,, x N Consider m-dimensional states (on the attractor): x t x t τ x t 2τ x t (m 1)τ For a proper embedding dimension m and embedding delay τ, these states make up a topologically equivalent reconstruction of the attractor. 12
Example: Sine Wave / Harmonic Oscillator Blackboard. 13
Example: Lorenz Attractor original reconstruction (τ =.8) 5 2 4 1 3 2 1 1 2 1 1 2 2 2 2 2 1 1 2 2 2 14
U(t) EEG (Wachzustand) t x(t) x(t+τ)
U(t) EEG (Epilepsiepatient) t x(t) x(t+τ)
U(t) EEG (epileptischer Anfall) t x(t) x(t+τ)
Theorems of Sauer, Yorke, and Casdagli Problem: Strange Attractors are only a subset of some n-manifold.. Fractal embedding theorems (Sauer, Yorke, and Casdagli). In the weak Whitney embedding theorem and Takens theorem, one can replace:. n-manifold A compact invariant set A n the box-counting dimension d b of A almost all (open and dense subset) almost all (probability 1) Advantage: d b < n 15
Dynamical Invariants Important characteristics of the dynamics are invariant under the embedding transformation: Lyapunov exponents dimensions entropy 16
Impact of the Delay: Sine Wave / Harmonic Oscillator Blackboard. 17
Impact of the Delay: Lorenz Attractor 2 τ =.1 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.2 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.3 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.4 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.5 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.6 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.7 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.8 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.9 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.1 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.11 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.12 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.13 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.14 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.15 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.16 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.17 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.18 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.19 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.2 1 1 2 2 1 1 2 2 2 18
Impact of the Delay: Lorenz Attractor 2 τ =.3 1 1 2 2 1 1 2 2 2 18
Choosing the Time Delay Requirement for an embedding: x t, x t τ, x t 2τ,, x t (m 1)τ not fully redundant. Aforementioned theorems: Almost every τ yields an embedding: Requirement for a good embedding: Minimal redundance of x t, x t τ, x t 2τ,, x t (m 1)τ (to unfold the attractor) Reasonably small τ (to avoid folding the attractor onto itself) Compare to: linear independence vs. orthogonality. 19
Zeros of the Autocorrelation Function Idea: R(x,τ) = x t and x t τ linearly independent on average. Choose τ as the first zero of the autocorrelation function. 2
First Minimum of the Mutual Information Mutual information I m (τ): Common information (redundancy) between x t and x t τ. To what extent does x t τ help to predict x t? Measure of independence / redundancy based on information theory (non-linear). (More on this in a few weeks) Choose τ as the first minimum of the mutual-information function. A. Fraser and H. Swinney, Independent coordinates for strange attractors from mutual information, Phys. Rev. A 33 (1986) 21
Autocorrelation vs. Mutual Information from A. Fraser and H. Swinney, Independent coordinates for strange attractors from mutual information, Phys. Rev. A 33 (1986) 22
Methods to Find the Embedding Delay zeros of the autocorrelation function minima of the mutual information many more No method is perfect or commonly agreed upon. Practically: Try at least two methods. Judge by further analysis. Alternative for m 3: Visually inspect the attractor. Keep (m 1)τ (time span in an embedded vector) constant. 23
Choosing the Embedding Dimension Aforementioned theorems: m = 2n + 1 or m = 2d b + 1 suffices. But: A priori, we do know neither n nor d b. This is the worst case a smaller m may also suffice: The surface of a torus (n = 2) can be embedded with m = 3. The Lorenz attractor (n = 3, d b 2) can be embedded with m = 3. Choosing m overly high may hamper further analyses (impact of noise, computational complexity). Need other ways to determine a good m. 24
Linear Dependence Idea: m is higher than necessary attractor only covers a subspace of reconstruction space (e.g, circle in m = 3) Check whether embedded vectors have full rank. Problems: Noise acts in all directions. Assumes linear dependence dependence may be non-linear. (e.g., bent circle in m = 3) 25
Assymptotic Invariants m too small wrong dynamical invariants (in general) m sufficient correct dynamical invariants Increase m until dynamical invariants converge. Difficulties: Criterion for convergence under real conditions (noise, finite data, ). Wrong (in general) is unpredictable. 26
False Nearest Neighbours Idea m too small trajectories intersect points close in reconstruction space that aren t close in actual phase space (false nearest neigbours) Raise m until false nearest neighbours vanish. M. B. Kennel, R. Brown, and H. D. I. Abarbanel, Determining minimum embedding dimension using a geometrical construction, Phys. Rev. A 45 (1992) 27
False Nearest Neighbours Practically Practically: Choose threshold ε for nearest neighbours. NN(m): Number of pairs of points in m-dimensional reconstruction space that are closer than ε. NN(m + 1) < NN(m) at least NN(m) NN(m + 1) false nearest neighbours in the m-dimensional reconstruction. Difficulties: Number of true nearest neighbours large and fluctuating (noise). 28
Methods to Determine the Embedding Dimension assymptotic invariants false nearest neighbours many more No method is perfect or commonly agreed upon. Practically: Try at least two methods. Judge by further analysis. Alternative for m 3: Visually inspect the attractor. If possible: embedding theorems as a sanity check. Keep (m 1)τ (time span in an embedded vector) constant. 29
Summary Delay embedding allows to reconstruct attractor from single observable. Parameters m and τ have to be carefully chosen. Reconstructed phase space may be used for: understanding prediction modelling Characteristics preserved by reconstruction: Lyapunov exponents dimensions entropy 3
Extensions multivariate time series different embedding delays for each component state-dependent embedding delays 31