Sjoerd Verduyn Lunel (Utrecht University)

Transcription

1 Universiteit Utrecht, February 9, 016 Wasserstein distances in the analysis of time series and dynamical systems Sjoerd Verduyn Lunel (Utrecht University)

2 Abstract A new approach based on Wasserstein distances, which are numerical costs of an optimal transportation problem, allows to analyze nonlinear phenomena in a robust manner. The long-term behavior is reconstructed from time series, resulting in a probability distribution over phase space. Each pair of probability distributions is then assigned a numerical distance that quantifies the differences in their dynamical properties. From the totality of all these distances a low-dimensional representation in a Euclidean space is derived. This representation shows the functional relationships between the dynamical systems under study. It allows to assess synchronization properties and also offers a new way of numerical bifurcation analysis.

3 Discrete Dynamical Systems Let X R d en f : X X and consider the discrete dynamical system starting from a given initial point x 0. x n+1 = f(x n ) A set A is called attracting with respect to U if for every neighborhood V of A, there exists a K = K(V ) such that f k (U) V for all k K. The basin of attraction of A is defined by B(A) = k 0 f k (U). If B(A) = X, then we call A a (global) attractor. A trajectory x = (x 1,..., x N ) generated by the dynamical system modeled by f can be viewed as a time series. In this talk we consider time series generated by f as well as generated directly from measurements.

4 Intermezzo Consider the iterative process x n+1 = x n + 1 x n. This process more than 4000 years old and nowadays is known as an application of Newton s method. Suppose that we start with x 0 = 1 then x 1 = x x 0 = 3. And the next step x = x x 1 = = Continuing this procedure yields the following table

5 Intermezzo II n x n approximation / / / / Notice that according to your calculator: = n x n approximation /

6 Intermezzo III Let a a number greater than zero and consider the iterative process x n+1 = x n a + a 1 x n. Choose your favorite a, start with x 0 = 1, and compute x, x 3 and x 4. There are also other interesting iterating systems. Let r be a number between 1 en 4 and consider the iterative process x n+1 = rx n (1 x n ). Choose your favorite r, start with x 0 between 0 and 1, for example, x 0 = 1/, and compute x, x 3 and x 4.

7 The Hénon map The this talk the running example in case the underlying dynamical system is known will be the Hénon map. The Hénon map is defined by ( ) xn+1 y n+1 = H ( xn ), n 0, y n where x 0 en y 0 are given initial conditions and H : R R is given by ( ( ) x 1 + y ax. y) bx Here a en b are parameters. This map was introduced in 1976 by Hénon to study (strange) attractors. If a = 1.4 and b = 0.3 the system exhibits chaotic behavior. Time series data from a real world system implicitly defines a dynamical system given by the information contained in repeated measurements, and delay coordinate reconstruction allows to represent these trajectories in Euclidean space.

8 Intermezzo IV The aim is to apply our algorithm directly to data, i.e., measurements as time series. But we can also use a model to produce a time series. Such a time series we call a synthetic time series, and allows us to test our algorithm theoretical. Consider the Hénon map H. Start with x 0 = 1 en y 0 = 1 and a = 1.4 and b = 0.3. Compute x 1 and y 1 by ( x1 y 1 ) ( ) (1) = = 0.3 ( ) and x en y by ( x y ) = ( ) (0.6) 0.18 = ( ) We can plot the points ( x0 ), y 0 ( x1 ), y 1 ( x ),... y in the plane. As in the example, this sequence approaches a geometrical object: the attractor.

9 Intermezzo V We can plot the attractor by plotting the tails of a long time series. From the theory, it however follows that we can reconstruct the attractor also from a piece of the tail of time series given by only by the x coordinate of the points (x, y). This method, plotting blocks of the tail of a time series, we can also do if we only have time series from measurements. This approach will be explained next.

10 Attractor reconstruction I Given a time series x = (x 1,..., x N ) of N measurements of a single observable X, a trajectory of a dynamical system is reconstructed by mapping each consecutive block x [i] = (x i, x i+q,..., x i+(k 1)q ) of k values, sampled at discrete time intervals q, into a single point x [i] in a Euclidean space Ω = R k. This map defines the delay-coordinate map F into the reconstructed phase space Ω. Sauer, Yorke and Casdagli (1991) extending work by Takens (1981) showed, under mild assumptions, that almost every delay-coordinate map F : R N R k is one-to-one on A provided that the embedding dimension k is larger than twice the box counting dimension of A. Also, any manifold structure within A will be preserved in F (A). The optimal value of the lag q and of the embedding dimension k is a priori unknown but can be estimated from the data.

11 Attractor reconstruction II This methodology of attractor reconstruction by delay embedding

12 Attractor reconstruction III The result of the embedding process is a discrete trajectory in reconstructed phase space Ω = R k. This trajectory can be interpreted as a probability measure µ on Ω, where µ[a] = 1 N N i=1 δ x[i] [A], A Ω, is the time average of the characteristic function of the points in phase space visited. In the limit N the measure µ is invariant under the dynamics and under mild assumptions the invariant measure is unique and its support contains an attractor (Lasota and Mackey, 004). If a dynamical system underlying the time series is available, subdivision methods allow to approximate the attractor and its natural measure with arbitrary precision. We shall illustrate this for the Hénon map.

13 The Hénon map - a time series For the parameters a = 1.0 en b = 0.54 and the initial condition x 0 = 0.1 en y 0 = 0.0 the time series for the x-coordinate is given by Henon system: Time series #1 Henon system: Time series #1 (detail) x Value x Value Time Time Figuur 1: Time series of the x-coördinate with x 0 = 0.1 en y 0 = 0.0 The time series consists of 3000 iterations and is relatively short - this is the situation one often encounters in practice.

14 The Hénon map reconstruction of the attractor Reconstruction of the attractor using the generated time series Henon system: Reconstructed attractor #1 x[n+1] x[n] Figuur : The attractor corresponding to the parameters a = 1.0 en b = The first 1000 entries of the time series are considered transient and the delay reconstruction is based on the entries [ ]. Zie Kant/Schreiber, Nonlinear time series analysis, Cambridge.

15 The Hénon map - reconstruction of the attractor Here are some reconstructed attractors for different parameter values. Henon system: Reconstructed attractor #3 Henon system: Reconstructed attractor #4 x[n+1] x[n+1] x[n] x[n] Figuur 3: The reconstructed attractor for, respectively, a = 1.0,b = 0.5 en a = 1.0, b = Note that the chaotic behavior disappears for last set of parameter values and that we have periodic behavior.

16 The Hénon map - the subdivision algorithm Subdivision #4 of Henon attractor Subdivision #6 of Henon attractor Subdivision #8 of Henon attractor Subdivision #10 of Henon attractor y 0 y 0 y y x x x x Student Version of MATLAB Student Version of MATLAB Student Version of MATLAB Student Version of MATLAB 3 Subdivision #14 of Henon attractor 3 Subdivision #16 of Henon attractor 3 Subdivision #18 of Henon attractor 3 Subdivision #18 of Henon attractor y 0 y 0 y 0 y x x x x Student Version of MATLAB Student Version of MATLAB Student Version of MATLAB Student Version of MATLAB

17 Calculation of Wasserstein distances in the discrete case An example of an optimal transportation problem in the discrete case Figuur 4: Open circles correspond to the first measure, filled circles correspond to the second measure. Left panel: Initial configuration. Numbers indicate probability mass at each point. Right panel: An optimal transportation plan with Wasserstein distance W 3.1. The numbers next to the arrows indicate how much probability mass is transported from the first measure to the second measure. Since the algorithms for the transportation problem have at least a quadratic dependence on sample size. A practical solution is to resample smaller subseries from the reconstructed trajectory and to estimate the Wasserstein distances multiple times, bootstrapping its expected value. This reduces the computational load.

18 Distance matrices We assume that the distance information is presented in the form of a single matrix M whose entries M ij = W (µ i, µ j ) represent the distances between the family of dynamical systems under consideration. Multidimensional scaling (MDS) techniques are used to model distance data as points in a geometric (usually Euclidean) space. In our present application to dynamical systems, each point in this space represents a single dynamical system and the space can be interpreted as the space of (the totality of)their possible dynamical behavior. We call this space the behavior space. Principle 1. The reconstructed behavior space, i.e., the MDS coordinates derived from a distance matrix, is the object at which all (statistical) analysis starts.

19 Methodology A summary of the methodology Dynamical systems Measurement / Projection Time series Delay reconstruction Probability distributions Distance matrix Wasserstein distances Multidimensional scaling Behavior representation Statistical analysis Parameter changes? Clustering Classification Coupling strength? Permutation tests Cross-validation Significance? Accuracy?

20 Visualization of parameter changes One of the most interesting aspects of the distance based analysis that we have introduced is the possibility to visualize changes in dynamical behavior with respect to parameter changes, similar to a bifurcation analysis. The parameters a, b of the Hénon map were varied, with a ranging from 1.4 to 0.7 in fourteen steps of 0.05, and b ranging from 0.3 to 0.0 in steps of 0.0. The invariant measures of the x-variable, corresponding to a trivial embedding dimension k = 1, are shown density of x density of x a b = b a = 1.4 Figuur 5: Left panel: Variation in parameter a with constant b = 0.3. Right panel: Variation in parameter b with constant a = 1.4. Darker shade indicates large time averages, and lighter shade smaller time averages.

21 Hénon map: Visualization of parameter changes II The left panel shows a two-dimensional projection of behavior space, i.e., of the Wasserstein distances of the respective dynamical systems. The distinct behavior of these systems, with respect to parameter changes, is clearly visible. Larger deviations of the parameters from (a 0, b 0 ) = (1.4, 0.3) result in points that are farther away from the point Figuur 6: Two-dimensional MDS representation of Wasserstein distances. Squares correspond to variation in the first parameter a, Triangles to variation in the second parameter b. The circles around the points corresponding to a = 1.4, b = 0.3 have radius and which are the mean self-distances.

22 Lung diseases - Background Both asthma and the condition known as chronic obstructive pulmonary disease (COPD) are obstructive lung diseases that in the early stages they show similar symptoms, rendering correct diagnosis difficult. As different treatments are needed, this is of considerable concern. An important diagnostics tool is the forced oscillation technique (FOT), as it allows to assess lung function non-invasive. By superimposing a range of pressure oscillations on the ambient air and analyzing the response of the airway systems, a number of parameters can be estimated that describe the mechanical properties of airway tissue. In particular, for each forcing frequency ω, a complex input impedance Z(ω) is available as time series, discretely sampled during a short period of tidal breathing. The real part R(ω) represents the airway resistance and the imaginary part Xω) the airway reactance.

23 Lung diseases - The data Clinicians usually only use the mean values R(ω) and X(ω) of these parameters, averaged over the measurement period. Clearly there is a lot more (dynamical) information contained in the time series as the example time series of R(ω) and X(ω) show cmh O L s cmh O L s time [s] time [s] Figuur 7: Example time series of respiratory resistance R(8) (upper curves) and respiratory reactance X(8) (lower curves) by forced oscillation technique during thirty seconds of tidal breathing. Left panel: A patient with mild asthma. Right panel: A patient with mild to severe COPD. The horizontal lines indicate the mean values used routinely in clinical assessment.

24 Lung diseases - Discrimination by Wasserstein distances I The motivation for the application of Wasserstein distances to this dataset is the assumption that the two lung diseases affect the temporal dynamics of input impedance in distinct ways. Considering asthma and COPD as dynamical diseases, we assume an underlying dynamical systems with different parameters for the different diseases. Although these parameters are not accessible, it is then possible to discriminate the two diseases using the Wasserstein distances between the data to quantify the differences in dynamics. For simplicity, we only consider here a two-dimensional reconstructing here, where the time series of R(8) and X(8) were combined into a series of two-dimensional vectors with trivial embedding dimension k = 1, trivial lag q = 1, and a length of about 1000 values (recorded at 16 Hz, the Nyquist frequency for the 8 Hz forced oscillation, concatenating all 1 measurements into one long series per patient). We consider the distribution of these points in Ω = R as an approximation of the invariant measure of the underlying dynamical system.

25 Lung diseases - Discrimination by Wasserstein distances II For reference we show the results for the squared sum of differences d ij = ( ( X i (8) X j (8)) + ( R i (8) R j (8)) ) 1/ in means (not the Wasserstein distances) A B C P= Figuur 8: Panel A: Two dimensional MDS reconstruction for patients suffering from asthma (open circles) and COPD (filled squares). Panel B: Strain values against reconstruction dimension. Panel C: Some statistics for the two classes.

26 Lung diseases - Discrimination by Wasserstein distances III The results using the Wasserstein distances are shown below A B C P=3e Figuur 9: Panel A: Two dimensional MDS reconstruction for patients suffering from asthma (open circles) and COPD (filled squares). Panel B: Strain values against reconstruction dimension. Panel C: Some statistics for the two classes. The classification by Wasserstein distances is noticeable better. From this we conclude that the dynamical information contained in the fluctuations of respiratory impedance contains valuable clinical information.

27 Lung diseases - Summary The results presented for asthma and COPD patients show that the Wasserstein distances are able to capture differences of shape of the long-term behavior of real-world time series. Even for the trivial embedding shown and a low number of bootstrap samples, the Wasserstein distances allow to classify a large proportion of the lung diseases correctly. In fact, these result are the best known classification results (at single FOT frequency) of these two lung diseases known in the literature.

28 Conclusion Uit Mackey, M.C.; Milton, J.G.: Dynamical Diseases, Ann. N.Y. Acad. Sci. 504 (1987), 16-3: A dynamical disease is defined as a disease that occurs in an intact physiological control system operating in a range of control parameters that leads to abnormal dynamics. The signature of a dynamical disease is a change in the qualitative dynamics of some observable nature as one or more parameters are changed. This is precisely the situation that we discussed in this talk and showed that Wasserstein distances can potentially be effectively used even in noisy environment to detect the signature of a dynamical disease. Muskulus-VL, J Applied Physiology (010) Muskulus-VL, Physica D (011) Muskulus-VL, In progress Please contact me for further information.

29 Nationale Wetenschapsagenda