Bayesian Dynamic Mode Decomposition

Transcription

1 Bayesian Dynamic Mode Decomposition Naoya Takeishi, Yoshinobu Kawahara,, Yasuo Tabei, Takehisa Yairi Dept. of Aeronautics & Astronautics, The University of Tokyo The Institute of Scientific and Industrial Research, Osaka University RIKEN Center for Advanced Intelligence Project (AIP) 22 August 2017, Melbourne

2 Motivation: Analysis of dynamical systems Various types of complex phenomena can be described in terms of (nonlinear) dynamical systems. x t+1 = f(x t ), x M (state space) When f is nonlinear, analysis based on trajectories of x is difficult. N. Takeishi (The Univ. of Tokyo) Bayesian DMD 2 of 14

3 Operator-theoretic view of dynamical systems Definition (Koopman operator [Koopman 31, Mezić 05]) Koopman operator (composition operator) K represents time-evolution of observables (i.e., observation function) g : M R or C. Kg(x) = g(f(x)), g F (function space) K describes temporal evolution of function (infinite-dimensional vector) instead of the finite-dimensional state vector. Defining K, we can lift the analysis of nonlinear dynamical systems into a linear (but infinite-dimensional) regime! Since K is linear, we can analyze dynamics using the spectra of K. N. Takeishi (The Univ. of Tokyo) Bayesian DMD 3 of 14

4 Koopman mode decomposition (KMD) Eigenvalues and eigenfunctions of K: K ϕ i (x) = λ i ϕ i (x) for i = 1, 2,.... Projection of g(x) to span{ϕ 1 (x), ϕ 2 (x),... } (i.e., transformation to a canonical form). Since ϕ is eigenfunction, Coefficients are called Koopman modes. g(x) = ϕ i (x) v i g(x t ) = i=1 i=1 λ t i ϕ i (x 0 )v i }{{}, (KMD) w i where λ i = decay rate of w i, λ i = frequency of w i. A numerical realization of KMD is dynamic mode decomposition (DMD) [Rowley+ 09, Schmid 10, Tu+ 14]. N. Takeishi (The Univ. of Tokyo) Bayesian DMD 4 of 14

5 Dynamic mode decomposition (DMD) Assumption (K-invariant subspace [Budišić+ 12]) Dataset is generated with a set of observables [ g(x) = g 1 (x) g 2 (x) g n (x) that spans (approximately) K-invariant subspace. Then, KMD can be (approximately) realized by DMD. Algorithm (DMD [Tu+ 14]) Input time-series (y 0,..., y m ) s.t. y t = g(x t ) Output eigenvalues {λ}, eigenfunctions {ϕ}, and modes {w} 1. Estimate a linear model y t+1 Ay t. 2. On A, compute eigenvalues λ i and right-/left-eigenvectors w i, z H i. 3. Compute ϕ i,t = z H i y t. ] T N. Takeishi (The Univ. of Tokyo) Bayesian DMD 5 of 14

6 Quasi-periodic modes extraction by KMD/DMD Review: KMD/DMD computes the decomposition of time-series into modes w i that evolve with frequency λ i and decay rate λ i. wi is termed dynamic modes. g(x t ) i=1 λ t i w i Example (2D fluid flow past a cylinder) Flow past a cylinder is universal in many natural/engineering situations. DMD w 1 + w 2 + w 3 + w 4 + w 5 + N. Takeishi (The Univ. of Tokyo) Bayesian DMD 6 of 14

7 Other applications of KMD/DMD Lots of applications in a wide range of domains fluid mechanics [Rowley+ 09, Schmid 10, & many more], neuroscience [Brunton+ 16], image processing [Kutz+ 16, Takeishi+ 17], analysis of power systems [Raak+ 16, Susuki+ 16], epidemiology [Proctor&Eckhoff 15], optimal control [Mauroy&Goncalves 16], finance [Mann&Kutz 16], medical care [Bourantas+ 14], robotics [Berger+ 15], etc. N. Takeishi (The Univ. of Tokyo) Bayesian DMD 7 of 14

8 Issue DMD relies on linear modeling g(x t+1 ) Ag(x t ) and eigendecomposition of A. So it lacks an associated probabilistic/bayesian framework, by which we can consider observation noise explicitly, perform a posterior inference, consider DMD extensions in a unified manner, etc. Let s do it! analogously to PCA s formulation as probabilistic/bayesian PCA [Tipping&Bishop 99, Bishop 99] N. Takeishi (The Univ. of Tokyo) Bayesian DMD 8 of 14

9 Proposed method (1/2): Probabilistic DMD Dataset: snapshot pairs with observation noise ( ) D = (y 0,1, y 1,1 ),..., ( y 0,t, y 1,t ),..., (y 0,m, y 1,m ), where y 0,t = g(x t ) + e 0,t and y 1,t = g(x t+ t ) + e 1,t, Definition (Generative model of probabilistic DMD) y 0,t CN y 1,t CN ( k ) i=1 ϕ t,iw i, σ 2 I ( k ) i=1 λ iϕ t,i w i, σ 2 I ϕ t,i CN (0, 1) If k = n and σ 2 0, the MLE of (λ, w) coincides with DMD s solution. N. Takeishi (The Univ. of Tokyo) Bayesian DMD 9 of 14

10 Proposed method (2/2): Bayesian DMD Definition (Prior on parameters for Bayesian DMD) w i vi,1:n 2 CN ( 0, diag ( vi,1, 2..., vi,n)) 2, v 2 i,d InvGamma (α v, β v ) λ i CN (0, 1) σ 2 InvGamma (α σ, β σ ) v 2 i,1:n w i y 0,t 2 k i y 1,t ' t m For a posterior inference, a Gibbs sampler can be constructed easily. N. Takeishi (The Univ. of Tokyo) Bayesian DMD 10 of 14

11 Extension example: Sparse Bayesian DMD Definition (Prior on parameters for sparse Bayesian DMD) w i vi,1:n 2 CN ( 0, σ 2 diag ( vi,1, 2..., vi,n)) 2, v 2 i,d Exponential(γi 2 /2) λ i CN (0, 1) σ 2 InvGamma (α σ, β σ ) v 2 i,1:n w i y 0,t 2 k i y 1,t ' t m We can extend the model in a unified Bayesian manner. N. Takeishi (The Univ. of Tokyo) Bayesian DMD 11 of 14

12 Numerical example (1) Example (Fixed-point attractor) Generate data by y t = λ t 1 [ ] T [ T λ t 2 2 2] + et, where e is Gaussian observation noise. True eigenvalues are λ 1 = 0.9 and λ 2 = 0.8. Re( ~ 61) truth DMD TLS-DMD < Re( ~ 62) < N. Takeishi (The Univ. of Tokyo) Bayesian DMD 12 of 14

13 Numerical example (2) Example (Limit-cycle attractor) Generate data from Stuart Landau equation r t+1 = r t + t(µr t r 3 t ), θ t+1 = θ t + t(γ βr 2 t ), and Gaussian observation noise. True (continuous-time) eigenvalues lie on the imaginary axis. Re(log(6)="t) DMD TLS-DMD BDMD (average) Im(log(6)="t) N. Takeishi (The Univ. of Tokyo) Bayesian DMD 13 of 14

14 Analysis of dynamical systems based on Koopman operator is a useful tool. Kg(x) = g(f(x)) Dynamic mode decomposition (DMD) is a numerical method for Koopman analysis. In this work, we developed probabilistic & Bayesian DMDs to consider observation noise, infer posterior distribution, v 2 i,1:n wi y 0,t 2 extend DMD in a unified manner, etc. k i y 1,t ' t m Implementation available at N. Takeishi (The Univ. of Tokyo) Bayesian DMD 14 of 14