Overview of sparse system identification

Transcription

1 Overview of sparse system identification J.-Ch. Loiseau 1 & Others 2, 3 1 Laboratoire DynFluid, Arts et Métiers ParisTech, France 2 LIMSI, Université d Orsay CNRS, France 3 University of Washington, Seattle, USA European Workshop on R.O.M for Industrial Applications October 2017

2 J.-Ch. Loiseau s-r.o.m. 1 of 43

3 J.-Ch. Loiseau s-r.o.m. 2 of 43 Overview What is reduced-order modeling? Reduced-order modeling (R.O.M) is a technique for reducing the computational complexity of mathematical models in numerical simulations. By reducing the state space s dimension, an approximation of the original model is computed. This R.O.M can then be evaluated with lower accuracy but in significantly less time. Wikipedia

4 J.-Ch. Loiseau s-r.o.m. 3 of 43 Overview Step 1 : Data Acquisition Examples: Direct Numerical Simulation (DNS), Large Eddy Simulation (LES), Time-resolved PIV visualizations,...

5 J.-Ch. Loiseau s-r.o.m. 4 of 43 Overview Step 2 : Dimensionality reduction Examples: Fourier Mode Decomposition, Proper Orthogonal Decomposition (POD), Dynamic and Koopman Mode Decomposition (DMD), Resolvent modes,...

6 J.-Ch. Loiseau s-r.o.m. 5 of 43 Overview Step 3 : Galerkin projection The governing equations of the high-dimensional system read d Q = LQ + N (Q) dt Assuming Q(x, t) U(x)a(t), a reduced-order model governing the evolution of a(t) can be obtained by Galerkin projection U T U d dt a = UT LUa + U T N (Ua) which can be rewritten as M d dt a = La + N (a).

7 J.-Ch. Loiseau s-r.o.m. 6 of 43 Illustration of POD-Galerkin R.O.M. Application to the shear-driven cavity flow

8 J.-Ch. Loiseau s-r.o.m. 7 of 43 POD-Galerkin R.O.M. Application to the shear-driven cavity flow Two-dimensional shear-driven cavity flow is becoming a standard benchmark in fluid dynamics. Non-trivial sequence of bifurcations as Re. Multiple length scales and time scales phenomena. Practical applications in aeronautics. Figure: Instantaneous vorticity colormap at Re = 7500 (based on cavity s depth).

9 J.-Ch. Loiseau s-r.o.m. 8 of 43 POD-Galerkin R.O.M. Application to the shear-driven cavity flow t PSD ω

10 J.-Ch. Loiseau s-r.o.m. 9 of 43 POD-Galerkin R.O.M. Proper Orthogonal Decomposition POD can be formulated as an SVD. Key advantage of SVD is its quantification of the truncation error. Flow fields can be reconstructed with an accuracy 90% using only the first 6 POD modes. % of explained variance Number of singular values

11 J.-Ch. Loiseau s-r.o.m. 10 of 43 POD-Galerkin R.O.M. Proper Orthogonal Decomposition

12 J.-Ch. Loiseau s-r.o.m. 11 of 43 POD-Galerkin R.O.M. Proper Orthogonal Decomposition POD analysis reveals the low-rank structure of the dynamics. Modes 1 to 4 capture the coherent structures along the shear layer. Modes 5 and 6 capture the inner-cavity coherent structures.

13 J.-Ch. Loiseau s-r.o.m. 12 of 43 POD-Galerkin R.O.M. Running the R.O.M v1, v1, v3, v3, v5, v5, t t (a) (b) Figure : (a) Short-time and (b) Long-time evolution of the amplitudes of the leading POD modes predicted by the POD-Galerkin projection R.O.M.

14 J.-Ch. Loiseau s-r.o.m. 13 of 43 POD-Galerkin R.O.M. Summary Despite the low-rank structure, a naive Galerkin projection leads to a R.O.M with limited prediction horizon. Including high-order modes (small scale structures) to properly capture the energy cascade does not necessarily improve the long-term behavior of the R.O.M. Several techniques exist to overcome such problem. All of them however require deep knowledge of the physics of the problem and can be mathematically challenging. Major limitation of Galerkin projection The governing equations need to be known a priori!

15 Sparse system identification J.-Ch. Loiseau s-r.o.m. 14 of 43

16 Overview What is system identification? The field of system identification uses statistical methods to build mathematical models of dynamical systems from measured data. Wikipedia White-box Knowing the model, estimate its parameters from data. Gray-box Given a generic model structure, estimate its parameters from data. Black-box Determine the model structure and estimate its parameters from data. J.-Ch. Loiseau s-r.o.m. 15 of 43

17 J.-Ch. Loiseau s-r.o.m. 16 of 43 Overview What is sparse system identification? Non sunt multiplicanda entia sine necessitate William of Ockham A model should be as simple as possible no simpler. A. Einstein

18 J.-Ch. Loiseau s-r.o.m. 17 of 43 Overview Which techniques? Over the years, a large number of different techniques have been developed for the identification of nonlinear dynamical systems : Volterra series, Block-structured models, Neural networks, NARMAX models. They all rely on the same four steps : 1. Data gathering, 2. Model postulate, 3. Parameters identification, 4. Model validation.

19 J.-Ch. Loiseau s-r.o.m. 18 of 43 SINDy Brunton et al., PNAS 2016

20 J.-Ch. Loiseau s-r.o.m. 19 of 43 Galerkin Regression Overview POD-Galerkin projection R.O.M uses Σ to select the projection basis U(x), but entirely neglects the temporal information in V H (t).

21 J.-Ch. Loiseau s-r.o.m. 20 of 43 Galerkin Regression Overview 0.05 v1, v3, v5, t Figure : Time-evolution of the amplitude of the leading POD modes.

22 J.-Ch. Loiseau s-r.o.m. 21 of 43 Galerkin Regression Looking for nonlinear correlations POD modes are linearly uncorrelated but can be nonlinearly correlated. Possibility to further reduce the dimension of the R.O.M. if strong nonlinear correlations exist v v v 5 Figure : Illustration of nonlinear correlation between v 3 and v 5 with v 1.

23 J.-Ch. Loiseau s-r.o.m. 22 of 43 Galerkin Regression Looking for nonlinear correlations Randomized Dependence Coefficient (RDC) is a new metric for nonlinear correlations. v 1 v 2 v 3 v 4 v 5 v 6 v 1 v 2 v 3 v 4 v 5 v 6 v 1 v 2 v 3 v 4 v 5 v 6 Pearson s ρ RDC

24 J.-Ch. Loiseau s-r.o.m. 23 of 43 Galerkin Regression Model postulate PCA: 6 modes are required to reconstruct the flow field to 95% accuracy. In practice, only four of these modes are required from a dynamical point of view. The remaining two are slaved to these driving modes. The model we postulate reads da = f (a, b) dt b = h(a) where a is the amplitude of the driving modes, b that of the slaved ones, and h(a) is a manifold equation.

25 J.-Ch. Loiseau s-r.o.m. 24 of 43 Galerkin Regression Basis pursuit problem Given time-series of a, construct a library of candidate (nonlinear) functions for the r.h.s Θ(a) = [ 1 a P 2 (a) P N (a) ] The unknown system can be recast as da dt = Θ(a)ξ where ξ is a sparse vector of coefficients determining which terms from the library Θ(a) are actually active in f (a, h(a)).

26 J.-Ch. Loiseau s-r.o.m. 25 of 43 Galerkin Regression Basis pursuit problem The sparse vector can be identified by solving the following sparse regression problem minimize ξ Least-Squares fit {}}{ ȧ Θ(a)ξ λ ξ 1 }{{} Sparsity promotion It is a convex optimization problem which can be solved very efficiently with standard open-source libraries.

27 J.-Ch. Loiseau s-r.o.m. 26 of 43 Galerkin Regression Overview Illustration of system identification using SINDy. From Brunton et al., PNAS, 2015.

28 J.-Ch. Loiseau s-r.o.m. 27 of 43 Galerkin Regression Dynamical system The Galerkin Regression model identified reads da dt = La + Q(a) + C(a). L is a linear operator governing the linearized dynamics of the system around the mean flow. Q(a) describes the quadratic interactions between the driving modes. C(a) is a cubic term describing the influence of slaved (i.e. truncated) modes onto the driving ones.

29 J.-Ch. Loiseau s-r.o.m. 28 of 43 Galerkin Regression Dynamical system After some minor simplifications, the identified model can be recast as ä 1 + q(a 1 )ȧ 1 + ω 2 0a 1 = f (a 1, a 3 ) ä 3 + p(a 3 )ȧ 3 + ω 2 1a 3 = g(a 1, a 3 ) Despite its apparent complexity, the flow can simply modeled as a set of two nonlinearly coupled quasi-harmonic oscillators.

30 J.-Ch. Loiseau s-r.o.m. 29 of 43 Galerkin Regression Manifold equations The identified manifold equations read b 1 = 0.58(a 2 1 a 2 2) a 1 a 2 b 2 = 0.35(a 2 1 a 2 2) 1.16a 1 a 2. Although the evolutions of higher-harmonic shear-layer modes are linearly uncorrelated to that of the dominant modes, these evolutions turn out to be strongly nonlinearly correlated.

31 J.-Ch. Loiseau s-r.o.m. 30 of 43 Galerkin Regression Comparisons 2 DNS Galerkin Regression 2 DNS Galerkin Regression a1 0 a b1 0 b a3 0 a t t (a) (b) Figure : (a) Short-time and (b) Long-time evolution of the amplitudes of the leading POD modes predicted by the POD-Galerkin regression R.O.M.

32 J.-Ch. Loiseau s-r.o.m. 31 of 43 Galerkin Regression Comparisons (a) Direct Numerical Simulation (b) Reduced-Order Model

33 J.-Ch. Loiseau s-r.o.m. 32 of 43 Galerkin Regression Summary Reconstructing the flow field (kinematics) and modeling its dynamics are two different problems. SINDy allows one to identify efficient and very low-dimensional R.O.M. Takes full advantage of possibly existing nonlinear correlations (manifold learning). Lowest-dimensional description of the dynamics. Not need prior knowledge of the high-dimensional governing equations! Yet, we easily identify a human-interpretable model.

34 Conclusions and perspectives J.-Ch. Loiseau s-r.o.m. 33 of 43

35 J.-Ch. Loiseau s-r.o.m. 34 of 43 Conclusions and perspectives Additional examples of SINDy a a 1 Low-order model describing the saturation of globally unstable flows. [ ] [ ] [ ] d a1 0 ω a1 = dt a 2 ω σ(1 a1 2 a2 2 ) a 2

36 J.-Ch. Loiseau s-r.o.m. 35 of 43 Conclusions and perspectives Additional examples of SINDy Despite its complexity, the dynamics of the buffet can be modeled like that of the cylinder flow. [ ] [ ] [ ] d a1 0 ω a1 = dt a 2 ω σ(1 a1 2 a2 2 ) a 2 Illustration of buffet on an airfoil at Re = and Ma = 0.73 using a RANS solver This link may significantly simplify the nonlinear control of this flow.

37 Conclusions and perspectives Additional examples of SINDy I Successfully applied to the Fluidic Pinball., Benchmark for MIMO nonlinear flow control., Open-Source platform (Octave or Python). DNS Low-order model (1) CL J.-Ch. Loiseau (3) (1) CL System ID applied directly to time-traces of the lift coefficients., Experimentally realistic R.O.M strategy., Excellent descriptive capabilities. CL (3) CL CL (2) I (2) CL s-r.o.m. 36 of 43

38 J.-Ch. Loiseau s-r.o.m. 37 of 43 Conclusions and perspectives Extension for control purposes SINDy has been extended to identify low-order nonlinear models including control inputs da = f (a, b) dt b = K(a) which can then be used for real-time control purposes (i.e. drag reduction, mixing enhancement,...). Current developments aim at interfacing SINDYc with Machine-Learning Control algorithms.

39 J.-Ch. Loiseau s-r.o.m. 38 of 43 Conclusions and perspectives Extension for varying parameters SINDy can be naturally extended to identify parametrized systems da dt = f (a, µ) where µ is the vector of parameters. Key challenge is to make sure that the meaning of a i does not change as µ varies (i.e. no mode switching)

40 J.-Ch. Loiseau s-r.o.m. 39 of 43 Conclusions and perspectives Extension for PDE identification

41 J.-Ch. Loiseau s-r.o.m. 40 of 43 Conclusions and perspectives Take-away Messages Sparse regression allows you to identify a low-order model of your system that include the influence of truncated modes. Look for nonlinear correlations in your data, not just linear ones! In all cases we have investigated, Galerkin Regression has outperformed Galerkin projection.

42 J.-Ch. Loiseau s-r.o.m. 41 of 43 Conclusions and perspectives Open questions A lot of material and technicalities have not been covered in this talk, including but not limited to: Is POD really the best dimensionality reduction technique? How large and diverse should the library Θ be for Galerkin Regression? How to select the best model balancing parsimony and accuracy? What about data contaminated with non-gaussian/non-white noise? How to quantify uncertainty of the model s estimated parameters?

43 J.-Ch. Loiseau s-r.o.m. 42 of 43 Sparse reduced-order modeling If you want to know more

44 J.-Ch. Loiseau s-r.o.m. 42 of 43 Thank you for your attention. Any questions?

45