System identification and control with (deep) Gaussian processes. Andreas Damianou

Transcription

1 System identification and control with (deep) Gaussian processes Andreas Damianou Department of Computer Science, University of Sheffield, UK MIT, 11 Feb. 2016

2 Outline Part 1: Introduction Part 2: Gaussian processes GPs as infinite dimensional Gaussian distributions Part 3: In practice Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

3 Outline Part 1: Introduction Part 2: Gaussian processes GPs as infinite dimensional Gaussian distributions Part 3: In practice Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

4 General area of interest Dynamical systems. Non-linear models; models with exogenous inputs (NARX) Model-based, data-driven approaches for regressive and auto-regressive inference.

5 Examples Through a model, we wish to learn to: perform free simulation by learning patterns coming from the latent generation process (a mechanistic system we do not know) perform inter/extrapolation in time-series data which are very high-dimensional (e.g. video) detect outliers in data coming from a dynamical system optimize policies for control based on a model of the data.

6 Data-driven Data driven: Learn from data by exploiting patterns through probabilistic modelling. Pros: Complex situations where no ODEs are present etc. Prior probabilities can to some degree incorporate side knowledge. Principled handling of noise / uncertainty.... Cons: More difficult to incorporate mechanistic knowledge (although there s work attempting to do this). Relies on the way the model is optimized (local minima, approximations, computational inefficiencies, numerical problems...)...

7 Model-based approach Mechanistic model challenge: create a model which is as simple as possible but also as close to reality as possible. Probabilistic model challenge: enrich the statistical properties of the model: robustness to outliers, representation learning (e.g. deep, auto-encoders)

8 Why Model with Gaussian process Uncertainty quantification learn from few data attractive analytical properties Bayesian framework: make modelling assumptions explicit.

9 Cool stuff you can do with GPs #1 Model-based policy learning (Cart-pole) (Unicycle) Work by: Marc Deisenroth, Andrew McHutchon, Carl Rasmussen

10 Outline Part 1: Introduction Part 2: Gaussian processes GPs as infinite dimensional Gaussian distributions Part 3: In practice Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

11 Introducing Gaussian Processes: A Gaussian distribution depends on a mean and a covariance matrix. A Gaussian process depends on a mean and a covariance function.

12 Infinite model... but we always work with finite sets! Let s start with a multivariate Gaussian: p(f 1, f 2,, f }{{} s, f s+1, f s+2,, f N ) N (µ, K). }{{} f A f B with: µ = [ µa µ B ] [ ] KAA K and K = AB K BA K BB Marginalisation property: p(f A, f B ) N (µ, K). Then: p(f A ) = p(f A, f B )df B = N (µ A, K AA ) f B

13 Infinite model... but we always work with finite sets! Let s start with a multivariate Gaussian: p(f 1, f 2,, f }{{} s, f s+1, f s+2,, f N ) N (µ, K). }{{} f A f B with: µ = [ µa µ B ] [ ] KAA K and K = AB K BA K BB Marginalisation property: p(f A, f B ) N (µ, K). Then: p(f A ) = p(f A, f B )df B = N (µ A, K AA ) f B

14 Infinite model... but we always work with finite sets! In the GP context: µ X µ = and K = [ KXX ]

15 Posterior is also Gaussian! p(f A, f B ) N (µ, K). Then: p(f A f B ) = N (µ A + K AB K 1 BB (f B µ B ), K AA K AB K 1 BB K BA) In the GP context this can be used for inter/extrapolation: f f 1,, f N GP post p(f f 1,, f N ) = p(f(x ) f(x 1 ),, f(x N )) But where is K.. coming from in GPs? N (K K 1 f, K, K K 1 K )

16 Posterior is also Gaussian! p(f A, f B ) N (µ, K). Then: p(f A f B ) = N (µ A + K AB K 1 BB (f B µ B ), K AA K AB K 1 BB K BA) In the GP context this can be used for inter/extrapolation: f f 1,, f N GP post p(f f 1,, f N ) = p(f(x ) f(x 1 ),, f(x N )) But where is K.. coming from in GPs? N (K K 1 f, K, K K 1 K )

17 Posterior is also Gaussian! p(f A, f B ) N (µ, K). Then: p(f A f B ) = N (µ A + K AB K 1 BB (f B µ B ), K AA K AB K 1 BB K BA) In the GP context this can be used for inter/extrapolation: f f 1,, f N GP post p(f f 1,, f N ) = p(f(x ) f(x 1 ),, f(x N )) But where is K.. coming from in GPs? N (K K 1 f, K, K K 1 K )

18 Covariance samples and hyperparameters k(x, x ) = α exp ( γ 2 (x x ) (x x ) ) The hyperparameters of the cov. function define the properties (and NOT an explicit form) of the sampled functions

19 Incorporating Gaussian noise is tractable So far we assumed: f = f(x) Assuming that we only observe noisy versions y of the true outputs f: y = f(x) + ɛ, ɛ N (0, σ 2 )

20 Fitting the data (shaded area is uncertainty)

21 Fitting the data - Prior Samples

22 Fitting the data

23 Fitting the data

24 Fitting the data - more noise

25 Fitting the data - no noise

26 Fitting the data - Posterior samples

27 Fitting the data

28 Fitting the data

29 Fitting the data

30 Fitting the data

31 Fitting the data

32 Fitting the data

33 Part 1: Take-home messages Gaussian processes as infinite dimensional Gaussian distributions can be used as priors over functions Non-parametric: training data act as parameters Principled handling of uncertainty

34 Outline Part 1: Introduction Part 2: Gaussian processes GPs as infinite dimensional Gaussian distributions Part 3: In practice Autoregressive Dynamics Going deeper: Deep Recurrent Gaussian Process Regressive dynamics with deep GPs

35 NARX model A standard NARX model considers an input vector x i R D comprised of L y past observed outputs y i R and L u past exogenous inputs u i R: x i = [y i 1,, y i Ly, u i 1,, u i Lu ], y i = f(x i ) + ɛ (y) i, ɛ (y) i Latent auto-regressive GP model: N (ɛ (y) i 0, σy), 2 x i = f(x i 1,, x i Lx u i 1,, u i Lu ) + ɛ (x) i, y i = x i + ɛ (y) i, Contribution 1: Simultaneous auto-regressive and representation learning. Contribution 2: Latents avoid the feedback of possibly corrupted observations into the dynamics. Mattos, Damianou, Barreto, Lawrence, 2016

36 NARX model A standard NARX model considers an input vector x i R D comprised of L y past observed outputs y i R and L u past exogenous inputs u i R: x i = [y i 1,, y i Ly, u i 1,, u i Lu ], y i = f(x i ) + ɛ (y) i, ɛ (y) i Latent auto-regressive GP model: N (ɛ (y) i 0, σy), 2 x i = f(x i 1,, x i Lx u i 1,, u i Lu ) + ɛ (x) i, y i = x i + ɛ (y) i, Contribution 1: Simultaneous auto-regressive and representation learning. Contribution 2: Latents avoid the feedback of possibly corrupted observations into the dynamics. Mattos, Damianou, Barreto, Lawrence, 2016

37 Robustness to outliers Latent auto-regressive GP model: x i = f(x i 1,, x i Lx u i 1,, u i Lu ) + ɛ (x) i, y i = x i + ɛ (y) i, ɛ (x) i ɛ (y) i N (ɛ (x) i 0, σx), 2 N (ɛ (y) 0, τ 1 i i ), τ i Γ(τ i α, β), Contribution 3: Switching-off outliers by including the above Student-t likelihood for the noise. Mattos, Damianou, Barreto, Lawrence, 2016

38 Robust GP autoregressive model: demonstration (a) Artificial 1. (b) Artificial 2. Figure: RMSE values for free simulation on test data with different levels of contamination by outliers.

39 (c) Artificial 3. (d) Artificial 4. (e) Artificial 5.

40 Going deeper: Deep Recurrent Gaussian Process u x (1) x (2) x (H ) y Figure 1: RGP graphical model with H hidden layers. x is the lagged latent function values augmented with the lagged exogenous inputs. Mattos, Dai, Damianou, Barreto, Lawrence, 2016

41 Inference is tricky... log p(y) N L 2 Tr H+1 h=1 ( ( K (H+1) z log 2πσ 2 h 1 ) 1 Ψ (H+1) (σH+1 2 y Ψ (H+1) )2 1 { ( H + 1 N 2σ 2 h=1 h i=l K (h) z 1 2 K(h) + 1 ( µ (h)) (h) Ψ 2(σh 2)2 1 N ) i=l+1 x (h) i ( q 2σ 2 H+1 ) ) ( y y + Ψ (H+1) 0 K z (H+1) ( K z (H+1) + 1 Ψ (H+1) σh ( λ (h) i + µ (h)) µ (h) + Ψ (h) 0 Tr z + 1 Ψ (h) σh 2 2 ( K z (h) + 1 ) 1 ( Ψ (h) σh 2 2 ( ) L log q + x (h) i x (h) i i=1 x (h) i 1 2 K(H+1) z + 1 σh+1 2 ) 1 ( ) y Ψ (h) 1 ( q x (h) i Ψ (H+1) 1 ( ( K (h) z ) µ (h) ) ( log p Ψ (H+1) 2 ) 1 Ψ (h) 2 x (h) i ) }. ) )

42 Results Results in nonlinear systems identification: 1. artificial dataset 2. drive dataset: by a system with two electric motors that drive a pulley using a flexible belt. input: the sum of voltages applied to the motors output: speed of the belt.

43 RGP GPNARX MLP-NARX LSTM

44 RGP GPNARX MLP-NARX LSTM

45 Avatar control Figure: The generated motion with a step function signal, starting with walking (blue), switching to running (red) and switching back to walking (blue) Videos: Switching between learned speeds Interpolating (un)seen speed Constant unseen speed

46 Regressive dynamics with deep GPs t x h f Instead of coupling f s by encoding the Markov property, we couple them by coupling the f s inputs through another GP with time as input. y = f(x) + ɛ x GP(0, k x (t, t)) f GP(0, k f (x, x)) y Damianou, Titsias, Lawrence, 2011

47 Dynamics Dynamics are encoded in the covariance matrix K = k(t, t). We can consider special forms for K. Model individual sequences Model periodic data (missa) (dog) (mocap)

48 Summary Data-driven, model-based approach to control problems Gaussian processes: Uncertainty quantification / propagation gives an advantage Deep Gaussian processes: Representation learning + dynamics learning Future work: Deep Gaussian processes + mechanistic information; consider real applications. Thanks!

49 Thanks Thanks to my co-authors: Neil Lawrence, Cesar Lincoln Mattos, Zhenwen Dai, Javier Gonzalez, Guilheme Barreto Thanks to George Karniadakis, Themis Sapsis, Paris Perdikaris for hosting me