1 / 31 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 2015.

Transcription

1 Outline Part 4 Nonlinear system identification using sequential Monte Carlo methods Part 4 Identification strategy 2 (Data augmentation) Aim: Show how SMC can be used to implement identification strategy 2 Data augmentation. Thomas Schön Division of Systems and Control Department of Information Technology Uppsala University. thomas.schon@it.uu.se, www: user.it.uu.se/~thosc2. Summary of Part 3 2. Identification strategy 2 Data augmentation 3. Maximum likelihood via Expectation Maximization (EM) 4. Sampling state trajectories using Markov kernels 5. Bayesian identification using Gibbs sampling 6. Some of our current research activities (if there is time) Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Summary of Part 3 Identification strategy 2 data augmentation We introduced an auxiliary variable u = (x :T, a 2:T ), u ψ θ (u y :T ). and considered an extended target distribution φ(θ, u y :T ) = p θ,u(y :T )ψ θ (u y :T )p(θ). p(y :T ) Could construct a standard MH algorithm that operates in the non-standard extended space Θ X NT {,..., N} N(T ). p(θ y :T ) is recovered exactly as the marginal of the extended target distribution φ(θ, u y :T ), despite the fact that we employ an SMC approximation of the likelihood using a finite number of particles N. Motivation: If we had access to the complete likelihood p θ (x :T, y :T ) = µ θ (x ) T t= the problem would be much easier. T g θ (y t x t ) f θ (x t+ x t ) Key idea: Treat the state sequence x :T as an auxiliary variable that is estimated together with the parameters θ. The data augmentation strategy breaks the original problem into two new and closely linked problems. t= Intuitively: Alternate between updating θ and x :T. 2 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.

2 Maximum likelihood via EM. Expectation maximization (EM) employs the complete likelihood p θ (x :T, y :T ) as a substitute for the observed likelihood p θ (y :T ). They are of course related, p θ (x :T, y :T ) = p θ (x :T y :T )p θ (y :T ). It can be shown that by iteratively maximizing Q(θ, θ[k]) log p θ (x :T, y :T )p θ[k] (x :T y :T )dx :T = E θ[k] [log p θ (x :T, y :T ) y :T ], w.r.t. θ, the obtained sequence {θ[k]} k will result in a monotonical increase in likelihood values. SMC is used to approximate the JSD p θ[k] (x :T y :T ). 4 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Ex blind Wiener identification (I/III) Using EM and particle smoothing together Algorithm EM for identifying nonlinear dynamical systems. Initialise: Set k = and choose an initial θ. 2. While not converged do: (a) Expectation (E) step: Compute Q(θ, θ k ) = log p θ (x :T, y :T ) p θk (x :T y :T ) }{{} dx :T using sequential Monte Carlo (particle smoother). (b) Maximization (M) step: Compute θ k+ = arg max θ Θ (c) k k + Q(θ, θ k ) Thomas B. Schön, Adrian Wills and Brett Ninness. System Identification of Nonlinear State-Space Models. Automatica, 47():39-49, January 2. 5 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Ex blind Wiener identification (II/III) e,t u t L x t+ = ( A z t h (z t, β) h 2 (z t, β) Σ e 2,t Σ y,t y 2,t B ) ( ) x t, u u t N (, Q), t z t = Cx t, y t = h(z t, β) + e t, e t N (, R). Id. problem: Find L, β, r, and r 2 based on {y,:t, y 2,:T }. Second order LGSS model with complex poles. Results obtained using T = samples. Employ the EM-PS with N = particles. The plots are based on realisations of data. Nonlinearities (dead zone and saturation) shown on next slide. Gain (db) Normalised frequency Bode plot of estimated mean (black), true system (red) and the result for all realisations (gray). 6 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.

3 Ex blind Wiener identification (III/III) Outline Part 4 Output of nonlinearity Input to nonlinearity Output of nonlinearity Input to nonlinearity 2 Estimated mean (black), true static nonlinearity (red) and the result for all realisations (gray). Aim: Show how SMC can be used to implement identification strategy 2 Data augmentation.. Summary of Part 3 2. Identification strategy 2 Data augmentation 3. Maximum likelihood via Expectation Maximization (EM) 4. Sampling state trajectories using Markov kernels 5. Bayesian identification using Gibbs sampling 6. Some of our current research activities (if there is time) Adrian Wills, Thomas B. Schön, Lennart Ljung and Brett Ninness. Identification of Hammerstein-Wiener Models. Automatica, 49(): 7-8, January / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Gibbs sampler for SSMs Sampling based on SMC Aim: Compute p(θ, x :T y :T ). MCMC: Gibbs sampling (blocked) for SSMs amounts to iterating Draw θ[m] p(θ x :T [m ], y :T ); OK! Draw x :T [m] p(x :T θ[m], y :T ). Hard! With P ( x :T = xi :T ) w i T we get, x :T.5.5 approx. p(x :T θ, y :T ). Problem: p(x :T θ[m], y :T ) not available! State Idea: Approximate p(x :T θ[m], y :T ) using a sequential Monte Carlo method! Time / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.

4 Problems and a solution PMCMC Problems with this approach, Based on a PF approximate sample. Does not leave p(x :T θ, y :T ) invariant! Relies on large N to be successful. A lot of wasted computations. To get around these problems, The idea underlying PMCMC is to make use of a certain SMC sampler to construct a Markov kernel leaving the joint smoothing distribution p(x :T θ, y :T ) invariant. This Markov kernel is then used in a standard MCMC algorithm (e.g. Gibbs, results in the Particle Gibbs (PG)). Use a conditional particle filter (CPF). One prespecified reference trajectory is retained throughout the sampler. SMC is used to build an MCMC kernel with p(x :t θ, y :t ) as its stationary distribution without introducing any systematic errors! Christophe Andrieu, Arnaud Doucet and Roman Holenstein, Particle Markov chain Monte Carlo methods, Journal of the Royal Statistical Society: Series B, 72: , 2. 2 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Three SMC samplers Three SMC samplers leaving p(x :T θ, y :T ) invariant:. Conditional particle filter (CPF) Christophe Andrieu, Arnaud Doucet and Roman Holenstein, Particle Markov chain Monte Carlo methods, Journal of the Royal Statistical Society: Series B, 72: , CPF with backward simulation (CPF-BS) N. Whiteley, Discussion on Particle Markov chain Monte Carlo methods, Journal of the Royal Statistical Society: Series B, 72(3), 36 37, 2. Conditional particle filter (CPF) Let x :T = (x,..., x T ) be a fixed reference trajectory. At each time t, sample only N particles in the standard way. Set the N th particle deterministically: x N t = x t. N. Whiteley, C. Andrieu and A. Doucet, Efficient Bayesian inference for switching state-space models using discrete particle Markov chain Monte Carlo methods, Bristol Statistics Research Report :4, 2. Fredrik Lindsten and Thomas B. Schön. On the use of backward simulation in the particle Gibbs sampler. Proc. of the 37th Internat. Conf. on Acoustics, Speech, and Signal Processing (ICASSP), Kyoto, Japan, March CPF with ancestor sampling (CPF-AS) Fredrik Lindsten, Michael I. Jordan and Thomas B. Schön, Particle Gibbs with ancestor sampling, arxiv:4.64, 24. Fredrik Lindsten, Michael I. Jordan and Thomas B. Schön, Ancestor sampling for particle Gibbs, Advances in Neural Information Processing Systems (NIPS) 25, Lake Tahoe, NV, US, December, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. CPF causes us to degenerate to the something that is very similar to the reference trajectory, resulting in slow mixing. 5 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.

5 CPF vs. CPF-AS motivation BS is problematic for models with more intricate dependencies. Reason: Requires complete trajectories of the latent variable in the backward sweep. Solution: Modify the computation to achieve the same effect as BS, but without an explicit backwards sweep. Implication: Ancestor sampling opens up for inference in a wider class of models, e.g. non-markovian SSMs, PGMs and BNP models. CPF-AS intuition Let x :T = (x,..., x T ) be a fixed reference trajectory. At each time t, sample only N particles in the standard way. Set the N th particle deterministically: x N t = x t. Generate an artificial history for x N t by ancestor sampling. Ancestor sampling is conceptually similar to backward simulation, but instead of separate forward and backward sweeps, we achieve the same effect in a single forward sweep. 6 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. CPF-AS algorithm CPF-AS causes us to degenerate to something that is very different from the reference trajectory, resulting in better mixing. 7 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. The PGAS Markov kernel Algorithm CPF-AS, conditioned on x :T. Initialize (t = ): (a) Draw x i r θ, (x i ), for i =,..., N. (b) Set x N = x. (c) Set w i = W θ, (x i ). 2. For t = 2 to T do: (a) Draw a i t C({w j t } N j= ), for i =,..., N. (b) Draw x i t r θ,t (x t x ai t :t ), for i =,..., N. (c) Set x N t = x t. (d) Draw a N t with P (() a N t = i) w i t γ θ,t (x i :t,x t:t ) γ θ,t (x i :t ). (e) Set x i :t = {x ai t :t, xi t} and w i t = W θ,t (x i :t).. Run CPF-AS(x :T ) targeting p(x :T θ, y :T ). 2. Sample x :T with P ( x :T = xi :T ) w i T. Maps x :T stochastically into x :T Implicitly defines an ergodic Markov kernel (Pθ N ) referred to as the PGAS (particle Gibbs with ancestor sampling) kernel. Theorem For any number of particles N and θ Θ, the PGAS kernel Pθ N leaves p(x :T θ, y :T ) invariant, p(dx :T θ, y :T ) = Pθ N (x :T, dx :T )p(dx :T θ, y :T ) 8 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.

6 PGAS for Bayesian learning Toy example stochastic volatility (I/II) Bayesian learning: Gibbs + CPF-AS = PGAS Algorithm Particle Gibbs with ancestor sampling (PGAS). Initialize: Set {θ[], x :T []} arbitrarily. 2. For m, iterate: (a) Draw θ[m] p(θ x :T [m ], y :T ). (b) Run CPF-AS(x :T [m ]), targeting p(x :T θ[m], y :T ). (c) Sample with P (x :T [m] = x i :T ) wi T. ACF Consider the stochastic volatility model, x t+ =.9x t + w t, w t N (, θ), ( ) y t = e t exp 2 x t, e t N (, ). Let us study the ACF for the estimation error, θ E [θ y :T ] PG, T = N=5 N=2 N= N= ACF PG-AS, T = N=5 N=2 N= N= Lag Lag 2 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Toy example stochastic volatility (II/II) Example semiparametric Wiener model PG PGAS v t e t Update frequencey of xt N = 5 N = 2. N = N = Time (t) Update frequencey of xt N = 5 N = 2. N = N = Time (t) Plots of the update rate of x t versus t, i.e. the proportion of iterations where x t changes value. This provides another comparison of the mixing. z t u t L g( ) Σ Parametric LGSS and a nonparametric static nonlinearity: x t+ = ( A B ) ( ) xt + v }{{} u t, v t N (, Q), t Γ z t = Cx t. y t = g(z t ) + e t, y t e t N (, R). 22 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.

7 Example semiparametric Wiener model Example semiparametric Wiener model Show movie Parameters : θ = {A, B, Q, g( ), r}. Bayesian model specified by priors Conjugate priors for Γ = [A B], Q and r, p(γ, Q) = Matrix-normal inverse-wishart p(r) = inverse-wishart Gaussian process prior on g( ), g( ) GP(z, k(z, z )). Inference using PGAS with N = 5 particles. T = measurements. We ran 5 MCMC iterations and discarded 5 as burn-in. Γ g( ) u :T x :T y :T Q r Magnitude (db) Phase (deg) 2 2 True Posterior mean 99 % credibility Frequency (rad/s) Bode diagram of the 4th-order linear system. Estimated mean (dashed black), true (solid black) and 99% credibility intervals (blue). h(z) True Posterior mean 99 % credibility z Static nonlinearity (non-monotonic), estimated mean (dashed black), true (black) and the 99% credibility intervals (blue). Fredrik Lindsten, Thomas B. Schön and Michael I. Jordan. Bayesian semiparametric Wiener system identification. Automatica, 49(7): , July / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Some of our current research activities Inference in probabilistic graphical models Joint work with (alphabetical order): Christian A. Naesseth (Linköping University), John Aston (University of Cambridge), Alexandre Bouchard-Côté (University of British Columbia). Adam M. Johansen (University of Warwick), Bonnie Kirkpatrick (University of Miami), Fredrik Lindsten (University of Cambridge), Arno Solin (Aalto University), Andreas Svensson (Uppsala University), Simo Särkkä (Aalto University) and Johan Wågberg (Uppsala University). Constructing an artificial sequence of intermediate target distributions for an SMC sampler is a powerful (and quite possibly underutilized) idea. x 4 x 5 x x 2 x 3 y y 2 y 3 Christian A. Naesseth, Fredrik Lindsten and Thomas B. Schön, Sequential Monte Carlo methods for graphical models. Advances in Neural Information Processing Systems (NIPS) 27, Montreal, Canada, December, 24. Fredrik Lindsten, Adam M. Johansen, Christian A. Naesseth, Bonnie Kirkpatrick, Thomas B. Schön, John Aston and Alexandre Bouchard-Côté. Divide-and-Conquer with Sequential Monte Carlo. arxiv: , June / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.

8 SMC in high dimensions A nonparametric model GP-SSM The bootstrap PF suffers from weight collapse in high-dimensional settings. This degeneracy can be reduced by using so-called fully adapted proposals. k k k + We can mimic the efficient fully adapted proposals for arbitrary latent spaces and structures in high-dimensional models. Approximations the proposal distribution and use a nested coupling of multiple SMC samplers and backward simulators. Consider the Gaussian Process SSM (GP-SSM): x t+ = f(x t ) + w t, s.t. f(x) GP(, κ θ,f (x, x )), y t = g(x t ) + e t, s.t. g(x) GP(, κ θ,g (x, x )), The model functions f and g are assumed to be realizations from a Gaussian process prior We will identify this model by inferring the posterior distribution The PGAS construction is used. p(f, g, Q, R, θ y :T ). Christian A. Naesseth, Fredrik Lindsten and Thomas B. Schön, Nested sequential Monte Carlo. In Proceedings of the 32nd International Conference on Machine Learning (ICML), Lille, France, July, 25. Andreas Svensson, Arno Solin, Simo Särkkä and Thomas B. Schön, Computationally Efficient Bayesian Learning of Gaussian Process State Space Models. In submitted, available on arxiv tomorrow, June, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. The aim of these lectures The aim of these lectures is to provide an introduction to the theory and application of sequential Monte Carlo (SMC) methods for nonlinear system identification (and state estimation) problems. When it comes to SMC we will focus on the particle filter. After this course you should be able to derive your own SMC based algorithms allowing you to solve nonlinear system identification and state estimation problems. If you need inspiration I have provided some exercises here user.it.uu.se/~thosc2/courses.html 3 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25. Conclusions. Provided two key strategies and four different approaches for system identification in nonlinear SSMs. 2. Showed how SMC can be used to implement these strategies. 3. (Hopefully) conveyed the intuition underlying SMC. 4. SMC is applicable to many problems, not just SSMs via PF. Manuscript is also available (just ask me for a draft) Thomas B. Schöon and Fredrik Lindsten. Learning of dynamical systems Particle filters and Markov chain methods, 25. PhD course on the topic is available here user.it.uu.se/~thosc2/cids.html There will be an invited session on SMC at SYSID in Beijing (China) in October. Fast moving research area offering lots of opportunities! 3 / 3 Identification of nonlinear dynamic systems (guest lectures) Brussels, Belgium, June 8, 25.