A Constraint Generation Approach to Learning Stable Linear Dynamical Systems

Size: px

Start display at page:

Download "A Constraint Generation Approach to Learning Stable Linear Dynamical Systems"

Osborn George
6 years ago
Views:

1 A Constraint Generation Approach to Learning Stable Linear Dynamical Systems Sajid M. Siddiqi Byron Boots Geoffrey J. Gordon Carnegie Mellon University NIPS 2007 poster W22

2 steam

Application: Dynamic Textures 1 steam river fountain Videos of moving scenes that exhibit stationarity properties Dynamics can be captured by a low-dimensional model Learned models

3 Application: Dynamic Textures 1 steam river fountain Videos of moving scenes that exhibit stationarity properties Dynamics can be captured by a low-dimensional model Learned models can efficiently simulate realistic sequences Applications: compression, recognition, synthesis of videos 1 S. Soatto, D. Doretto and Y. Wu. Dynamic Textures. Proceedings of the ICCV, 2001

4 Linear Dynamical Systems Input data sequence

5 Linear Dynamical Systems Input data sequence

6 Linear Dynamical Systems Input data sequence Assume a latent variable that explains the data

7 Linear Dynamical Systems Input data sequence Assume a latent variable that explains the data

8 Linear Dynamical Systems Input data sequence Assume a latent variable that explains the data Assume a Markov model on the latent variable

9 Linear Dynamical Systems Input data sequence Assume a latent variable that explains the data Assume a Markov model on the latent variable

10 Linear Dynamical Systems 2 State and observation models: Dynamics matrix: Observation matrix: 2 Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Trans.ASME-JBE

11 Linear Dynamical Systems 2 This talk: - Learning LDS parameters from data while ensuring a stable dynamics matrix A more efficiently and accurately than previous methods State and observation models: Dynamics matrix: Observation matrix: 2 Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Trans.ASME-JBE

12 Learning Linear Dynamical Systems Suppose we have an estimated state sequence

13 Learning Linear Dynamical Systems Suppose we have an estimated state sequence Define state reconstruction error as the objective:

14 Learning Linear Dynamical Systems Suppose we have an estimated state sequence Define state reconstruction error as the objective: We would like to learn such that i.e.

15 Subspace Identification 3 Subspace ID uses matrix decomposition to estimate the state sequence 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

16 Subspace Identification 3 Subspace ID uses matrix decomposition to estimate the state sequence Build a Hankel matrix D of stacked observations 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

17 Subspace Identification 3 Subspace ID uses matrix decomposition to estimate the state sequence Build a Hankel matrix D of stacked observations 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

18 Subspace Identification 3 In expectation, the Hankel matrix is inherently low-rank! 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

19 Subspace Identification 3 In expectation, the Hankel matrix is inherently low-rank! 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

20 Subspace Identification 3 In expectation, the Hankel matrix is inherently low-rank! 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

21 Subspace Identification 3 In expectation, the Hankel matrix is inherently low-rank! Can use SVD to obtain the low-dimensional state sequence 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

22 Subspace Identification 3 In expectation, the Hankel matrix is inherently low-rank! Can use SVD to obtain the low-dimensional state sequence For D with k observations per column, = 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

23 Subspace Identification 3 In expectation, the Hankel matrix is inherently low-rank! Can use SVD to obtain the low-dimensional state sequence For D with k observations per column, = 3 P. Van Overschee and B. De Moor Subspace Identification for Linear Systems. Kluwer, 1996

24 Lets train an LDS for steam textures using this algorithm, and simulate a video from it! = [ ] x t 2 R 40

25 Simulating from a learned model x t The model is unstable t

26 Notation λ 1,, λ n : eigenvalues of A ( λ 1 > > λ n ) ν 1,,ν n : unit-length eigenvectors of A σ 1,,σ n : singular values of A (σ 1 > σ 2 > σ n ) S λ : matrices with λ 1 1 S σ : matrices with σ 1 1

27 Stability a matrix A is stable if λ 1 1, i.e. if

28 Stability a matrix A is stable if λ 1 1, i.e. if

29 Stability a matrix A is stable if λ 1 1, i.e. if λ 1 = 0.3 λ 1 = 1.295

30 Stability a matrix A is stable if λ 1 1, i.e. if λ 1 = 0.3 x t (1), x t (2) λ 1 = x t (1), x t (2)

31 Stability a matrix A is stable if λ 1 1, i.e. if λ 1 = 0.3 x t (1), x t (2) λ 1 = x t (1), x t (2) We would like to solve s.t.

32 Stability and Convexity But S λ is non-convex!

33 Stability and Convexity But S λ is non-convex! A 1 A 2

34 Stability and Convexity But S λ is non-convex! A 1 A 2 Lets look at S σ instead S σ is convex S σ µ S λ

35 Stability and Convexity But S λ is non-convex! A 1 A 2 Lets look at S σ instead S σ is convex S σ µ S λ

36 Stability and Convexity But S λ is non-convex! A 1 A 2 Lets look at S σ instead S σ is convex S σ µ S λ Previous work 4 exploits these properties to learn a stable A by solving the semi-definite program s.t. 4 S. L. Lacy and D. S. Bernstein. Subspace identification with guaranteed stability using constrained optimization. In Proc. of the ACC (2002), IEEE Trans. Automatic Control (2003)

37 Lets train an LDS for steam again, this time constraining A to be in S σ

38 Simulating from a Lacy-Bernstein stable texture model x t t Model is over-constrained. Can we do better?

39 Our Approach Formulate the S σ approximation of the problem as a semi-definite program (SDP) Start with a quadratic program (QP) relaxation of this SDP, and incrementally add constraints Because the SDP is an inner approximation of the problem, we reach stability early, before reaching the feasible set of the SDP We interpolate the solution to return the best stable matrix possible

40 The Algorithm

41 The Algorithm objective function contours

42 objective function contours The Algorithm A 1 : unconstrained QP solution (least squares estimate)

43 objective function contours The Algorithm A 1 : unconstrained QP solution (least squares estimate)

44 objective function contours The Algorithm A 1 : unconstrained QP solution (least squares estimate) A 2 : QP solution after 1 constraint (happens to be stable)

45 objective function contours The Algorithm A 1 : unconstrained QP solution (least squares estimate) A 2 : QP solution after 1 constraint (happens to be stable) A final : Interpolation of stable solution with the last one

46 objective function contours The Algorithm A 1 : unconstrained QP solution (least squares estimate) A 2 : QP solution after 1 constraint (happens to be stable) A final : Interpolation of stable solution with the last one A previous method : Lacy Bernstein (2002)

47 Lets train an LDS for steam using constraint generation, and simulate

48 Simulating from a Constraint Generation stable texture model x t t Model captures more dynamics and is still stable

49 Least Squares Constraint Generation

50 Algorithms: Empirical Evaluation Constraint Generation CG (our method) Lacy and Bernstein (2002) LB-1 finds a σ 1 1 solution Lacy and Bernstein (2003) LB-2 solves a similar problem in a transformed space Data sets Dynamic textures Biosurveillance baseline models (see paper)

52 Steam video Reconstruction error % decrease in objective (lower is better) number of latent dimensions

53 Running time Steam video Running time (s) number of latent dimensions

54 Conclusion A novel constraint generation algorithm for learning stable linear dynamical systems SDP relaxation enables us to optimize over a larger set of matrices while being more efficient

Conclusion A novel constraint generation algorithm for learning stable linear dynamical systems SDP relaxation enables us to optimize over a

55 Conclusion A novel constraint generation algorithm for learning stable linear dynamical systems SDP relaxation enables us to optimize over a larger set of matrices while being more efficient Future work: Adding stability constraints to EM Stable models for more structured dynamic textures

56 Thank you!

58 Subspace ID with Hankel matrices Stacking multiple observations in D forces latent states to model the future = e.g. annual sunspots data with 12-year cycles k = 1 k = 12 t First 2 columns of U t

59 Stability and Convexity The state space of a stable LDS lies inside some ellipse

60 Stability and Convexity The state space of a stable LDS lies inside some ellipse The set of matrices that map a particular ellipse into itself (and hence are stable) is convex

61 Stability and Convexity The state space of a stable LDS lies inside some ellipse The set of matrices that map a particular ellipse into itself (and hence are stable) is convex If we knew in advance which ellipse contains our state space, finding A would be a convex problem. But we don t???

62 Stability and Convexity The state space of a stable LDS lies inside some ellipse The set of matrices that map a particular ellipse into itself (and hence are stable) is convex If we knew in advance which ellipse contains our state space, finding A would be a convex problem. But we don t??? and the set of all stable matrices is non-convex

A Constraint Generation Approach to Learning Stable Linear Dynamical Systems

A Constraint Generation Approach to Learning Stable Linear Dynamical Systems Sajid M Siddiqi Robotics Institute Carnegie-Mellon University Pittsburgh, PA 15213 siddiqi@cscmuedu Byron Boots Computer Science