State Estimation of Linear and Nonlinear Dynamic Systems

Size: px

Start display at page:

Download "State Estimation of Linear and Nonlinear Dynamic Systems"

Chad Palmer
6 years ago
Views:

1 State Estimation of Linear and Nonlinear Dynamic Systems Part IV: Nonlinear Systems: Moving Horizon Estimation (MHE) and Particle Filtering (PF) James B. Rawlings and Fernando V. Lima Department of Chemical and Biological Engineering University of Wisconsin Madison AICES Regional School RWTH Aachen March 7, 28 Rawlings and Lima (UW) MHE and PF AICES / 45 Outline The Challenge of Nonlinear Estimation 2 Moving Horizon Estimation (MHE) 3 Particle Filtering (PF) 4 Combining PF and MHE 5 Conclusions 6 Further Reading Rawlings and Lima (UW) MHE and PF AICES 2 / 45

2 The Challenge of Nonlinear Estimation Linear Estimation Nonlinear Estimation Estimation Possibilities: one state is the optimal estimate 2 infinitely many states are optimal estimates (unobservable) Estimation Possibilities: one state is the optimal estimate 2 infinitely many states are optimal estimates (unobservable) 3 finitely many states are locally optimal estimates Rawlings and Lima (UW) MHE and PF AICES 3 / 45 Full Information Estimation Nonlinear model, Gaussian noise, x(k + ) = F (x, u) + G(x, u)w y(k) = h(x) + v The trajectory of states X (T ) := {x(),... x(t )} Maximizing the conditional density function max p X Y (X (T ) Y (T )) X (T ) Rawlings and Lima (UW) MHE and PF AICES 4 / 45

3 Equivalent Optimization Problem Using the model and taking logarithms min X (T ) V (x()) + T j= L w (w(j)) + T j= L v (y(j) h(x(j))) subject to x(j + ) = F (x, u) + w (G(x, u) = I ) V (x) := log(p x() (x)) L w (w) := log(p w (w)) L v (v) := log(p v (v)) Rawlings and Lima (UW) MHE and PF AICES 5 / 45 What do we do when we have a new measurement? New measurement at time T+ Measurement Estimate y T Resolve optimization problem with T + stages. = Size of optimization increases with time. Employ moving horizon approximation. = Bound size of optimization with approximate estimator. Rawlings and Lima (UW) MHE and PF AICES 6 / 45

4 Adding new Observations to the Estimation Problem It occasionally happens that after we have completed all parts of an extended calculation on a sequence of observations, we learn of a new observation that we would like to include. In many cases we will not want to have to redo the entire elimination but instead to find the modifications due to the new observation in the most reliable values of the unknowns and in their weights. C.F. Gauss, 823 G.W. Stewart Translation, 995, p. 9. Rawlings and Lima (UW) MHE and PF AICES 7 / 45 Moving Horizon Estimation Moving Estimation Window Discard old measurement New measurement at time T+ Measurement Estimate MH Estimate y T+ In the Moving Horizon Estimation(MHE) strategy The most recent N states are considered Rawlings and Lima (UW) MHE and PF AICES 8 / 45

5 Arrival Cost and Moving Horizon Estimation Most recent N states X (T N : T ) := {x(t N)... x(t )} Optimization problem min V T N(x(T N)) X (T N:T ) }{{} arrival cost + subject to x(j + ) = F (x, u) + w. T j=t N L w (w(j))+ T j=t N L v (y(j) h(x(j))) Rawlings and Lima (UW) MHE and PF AICES 9 / 45 What does moving horizon estimation have to offer? linear model Gaussian noise stability linear model general noise inequality constraints stability nonlinear model Gaussian noise nonlinear model general noise inequality constraints stability = Kalman Filter = MHE } = Extended Kalman Filter = MHE Rawlings and Lima (UW) MHE and PF AICES / 45

6 Literature Summary Data Reconciliation/Moving Horizon Estimation Liebman et al. (992), Kim et al. (99), Bequette (99), Ramamurthi et al. (993), Tjoa and Biegler (99), Albuquerque and Biegler (996), Marquardt et al. (M hamdi et al., 996; Binder et al., 22)... Moving Horizon Observers Jang et al. (986), Zimmer (994), Michalska and Mayne (995), Moraal and Grizzle (995) Constrained Moving Horizon Estimation Meadows et al. (993): Linear constrained estimation Muske and Rawlings (995): Linear and nonlinear MHE Robertson and Lee (Robertson et al., 996; Robertson and Lee, 22): Linear and nonlinear MHE, constraints, truncated distributions Tyler and Morari (996): Linear MHE, constraints Findeisen (997): Linear MHE, constraints Rao, Rawlings, Mayne, Lee (Rao et al., 23, 2; Rao and Rawlings, 22; Michalska and Mayne, 995): Linear and nonlinear MHE, constraints Rawlings and Lima (UW) MHE and PF AICES / 45 Moving Horizon Approximation Moving Estimation Window Discard old measurement New measurement at time T+ Measurement Estimate MH Estimate y T+ What are the consequences of neglecting old data? Sensitivity to noise, high gain estimator. Divergence. Rawlings and Lima (UW) MHE and PF AICES 2 / 45

7 Potential pitfalls of neglecting past data y Measurement Estimate MH Estimate By neglecting or too weakly weighting the past data, the estimator may be sensitive to outliers or noise. = Account for past data using approximate statistic. Rawlings and Lima (UW) MHE and PF AICES 3 / 45 Potential pitfalls of improperly approximating past data Account for past data by penalizing deviation from past estimate y Measurement Estimate MH Estimate By weighting the past data or the prior information too strongly, the estimator may be unable to keep up with data. Estimator divergence may result. = We require some forgetting to improve the robustness. Rawlings and Lima (UW) MHE and PF AICES 4 / 45

8 How are full information and moving horizon estimation related? = Forward Dynamic Programming T Φ T = Γ(x x) + L(w k, v k ) k= T N = Γ(x x) + L(w k, v k ) k= } {{ } Cost associated with arriving at x T N Arrival Cost Ξ T N (x T N ) + T T N L(w k, v k ) }{{} Uniquely determined by x T N and {w k } T k=t N Fixed Horizon Estimation Problem Rawlings and Lima (UW) MHE and PF AICES 5 / 45 Arrival Cost Ξ T Estimate Restricted Estimate Measurement x T Rawlings and Lima (UW) MHE and PF AICES 6 / 45

9 Forward Dynamic Programming Structure Arrival Cost Ξ T N Fixed Horizon Full Information T N T Rawlings and Lima (UW) MHE and PF AICES 7 / 45 Moving Horizon Estimation Optimization Problem min Θ T = {x k } T k=t N L(w k, v k ) + Γ T N (x T N x T N T N ) }{{} prior information arrival cost Initial penalty Γ T N summarizes past data by penalizing deviation away from past estimate. If the initial penalty is equal to the arrival cost, then the full information and moving horizon estimation problems are equivalent. Rawlings and Lima (UW) MHE and PF AICES 8 / 45

10 Comments For unconstrained linear systems with quadratic objectives, we can calculate the arrival cost with the Kalman filter covariance. Moving horizon estimation reduces to Kalman filtering. For constrained linear systems with quadratic objectives, we can globally lower bound the arrival cost with the Kalman filter covariance. When the system is nonlinear, we cannot in general calculate a globally lower bound to the arrival cost with the exception of the trivial choice: Γ T =. One solution: Generate lower bound online (Rao, 2). Rawlings and Lima (UW) MHE and PF AICES 9 / 45 One year from now I try to reconstruct my travel to Aachen Flew from Madison to a larger airport. Data: I don t remember very well, but it was a short flight... Flew from that larger airport to Frankfurt. Data: I remember this flight very well. It took forever; it was about 7 hours... overbound Madison actual underbound St. Louis 3 /2 Chicago /4 Milwaukee Minneapolis Detroit 7 6 * 8 6 Frankfurt Rawlings and Lima (UW) MHE and PF AICES 2 / 45

11 Arrival Cost Approximation Current Research in MHE The statistically correct choice for the arrival cost is the conditional density of x(t N) Y (T N ) V T N (x) = log p x(t N) Y (x Y (T N )) Arrival cost approximations (Rao et al., 23) uniform prior (and large N) EKF covariance formula MHE smoothing Rawlings and Lima (UW) MHE and PF AICES 2 / 45 Particle filtering sampled densities p s (x) = s w i δ(x x i ) x i samples (particles) w i weights i=.5.4 p(x).3.2 p(x) p s (x) x Exact density p(x) and a sampled density p s (x) with five samples for ξ N(, ) Rawlings and Lima (UW) MHE and PF AICES 22 / 45

12 Convergence cumulative distributions.8 P(x) P(x) P s (x) Corresponding exact P(x) and sampled P s (x) cumulative distributions x Rawlings and Lima (UW) MHE and PF AICES 23 / 45 Importance sampling In state estimation, p of interest is easy to evaluate but difficult to sample. We choose an importance function, q, instead. When we can sample p, the sampled density is { p s = x i, w i = } p sa (x i ) = p(x i ) s When we cannot sample p, the importance sampled density p s (x) is { p s = x i, w i = } p(x i ) p is (x i ) = q(x i ) s q(x i ) Both p s (x) and p s (x) are unbiased and converge to p(x) as sample size increases (Smith and Gelfand, 992). Rawlings and Lima (UW) MHE and PF AICES 24 / 45

13 p(x) Density to be sampled p(x). q(x) x x Importance function that can be sampled q(x). Rawlings and Lima (UW) MHE and PF AICES 25 / q(x) x Importance function q(x) and its histogram based on 5 samples p(x) x Exact density p(x) and its histogram based on 5 importance samples. Rawlings and Lima (UW) MHE and PF AICES 26 / 45

14 Importance sampled particle filter (Arulampalam et al., 22) p(x(k + ) Y (k + )) = {x i (k + ), w i (k + )} x i (k + ) is a sample of q(x(k + ) x i (k), y(k + )) w i (k + ) = w i (k) p(y(k + ) x i(k + ))p(x i (k + ) x i (k)) q(x i (k + ) x i (k), y(k + )) The importance sampled particle filter converges to the conditional density with increasing sample size. It is biased for finite sample size. Rawlings and Lima (UW) MHE and PF AICES 27 / 45 Research challenge placing the particles Optimal importance function (Doucet et al., 2). Restricted to linear measurement y = Cx + v. Resampling Curse of dimesionality Rawlings and Lima (UW) MHE and PF AICES 28 / 45

15 Optimal importance function 2 5 k = 5 k = 4 k = 3 x 2 (k) k = 2 5 k = k = Particles locations versus time using the optimal importance function; 25 particles. Ellipses show the 95% contour of the true conditional densities before and after measurement. x (k) Rawlings and Lima (UW) MHE and PF AICES 29 / 45 Resampling ➀ w w + w 2 w + w 2 + w 3 ➁ ➂ How to resample without bias x = x x 2 = x 3 x 3 = x 3 Partition [, ] with original sample weights, w i. Arrows depict the outcome of drawing three uniformly distributed random numbers. Sample x 2 is discarded and sample x 3 is repeated twice in the resample. The new sample s weights are w = w 2 = w 3 = /3. Rawlings and Lima (UW) MHE and PF AICES 3 / 45

16 Resampling Original sample Resample state weight state weight x w = 3 x = x w = 3 x 2 w 2 = x 2 = x 3 w 2 = 3 x 3 w 3 = 6 x 3 = x 3 w 3 = 3 The properties of the resamples are summarized by { wj, x p re ( x i ) = i = x j, x i x j w i = /s, all i Rawlings and Lima (UW) MHE and PF AICES 3 / 45 Resampling 2 5 k = 5 k = 4 k = 3 x 2 (k) k = 2 5 k = k = Particles locations versus time using the optimal importance function with resampling; 25 particles. x (k) Rawlings and Lima (UW) MHE and PF AICES 32 / 45

17 The MHE and particle filtering hybrid approach Hybrid implementation Use the MHE optimization to locate/relocate the samples Use the PF to obtain fast state estimates between MHE optimizations Rawlings and Lima (UW) MHE and PF AICES 33 / 45 Application: Semi-Batch Reactor Reaction: 2A B k =.6 Measurement is C A + C B x = [ 3 ] T A, B F i, C A, C B V dc A dt dc B dt = 2kC 2 A + F i V C A t =. = kc 2 A + F i V C B Noise covariances Q w = diag (. 2,. 2 ) and R v =. 2 Bad Prior: x = [. 4.5 ] T with a large P Unmodelled Disturbance: C A, C B is pulsed at t k = 5 Rawlings and Lima (UW) MHE and PF AICES 34 / 45

18 Using only MHE MHE implemented with N = 5(t =.5) and a smoothed prior MHE recovers robustly from poor priors and unmodelled disturbances Concentrations, ca, cb estimates y c B Measurement, y c A time Rawlings and Lima (UW) MHE and PF AICES 35 / 45 Using only particle filter Particle filter implemented with the Optimal importance function: p(x k x k, y k ), 5 samples, Resampling The PF samples never recover from a poor x. Not robust Concentrations, ca, cb estimates y c B Measurement, y c A time Rawlings and Lima (UW) MHE and PF AICES 36 / 45

19 MHE/PF hybrid with a simple importance function Importance function for PF: p(x k x k ), 5 samples The PF samples recover from a poor x and the unmodelled disturbance only after the MHE relocates the samples Concentrations, ca, cb estimates y 6 c B c A time 4 2 Measurement, y Rawlings and Lima (UW) MHE and PF AICES 37 / 45 MHE/PF hybrid with an optimal importance function The optimal importance function: p(x k x k, y k ), 5 samples MHE relocates the samples after a poor x, but samples recover from the unmodelled disturbance without needing the MHE Concentrations, ca, cb estimates y c B Measurement, y c A time Rawlings and Lima (UW) MHE and PF AICES 38 / 45

20 Conclusions Optimal state estimation of the linear dynamic system is the gold standard of state estimation. MHE is a good option for linear, constrained systems. The classic solution for nonlinear systems, the EKF, has been superseded. The UKF, for example, is an easily implemented alternative worth evaluating. MHE and particle filtering are higher-quality solutions for nonlinear models. MHE is robust to modeling errors but requires an online optimization. PF is simple to program and fast to execute but may be sensitive to model errors. They require more user experience to set up properly and more computational resources to execute. The payoff can be substantial, however. Hybrid MHE/PF methods can combine these complementary strengths. Rawlings and Lima (UW) MHE and PF AICES 39 / 45 Future challenges Process systems are typically unobservable or ill-conditioned, i.e. nearby measurements do not imply nearby states. We must decide on the subset of states to reconstruct from the data an additional part to the modeling question. Nonlinear systems produce multi-modal densities. We need better solutions for handling these multi-modal densities in real time. Rawlings and Lima (UW) MHE and PF AICES 4 / 45

21 Acknowledgments Murali Rajamani of BP Professor Bhavik Bakshi of OSU for helpful discussion. NSF grant #CNS 5447 PRF grant #4332 AC9 Rawlings and Lima (UW) MHE and PF AICES 4 / 45 Further Reading I J. Albuquerque and L. T. Biegler. Data reconciliation and gross-error detection for dynamic systems. AIChE J., 42(): , 996. M. S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp. A tutorial on particle filters for online nonlinear/non-gaussian Bayesian tracking. IEEE Trans. Signal Process., 5 (2):74 88, February 22. B. W. Bequette. Nonlinear predictive control using multi-rate sampling. Can. J. Chem. Eng., 69:36 43, February 99. T. Binder, L. Blank, W. Dahmen, and W. Marquardt. On the regularization of dynamic data reconciliation problems. J. Proc. Cont., 2(4): , 22. A. Doucet, S. Godsill, and C. Andrieu. On sequential Monte Carlo sampling methods for Bayesian filtering. Stat. and Comput., :97 28, 2. P. Findeisen. Moving horizon state estimation of discrete time systems. Master s thesis, University of Wisconsin Madison, 997. S.-S. Jang, B. Joseph, and H. Mukai. Comparison of two approaches to on-line parameter and state estimation of nonlinear systems. Ind. Eng. Chem. Proc. Des. Dev., 25:89 84, 986. Rawlings and Lima (UW) MHE and PF AICES 42 / 45

22 Further Reading II I. Kim, M. Liebman, and T. Edgar. A sequential error-in-variables method for nonlinear dynamic systems. Comput. Chem. Eng., 5(9):663 67, 99. M. Liebman, T. Edgar, and L. Lasdon. Efficient data reconciliation and estimation for dynamic processes using nonlinear programming techniques. Comput. Chem. Eng., 6 (/): , 992. E. S. Meadows, K. R. Muske, and J. B. Rawlings. Constrained state estimation and discontinuous feedback in model predictive control. In Proceedings of the 993 European Control Conference, pages European Automatic Control Council, June 993. A. M hamdi, A. Helbig, O. Abel, and W. Marquardt. Newton-type receding horizon control and state estimation. In Proceedings of the 996 IFAC World Congress, pages 2 26, San Francisco, California, 996. H. Michalska and D. Q. Mayne. Moving horizon observers and observer-based control. IEEE Trans. Auto. Cont., 4(6):995 6, 995. P. E. Moraal and J. W. Grizzle. Observer design for nonlinear systems with discrete-time measurements. IEEE Trans. Auto. Cont., 4(3):395 44, 995. Rawlings and Lima (UW) MHE and PF AICES 43 / 45 Further Reading III K. R. Muske and J. B. Rawlings. Nonlinear moving horizon state estimation. In R. Berber, editor, Methods of Model Based Process Control, NATO Advanced Study Institute series: E Applied Sciences 293, pages , Dordrecht, The Netherlands, 995. Kluwer. Y. Ramamurthi, P. Sistu, and B. Bequette. Control-relevant dynamic data reconciliation and parameter estimation. Comput. Chem. Eng., 7():4 59, 993. C. V. Rao. Moving Horizon Strategies for the Constrained Monitoring and Control of Nonlinear Discrete-Time Systems. PhD thesis, University of Wisconsin Madison, 2. C. V. Rao and J. B. Rawlings. Constrained process monitoring: moving-horizon approach. AIChE J., 48():97 9, January 22. C. V. Rao, J. B. Rawlings, and J. H. Lee. Constrained linear state estimation a moving horizon approach. Automatica, 37():69 628, 2. C. V. Rao, J. B. Rawlings, and D. Q. Mayne. Constrained state estimation for nonlinear discrete-time systems: stability and moving horizon approximations. IEEE Trans. Auto. Cont., 48(2): , February 23. D. G. Robertson and J. H. Lee. On the use of constraints in least squares estimation and control. Automatica, 38(7):3 24, 22. Rawlings and Lima (UW) MHE and PF AICES 44 / 45

23 Further Reading IV D. G. Robertson, J. H. Lee, and J. B. Rawlings. A moving horizon-based approach for least-squares state estimation. AIChE J., 42(8): , August 996. A. F. M. Smith and A. E. Gelfand. Bayesian statistics without tears: A sampling-resampling perspective. Amer. Statist., 46(2):84 88, 992. I. B. Tjoa and L. T. Biegler. Simultaneous strategies for data reconciliation and gross error detection of nonlinear systems. Comput. Chem. Eng., 5():679 69, 99. M. L. Tyler and M. Morari. Stability of constrained moving horizon estimation schemes. Preprint AUT96 8, Automatic Control Laboratory, Swiss Federal Institute of Technology, 996. G. Zimmer. State observation by on-line minimization. Int. J. Control, 6(4):595 66, 994. Rawlings and Lima (UW) MHE and PF AICES 45 / 45

State Estimation using Moving Horizon Estimation and Particle Filtering

State Estimation using Moving Horizon Estimation and Particle Filtering James B. Rawlings Department of Chemical and Biological Engineering UW Math Probability Seminar Spring 2009 Rawlings MHE & PF 1 /