The Entropy Penalized Minimum Energy Estimator

Transcription

1 The Entropy Penalized Minimum Energy Estimator Sergio Pequito, A. Pedro Aguiar Adviser Diogo A. Gomes Co-Adviser Instituto Superior Tecnico Institute for System and Robotics Department of Electronic and Computer Engineering Instituto Superior Tecnico Department of Mathematics Abstract: This paper addresses the state estimation problem of nonlinear systems. We formulate the problem using a minimum energy estimator MEE approach and propose an entropy penalized scheme to approximate the viscosity solution of the Hamilton-Jacobi equation that follows from the MEE formulation. We derive an explicit observer algorithm that is iterative and filtering-like, which continuously improves the state estimation as more measurements arise. In addition, we propose a computationally efficient procedure to estimate the state by preforming an approximation of the nonlinear system along the trajectory of the estimate. In this case, for the first and second order approximations of the state equation, we derive a closed-form iterative solution for the Hessian of the entropy-like version of the optimal cost function of the MEE. We illustrate and contrast the performance of our algorithms with the extended Kalman filter using specific nonlinear examples with the feature that the do not converge to the correct value. Keywords: Nonlinear Observer, Entropy Penalized, Minimum Energy Estimator, Viscosity Solutions Contents Introduction The Entropy Penalized Minimum Energy Estimator. The Minimum Energy Estimator. Viscosity Solutions for Hamilton-Jacobi equations 3.3 The Entropy Penalized method 3.4 The Entropy Penalized Minimum Energy Estimator 3 3 The entropy penalized MEE for systems 3 3. Convergence of Entropy Penalized Minimum Energy Estimator 4 3. First order approximation Second order approximation 4 4 Simulation results 5 4. One dimensional example 5 4. Two dimensional example Discussion of Results 6 5 Conclusions and Further Research 6 Research supported in part by PTDC/EEA-ACR/65996/6 FCT NAV and PTDC/EEA-ACR/67/6 FCT DENO.. INTRODUCTION The determination in real time of an estimative of the state of a given nonlinear system from partial and noisy measurements of the inputs and outputs and inexact knowledge of the initial condition has been a fundamental and a challengeable problem in theory and applications of control systems Schon 6,Kailath et al.,c.evans 6,Gelb 974. By far, the extended Kalman filter is the most widely used method for estimating the state. It is obtained by linearizing the nonlinear dynamics and the observation along the trajectory of the estimatekhalil 4,Jazwinski 97,Maybeck 979,Grewal and Andrews,Gelb 974. However, since it is only a local method, it often fails to converge. Several nonlinear observers using deterministic and stochastic approaches can be found in the literature Wei Kang 7,Jazwinski 97,Gelb 974,Nagpal and Khargonekar 997,Krener 6,Charalambous et al. 994,Basar,J.P. Gauthier and Othman Jun 99: Lyapunov-like, Luenberger-like, high-gain observers, sliding-mode observers, optimizationbased, etc. Particular interesting classes of optimal nonlinear observers are the minimum energy estimator MEE and the closely related H estimator. The MEE were first proposed by Motersen 968 and further improved by Hijab 98. In Krener 6 is proven convergence of the MEE provided that the system is uniformly observable for every input.

2 In the MEE approach one tries to find an observer that computes an estimate ˆxt of the state xt that is compatible with the system s dynamics and measured outputs and minimizes the energy of the noise and disturbances. One important feature of this approach is that if the system is linear, than one would obtain precisely the Kalman-Bucy filter. However, in general, both minimum-energy and H state estimators for nonlinear systems lead to infinitedimensional observers whose state evolves according to a first-order nonlinear PDE of Hamilton-Jacobi HJ type, driven by the observations. To the best of our knowledge, besides linear systems, the only class of systems reported in the literature one can have a closedform solution that is filtering-like and iterative are stateaffine systems with implicit outputs Aguiar and Hespanha 6,Aguiar and Hespanha 9. Motivated by the above considerations, this paper addresses the problem of computing the viscosity solution Jx,t of the HJ PDE that arises in the minimum energy estimator formulation. We propose an alternative strategy compared to the traditional ones of computing directly Jx, t via finite elements that consists in the discretization of the estimation problem and an addition of a suitable entropy functional. This functional regularizes the evolution of the discrete value function by acting as a viscosity term. This strategy follows from recent results on entropy penalization methods for HJ equations described in Gomes and Valdinoci 7. The main contributions of this paper are twofold. First, we derive an explicit observer algorithm, called entropy penalized minimum energy estimator that is iterative and filtering-like, which continuously improves the state estimation as more measurements arise. Second, we propose a computationally efficient procedure to estimate the state. In this case, for the first and second order approximations of the state equation, we derive a closed-form iterative solution for the Hessian of the entropy-like version of the optimal cost function of the MME. We illustrate and contrast the performance of our algorithms with the extended Kalman filter using specific nonlinear examples with the feature that the do not converge to the correct value. The paper is organized as follows. Section introduces the entropy penalized minimum energy estimator MEE. In Section 3, we derive the explicit iterative formulas of the entropy penalized MEE when the output equation of the plant is linearized and the state equation is replaced by a first or second order approximation. Section 4 illustrates the performance of the entropy penalized MEE through computer simulations for two highly nonlinear systems. Conclusions and suggestions for further research are presented in Section 5.. THE ENTROPY PENALIZED MINIMUM ENERGY ESTIMATOR Consider the problem of estimating the current state x R n of a nonlinear system ẋ = fx + gxw, x = x a y = hx + v, b f : R n R n, g : R n R n m, h : R n R p, y R p denotes the measured output, w R m an input disturbance that cannot be measured and v R p the measurement noise affecting the output. The initial condition x and the signals w and v are assumed deterministic but unknown. For simplicity, we have considered a system without control signal since for estimator design purpose this signal is assumed to be known.. The Minimum Energy Estimator The minimum energy estimator produces an estimate for the state x that is compatible with the system s dynamics and measured outputs for noise v and disturbances w with lowest integral-square-norm Krener 6,Aguiar and Hespanha 6. The optimal state estimate ˆx at time t is defined as ˆxt = arg min Jz,t, z Rn the cost functional is given by Jx,t = x ˆx Q t + min wτ + yτ hxτ dτ, w:[,t] 3 x ˆx Q = x ˆx T Q x ˆx, Q >, and ˆx encodes a priori information about the state. Under reasonable assumptions in see details in Krener 6,Krener and Duarte 996, Jx, t is locally Lipschitz continuous and is a viscosity solution of the HJ PDE J t x,t + J x x,tfx + JT x x,t Γ yt hx =, 4 Γx = gxg T x, and J t and J x denote the partial derivatives of J w.r.t. t and x, respectively. Note that and 4 define an infinite dimensional observer with state J,t. To rewrite it in a filtering like form similar to a Kalman filter we follow the steps proposed in Krener and Duarte 996,Krener 6 by supposing that Jx, t is a smooth solution to 4. In this case, since ˆxt is a minimum of Jx,t then J x ˆxt,t =, 5a J xx ˆxt,t ˆxt + J xt ˆxt,t =. 5b Taking partial derivatives of 4 w.r.t. x, evaluating at ˆxt,t, and using 5a it follows that J tx ˆxt,t+J xx ˆxt,tfˆx + h T x ˆxt y hˆxt =. From these two last equations we finally arrive to the dynamic observer equation ˆxt = fˆx + J xx ˆx,th T x ˆx y hˆx. 6 It is important to stress that we still have the problem of computing Jx,t or more precisely the Hessian of Jx,t. To solve this problem, we propose an entropy penalized scheme to approximate the viscosity solution of the HJ PDE 4 following the procedure in Gomes and Valdinoci 7. Before we introduce this method we present the definition of viscosity solutionbardi and Capuzzo-Dolcetta 997,Fleming and Soner 5,Cannarsa and Sinestrari 3.

3 . Viscosity Solutions for Hamilton-Jacobi equations J t denotes the time discretization of J, i the iterative step, and L the Lagrangian given by Definition. Assume that ψ : R n R is a continuous function. Let Ω R n be an open set and fix x Ω. The Lx,w = w + y hx. 8 set D + ψx of super-differentials of ψ at the point x is As in Gomes and Valdinoci 7, 7 can be written as defined as { } D + ψx = p R n ψy ψx p.y x : limsup. y x y x J t x,i+ = min [J t x + t ẋ,i + t Lx,w]dµw µ R Similarly, the set of sub-differentials of ψ at a point x is m 9 defined as { } denotes the space of the probability measures D ψx = p R n ψy ψx p.y x : limsup. on R m. We now add an entropy functional to regularize y x y x the evolution of the discrete value function which acts as a viscosity term. For this purpose, let ǫ > be a small parameter and D the set of probability densities on R m, Applying the previous ideas to partial differential equations PDE of the type i.e., D = γ L R m γw a.e., γwdw =. Hx,ψx, ψx =, R m H : Ω R R n R is a continuous nonlinear function, usually denoted by Hamiltonian and ψ is the gradient of ψ the following result holds Bardi and Capuzzo- Dolcetta 997,Fleming and Soner 5,Cannarsa and Sinestrari 3 Lemma. A function ψ CΩ is a viscosity subsolution of Hx,ψx, ψx = if Hx,ψx,p, for every x Ω, p D + ψx. Similarly, ψ CΩ is a viscosity supersolution of Hx,ψx, ψx = if Hx,ψx,p, for every x Ω, p D ψx. We can now state the following result Krener 6,Krener and Duarte 996 Theorem 3. The function Jx,t defined by Jx,t = x ˆx Q + min w:[,t] t wτ + yτ hxτ dτ, is a viscosity solution of the Hamilton Jacobi PDE J t x,t+j x x,tfx+ JT x x,t Γ yt hx =, and satisfies the initial condition J x,t..3 The Entropy Penalized method Consider the a discretization in time, with time step t = t k+ t k of the cost functional 3 and use the Lax- Hopf formula Cannarsa and Sinestrari 3 to obtain J t x,i + = min w R m [ J t x + t ẋ,i + t Lx,w ], 7 Then, applying the entropy penalized scheme to 9 we obtain [ J t x,i + = min J t x + t ẋ,i γ D R m + t Lx,w + εln γw ] γwdw. Using the results in Gomes and Valdinoci 7, we can then conclude that is equivalent to the following explicit iteration scheme J t x+ t ẋ,i+ t Lx,w J t x,i + = εln ε dw. R m e.4 The Entropy Penalized Minimum Energy Estimator We call the system composed by 6, or a discretization of it, given by [ ˆx i+ = ˆx i + t fˆxi + J t xx ˆx,ih T x ˆx i [y hˆx i ] ] together with the scheme to approximate the value function Jx,t as the entropy penalized minimum energy estimator. In the next section, for the first and second order approximations of the state equation and for the linearized output equation, we present explicitly iterative formulas to compute J t x,i and its Hessian J t xx x,i. 3. THE ENTROPY PENALIZED MEE FOR SYSTEMS In this section, we restrict system to ẋ = fx + Gw, x = x, a y = Hx + v, b G R n m is assumed to be a Hermitian matrix, and H is the Jacobian matrix of h around the estimated state ˆx, that is, H = h x x=ˆx. In the following, we derive iterative formulas using the entropy penalized scheme to compute the approximation of the viscosity solution of

4 the HJ PDE 4. In Section 3. we present the proof of convergence for the. In Section 3., we consider a first order approximation of the state equation in the estimate point ˆx and compute an approximation of the cost function valid in a neighborhood of that point. In Section 3.3, we further improve the results by using a second order approximation. 3. Convergence of Entropy Penalized Minimum Energy Estimator In this subsection we provide the proof of convergence of the entropy penalized minimum energy estimator. First of all we need the following lemma that uses the fact that the EP method converges uniformly to the solution. Lemma 4. Let J t the EP approximation of J defined in 4. Then J t J uniformly in x as t. Furthermore, there exists a function γt such that γ as t, and J t J γt, t The following result establish the convergence of the estimate ˆxt. Theorem 5. Assume the following discounted cost function J x,t = + t inf w.,v.,z. e αt x ˆx Q e αt s ws + yτ hxτ ds, and suppose that the system is uniformly observable for any input. Then xt ˆxt as t. If α > then the convergence is exponential. Proof See Pequito First order approximation Theorem 6. Consider system with f replaced by a first order approximation fx = fˆx + f x ˆxx ˆx and set the initial condition of the value function J t x, as J t x, = x µ Q, µ = ˆx and Q is a positive definite matrix. Then, the cost function J t x,i at each time step i is quadratic and its Hessian J t xx x,i can be written as J t xx x,i = x µ i Qi, xx µ i = Q i T Θ T i Λ i Θ i t y H T, th T H, Q i = Θ T i Λ i Θ i + Θ i = I + t f x ˆx T, Θ i = t f ˆx t f x ˆx T ˆx µ i, Λ i = Q i Q i GU i tg T Q T i, U i = tg T Q i G + ti. Proof See Pequito Second order approximation In this section we improve our method by performing a second order approximation. In this case fx is replaced by f x = f ˆx + f x ˆx x ˆx + x ˆxT f xx ˆx x ˆx x ˆx T f x ˆxT f xx ˆx x ˆx = xx ˆx x ˆx. x ˆx T fxx n ˆxn x ˆx and the upper script refers to the coordinate index, since f is a vector valued function. Before we proceed, the following elementary lemma is needed. Lemma 7. The integral M+T ε dw...dw m, R m e M = w + c T P w + c, T = Λ T WΛ, Λ = w + c T f xx ˆx w + c, ǫ >, W,P R m m are definite positive matrices, and c j are suitable constant vectors, is absolutely convergent. Theorem 8. Consider system with f replaced by the second order approximation and set the boundary condition of the value function J t x, as J t x, = x µ Q, µ = ˆx and Q is a definite positive matrix. Then, the cost function J t x,i at each time step i is of order four and its Hessian J t xx x,i can be computed using the following iterative method. For i =, J t xx x,i = α T i For i, Q tq GU G T Q T U = J t xx x,i = αi + tx T H T Hx. xx tg T Q G + ti Ξ T i Ω i Ξ i + tx T H T Hx, xx

5 Ξ i = α i + with Ω i t y T H T 4 ty T HGG T C H T, Ω i = th T H 4 4 th T HGG T H T H, α i = x + t f ˆx + f x ˆx T x ˆx + t x ˆxT f xx ˆx x ˆx. Proof See Pequito SIMULATION RESULTS e Fig.. D example - Second Order Approximation The performance of the entropy penalized minimum energy estimator is now evaluated through two examples the extended Kalman filter diverges. The first example is an one dimensional system and the second one is a two dimensional system. In both cases we illustrate the first and second order approximation and contrast the performance with the. In the following, e i = ˆx i x i denotes the estimation error in the coordinate i and the absolute error One dimensional example Consider the system ẋ =.x x y = x we set as initial condition x =, ˆx =., an initial boundary condition J x = x, and a time step of t =.. Figure shows the time evolution of the estimation errors of the using a first approximation and the. Note that contrary to our estimator, the state estimate of the do not converge to the correct value. Figure illustrates the case for the using a second order approximation. The absolute value of e is displayed in Fig. 3 and 4. From these figures, one can see that the transient of the second-order is faster Fig. 3. D example - Absolute First Approximation e Fig. 4. D example - Absolute Second Approximation 4. Two dimensional example Fig.. D example - First Order Approximation We now present the simulation results for the second-order highly nonlinear system ẋ = 4x x x + x x ẋ = x 4x x + x x y = [x,x ] T In this simulation, the initial condition for both estimators is ˆx =.,. while the real system starts at x =,.

6 The initial boundary condition for the is J x = x T x and the time step is t =.. Figures 5-8 display the results obtained, which agree with the one-dimensional case e.8.6 e Fig. 5. D example - First Order Approximation Fig. 8. D example - Absolute Second Approximation e e e Fig. 6. D example - Second Order Approximation Fig. 7. D example - Absolute First Approximation The MEE is a particular case of the H observer, see for instance Krener 997. To provide an insight about the robustness of the, Fig. 9 shows the case when there exist uniform measurement errors in the interval [.5,.5]. As one can see, even in the presence of significant noise the do not diverge and the estimation errors converge to a small neighborhood of zero. e Fig. 9. D example - Second Order Approximation with Uniform s 4.3 Discussion of Results The derived in this paper has the desired feature of being iterative and filtering-like, which continuously improves the state estimate as more measurements arise. In fact, it resembles the see expression 6. However, there is a main difference. In the, it is the Riccati equation that provides an estimative of the evolution of the value function, which may give bad results if the system is highly nonlinear. In approach proposed in the paper, the fact that the system is nonlinear is taken into consideration by computing an estimative of the evolution of an approximation of the viscosity solution. The simulations show that the is less sensitive to the nonlinear nature of the plant compared with the. The drawback is that the convergence speed of the in some cases is slightly slower than the when this one converge. 5. CONCLUSIONS AND FURTHER RESEARCH We have considered the state estimation problem of nonlinear systems using a minimum energy estimator approach combined with an entropy penalized scheme to approximate the viscosity solution of the Hamilton-Jacobi equation. We derived a computationally efficient procedure to estimate the state and the Hessian of the value function for the first and even second order approximations of the state equation and linearized output equation. The simu-

7 lation results illustrate and contrast the good performance of our algorithms compared with the. Note that we were able to present the solution in the cases that is known to diverge. Future work will address the development of new techniques based in Preparata and Shamos 985,Anderson 999,Cannarsa and Sinestrari 3 to compute the value function using Monte Carlo approximation methods for the entropy penalized method. REFERENCES Aguiar, A.P. and Hespanha, J.P. 6. Minimum-energy state estimation for systems with perspective outputs. 5, 6 4. Aguiar, A.P. and Hespanha, J.P. 9. Robust filtering for deterministic systems with implicit outputs. 584, Anderson, E.C Monte Carlo Methods and Importance Sampling - Lectures. Berkeley University, CA, USA. Bardi, M. and Capuzzo-Dolcetta, I Optimal control and viscosity solutions oh Hamilton-Jacobi-Bellman equations. Birkhuser - Boston, Boston, USA. Basar, T.. Paradigms for robustness in controller and filter desingns. J. Macroeconomic Dynamics. Cannarsa, P. and Sinestrari, C. 3. Semiconcave Functions, Hamilton-Jacobi Equations, and Optimal Control, volume Progress in Nonlinear Differential Equations and Their Applications - Volume 58. Birkhauser. C.Evans, L. 6. An Introduction to Mathematical Optimal Control Theory. University of California - Berkeley, Berkeley, USA. Charalambous, C., James, M., Flemming, W., Mceneaney, W., Sensitive, R., Basar, T., Bernhard, P., Birkhauser, B., and Isidori, A Risk-sensitive control, differential games, and limiting problems in infinite dimensions. In in Proceedings of 33rd IEEE Conference on Decision and Control, Lake Buena, Fleming, W.H. and Soner, H.M. 5. Controlled Markov Processes and Viscosity Solutions Stochastic Modeling and Applied Probability. Springer. Gelb, A Applied optimal estimation. MIT Press. Gomes, D.A. and Valdinoci, E. 7. Entropy penalization methods for hamilton-jacobi equations. Volume 5, Grewal, M.S. and Andrews, A.P.. Kalman Filtering : Theory and Practice Using MATLAB. Wiley- Interscience. Hijab, O. 98. Minimum Energy Estimation. Univ. of California, Berkeley. Jazwinski, A.H. 97. Stochastic Processes and Filtering Theory. Academic Press. J.P. Gauthier, H.H. and Othman, S. Jun 99. A simple observer for nonlinear systems applications to bioreactors. Kailath, T., Sayed, A.H., and Hassibi, B.. Linear Estimation Prentice Hall Information and System Sciences Series. Prentice Hall. Khalil, H.K. 4. Nonlinear Systems - Second Edition. Prentice Hall, New Jersey. Krener, A.J Necessary and sufficient conditions for nonlinear worst case Krener, A. and Duarte, A A hybrid computational approach to nonlinear estimation. Krener, A.J. 6. The convergence of the minimum energy estimator. Department of Mathematics, University of California. Maybeck, P.S Stochastic models, estimation and control. Volume I. Motersen, R.E Maximum likelihood recursive nonlinear filtering. J. Optimization Theory and Applic.,, Nagpal, K.M. and Khargonekar, P.P Filtering and smoothing in an h-infinity setting. IEEE Transactions on Automatic Control, 36. Pequito, S. 9. The Entropy Penalized Minimum Energy Estimator. Instituto Superio Técnico, Technical University of Lisbon, Lisbon. Preparata, F.P. and Shamos, M.I Computational Geometry: An Introduction. Springer-Verlag, Berlin, Germany. Schon, T.B. 6. Estimation of Nonlinear Dynamic Systems - Theory and Applications. Linkoping Studies in Science and Technology. Wei Kang, Mingqing Xiao, C.R.B. 7. New trends in nonlinear dynamics and control, and their applications. Springer.