Event-triggered Model Predictive Control with Machine Learning for Compensation of Model Uncertainties

Transcription

1 Event-triggered Model Predictive Control with Machine Learning or Compensation o Model Uncertainties Jaehyun Yoo, Adam Molin, Matin Jaarian, Hasan Esen, Dimos V. Dimarogonas, and Karl H. Johansson Abstract As one o the extensions o model predictive control (MPC), event-triggered MPC takes advantage o the reduction o control updates. However, approaches to eventtriggered MPCs may be subject to requent event-triggering instants in the presence o large disturbances. Motivated by this, this paper suggests an application o machine learning to this control method in order to learn a compensation model or disturbance attenuation. The suggested method improves both event-triggering policy eiciency and control accuracy compared to previous approaches to event-triggered MPCs. We employ the radial basis unction (RBF) kernel based machine learning technique. By the universial approximation property o the RBF, which imposes an upper bound on the training error, we can present the stability analysis o the learningaided control system. The proposed algorithm is evaluated by means o position control o a nonholonomic robot subject to state-dependent disturbances. Simulation results show that the developed method yields not only two times less event triggering instants, but also improved tracking perormance. I. INTRODUCTION Model predictive control (MPC), which is one o modern control methods, derives control action by solving an optimal control problem based on the inormation about the dynamics and the current state o the plant. On the other hand, event-triggered control is a sampled data control scheme and requires the executions only when the desired control speciication cannot be guaranteed. The application o eventtriggered strategies to model predictive control (MPC) has been drawing increasing attention, due to advantages o the reduction o control updates and the capability to deal with nonlinearities and constraints. The assumption used or general event-triggered policies is the input-to-state stability (ISS) o the plant. The ISS property is used to ind suicient conditions or triggering, in the case o uncertain nonlinear systems with additive disturbance [] [4], where an upper bound o the disturbance is known. Maximum upper bound on the disturbance is the major criterion or judging an event-trigger. The major adverse aspect o these event-triggered MPCs is that the triggering occurs too requently when the bound is large, which is a common situation in practice. *This work has been supported in part by the Knut and Alice Wallenberg Foundation, the Swedish Research Council (VR), the Swedish Foundation or Strategic Research including the SSF-NRF Sweden-Korea research program, and the Denso AUTOMOTIVE GmbH. J. Yoo, M. Jaarian, D. V. Dimarogonas, and K. H. Johansson are with the ACCESS Linnaeus Center and the School o Electrical Engineering, KTH Royal Institute o Technology, Stockholm, Sweden, s: {jaehyun,matinj,dimos,kallej@kth.se} A. Molin and H. Esen are with the Technical Research Department, DENSO AUTOMOTIVE Deutschland GmbH, Freisinger Str. 2, Echning, Germany, s: {a.molin,h.esen@denso-auto.de} Motivated by this problem, the main contribution o this paper is applying machine learning technique to compensate or the disturbance to improve triggering eiciency as well as control perormance. Many o disturbance rejection methods such as adaptive control require restrictions on the disturbance such as a known upper bound or a known structure (e.g, constant or harmonic) [5] [7]. The suggested learning-aided control approach can relax these restrictions and automatically model a compensator. Many works have applied machine learning techniques to enhance perormance o control systems by learning empirical model synthesis. In [8], [9], the unknown part o a dynamic model or disturbance is modelled by supervised learning. In [] [2], reinorcement learning is used to derive a control policy or a system with unknown dynamics through trialand-error interactions in a dynamic environment. In [3], [4], parametric controllers are automatically designed by machine learning technique while guaranteeing saety o robotic systems. Despite their eorts to exploit machine learning methods, they have only ocused on improving tracking perormance, but not provided stability analysis. In this paper, the distinctive eature is that we are capable o conducting a stability analysis despite the act that the control design is based on machine learning techniques. To this purpose, we adopt the radial basis unction (RBF) as a kernel unction that is a part o the the machine learning algorithm. As an RBF characteristics, it has the universial approximation property that serves as an upper bound on the training error. This property is used to guarantee stability o the suggested control system. As one o RBF-based machine learning algorithms, we employ least square support vector regression (LSSVR). Support vector machine (SVM) [5] has been widely used in a range o applications such as data mining, classiication, regression and time-series orecasting [6]. The LSSVR developed rom [7] is modiied rom the standard SVM. This reormulation simpliies the optimization in a way that the solution is characterized by a linear system while standard SVM solves an optimization in every iteration step by interior point methods. In particular, when machine learning is used or a control system, computational issues are important because control inputs should be properly provided in time. Thus, LSSVR can be one o suitable applications to combine with control systems. The proposed algorithm is evaluated to position control o a nonholonomic robot and is compared to a standard eventtriggered MPC. The simulation results show the learning eect on control perormance, triggering condition, and

2 disturbance compensation under the event-triggered MPC ramework. Also, we can see that smooth control inputs are generated rom the suggested control method, while oscillated control inputs are shown or a standard event-triggered MPC scheme. As a result, the developed method yields not only two times less event triggering instants, but also improved tracking perormance. The rest o this paper is organized as ollows. Section II presents the system description and summarizes contributions. Section III describes the control problem ormulation under nonlinear MPC with some assumptions. Section IV introduces the suggested learning-based event-triggered MPC. Section V describes the LSSVR machine learning algorithm. Section VI and Section VII show simulation results and concluding remarks. II. SYSTEM DESCRIPTION Consider the nonlinear discrete-time dynamic system x k+ = (x k, u k, d k ), () where x k R n denotes the system state, u k R m is the control input, and d k R n is the additive disturbance. The system is subject to constraints on the state and on the control variables x k X, u k U. We assume that the uncertainty d k is a unction o x k and u k satisying: x k+ = (x k, u k ) + d(x k, u k ), (2) with known (x k, u k ) and unknown disturbance d(x k, u k ). This paper uses model predictive control (MPC) or tracking. The control input sequence u(k + j k) is calculated over a inite horizon j N. To counteract uncertainty d(x k, u k ), we aim to design the disturbance compensator g(x k, u k ) such that d(x k, u k ) g(x k, u k ) = ε k ε, (3) where the uncertainty compensation error is bounded by ε >, and we assume that the ull state x k is measurable. The main contribution o this paper is applying a machine learning technique to model g(x k, u k ). The bound in (3) can be obtained by the property o the universal approximation [8] when applying the radial basis unction (RBF) as a kernel unction used in the machine learning algorithm. The predictive state sequence should be estimated or calculating the optimal control in the MPC scheme. Taking the model g(x k, u k ) into account, the predicted state ˆx(k + j k) or j is given by ˆx(k + j k) = (ˆx(k + j k), u(k + j k)) + g(ˆx(k + j k), u(k + j k)). (4) This prediction model in (4) is dierent to the prediction model o the previous works [] [3] which use only the nominal model, i.e the irst term o the right hand in (4). The extension to event-triggered MPC is considered with disturbance compensator. The idea behind the event-triggered MPC is to trigger a solution process o the optimal control problem, only when it is needed. Otherwise, it keeps using the control input calculated at the last event-trigger instant. The updates o the control law depend on the error o the actual and the predicted states o the system. Thereore, the estimation by the disturbance compensation in (4) aects the triggering condition as well as the control input calculation. III. CONTROL PROBLEM FORMULATION In this section, the problem ormulation o nonlinear MPC is presented. The design and analysis o the proposed controller is provided along with some assumptions that are necessary to achieve stability o the closed-loop system. A. System assumptions Assumption : It is assumed that (, ) = and (x, u) is locally Lipschitz in the domain X U with Lipschitz constant L. Let us deine the preidction model ˆ(x k, u k ) = (x k, u k )+ g(x k, u k ) rom (4). Then ˆ(x k, u k ) is also locally Lipschitz, given by: ˆ(x, u) ˆ(x 2, u) L ˆ x x 2. (5) Remark : Lipschitz assumption or the nominal model with Lipschitz constraint L is standard or guaranteeing ISS (input-to-state stability). Without loss o generality, the RBF unction g satisies Lipschitz condition with constraint L g. Because both and g satisy Lipschitz condition, a linear combination o them, i.e., ˆ, is also Lipschitz with Lipschitz constraint L ˆ = L + L g. Remark 2: The event-triggered MPC algorithms presented in [] [3] assume that the addictive disturbance is bounded such that d k γ. (6) Ater applying a designed compensator, the bound o the disturbnace error ε can be smaller than γ: ε γ. (7) The more eicient event-triggering condition is derived by the relationship (7) in comparison with the existing works that should set the bound γ in (6) high enough in practice. In comparison with the existing works, the machine learning based adaptive method can compensate or the uncertainty regardless o its magnitude and structure once its pattern is recognized. We note e as the Euclidean norm o dierence between the true state and the predicted state, given by e(k + j k) = x k+j ˆx(k + j k). (8) Note that at the current k, e(k k ) is measurable, while e(k+j k ) or j are non-measurable predictions. Both measurable and non-measurable variables are necessary to derive the event-triggering condition. The non-measurable prediction has a bound as in the ollowing lemma. Lemma : The error e(k + j k) or j is bounded by

3 e(k + j k) L j e(k + k) + Lj L ε Lj L ε (9) Proo: By using the triangle inequality and the Lipschitz condition o the model, we can obtain the ollowing recursion x k+ ˆx(k + k) = x k+ (ˆx(k k), u k ) g(ˆx(k k), u k ) = x k+ (x k, u k ) g(x k, u k ) = d(x k, u k ) g(x k, u k ) = ε k ε x k+j ˆx(k + j k) L j L j ε k + L j 2 Lj L ε. ε k+ + L ε k+j e(k + k) + Lj L ε Lemma 2: The error o predictions between current and previous s is bounded such that ˆx(k + j k) ˆx(k + j k ) L jˆ e(k k ) () Proo: ˆx(k k) ˆx(k k ) = e(k k ) ˆx(k + k) ˆx(k + k ) = (ˆx(k k), u k ) + g (ˆx(k k), u k ) (ˆx(k k ), u k ) g (ˆx(k k ), u k ) L ˆ ˆx(k k) ˆx(k k ) = L ˆ e(k k ) ˆx(k + j k) ˆx(k + j k ) L jˆ e(k k ). Lemma and Lemma 2 are used to analyze stability and easibility o the control system. We note that their proos are similar to the proos shown in [2] [4]. Thus, the remaining proos in this paper will be condensed because o the similarity to the existing works [2] [4]. However, we highlight the dierences to the existing works by Remarks 2, 3, and 4. B. MPC ormulation MPC is a control strategy that calculates predictions o current and uture control inputs by solving an online inite horizon optimal control problem (FHOCP). The current and predictive states and control inputs are denoted in vector ormat as X(k) = {ˆx(k + i k)} i=n i=, U(k) = {u(k + i k)} i=n i=, where N is length o prediction horizon, and the initial state is given such that ˆx(k k) = x k. The FHOCP is ormulated in the ollowing: min J(X(k), U(k)) = U(k) N min U(k) i= L (ˆx(k + i k), u(k + k)) + V (ˆx(k + N k)), subject to ˆx(k + j + k) = ˆ (ˆx(k + j k), u(k + j k)) X j u(k + j k) U, ˆx(k + N k) X, j =,..., N, () where X denotes the terminal constraint set, L( ) is the running cost unction, and V ( ) is the terminal cost unction. The ollowing assumptions are required to guarantee control stability and easibility. Assumption 2: The running cost L(x, u) is Lipschitz continuous in X U, with a Lipschitz constant L c. Also, It satisies L(, ) = and there are positive integers α > and ω, such that L(x, u) α (x, u) ω. Assumption 3: There exists a local stabilising controller h(x) or the terminal set X in the sense that V ((x, h(x))) V (x) L(x, h(x)) or x Φ, and V is Lipschitz in Φ with Lipschitz constant L V. The compact set Φ is given by Φ = {x R n : V (x) α Φ } such that Φ X N m or m =,..., N. The inal set X = {x R n : V (x) α v } is such that x Φ, (x, h(x)) X. The constraint set X j in () is made to guarantee that there is a robust positively invariant set or the closed-loop system where a solution o the FHOCP exists. By Lemma, the set X j is deined by X j = X B j (ε), { with B j (ε) = x R n : x Lj L ε } (2) and the set operator denotes the Pontryagin dierence. Remark 3: From (2), as the set B j (ε) becomes larger, the constraint set X j becomes smaller so that easibility o the FHOCP decreases. Let us call the relationship in (7), which denotes that the set B j (ε) in (2) { established by ε is much smaller than} the set B j (γ) = x R n : x γ (L j )/(L ) that is deined in the existing works [2] [4]. Thereore, a degree o the easibility o the proposed event-triggered MPC is improved as large as the dierence between γ and ε. IV. LSSVR-BASED EVENT-TRIGGERED MPC In this section, the triggering condition o the suggested event-triggered MPC control is presented with guarantee o easibility and stability. A. Control law and easibility analysis In the classic time-triggered MPC strategy, the FHOCP is solved at every. In the event-triggered setup, the

4 rest o the predictions might be used until a new event occurs or the lapse o the horizon time. Suppose that we obtained the optimal solution U (k ) = {u (k k ),..., u (k + N 2 k )} and the corresponding optimal cost J (k ) rom the last k. Then, the control sequence U(k+m) = {u (k+m k+m),..., u (k+m+n k+m)} or s m =,..., N is given by: u (k + j k + m) = (3) { u (k + j k ) or j = m,..., N 2, h(ˆx(k + N k + m)) or j = m + N. From the control law (3), instead o solving the FHOCP at k + m, the control input U(k + m) is applied to check stability and decide i an event is triggered. Analysis o easibility to ensure existence o a solution satisying all the constraints or the FHOCP, is given the ollowing lemma. Lemma 3: A system described by (2) is assumed to satisy all Assumptions -3. Then, the FHOCP is easible i the learning error ε is bounded by: ε (α Φ α v ) L V L N. (4) ˆ Proo: We can ollow the easibilty proo described in [3] by applying Lemma and Lemma 2. B. Stability analysis and triggering condition Let the optimal cost at k be J (k ) and the costs o the easible sequence be indicated by J(k + j) or j =,..., N. Then, the dierences o these costs are given by J j = J(k + j) J (k ). Theorem : Consider a system described by (2) and assume Assumptions -3. Then, with the control law (3), J j is bounded by: or j =, J L Z e(k k ) α x k w (5) or j N, J j L Zj Lj j L ε α x k+j i w, (6) where L Zj = L V L (N ) j ˆ i= L (N ) j ˆ + L c L ˆ. (7) Thus, according to input-to-state stability (ISS) [9], the optimal cost is an ISS-Lyapunov unction o the closed loop system. Thus, the closed-loop system is input-to-state stable. Proo: We can ollow the procedures o Theorems and 2 described in [2] by applying our Lemma and Lemma 2. Consequently, the triggering rule is deined to maintain the stability, which is ensured when J i is strictly decreasing: J j+ J j. (8) By (5) (8), the triggering condition can be stated in the ollowing. Triggering rule: or j =, or j N, L Z e(k k ) σ α x k w (9) L Zj Lj j L ε σ α x k i+j w (2) i= and ( L Zj Lj ) L L Z j Lj ε σ α x k w, L (2) with < σ <, and L Zj rom (7). In case o j =, the measurable error e(k k ) = x k ˆx(k k ) is used or the triggering condition, where ˆx(k k ) is obtained by the learning-based estimation. Aterwards, or predictive period j N, the triggering condition depends on the upper bound ε. Remark 4: Relatively large values on the let-hand terms in (2) and (2) than the right-hand terms cause requent triggering instants, which is one o the practical problems in conventional event-triggered MPC [2] [4]. In this paper, however, the proposition that ε is small enough by uncertainty compensation can prevent the requent triggering instants. To summarize, suppose that the FHOCP is solved at k. The FHOCP is triggered at k + j i (9) or (2)-(2) is violated. When they are satisied or all j N, the FHOCP is triggered at the last horizon step k + N. V. COMPENSATOR DESIGN BASED ON MACHINE LEARNING This section describes how to design the compensator g(x k, u k ) through a machine learning technique. Machine learning techniques commonly use a kernel unction that transorms data in raw representation into eature vector representation, which causes big impact to learning perormance. The most popular kernel unction is the radial basis unction (RBF). In this paper, we employ least square support vector regression (LSSVR) as the learning bodyrame and RBF as the kernel unction. The basic idea o LSSVR is to map the data x R M to a higher dimensional eature space H (reproducing kernel Hilbert space) using a nonlinear mapping φ(x) : R M H R h, and then ind the relationship between the scalar target variable y and the explanatory variable x (i.e., linear regression) in the kernel space. In other words, given a training set o l training samples {(x i, y i )} l, it maps the training samples to a new data set {(φ(x i ), y i )} l with the nonlinear mapping φ( ).

5 8 7 6 state triggering triggering 2 Reerence [x, y, θ ]=[5, 8, π] Event-triggered (E-T) MPC LSSVR-based E-T MPC (a) Compared tracking results (b) Standard event-triggered MPC (c) Learning-based event-triggered MPC 2.5 v k ω k 2.5 v k ω k.2.5 disturbance d k compensation g k. control input control input disturbance (d) Standard event-triggered MPC -2 (e) Learning-based event-triggered MPC -.5 () Uncertainty and its estimation Fig. : Simulation results o the proposed algorithm in comparison to the standard event-triggered MPC [2], with respect to (a): tracking results, (b) and (c): event-triggering instants, (d) and (e): control inputs, and (): uncertainty prediction. Consider the linear regression model: g(x) = w, φ(x) + b, w R h, b R, where w and b are the coeicients, which are estimated by the ollowing optimization problem: min w,b,ξ 2 w 2 + c 2 ξi 2, s.t. ξ i = y i g(x i ), where the constant c > is weight parameter. A Lagrangian to solve the optimization is given by: L(w, b, ξ; α) = 2 wt w + 2 c ξ 2 i { ( α i y w T φ(x i ) + b ) } ξ i, where α i R are the Lagrange multipliers, and the KKT (Karush-Kuhn-Tucker) conditions are given by: w = = w = b = = α i φ(x i ), α i =, ξ i = = α i = cξ i, α i = = y ( w T φ(x i ) + b ) ξ i. Ater elimination o the variables w and e, we can obtain the ollowing solution orm: k(x, x ) + k(x, x ) b c α k(x l, x ) + k(x l, x l ) +. α l c c = [ y y l ] T, where k(x i, x j ) = φ(x i ) T φ(x j ) or i, j =,..., l is a kernel unction. Finally, by Mercer s theorem [5], the itting

6 unction as the output o LSSVR is given by: g(x) = α i k(x, x i ) + b. In this paper, the kernel unction is deined by radial basis unction (RBF): k(x i, x j ) = exp (c x i x /c 2 ), where c and c 2 are weight parameters to control strength and smoothness o the kernel unction. Thereore, the LSSVR output is rewritten by: g(x) = α i exp (c x i x /c 2 ) + b. (22) The RBF-based LSSVR (22) has a property o the universal approximation by the ollowing lemma. Lemma 4: For any given continuous real unction ḡ(x) on a compact set L and arbitary ε >, there exists on approximation unction g(x) ormed by (22) such that sup x L ḡ(x) g(x) < ε. (23) Proo: See [8]. VI. SIMULATION RESULTS For the simulation study, we consider the position control o a nonholonomic robot including state-dependent disturbance, given by: r x k+ = r x k + ( + δ)v k T cos θ k, (24) r y k+ = r y k + v kt sin θ k, θ k+ = θ k + ω k T. In this case, the uncertainty in (2) is deined as d(x k, u k ) = δv k T cos θ k that is only restricted on x-axis state. The vector u = [v, w] is the control input. The state vector x = [r x, r y, θ] T is comprised by the 2-D position o the robot (r x, r y ) and the orientation θ. The uncertainty δ =.4 is assumed constant. The constraints o the state and input are given by r x, r y < and v, w < 2. Given reerence r, the running cost unction and terminal cost unction are given by L = x r 2 Q + u 2 Q 2, and V = x 2 Q 3 with Q = 2I 3 3, Q 2 =.5I 2 2, and Q 3 = I 3 3. The prediction horizon is set to N = 2 steps, and the time interval is set to T =.2 sec. The initial positions is (r, x r y, θ ) = (,, ) and the reerence goal is r = (5, 8, π). A. Perormance analysis We present some comparison results to the conventional event-triggered MPC [2]. From Fig. (a), the proposed control scheme outperorms the compared method due to the learning capability in compensating or the oscillating uncertainty. Figs. (b) and (c) depict the triggering instants o both algorithms. Among 6 s, 26 and 3 triggering instants occur, respectively. From Figs. (d) and (e), the control inputs are shown. The suggested method yields the smooth control action, while the compared method barely approaches the reerence by the oscillating control input. The better control perormance as well as less triggering instants are caused by the accurate uncertainty estimation as shown in Fig. (). VII. CONCLUSION This paper presented the LSSVR-based event-triggered MPC strategy. It has guaranteed stability and easibility o the control system. Simulation results showed that the developed control scheme yields two times less event triggering instants and better tracking perormance or a unicycle vehicle with model uncertainty, compared with a standard event-triggered MPC. REFERENCES [] D. Lehmann, E. Henriksson, and Karl H. Johansson. Event-triggered model predictive control o discrete-time linear systems subject to disturbances. European Control Conerence, 23. [2] A. Eqtami, D. V. Dimarogonas, and K. J. Kyriakopoulos. Eventtriggered strategies or decentralized model predictive controllers. IFAC Proceedings, 44. (2): [3] K. Hashimoto, S. Adachi, and D. V. Dimarogonas. Distributed aperiodic model predictive control or multi-agent systems. IET Control Theory and Applications 9. (24): -2. [4] D. L. Marruedo, T. Alamo, and E. F. Camacho. Input-to-state stable MPC or constrained discrete-time nonlinear systems with bounded additive uncertainties. IEEE Conerence on Decision and Control, 22. [5] D. Cabecinhas, R. Cunha, and C. Silvestre. A nonlinear quadrotor trajectory tracking controller with disturbance rejection. Control Engineering Practice 26 (24): -. [6] E. Xargay, R. choe, N. Hovakimyan, and I. Kaminer. Convergence o a PI coordination protocol in networks with switching topology and quantized measurements., IEEE Conerence on Decision and Control, 22. [7] M. Jaarian. Robust consensus o unicycles using ternary and hybrid controllers., International Journal o Robust and Nonlinear Control (27). [8] A. Punjani and P. Abbeel. Deep learning helicopter dynamics models. IEEE International Conerence on Robotics and Automation, 25. [9] S. Bansal, A. K. Akametalu, F. J. Jian, F. Laine, and C. J. Tomlin. Learning quadrotor dynamics using neural network or light control. IEEE Conerence on Decision and Control, 26. [] J. Ko, D. J. Klein, D. ox, and D. Haehnel. Gaussian processes and reinorcement learning or identiication and control o an autonomous blimp. IEEE International Conerence on Robotics and Automation, 27. [] T. Zhang, G. Kahn, S. Levine, and P. Abbeel. Learning deep control policies or autonomous aerial vehicles with MPC-guided policy search. IEEE International Conerence on Robotics and Automation, 26. [2] D. Sadigh, E. S. Kim, S. Coogan, S. S. Sastry, and S. A. Seshia. A learning based approach to control synthesis o markov decision processes or linear temporal logic speciications. IEEE Conerence on Decision and Control, 24. [3] F. Berkenkamp, A. P. Schoellig, and A. Krause. Sae controller optimization or quadrotors with Gaussian processes. IEEE International Conerence on Robotics and Automation, 26. [4] A. K. Akametalu, J. F. Fisac, J. H. Gillula, S. Kaynama, M. N. Zeilinger, and C. J. Tomlin. Reachability-based sae learning with Gaussian processes. IEEE Conerence on Decision and Control, 24. [5] C. Cortes, and V. Vapnik. Support-vector networks. Machine Learning 2.3 (995): [6] J. Yoo, W. Kim, and H. J. Kim. Distributed estimation using online semi-supervised particle ilter or mobile sensor networks. IET Control Theory & Applications 9.3 (25): [7] J.A.K. Suykens, T. V. Gestel, and J. D. Brabanter. Least squares support vector machines. World Scientiic, 22. [8] J. Park and I. W. Sandberg. Universal approximation using radialbasis-unction networks. Neural Computation 3.2 (99): [9] D. Angeli, E. D. Sontag, and Y. Wang. A characterization o integral input-to-state stability. IEEE Transactions on Automatic Control 45.6 (2):