Piecewise-affine functions: applications in circuit theory and control

Piecewise-affine functions: applications in circuit theory and control Tomaso Poggi Basque Center of Applied Mathematics Bilbao 12/04/2013 1/46

Outline 1 Embedded systems 2 PWA functions Definition Classes and representation forms 3 Evaluation of PWA functions 4 Applications Model Predictive Control Virtual sensors (nonlinear state observers) Others 5 Conclusions Matlab software and toolboxes Open issues 2/46

Embedded Systems An embedded system is a computer system designed for executing specific tasks within a larger system We can find them in Mobile phones, digital cameras, mp3 players,... Home appliances (microwave ovens, washing machines,...) Cars, planes, trains,... Embedded systems contain processing cores that are either microprocessors or digital circuits Digital Signal Processor (DSP) 3/46

Digital Signal Processor VS Personal Computer Personal Computer Digital Signal Processor General purpose High power/frequency No deterministic execution time Complex operations Specific task Low power/frequency Real Time Simple operations Available operations on DSP (Fixed point arithmetic) Arithmetic: +,, Comparisons: =, >, <,, Boolean: NOT, AND, OR, XOR Memory 4/46

Algorithms for DSP Requirements: Fast execution Limited resources Low-level programming Strategy On-line execution Off-line design 5/46

Algorithms for DSP Requirements: Fast execution Limited resources Low-level programming Strategy On-line execution Maintain simplicity Off-line design All the hard math here 5/46

Outline 1 Embedded systems 2 PWA functions Definitions Classes and representation forms 3 Evaluation of PWA functions 4 Applications Model Predictive Control Virtual sensors (nonlinear state observers) Others 5 Conclusions Matlab software and toolboxes Open issues 6/46

Polytopes and Simplexes (1/2) Definition Polytope in R n : intersection of m halfspaces (boundaries) Ω= { x R n : h jx+ k j 0, j = 1,...,m } Boundary: (n 1)-dimensional hyper-plane in the form h jx+ k j = 0 Alternative definition: convex hull of vertices { Ω= x : x = m i=1 µ i v i ; 0 µ i 1,i = 1,...,m; m i=1 µ i = 1 } Simplex in R n : a polytope defined by m=n+1 vertices v i 7/46

Polytopes and Simplexes (2/2) Examples Polytopes Simplexes n=2 n=3 v 3 v 2 v 0 v 1 v 0 v 1 v 2 v 0 v 1 n=1 n= 2 n=3 8/46

Piecewise-affine functions f PWA : D R n R is piecewise-affine (PWA) if its domain is partitioned in M polytopes D = M k=1 Ω k, Ω k Ω h = 0, k h it is linear affine over each polytope f PWA (x)=f k x+ g k, x Ω k Partition 15 polytopes 2 1.5 1 0.5 0 1.5 1 0.5 0 0.5 1 x 1 1 0.5 x 2 0 0.5 1 1.5 9/46

Piecewise-affine functions Special case: Lattice form Any continuous PWA function can be expressed as 1 f PWA (x)= min 1 k M max 1 h M φ kh =1 { f h x+ g h where Φ=[φ kh ] is a zero-one structure matrix: } 1, f k φ kh = x+ g k f h x+ g h, x Ω k 0, otherwise Non need to define/store the partition explicitly! 1 Tarela and Martínez [1999] 10/46

Piecewise-affine functions Special case: regular partitions Simplicial Rectangular Used in approximation/identification problems 2 Efficient evaluation algorithms 2 Poggi et al. [2011], Genuit et al. [2012] 11/46

Simplicial PWA functions (1/2) Simplicial partition Simplicial partition of D ={x R n : a i x i b i } obtained by: x 2 b 2 x 1 a 2 a 1 b 1 12/46

Simplicial PWA functions (1/2) Simplicial partition Simplicial partition of D ={x R n : a i x i b i } obtained by: dividing each dimensional component in m i sub-intervals hyper-rectangles x 2 N v vertices b 2 x 1 a 2 a 1 b 1 12/46

Simplicial PWA functions (2/2) Representation Any PWAS function can be expressed: 1 Using the classical representation 2 As a sum of N v elements f S (x)=f j x+ g j, x S j f S (x)= N v k=1 c k φ k (x)=c φ(x) N v weighting coefficients N v basis functions 3 By linear interpolation of the f S values at the n+1 simplex vertices f S (x)= n j=0 µ j f S (v j ) 13/46

Simplicial PWA functions (2/2) Representation Any PWAS function can be expressed: 1 Using the classical representation f S (x)=f j x+ g j, x S j 2 As a sum of N v elements numerical/analytical calculations f S (x)= N v k=1 c k φ k (x)=c φ(x) N v weighting coefficients N v basis functions 3 By linear interpolation of the f S values at the n+1 simplex vertices DSP implementation f S (x)= n j=0 µ j f S (v j ) 13/46

α-basis PWA Radial Basis Functions with fixed support N v hyper-pyramids (pyramids in 2D) centred at the N v vertices Key property: 1, if k = i α k (v i )= c k = f S (v k ) 0, if k i 1 z2 αk(x) 3 Simplex 2 1 vk 0 0 1 2 3 z1 14/46

α-basis PWA Radial Basis Functions with fixed support Example: m 1 = m 2 = 3 N v = 16 14/46

Function approximation/identification with PWAS functions original function approximated function f(x) f S (x) P W A 6 4 2 fpwl 0 2 4 6 3 2 1 0 x2 1 2 3 3 2 1 x1 0 1 2 3 Two types of problems Approximation: f is known (analytically) Identification: some samples of f are available 15/46

Function approximation/identification with PWAS functions original function approximated function f(x) f S (x) 4 2 0 2 4 6 6 P W A fpwl 6 4 2 0 2 4 6 2 1 0 1 2 2 1 0 1 2 3 2 1 0 x2 1 2 3 3 2 1 x1 0 1 2 3 Two types of problems Approximation: f is known (analytically) Identification: some samples of f are available 15/46

Calculation of the value of a PWA function Evaluation algorithms suitable for fast calculations Partition dependent: PWA: calculate f PWA (x) 1 Find the index i such that x Ω i 2 Evaluate the affine expression f i x+ g i Lattice PWA: calculate f PWA (x) 1 Efficient only in limited circumstances PWAS: calculate f PWAS (x) 1 Scaling 2 Linear interpolation 17/46

Calculation of the value of a PWA function Evaluation algorithms suitable for fast calculations Partition dependent: PWA: calculate f PWA (x) Point Location Problem 1 Find the index i such that x Ω i 2 Evaluate the affine expression f i x+ g i Lattice PWA: calculate f PWA (x) 1 Efficient only in limited circumstances PWAS: calculate f PWAS (x) 1 Scaling 2 Linear interpolation 17/46

Calculation of the value of a PWA function Evaluation algorithms suitable for fast calculations Partition dependent: PWA: calculate f PWA (x) Point Location Problem 1 Find the index i such that x Ω i 2 Evaluate the affine expression f i x+ g i Lattice PWA: calculate f PWA (x) Apply definition as is 1 Efficient only in limited circumstances PWAS: calculate f PWAS (x) 1 Scaling 2 Linear interpolation 17/46

Calculation of the value of a PWA function Evaluation algorithms suitable for fast calculations Partition dependent: PWA: calculate f PWA (x) Point Location Problem 1 Find the index i such that x Ω i 2 Evaluate the affine expression f i x+ g i Lattice PWA: calculate f PWA (x) Apply definition as is 1 Efficient only in limited circumstances PWAS: calculate f PWAS (x) Khun s decomposition 1 Scaling 2 Linear interpolation 17/46

Point Location Problem (1/2) Combinatorial solution Given M e boundaries, find i such that x Ω i Combinatorial algorithm for j = 1 to M e do compare x with e j : e1 e4 h j x+ k j 0 Ω1 Ω3 x end for e2 Ω2 Ω5 e3 Ω4 18/46

Point Location Problem (2/2) Binary search tree Clever choice of the sequence of edges The tree is built off-line 3 The tree is explored on-line by a DSP e1 e4 root e1 Ω1 Ω3 > x e3 e4 e2 Ω2 Ω5 > > Ω4 e2 Ω5 Ω3 e3 > Ω4 Ω2 Ω1 3 Tøndel et al. [2002], Fuchs et al. [2010] 19/46

Point Location Problem (2/2) Binary search tree Clever choice of the sequence of edges The tree is built off-line 3 The tree is explored on-line by a DSP e1 e4 root Ω1 Ω3 e 1 : e1 > x e3 e4 e2 Ω2 Ω5 > > Ω4 e2 Ω5 Ω3 e3 > Ω4 Ω2 Ω1 3 Tøndel et al. [2002], Fuchs et al. [2010] 19/46

Point Location Problem (2/2) Binary search tree Clever choice of the sequence of edges The tree is built off-line 3 The tree is explored on-line by a DSP e1 e4 root Ω1 Ω3 e 1 : h 1 x+ k 1 > 0 e1 > x e3 e4 e2 Ω2 Ω5 > > Ω4 e2 Ω5 Ω3 e3 > Ω4 Ω2 Ω1 3 Tøndel et al. [2002], Fuchs et al. [2010] 19/46

Point Location Problem (2/2) Binary search tree Clever choice of the sequence of edges The tree is built off-line 3 The tree is explored on-line by a DSP e1 e4 root Ω1 Ω3 e 1 : h 1 x+ k 1 > 0 e1 > x e3 e4 e2 Ω2 Ω5 e 4 : > > Ω4 e2 Ω5 Ω3 e3 > Ω4 Ω2 Ω1 3 Tøndel et al. [2002], Fuchs et al. [2010] 19/46

Point Location Problem (2/2) Binary search tree Clever choice of the sequence of edges The tree is built off-line 3 The tree is explored on-line by a DSP e1 e4 root Ω1 Ω3 e 1 : h 1 x+ k 1 > 0 e1 > x e3 e4 e2 Ω2 Ω5 e 4 : h 4 x+ k 4 > 0 > > Ω4 e2 Ω5 Ω3 e3 > Ω4 Ω2 Ω1 3 Tøndel et al. [2002], Fuchs et al. [2010] 19/46

Point Location Problem (2/2) Binary search tree Clever choice of the sequence of edges The tree is built off-line 3 The tree is explored on-line by a DSP e1 e4 root Ω1 Ω3 e 1 : h 1 x+ k 1 > 0 e1 > x e3 e4 e2 Ω2 Ω5 e 4 : h 4 x+ k 4 > 0 > > Ω4 e2 Ω5 Ω3 e3 Ω4 Ω 3 : f 3 x+ g 3 > Ω2 Ω1 3 Tøndel et al. [2002], Fuchs et al. [2010] 19/46

Point Location Problem (2/2) Binary search tree Clever choice of the sequence of edges The tree is built off-line 3 The tree is explored on-line by a DSP e1 e4 root Ω1 Ω3 e 1 : h 1 x+ k 1 > 0 e1 > x e3 e4 e2 Ω2 Ω5 e 4 : h 4 x+ k 4 > 0 > > Ω4 e2 Ω5 Ω3 e3 Ω4 Ω 3 : f 3 x+ g 3 > Ω2 Ω1 Only sums, multiplications and comparisons! 3 Tøndel et al. [2002], Fuchs et al. [2010] 19/46

Evaluation in lattice form Direct application of the formula f PWA (x)= min 1 k M max 1 h M φ kh =1 { f h x+ g h } Key result 4 : If some regions share the same affine expression Ω h,ω k, Ω h Ω k : f h = f k, g h = g k It is possible to remove rows and columns from Φ=[φ kh ] Different regions can be joined 4 Wen et al. [2009] 20/46

Evaluation of a PWAS function (1/2) Scaling Fast and simple evaluation algorithm exploiting 5 the regularity of the partition local linearity x i is scaled to z i [0,m i ] by a linear transformation x 2 x2 3 v k x2 2 D x2 1 x 1 x2 0 x 0 1 x 1 1 x 2 1 x 3 1 x 4 1 D z z u 2 k = T(v k ) 3 T 2 u k 1 0 0 1 2 3 4 z 1 5 Parodi et al. [2005] 21/46

Evaluation of a PWAS function (2/2) Interpolation Linear interpolation of the f S values at the n+1 simplex vertices f S (z)= n j=0 µ j c z j, c z j = f S (u z j ) z 2 3 2 z 1 z 0 0 1 2 3 4 z 1 u z 2 z δ 1 z i integer parts δ 2 u z 0 u z 1 z = n j=0 δ i = z i z i fractional parts µ j u z j µ j : obtained by ordering δ i u z j : vertices around z obtained from z and δ i 22/46

Complexity analysis PWA functions (point location problem) Memory: (M+ M e )(n+1) Time: O(log(M)n) Continuous PWA functions (lattice form) Memory: < M(n+1) Time: < O(M) PWAS functions Memory: N v Time: O(n) 23/46

PWA functions applications Model Predictive Control Lyapunov functions (hints) Virtual sensors (nonlinear state observers) Dynamical system modelling (hints) 25/46

Model Predictive Control Control of dynamical systems under constraints x 2 u(t) x(t+ 1)=f(x(t),u(t)) y(t)=g(x(t)) y(t) x0 x 1 Problem: find a control law that 26/46

Model Predictive Control Control of dynamical systems under constraints x 2 u(t) x(t+ 1)=f(x(t),u(t)) y(t)=g(x(t)) y(t) x0 u(t) = u(x(t)) x 1 DSP Problem: find a control law that regularises the system to the origin 26/46

Model Predictive Control Control of dynamical systems under constraints x 2 um u(t) um x(t+ 1)=f(x(t),u(t)) y(t)=g(x(t)) ym y(t) ym x0 u(t) = u(x(t)) x 1 DSP Problem: find a control law that regularises the system to the origin fulfils constraints on u(t), x(t) and y(t) Solution: Receding Horizon Control 1 At time t calculate N control moves u 0,...,u N 1 2 Apply u 0 and discard the remaining control moves 3 At time t+ 1 repeat the procedure 26/46

Model Predictive Control Formulation of the optimisation problem 6 N control moves obtained by optimisation N 1 min u 0,...,u N 1 k=0 L (x k,u k )+L F (x N ) x k+1 = f(x k,u k ), k = 0,...,N 1 x 0 = x(t) u k U,k = 0,...,N 1 x k X, k = 0,...,N 1 x N X N Input and state weight Final state weight System dynamics Feasibility constraints Constraint on final state (stability) 6 Alessio and Bemporad [2009] 27/46

Model Predictive Control Linear system, quadratic cost, linear constraints N 1 min u 0,...,u N 1 k=0 { Qx k 2 + Ru k 2 }+ Px N 2 x k+1 = Ax k + Bu k, k = 0,...,N 1 x 0 = x(t) H u u k k u, k = 0,...,N 1 H x x k k x, k = 0,...,N 1 H N x N k N Q and R are tuning parameters if P solution of the Riccati equation Stability is guaranteed 28/46

Explicit Model Predictive Control Linear system, quadratic cost, linear constraints Called z [u 0,...,u N 1 ], the optimisation problem can be rewritten as a Multi-parametric Quadratic Programming (mpqp) Key result: 7 1 min z 2 z Hz+ x (t)cz Gz W Sx(t) 0 There exists an explicit solution u(t)=u 0 = u PWA (x(t)) PWA function defined over the state space 7 Bemporad et al. [2002] 29/46

Model Predictive Control Other formulations and extensions One-norm min z N 1 k=0 { Qx k 1 + Ru k 1 }+ Px N 1 x k+1 = Ax k +Bu k, k = 0,...,N 1 Infinite-norm min z N 1 k=0 { Qx k + Ru k }+ Px N x k+1 = Ax k +Bu k, k = 0,...,N 1 x 0 = x(t) H uu k k u, k = 0,...,N 1 H x x k k x, k = 0,...,N 1 H N x N k N x 0 = x(t) H uu k k u, k = 0,...,N 1 H x x k k x, k = 0,...,N 1 H N x N k N PWA model min z N 1 k=0 { Qx k 2 + Ru k 2 }+ Px N 2 x k+1 = f PWA (x k,u k ), k = 0,...,N 1 x 0 = x(t) H uu k k u, k = 0,...,N 1 H x x k k x, k = 0,...,N 1 H N x N k N Tracking min z N 1 k=0 { Q(yk r k ) 2 + R(u k u k 1 ) 2 } x k+1 = Ax k +Bu k, k = 0,...,N 1 y k = Cx k, k = 0,...,N 1 x 0 = x(t) H uu k k u, k = 0,...,N 1 H x x k k x, k = 0,...,N 1 30/46

Approximated MPC with PWAS functions Given Solve 8 The optimal solution u 0 A PWAS function u S = c φ(x) defined over a simplicial partition min (u 0 (x) u S ) 2 dx c D Ĝc Ŵ Constraints Ĝc Ŵ ensure feasibility More efficient implementations in DSP 8 Poggi et al. [2011] 31/46

Model Predictive Control Closed-loop dynamics and stability Closed loop dynamics are given by x(t+ 1)=Ax(t)+Bu(t)=Ax+ B( f k x(t)+ g k )=f k x(t)+g k Closed-loop stability: Optimal MPC solution Final state constraints A posteriori analysis (Lyapunov function) Approximated MPC Further constraints must be inserted A posteriori analysis (Lyapunov function) 32/46

PWA Lyapunov Functions (1/2) If there exist a function V(x):D R, three real values α > 0, β > 0, λ (0,1), and an integer number p > 0 such that, for all x D and all k {1,...,M}, α x p V(x) β x p V(f k x+ g k ) λv(x) 0, x Ω k then, the origin of the system is asymptotically stable in D in case p = 1; exponentially stable in D in case p > 1. 33/46

PWA Lyapunov Functions (2/2) A PWA Lyapunov function V(x)=w k x+ z k, x Ω k can be found by solving the following Linear Programming 9 α v w h v+ z h β v v vert(ω h ) w k (f h v+ g h )+z k λ(w h (v) z h ) 0 v vert(ω hk ) : T hk = 1 where [T hk ] is the reachability matrix T hk { 1 if x Ωh s.t. f h x+ g h Ω k 0 otherwise Reachability matrix Ω hk ={x Ω h : f h x+ g h Ω k } Starting regions 9 Grieder et al. [2005] 34/46

Virtual sensors A virtual sensor is a nonlinear state observer x(t+ 1)=g(x(t),u(t)) S : y(t)=h y (x(t))+η(t) z(t) z(t)=h z (x(t))+ξ(t) u(t) S y(t) h, g y, g z unknown O DSP ẑ(t) u(t) R n u, y(t) R n y always available z(t) R available for t T ẑ(t) estimate of z(t), t > T 35/46

Virtual Sensors Standard two-step approach 1 Identify a model ˆ S for S from available data linear/nonlinear systems identification 2 Design a state observer from identified model e.g., Kalman filter, Extended Kalman filter,... Both step 1 and 2 could be hard to accomplish Alternative solution: single-step approach Identify the observer from data Direct Virtual Sensor 36/46

Direct Virtual Sensors (DVS) Identifying a virtual sensor from time series 10 Consider a time-window of size N: U t ={u(τ)} t τ=t N+1 Z t ={z(τ)} t τ=t N+1 Y t ={y(τ)} t τ=t N+1 Ẑ t ={ẑ(τ)} t τ=t N+1 DVS structure: dynamical PWAS system ẑ(t+ 1)=f S (U t,y t+1,ẑ t )=c φ(u t,y t+1,ẑ t ) Weights given by min c 10 Poggi et al. [2012] { } T 1 [z(t+ 1) f S (U t,y t+1,z t )] 2 t=n 37/46

Direct Virtual Sensors Convergence analysis: DVS VS two-step approach i) The vector of parameters guarantees the minimisation of the variance of the estimation error among all the virtual sensors [ with the same structure, i.e., f S = argmin E (z(t) ẑ(t)) 2]. f S ii) If it is possible to express the two-step observer in regression form as a particular realisation of the virtual sensor, one obtains that the performance of the DVS is better than or equal to that of the two-step observer. iii) If ii holds and there exists a set of parameters of the two-step observer that describes exactly the system, then the DVS is a minimum variance filter. 38/46

Other applications Dynamical Systems Modelling PWA dynamical systems arise in the following cases Linear system under Model Predictive Control Smooth maps approximated by PWA functions Circuit implementation of nonlinear systems 11 Hybrid Systems modelling Continuous dynamics described by linear difference equations; discrete dynamics described by finite state machines Equivalence with PWA systems 12 11 Poggi et al. [2009] 12 Bemporad [2004] 39/46

Conclusions PWA functions are a framework to solve many problems Optimal control State estimation Approximation/identification... They combine simplicity and complexity Simplicity easy to calculate Complexity approximation capabilities 41/46

Open issues Model Predictive Control Direct Approximated Explicit MPC Virtual Sensors A real application as a proof of concept Evaluation algorithms for DSP Simplicial non-uniform partitions Exploit lattice PWA form 42/46

Related Software Toolboxes for Matlab Multi-Parametric Toolbox (MPT) MPC solvers Computational geometry (polytopes) Lyapunov functions Hybrid Toolbox MPC solvers Hybrid systems modelling MOBY-DIC Toolbox Approximated Explicit MPC Lyapunov functions Virtual Sensors 43/46

Suggested bibliography I A. Alessio and A. Bemporad. A survey on explicit model predictive control. Lecture Notes in Control and Information Sciences, 384:345 369, 2009. A. Bemporad. Efficient conversion of mixed logical dynamical systems into an equivalent piecewise affine form. IEEE Transactions on Automatic Control, 49(5):832 838, 2004. A. Bemporad, M. Morari, V. Dua, and E.N. Pistikopoulos. The explicit linear quadratic regulator for constrained systems. Automatica, 38(1):3 20, 2002. A.N. Fuchs, C.N. Jones, and M. Morari. Optimized decision trees for point location in polytopic data setsapplication to explicit mpc. pages 5507 5512, 2010. B.A.G. Genuit, L. Lu, and W.P.M.H. Heemels. Approximation of explicit model predictive control using regular piecewise affine functions: An input-to-state stability approach. IET Control Theory and Applications, 6(8): 1015 1028, 2012. P. Grieder, M. Kvasnica, M. Baotic, and M. Morari. Stabilizing low complexity feedback control of constrained piecewise affine systems. Automatica, 41(10):1683 1694, 2005. M. Parodi, M. Storace, and P. Julián. Synthesis of multiport resistors with piecewise-linear characteristics: A mixed-signal architecture. International Journal of Circuit Theory and Applications, 33(4):307 319, 2005. T. Poggi, A. Sciutto, and M. Storace. Piecewise linear implementation of nonlinear dynamical systems: From theory to practice. Electronics Letters, 45(19):966 967, 2009. T. Poggi, A. Bemporad, A. Oliveri, and M. Storace. Ultra-fast stabilizing model predictive control via canonical piecewise affine approximations. IEEE Transactions on Automatic Control, 56(12):2883 2897, 2011. 44/46

Suggested bibliography II T. Poggi, M. Rubagotti, A. Bemporad, and M. Storace. High-speed piecewise affine virtual sensors. IEEE Transactions on Industrial Electronics, 59(2):1228 1237, 2012. J.M. Tarela and M.V. Martínez. Region configurations for realizability of lattice piecewise-linear models. Mathematical and Computer Modelling, 30(11-12):17 27, 1999. P. Tøndel, T.A. Johansen, and A. Bemporad. Computation and approximation of piecewise affine control laws via binary search trees. volume 3, pages 3144 3149, 2002. C. Wen, X. Ma, and B.E. Ydstie. Analytical expression of explicit mpc solution via lattice piecewise-affine function. Automatica, 45(4):910 917, 2009. 45/46

Thanks Questions and comments are welcome! 46/46