Moving Horizon Estimation for Staged QP Problems

Transcription

1 51s IEEE Conference on Decision and Conrol, Maui, HI, December 212 Moving Horizon Esimaion for Saged QP Problems Eric Chu, Arezou Keshavarz, Dimiry Gorinevsky, and Sephen Boyd Absrac his paper considers moving horizon esimaion (MHE) approach o soluion of saged quadraic programming (QP) problems. Using an insigh ino he consrained soluion srucure for he growing horizon, we develop a very accurae ieraive updae of he arrival cos in he MHE soluion. he updae uses a quadraic approximaion of he arrival cos and informaion abou he previously acive or inacive consrains. In he absence of consrains, he updae is he familiar Kalman filer in informaion form. In he presence of he consrains, he updae requires solving a sequence of linear sysems wih varying size. he proposed MHE updae provides very good performance in numerical examples. his includes problems wih l 1 regularizaion where opimal esimaion allows us o perform online segmenaion of sreaming daa. A. Problem saemen I. INRODUCION Given a series of quadraic forms g (u), g (u,v), where u,v R n and is an ineger index, consider a series of ime saged quadraic programming (QP) opimizaion problems for increasing g (z )+ =1 g (z 1,z ) F eq z = G eq z 1 +h eq F ineq z G ineq z 1 +h ineq = 1,...,, where z R n are decision variables; he marices F eq, F ineq have sizes compaible wih z and he vecors h eq, h ineq ; and denoes componen-wise for wo vecors. he opimizaion inerval [1, ] is furher called full horizon. Our goal is o provide a sequence of opimal soluions Z () = (z,...,z ) as increases. he sage coss g and he iniial cos funcion, g, are convex quadraic funcions ha can be wrien in he form [ ] [ ][ ] [ ] [ ] u R Q g (u,v) = u s u 2, (2) v M v r v Q (1) g (v) = v P v 2q v, (3) where marices R, Q, M, P R n,n provide he convexiy. he moivaion for considering he QP formulaion and nonlinear esimaion problem examples ha lead o such formulaion are presened below. In he absence of consrains, he ime saged QP is a sligh generalizaion of he opimal linear Gaussian esimaion problem. In linear Gaussian esimaion g (z 1,z ) = 1 2 y C z 2 Φ z A z 1 2 Ψ, (4) he auhors are wih he Informaion Sysems Laboraory, Elecrical Engineering Deparmen, Sanford Universiy, Sanford, CA , USA. {echu58,arezou,gorin,boyd}@sanford.edu. where y is he observed daa; z is he esimaed sae; A and C are he sae updae and he observaion marices, respecively; we use he noaion x 2 Q = x Qx, and Φ, Ψ are he inverse covariance marices of he observaion and process noise, respecively. As grows, he soluion for problem (4) can be recursively compued using he wellknown Kalman filer approach. his paper pursues a moving horizon esimaion (MHE) approach o solve he ime-saged QP (1). For each he MHE solves he problem V N (z N )+ = N+1 g (z 1,z ) F eq z = G eq z 1 +h eq F ineq z G ineq z 1 +h ineq = N +1,...,, (5) where N is fixed, and he ime inerval [ N +1, ] is called he receding horizon of lengh N. he arrival cos V N (z N ) in (5) is compued hrough a recursive updae. he arrival cos updae is he key conribuion of his paper and is described below. Unlike he recursive Kalman filer soluion, which is exac, he MHE soluion is approximae, because he arrival cos updae is approximae. his paper proposes a very accurae approximaion of he arrival cos. B. Moivaion ime-saged QP problems of he form (1) arise in many nonlinear esimaion applicaions, such as roboics, image processing, process monioring, navigaion, and oher filering and rend esimaion applicaions. his QP formulaion includes sae and observaion consrains, which makes i more versaile han he sandard Kalman filer. An imporan source of he ime-saged QP formulaions are problems relaed o compressed sensing or robus esimaion. Such problem impose penaly funcions o shape he error residuals. Common examples include he Huber and l 1 penaly funcion. One well-known example is oal-variaion denoising in which an l 1 penaly is used o promoe sparsiy in labeling daa segmens. We consider he oal-variaion denoising example in Secion III-B of his paper. ime-saged QPs also arise in order-resriced inference problems and bioequivalence problems in drug esing. For increasing, hese problems evenually require runcaing he size of he QP problem solved. he MHE offers a sysemaic approach o such sreaming daa processing. A brief discussion of some earlier work on MHE is presened in he nex subsecion. I is worh noing ha many (perhaps even mos) examples in he published MHE work consider QP formulaions ha can be described by (1).

2 C. Previous work Some of he imporan earlier work on MHE includes [1], [2], [3], [4], where he MHE problems and soluions are esablished as he (esimaion) dual o receding horizon conrol, or model predicive conrol (MPC). In [5] and laer work, he sabiliy of MPC is esablished by inroducing a quadraic approximaion for he cos-o-go a he erminal sae of he receding horizon. As a dual problem, he MHE abiliy o rack he full horizon soluion (sabiliy) requires adding a quadraic arrival cos penaly a he beginning of he receding horizon. For MPC, his line of reasoning is well discussed in [6], where furher references can be found. For MHE, a quadraic approximaion of he arrival cos leads o an exended Kalman filer (EKF) updae for he arrival cos. Such an updae is used in much of he exising work in MHE. In he presence of consrains, however, he quadraic approximaion of he arrival cos used in EKF migh be inaccurae. his is especially rue if he inequaliy consrains swich beween being acive and inacive a he opimal soluion. As a resul, uning he EKF updae in MHE is more of an ar han science. I is well recognized ha improperly uned arrival cos updae can render he MHE updae unsable [7]. his paper proposes a new approach o approximaing he arrival cos ha is aware of he consrains changing from acive o inacive as he opimizaion horizon moves. he earlier paper [8] discussed a special case of he MHE approach proposed in his paper in applicaion o isoonic regression esimaion. he MHE arrival cos updae in [8] uses knowledge abou he consrains being acive or inacive based on he previous MHE updae. his paper develops an exension of his idea o a general class of QP-represenable MHE problems. D. Conribuion of he paper his paper esablishes a class of saged QP problems (1) (3). his class includes many MHE problems in paricular he problems wih l 1 regularizaion where he opimal MHE esimaion allows us o perform on-line segmenaion of sreaming daa. wo examples of such problems are considered in Secion III. he conribuion of he paper is wofold: firs, we inroduce he approximaion hypohesis ha he opimal soluion of a ime-saged QP problem deep inside he horizon (i.e., far back enough in he filer hisory) is insensiive o he las sage of he problem. In paricular, he acive se of he inequaliy consrains becomes fixed afer a paricular ime as he problem horizon grows. his hypohesis holds for mos problems in pracice. Second, we inroduce an ieraive updae of he arrival cos in he MHE soluion. he ieraive updae uses a quadraic approximaion of he arrival cos and acive/inacive consrain informaion from he previous sep. In he absence of he consrains, he updae becomes he familiar Kalman filer in informaion form. In he presence of he consrains, he updae requires solving linear sysems of varying size. his paper demonsraes ha our approximaion hypohesis holds well in pracice and he proposed arrival cos updae leads o good esimaes in our numerical examples. A heoreical proof of our approximaion hypohesis is no included in his paper; noe ha sabiliy proofs for MPC appear well afer MPC s adopion. II. MOVING HORIZON ESIMAION A. Exac arrival cos funcion Since minimizing he full problem is equivalen o minimizing over a parially d problem, he full horizon problem (1) (3) can be presened in he form (5). his requires using an arrival cosv N (z N ) = V N (z N) obained by parial minimizaion wih respec o he decision variables z,z 1,...,z N 1. V N (z N) = ( min g (z )+ ) = N z =1 g (z 1,z ),z 1,...,z N 1 F eq z = G eq z 1 +h eq F ineq z G ineq z 1 +h ineq = 1,..., N We will furher call V N (z N) in (6) he exac arrival cos a N o disinguish i from he approximae arrival cos inroduced below. A recursive updae for he exac arrival cos from 1 o can be obained by exending he parial minimizaion o he nex decision variable z. his yields he Bellman equaion V (u) = min v V 1(v)+g (u,v) F eq v = G eq v +h eq F ineq v G ineq v +h ineq, where u corresponds o z and v o z 1. B. Arrival cos and acive consrains Because of he inequaliy consrains in (6), he exac arrival cos V (u) is a complicaed piecewise quadraic funcion of is argumen. ransiions beween he quadraic segmens corresponds o he inequaliy consrains in (1) swiching beween being acive and inacive. Calculaing he arrival cos funcion in (6) involves solving an inequaliy consrained QP in he decision variablesz,z 1,...,z N 1. As he number N of decision variables grows, in mos cases we can no longer effecively calculae V N (u) in (6) for all u R n. he proposed approach is o use quadraic approximaion of he arrival cos in he viciniy of he opimal soluion Z () = (z,...,z ). If Z () were known ahead of ime, hen we could replace he affine inequaliy consrains in (6) wih equaliy consrains whenever we know ha he consrain is igh for Z (). When he inequaliy consrain is no igh for he soluion Z (), we could drop he consrain. For he consrains in (6), consider he index se A of he inequaliy consrains ha are acive a sep for he (6) (7) 2

3 soluion Z () of he full-horizon problem wih he horizon. he acive consrain se A can be described as A = {i F ineq i z = Gineq i z 1 +hineq i }, (8) for = 1,...,. Noe ha he he se of acive consrains A depends on he horizon. As grows, he se A of he consrains acive a ime sep may change. By replacing he consrains in (6) wih he affine equaliy consrains, he arrival cos becomes a soluion of equaliy consrained QP - a quadraic funcion. Unforunaely, finding he se of acive consrains A exacly requires solving he full-horizon inequaliy consrained QP in he firs place. We will nex discuss how a quadraic approximaion of he arrival cos funcion and he approximae se of he acive consrains A can be compued recursively wihou he need o solve he full horizon problem. C. Approximaion hypohesis he MHE updae relies on represening he full horizon problem (1) in he form (5), wheren is fixed and he horizon is growing. In wha follows, we assume ha N is fixed and > N. For each, he arrival cos is compued based on he MHE soluion and he arrival cos ha have been obained a he previous sep, for 1. Afer ha, he new MHE soluion o he N-sep QP problem is compued. he proposed MHE updae is based on he following approximaion hypohesis: for N large enough and N, A = A +d for any d >. his approximaion implies ha, as grows, he acive consrain se for N does no change and ha A = A, where A is independen of for large enough. Assuming ha A is known, he combined equaliy and inequaliy consrains in (6) for = 1,..., N can be changed o he equaliy consrains F z = G z 1 +h, (9) where he marices F, G, and h are defined as [ ] [ ] [ ] F eq G eq h eq F =, G =, h =, (1) F ineq A G ineq A h ineq A and F A denoes he sub-marix of F formed from he rows F i for all i A. he approximaion hypohesis is a sylized fac ha is no exacly rue, bu appears o hold in pracice for reasonably large N. As is shown below, i allows us o obain very accurae MHE approximaions of he exac full-horizon soluion. D. Updae of he approximae arrival cos Assuming ha he inroduced approximaion hypohesis holds and A is known, he approximae arrival cos V can be found via an approximae Bellman ieraion. By replacing he consrains in (7) wih he equaliy consrains (9), we ge he approximae Bellman updae as V (v) = min u V 1 (u)+g (u,v) F 1 v = G 1 u+h 1 (11) A each horizon, he proposed MHE approach solves he problem (5), where he (approximae) arrival cos V N is updaed according o (11). he arrival cos updae (11) depends on he acive consrain se A N hrough (1). A each horizon, we ake A N = A N = A N 1, he acive se for he soluion ha was compued a horizon 1. Such updae of he acive consrain se follows he approximaion hypohesis described in Secion II-C. Unlike he exac Bellman updae (7) ha has inequaliy consrains, he approximae updae (11) has affine equaliy consrains only. Saring wih V (v) = g (v), he approximae arrival cos funcion obained in such updae is always quadraic and can be presened in he form V (v) = v P v 2q v +α, (12) where marix P and vecor q fully define he approximae arrival cos; he scalar consan α has no impac on he opimizaion problem. he approximae arrival cos updae (11) can be formulaed as an updae for P, q in (12). Consider he parial minimizaion problem for he approximae Bellman ieraion (11), wherev 1 is given by (12) andg (u,v) is given by (2). he Karush-Kuhn-ucker (KK) condiions for opimaliy yield P 1 +R Q F Q M G u v = q 1 +s r, F G µ h Eliminaing he variables u and µ from hese condiions yields a quadraic form in v ha describes V (v) (12). he expressions forp andq can be exraced from his quadraic form. he obained recursive updae for P and q has he form ] Q [ P = M [ P 1 +R G F G ] Q [ q = r [ P 1 +R G F G ] 1 [ ] Q F ] 1 [ ] q 1 +s. h his, along wih (1) and he acive consrain se A N updae, fully specifies he arrival cos updae (11). In conras o he familiar Kalman filer, he marices updaed and invered in our equaions have varying size. he size of he marices F, G, and h changes depending on he number of he inequaliy consrains ha are acive a ime. E. MHE updae summary o recap, he proposed MHE approach approximaely solves he ime-saged QP problem (1) (3). A each horizon, i provides a soluion Z mhe () ha approximaes he opimal soluion Z (). For N, we solve he full problem (1). For > N, he N-sep QP problem (5) is solved wih he approximae arrival cos funcion V N (z N ) of he form (12), where marices P N and q N are defined recursively as described in Secion II-D. he soluion o he QP problem (5) can be obained using any suiable QP solver. For > N he N-sep soluion o (5) defines he las N decision vecors z mhe in Z mhe () = 3

4 (z mhe,zmhe 1,...,zmhe ). he decision vecors for < N are aken from Z mhe ( 1) such ha z mhe = zmhe 1. A. Kalman filer III. EXAMPLES Consider a linear Gaussian esimaion problem for a dynamical sysem x +1 = A x +w y = C x +v, where x is he ime-dependen sae vecor, y is he observaion vecor, and A and C are marices of appropriae sizes. he noise vecors w and v are assumed o be Gaussian wih disribuion N(,Ψ 1 ) and N(,Φ 1 ), respecively. he iniial sae x is assumed o have disribuion N(P 1 q,p 1 ). he problem of maximum likelihood esimaion of he sae z, given he observaions y has he form (1) wih no consrains, where g (z 1,z ) is described by (4), and can be presened in he form (2) wih M = Ψ +C Φ C, Q = Ψ A, R = A Ψ A, r = C Φ y, s =. In his example, here are no consrains and all approximaions are exac. he updae (7) for he (exac) arrival cos funcion V can be performed by he recursive ieraion P = Ψ +C Φ C Ψ A (P 1 +A Ψ A ) 1 A Ψ q = C Φ y +Ψ A (P 1 +A Ψ A ) 1 q 1. In his case, our approximaion hypohesis holds exacly (since here are no consrains), and any choice of N in he MHE formulaion (5) wih he exac arrival cos will yield he same soluion. hese recursive esimaion equaions provide informaion form of he Kalman filer, where P is he inverse covariance marix (informaion marix). he sandard Kalman filer corresponds o he updae wih N =, and provides he las poin of he opimal full-horizon soluion as z = P 1 q. B. oal variaion denoising Consider he esimaion problem x Px 2q x + =1 λ x x = y Cx 2 Φ, (13) where x R n, =,..., are he decision variables, y R m, =,..., are he raw daa, C R m n is he observaion marix, and Φ R m m is a posiive definie weigh marix. he l 1 -norm is used o induce sparsiy of changes in he soluion x and force segmenaion of he soluion ino several inervals wih consan x. Problem (13) is known as oal-variaion denoising and is imporan for many applicaions. o implemen he MHE updae of Secion II-E, we formulae an equivalen problem x Px 2q x + =1 λ1 a + = (y Cx ) Φ(y Cx ) a x x 1 a x x 1 = 1,...,. Using decision variable vecorsz = (x,a ), we can express his as he ime-saged QP problem z P z 2q z + =1 g (z 1,z ) F ineq z G ineq z 1, = 1,...,, where [ ] [ ] P +C P = ΦC q +C, q = Φy, [ ] [ ] I I I F ineq =, G I I ineq =. I he cos funcion g has he form (2) wih [ ] C M = ΦC, Q = R =, [ ] C r = Φy, s (λ/2)1 =. Using his represenaion, we solve an insance of problem (13) wih scalar sae x and scalar daa y, wih C = 1, Φ = 1,P =,q =, and = 2. We use he moving horizon of N = 2 and λ = 2. he ime series y, =,..., was obained by adding random noise o a piecewise consan signal. We implemen he MHE updae of Secion II-E o solve his oal variaion denoising problem. Figure 1 shows he raw daa y as dos and he sae x in he full-horizon soluion Z () for = 2 as a piece-wise consan dashed line. he uneven solid line in he plo corresponds o he x-componen of he final sae z mhe esimaed a each sep of he MHE updae. In his plo, he MHE soluion overlaps wih he full horizon soluion, we have z mhe = z wihin he accuracy of he compuaions. he verical lines in Figure 1 delineae change poins where some of he consrains change beween acive and inacive. C. l 1 rend filering We now consider he l 1 rend filering problem described in [9]. he problem can be formulaed as x Px 2q x + =2 λ x 2x 1 +x 2 + = (y x ) 2, (14) where x is a scalar decision variable and ime series y, =,..., represens he daa o be filered. his problem places an l 1 penaly on he second difference x 2x 1 +x 2 o make i sparse. he soluions of (14) follow a piecewise linear rend wih sparse kink poins. he segmenaion of he soluion ino affine inervals (sraigh 4

5 Fig. 1. MHE filering resuls for oal variaion denoising line segmens) faciliaes daa inerpreaion, compression, and forecasing. Similar o he previous subsecion, he problem can be ransformed ino a ime-saged QP. An equivalen form of problem (14) is x Px 2q x + =2 λa + = (y x ) 2 x 2x 1 +x 2 a = 2,...,. Wih he decision variable z = (x,x 1,a ), we can express he problem as z P z 2q z + =1 g (z 1,z ) F ineq z G ineq z 1 F eq z = G eq z 1, = 2,...,, where P = P +1, q = q +y [ ] [ ] F ineq =, G ineq 1 1 =, F eq = [ 1 ], G eq = [ 1 ]. he represenaion (2) for he cos funcion g is given by M = 1, Q = R =, r = y, s =. (λ/2) Figure 2 shows he resul of applying he MHE algorihm of Secion II-E o problem (14) wih λ = 5, P =, q =. he daa y, =,..., is shown as dos. he solid line shows he opimal filering soluion z. In his case, he MHE soluion z mhe = z (wihin he accuracy of he compuaions) for all =,...,2. he verical lines in Figure 2 delineae he change poins Fig. 2. MHE filering resuls for l 1 filer. = 5 = x x x x 2 4 Fig. 3. he rue V (x) (dashed) and he approximae V(x) (solid) arrival coss. IV. ACCURACY OF APPROXIMAION A. Accuracy of arrival cos approximaion Consider he oal-variaion denoising problem (13) discussed as Example 2 in Secion III-B. his is a onedimensional example wih he scalar sae x, scalar observaion y, C = 1, Φ = 1, and λ = 2. In his example, an MHE updae wih window of N = 2 is applied o daa ime series of he lengh = 2. We calculae he exac arrival cos funcion V ( ), =,...,2 by gridding, and compare i o he approximae arrival cos funcions, obained using our mehod. Figure 3 compares he exac arrival cos funcion V (x) and he proposed quadraic approximaion of he arrival cos funcion V (x) used in he MHE. he plos show hese wo value funcions for = 5 (lef) and = 15 (righ). On he x-axis, we plo he deviaion of he sae variable from he exac full horizon soluion. he y-axis is he arrival cos funcion value offse by a consan (which is inconsequenial for he opimizaion problem). We observe ha he exac arrival cos funcion V and he approximae arrival cos funcion V mach very well in he viciniy of he operaing poin x =. B. Accuracy of segmenaion Consider he l 1 rend filering problem (14), discussed in Secion III-C. In his formulaion, he proposed MHE updae algorihm provides a piecewise linear segmenaion of he underlying rend from sreaming daa. he segmenaion is 5

6 x mhe x Fig. 5. Comparison of he error beween he full horizon esimaes (x ) and he moving horizon esimaes (x mhe ) Fig. 4. Segmenaion for l 1 rend filering: a (blue) do indicaes a change poin where our MHE soluion agrees wih he full horizon soluion, a (red) x represens false negaives where our MHE soluion failed o deec a change poin, and a (green) o represens false posiives where our MHE soluion erroneously deeced a change poin. described by he change poins (kinks) in he soluion. Using he proposed MHE updae, we are able o obain online segmenaion of sreaming daa as he problem size grows ou of bounds for a bach soluion. Figure 4 shows he change poins deeced using he proposed MHE algorihm for he same simulaion resuls as shown in Figure 2. he change poins were deeced as poins where x 2x 1 + x 2 > ǫ, where ǫ is defined by he accuracy of he QP opimizaion soluion. he dos show he deeced change poins. he poins where he full-horizon soluion Z () and our MHE algorihm do no agree are indicaed wih an x (false negaive) or an o (false posiive). he horizonal axis represens he horizon lengh, while he verical axis shows ime inside he full horizon [1, ]. he doed diagonal represens he causaliy boundary: he soluion is obained for. he MHE filering soluion in Figure 4 does mach he full horizon soluion exremely accuraely. he differing change poins have he soluion ha is very close o he consrain, close o he QP solver accuracy limi. C. Accuracy of opimal soluion approximaion Checking he accuracy of he arrival cos approximaion in Secion IV-A needs o be complemened by verificaion of opimal soluion approximaion. Furhermore, if decision vecor z has dimension larger han 2, visualizing he value funcion is no pracical. We compare he performance of he MHE wih proposed arrival cos approximaion agains he performance of an esimaor ha solves he full-horizon problem a each sep. We again consider he oal variaion denoising example (13), bu we use a larger problem dimension, wih x and y having size of n = m = 5, C = I, Φ = I, and λ = 2. We consider an MHE updae wih window of N = 5 applied o he daa ime series y of he lengh = 2. Figure 5 shows he resuls. For each, we plo all n = 5 enries of he esimae obained. he las sae esimaed wih our MHE updae x mhe is ploed along he horizonal axis. he esimaion error x mhe x is shown on he verical axis. he small error observed is on he order of accuracy of he QP solver. Furhermore, he numerical QP soluion is more accurae for he MHE soluion ha requires o solve he problem of small size (5 seps) compared o he fullhorizon soluion (2 seps). his resul means ha he MHE performance wih he proposed approximaion of he arrival cos is pracically idenical o he full-horizon esimaor, a leas in his example. V. CONCLUSION his paper has presened a moving horizon esimaor (MHE) for saged QP problems suiable for sreaming daa processing. he MHE updae is based on he approximaion assumpion ha he acive consrain se remains fixed far enough in he pas from he las ime sage. his assumpion allows us o derive an explici Bellman updae for he arrival cos. he updae is a generalizaion of Kalman filer updae; wih a difference ha a each sep he dimensions of he updaed marices change depending on he number of acive inequaliy consrains. REFERENCES [1] H. Michalska and D. Mayne, Moving horizon observers and observerbased conrol, IEEE ransacions on Auomaic Conrol, no. 6, pp , Jun [2] P. Moraal and J. Grizzle, Observer design for nonlinear sysems wih discree-ime measuremens, IEEE ransacions on Auomaic Conrol, no. 3, pp , Mar [3] C. Rao, J. Rawlings, and D. Mayne, Consrained sae esimaion for nonlinear discree-ime sysems: Sabiliy and moving horizon approximaions, IEEE ransacions on Auomaic Conrol, no. 2, pp , 23. [4] G. C. Goodwin, M. M. Seron, and J. A. D. Dona, Consrained Conrol and Esimaion - An Opimisaion Approach. Springer Verlag, 25. [5] H. Michalska and D. Mayne, Receding horizon conrol of nonlinear sysems, IEEE ransacions on Auomaic Conrol, no. 5, pp , May 199. [6] J. Primbs and V. Nevisic, A new approach o sabiliy analysis of finie receding horizon conrol wihou end consrain, IEEE ransacions on Auomaic Conrol, no. 8, pp , Aug. 2. [7] J. H. Lee and N. L. Ricker, Exended kalman filer based nonlinear model predicive conrol, Indusrial and Engineering Chemisry Research, vol. 33, no. 6, pp , [8] D. Gorinevsky, Efficien filering using monoonic walk model, in American Conrol Conference, Jun. 28, pp [9] S. J. Kim, K. Koh, S. Boyd, and D. Gorinevsky, l 1 rend filering, SIAM Review, no. 2, pp , 29. 6