Underactuated Control and Distribution of Multi-Agent Systems in Stratified Flow Environments

Transcription

1 49th IEEE Conference on Decson an Control December 15-17, 1 Hlton Atlanta Hotel, Atlanta, GA, USA Uneractuate Control an Dstrbuton of Mult-Agent Systems n Stratfe Flow Envronments Robert H Krohn an Thomas R Bewley Abstract The present paper shows how vertcal actuaton alone may be use to effectvely control the spatal strbuton of moble vehcles ( balloons n ar, or rfters n water) n vertcally-stratfe backgroun flows Applyng teratve, ajont-base, moel prectve control (MPC) technques couple wth lnear quaratc regular (LQR) feeback, agent trajectores an feeback control strateges are etermne to ensure the esre termnal-tme spacng of multple agents Varatons n both startng locatons an sturbances are consere to llustrate the sgnfcant control authorty n ths agent separaton problem n both lamnar an turbulent backgroun flows, espte the fact that lnear controllablty s lost n certan lmtng cases The paper thus emonstrates a novel applcaton of feeback control theory to an emergng real-worl applcaton n mult-agent systems The results lay the grounwork for future applcatons n Lagrangan samplng of unerwater ecosystems as well as the effcent samplng of hurrcanes for the purpose of forecastng ther evelopment I INTRODUCTION Stues of both oceanc an atmospherc currents va Lagrangan motons have been wely ocumente an use for moelng an research 1] ] Hstorcally, these methos have rele on passve moton of buoyant vehcles, but more recently rfters wth controllable buoyancy have been use to perform Lagrangan an mass-transport stues, such as the nternatonal Argo Project 1] eploye to montor ocean currents on a global scale Recent work by the atmospherc research communty has establshe the capablty of actvely an effcently alterng the nternal ensty of a balloon, thus enablng buoyancy control n the atmosphere over extene peros of tme ] The ablty to actvely change buoyancy (an, thus, vertcal poston) n such strongly stratfe flow systems opens the oor to a host of nterestng new lowpower topology control problems The key ea of the mult-agent coornate control strategy propose n ths work s to leverage the stratfcaton of the flow envronment n orer to establsh an mantan a egree of control over the topology of the swarm Leveragng nonlnear moel prectve control (MPC) an the lnear quaratc regulator (LQR), new methos for the actve control of the spatal strbuton of several such buoyancyactuate agents have been evelope Ths buoyancy-control-n-stratfe-flow approach extens naturally to unerwater vehcles n coastal systems, as the coastal ocean s also characterze by a flow fel wth sgnfcant varablty n both space an tme II BACKGROUND To begn, the backgroun stratfe flow (Fg 1) s taken as a smple pressure-rven (Poseulle) flow between two Y1 Y Y-1 Fg 1 Flow veloctes of the basc flow consere n ths paper, whch s pressure-rven n the x recton an shear rven n the z recton mpermeable nfnte plates wth no-slp bounares n the streamwse recton x, shear-rven (Couette) flow n the spanwse recton z, an no flow n the vertcal recton y The channel half-wh an maxmum flow velocty n x an z are rescale to unty wthout loss of generalty Startng from ths physcal escrpton of the flow system n Cartesan coornates, the state of each agent s efne as ts poston n 3-space relatve to an arbtrarly-chosen orgn n the streamwse an spanwse rectons, an the channel centerlne n the vertcal recton Mathematcally, the agent forcng terms (non-mensonal relatve veloctes) can then be escrbe as a functons of the vertcal poston, y, an the vertcal velocty, f; that s, for the th agent: x y y + 1 f, (1) z y where f s efne as the system nput It s assume that the vertcal actuaton of the agent occurs suffcently quckly, relatve to the spee of the backgroun flow, that the agent can move vertcally (wthn the vertcal confnes of the channel) before movng sgnfcantly n the streamwse or spanwse rectons (state another way, we assume that the channel half-wh s small wth respect to the horzontal stances of nterest) Ths smplfcaton of the backgroun flow retans the stratfe nature of many natural flows, an presents enough nonlnear characterstcs n the moel to make the problem nterestng, ffcult, an relevant n the conseraton of harer problems n the envronment In the nterest of unform samplng, we focus on the relatve separaton of agents throughout the volume, rather than the nvual locaton of each agent n absolute coornates We thus turn our attenton to begn wth the relatve Y X Z /1/$6 1 IEEE 99

2 Y +1 Y -1 Z Fg Flow velocty profles of an alternatve flow confguraton cf Fg 1] resultng n the same smplfe nonlnear moel compare (4) to (7)] separaton of two such agents III PROBLEM FORMULATION As mentone above, we now focus on two-agent systems, 1,], an the agent strbuton s escrbe n terms of the relatve horzontal separaton of these two agents We thus ntrouce a change of varables (COV) whch focuses specfcally on ths separaton Startng from the ntal state varables ntrouce n (1), an ntroucng the specfe separaton target stances C x an C z, two new COV states ˆx an ŷ are efne such that: ˆx x 1 x +C x, ẑ z 1 z +C z X (a) (b) Note that ˆx ẑ correspons to the agents beng at the specfe separaton Now efne the ntermeate control varables u 1 an u as lnear combnatons of the agents vertcal postons, ] ] u1 y1 y, (3) u y 1 y an take the tme ervatve of the COV states equaton (), a smple nonlnear escrpton of the agent-separaton system s foun: ˆx ẑ] ] x1 x y 1 + y ] ] u1 u (4) z 1 z y 1 y u Interestngly, the nonlnear moel gven n (4) may also be erve for an alternatve flow confguraton wth a halfparabolc velocty profle n the streamwse recton an a lnear velocty profle n the spanwse recton (Fg ) In ths case, the state equaton governng the movement of agents n the system s gven by cf (1)]: x y z y y f y (5) Usng the same COV for the separaton of the agents as propose n (), reefnng the ntermeate control varables u 1 an u such that cf (3)] ] ] u1 y1 y, (6) u y 1 y an followng the same fferentaton processes as use before, t s agan foun cf (4)] that ˆx (y1 y ) y ẑ] 1 + ] ] y u1 u (7) y 1 y u That s, the same nonlnear moel s obtane, though the ntermeate control varables, u 1 an u, are efne fferently Curously, lnearzaton of (4) equvalently, (7)], takng u 1 ū 1 + ū 1, u ū + ū, etc, about the nomnal soluton (wth ū 1 ū, assumng prme quanttes are small) results mmeately n the observaton that ˆx / ; that s, ˆx s lnearly uncontrollable Some fnte oscllaton of u s requre n orer to gve control authorty on ˆx va the control varable u 1 Ths s an mmeate ncaton of the elcateness of the present system Despte the loss of lnear controllablty n ths lmt, t s seen that there s n fact a lot of control authorty n ths problem f fnte vehcle motons are allowe, as explore n the balance of ths paper For smplcty, the remaner of ths paper focuses on the flow escrbe n Fgure 1 an moele n the lamnar flow case by (1), wth the relevant COV efne by () an ntermeate control varables efne by (3) IV MODEL PREDICTIVE CONTROL The ntermeate control varables u 1 an u are relate to the tme ervatves of the agent nputs f 1 an f ; thus, u 1 an u shoul be kept suffcently smooth such that f 1 an f are suffcently small By ntroucng another change of varables, v u /, an penalzng an energy measure of the v n the control formulaton, ths s realy acheve: u1 u ] v1 v ] ] f1 f (8) f 1 f Fnally, the state vector, q, an control vector, v, are efne n orer to facltate the scusson of the relevant state-space formulaton of the present problem: ˆx q ẑ u 1 u, v v1 v ] (9) The constructon of the change of varables s closely te to the formulaton of an assocate cost functon, whch, when mnmze, rves the system to behave n a esre fashon Usng a quaratc moel that penalzes state an nput values, rvng the cost towar keeps physcal nputs small an rves agent separaton to specfe parameters (e x x 1 C x an z z 1 C z ) Thus, the cost functon, J(q,t), s efne wth a state weghtng matrx Q q, nput weghtng matrx Q v, an termnal-tme weghtng matrx Q T : J 1 ( q T Q q q+v T Q v v ) + 1 qt (T)Q T q(t) (1) An nput sequence s etermne usng an ajont-base teratve optmzaton technque to mnmze the cost functon wth respect to the nput 4] 5] Ths teratve scheme creates a moel-base prectve controller (MPC), whch s solve by a tme-base nput sequence v(t) that reuces the cost functon over a tme wnow (n ths case rvng the states towar an mnmzng the assocate nput costs) 991

3 Startng from the nonlnear state space equatons presente n 4, 8, an 9, the tangent lnear equaton of the system s foun by replacng the state wth perturbe state varables (such as ˆx ˆx+ ˆx ) an solvng for the perturbe varables: ˆx u u ẑ 1 + u 1u u v 1 v u 1 u q Aq + Bv (11) Aonally, ong the same perturbaton analyss to the cost functon, (1), a small change n the cost can be wrtten J ( q T Q q q + v T Q v v ) + q T (T)Q T q (T) (1) Ths perturbaton of the cost functon can be efne n terms of a graent wth respect to the system nput usng an ajont base analyss Frst efnng the weghte nner prouct, a,b R T ah b, an then usng an ajont entty: r,lq L r,q +b, (13a) whch s equvalent to r T Lq (L r) T q + b, (13b) for some, as yet, unefne operator, L, an ajont varable, r Then choosng to relate the state wth the nput by choosng Lq Bv, (14) the operator, L, s foun by pluggng nto (11) an solvng to get: L A (15) Returnng ths result to the ajont entty (13b), r T ( r A ) q (L r) T q + b, (16) whch can then be ntegrate by parts ( ) Z r T r T q T rt A q (L r) T q + b (17) whch means b r T q T (18a) ( ) L r + AT r (18b) Pluggng Eqs 14 an 18a nto the ajont entty, (16): r T Bv (L r) T q + r t q T (19) Now comparng (19) to the perturbe cost functon, (1), an then choosng to efne: an L r Q q q () r Q T q (1) the ajont entty s agan re-wrtten: R T rt Bv R T (Q qq) T q +(Q T q) T q R T qt Q q q q T Q T q () Ths means that the prerturbe cost functon can then be wrtten n terms of the left-han-se of (), resultng n a escrpton of the cost functon as an equaton of a perturbaton of the system nput, v, an the ajont varable, r: J B T r+q v v ] v (3) Whch leas to, va the efnton of the nner prouct, a graent of the cost functon wth respect to the system n put as a fferental equaton of the form: J DJ DJ Dv,v where Dv BT r+q v v (4) Resolvng the efnton of the ajont varable, r, (18) s combne wth the chose values of Eqs 18b an 1 such that: L r Q v r r A H r+q v v (5a) for < t < T, whch s subject to the ntal conons: r Q T q at t T (5b) Startng from the termnal efnton of the ajont fel, r Q T q, the graent s etermne va a backwars march, from t T, usng an RK4 march Usng a conjugate graent escent metho, n conjuncton wth a Brent lne search, a escent recton an a step sze are foun Movng the nput ownhll creates a new nput sequence, whch s then use to etermne a new, lower cost, trajectory path whch can form the bass of the next mnmzaton teraton Upon satsfacton of a prescrbe convergence crtera, the resultant COV nput sequence, v(t), satsfes a local mnmzaton of the cost functon Usng ths sequence an the nvertble relatonshp foun n (8), the physcal agent nput sequences, f 1 (t) an f (t), are etermne Applyng ths sequence to the actual -agent equatons of moton escrbe by (1), the eal physcal trajectores of the agents are foun Ths trajectory s efne as the screte, tme epenent ) state { x k, yk, zk } on the scretze nterval k 1, T h for each tme step, k, each agent,, a termnal tme, T, an a step sze, h Startng wth both agents at the orgn an ntally specfyng C x, C z 1, Q q, Q v, an Q T 1I (where I s an approprately sze entty matrx), separaton s acheve n the spanwse recton usng penalty terms that only weght termnal poston wthout regar towar the cost of ntermeate nput values (Q v ) or state postons (Q q ) Results of such ealze separaton etermne though MPC analyss are shown n Fg 3 In aonal to cross-flow separaton, changng the C x separaton constant allows control trajectores to be evelope that separate agents n both the x an z rectons (as shown 99

4 Depth, y x 1 3 Cross stream, z Downstream, x Agent Fg 3 Unt separaton n the spanwse, z, recton of a -agent system subject to ealze open channel flow ncates startng locaton an ncates fnal poston Intal postons at the orgn wth C x, C z 1, Q q, Q v, an Q T 1I Heght (y) Agent Fg 4 Separaton constants of C z 1 an C x 5, recton anof a -agent system subject to ealze open channel flow ncates startng locaton an ncates fnal poston Intal postons at the orgn wth C x, C z 1, Q q, Q v, an Q T 1I n Fg 4 whch uses the same ntal conons as Fg 3 but wth C x 5) It s worth notng that the eal nput values o not ve the agents back to the centerlne of the channel n the shown control sequence; whch was alreay note to be an unstable result To correct for ths problem, aonal gan can be ae to the control sequence (va ncreasng the Q v gan matrx) whch woul cause the control sequence to approach as small a evaton from the centerlne as possble over the trajectory wnow V TRAJECTORY TRACKING VIA LQR FEEDBACK Uner ealze crcumstances, the MPC analyss of trajectory generaton proves enough nformaton to reach a local mnmzaton of the cost functon an track to a reasonable soluton to the separaton problem The result, however, cannot account for varaton n startng locaton or 15 ntermttent perturbaton of the state urng the actual state evoluton The open loop controller s not enough to satsfy stablty uner more realstc flow parameters Introucton of a close-loop feeback stablzaton metho s necessary n orer to reject sturbances n backgroun nose an correct for varaton n state poston Usng the soluton to the lnear-quaratc regulator (LQR) problem, an optmal control feeback gan s foun whch can be use to stablze the system In orer to apply a lnear feeback, the system must be lnearze about the preferre state In the -agent system, each agent s lnearze about the eal trajectory evelope wthn the MPC framework Defnng the LQR-state, G, an nput, F, as: x 1 y 1 z 1 ị x k G y k z k ị x n y n z n ( y 1 ) + 1 f 1 y 1 G ( y k ) + 1 f k, F y k ( y n ) + 1 f n y n f 1 f k f n, (6) over the sequence of eal trajectory locatons, { x k, yk, zk }, an nputs, f k, for each agent erve by the MPC soluton over the tme sequence t 1,,,k,n] for n T h The trajectory-lnearze system s then expresse n the form G / A G + B F, where A an B are the assocate Jacoban matrces for each agent, : a 1 a A, a j y j, (7) 1 a n an b 1 b B, b j 1 (8) b n The feeback gan s solve va the tme-epenent soluton to the fferental Rccat equaton: X A H X + X A X B Q 1 B ˆf H X + Q X where X (T) Q term (9) The solutons, X X (t), are etermne va a backwarsn-tme march usng the RK4 scheme an the feebackaugmente nput, ˆf, s gven by the feeback gan, K, multple by the fference between the evolvng actual state poston gven n (1) an the a vector of MPC-erve eal state poston: 993

5 ˆf K x y x y z z where K Q 1 B ˆf H X (3) 5 Agent Once agan, the strength of the feeback gan can be controlle va the relatve magntues of the agonal weghtng matrces Q ˆf, a weght on the cost of the nput, Q X, a weght on the ntermeate perturbe state, an Q term, a weght on the termnal postonal varaton from eal The structure of the A an B matrces (7) an 8, respectvely] allow the Rccat equaton to be solve as a set of n T h smaller fferental equatons by marchng the system Heght (y) X k ) T ( ) T X k ak (a + k X k X k b k Q kˆf bk X k k + Q X (31) from t T k for an ncremente tme nterval k,h,h,,t h,t] Then solvng for the tme-nterval, k, epenent feeback gan, K k, an nput, ˆf k ˆf k K k where x k y k z k K k Q 1 ˆf x k y k z k : B H X k (3) Usng ths trajectory-lnearze feeback control, varatons n ntal conons or backgroun velocty flows can be neutralze Ths feeback effectvely tracks each agent back to ts MPC eal path but as computatonal cost wth the one tme marchng of the set of systems to etermne the value of K k for each tme nterval, k Startng wth a perturbe ntal state {x 1,y 1,z 1 1} an {x y z }, an seekng a unt separaton n the cross-stream, z, recton (C x an C z 1), the feeback gan successfully tracks astray agent 1 back to the eal trajectory as shown n Fg 5 The feeback gan also allows ntermttent backgroun sturbance rejecton Startng wth both agents at the orgn an seekng unt cross-stream separaton (as n Fg 3), the agent s feeback augmente nput s able to correct for the applcaton of an aonal shear term ( z/ z/ + 1) at t 1 an return to the eal path as shown n Fg 6 More mportantly, the soluton to the LQR problem proves sturbance rejecton for ranom nose that may occur contnually n the backgroun flow an agent nputs Moelng varaton n the state veloctes s acheve by augmentng (1) wth scale zero mean whte nose, w x/, w f, an w z/ at each screte functon call n the RK4 march In ths llustratve example usng LQR, the ae nose nee not necessarly be Gaussan: x k y k ( yk ) X k f z k k + Y w f y k + Z kwk z/ w k x/ (33) Fg 5 Trajectory trackng for a perturbe ntal state wth eal trajectores seekng unt separaton n the spanwse, z, recton of a -agent system subject to ealze channel flow ncates startng locaton an ncates fnal poston Intal poston {x 1,y 1,z 1 1} an {x,y,z } wth C x, C z 1, Q q, Q v, an Q T 1I Heght (y) x Agent Fg 6 Trajectory trackng for an ntermttently excte backgroun flow wth eal trajectores seekng unt separaton n the spanwse, z, recton of a -agent system subject to ealze channel flow ncates startng locaton an ncates fnal poston Intal postons at the orgn wth C x, C z 1, Q q, Q v, an Q T 1I The scalng factors, X k ( y k ) + 1 an Z k y k, are use to ensure that the maxmum value of the nose term s the same sze as the eal backgroun flow at the partcular epth y k Scalng n the vertcal velocty, y/, s one wth a constant such that Y maxf for the nput sequence, F, etermne va (6) Applyng such a scale nose to backgroun flow an agent nput, the feeback gan s suffcently powerful to contnually aapt the nput sequence of each agent so as to raw t back to the eal MPC trajectory as shown n Fg 7 VI CONCLUSIONS The use of buoyancy controlle agent moton, n conjuncton wth a known stratfe backgroun flow envron

6 Heght (y) x Agent Fg 7 Trajectory trackng for a scale zero mean whte nose excte backgroun flow wth eal trajectores seekng unt separaton n the spanwse, z, recton of a -agent system subject to ealze open channel flow ncates startng locaton an ncates fnal poston Intal postons at the orgn wth C x, C z 1, Q q, Q v, an Q T 1I ment, proves enough control authorty to create separaton between several agents n the same flow Several types of backgroun flow can be smlarly characterze, an calculate solutons to the control problem unquely mappe back to form fferent physcal solutons uner the same mathematcal guse Lmte to a efne tme wnow, an teratve approach s use to solve the general separaton problem Defnng an ajont of the agent state as presente n (5a), a graent of the cost functon wth respect to the current nput s etermne n (1) Usng a escent metho to etermne an approprate step sze, the cost functon s mnmze by movng the nput, v, by the resultant step sze an recton The process s repeate as necessary to ensure convergence of the functon, J, to a local mnmum The soluton to the MPC problem (11) s a vector of nputs to the COV system, v, whch mnmze the esre cost functon Ths resultant COV nput s then mappe back to the esre backgroun flow, va (8), an a sequence of actual agent nputs s recovere Uner the eal backgroun flow consere n the mathematcal constructon of the MPC problem n (1), ths nput results n the esre spacng of the agents n the streamwse an spanwse rectons Ths soluton works uner an ealze backgroun flow, s purely open loop, an has no sturbance rejecton capablty By lnearzng the state of each agent about ts MPCerve eal trajectory as one n (7) an (8), a feeback gan s calculate by solvng the fferental Rccat equaton presente n (9) an (3) Augmentng the MPC-erve nput va ths LQR gan, perturbe system states, varaton n startng locatons, an ntermttent nose n backgroun flows are correcte for wthn the orgnal trajectory tmeframe (Fgs 5, 6, an 7 respectvely) 5 VII FUTURE WORK The major cost of the presente MPC calculaton s the Runge-Kutta march of the state an ajont over the tme wnow Each teraton of the cost functon optmzaton nvolves both a full state an a full ajont march over the wnow n orer to prouce a graent Therefore, t s mportant that the numercal processes assocate wth the cost functon mnmzaton presente n Eqs 1-5 lmt, as much as possble, the requre number steps to reach a mnmum value Whle the conjugate graent escent metho n conjuncton wth a Brent lne search metho s aequate, t can requre a large number of functon calls an be computatonally expensve There are several numercal methos of nterest, notably varatons on reuce memory versons of the Broyen-Fletcher-Golfarb-Shanno (BFGS) escent metho 7], whch have been shown n prelmnary results requre substantally fewer functon evaluatons uner smlar mnmzaton accuracy constrants The current formulaton of the LQR feeback s base on the assumpton of perfect state knowlege Relaxng ths assumpton, a lnear-quaratc Gaussan (LQG) controller can be formulate by moelng usng the Kalman estmate n place of the state n the LQR problem formulaton 6] Usng rect numercal smulaton of ncompressble Naver-Stokes, smulatons wth equvalent flow structures to the eal case presente n (1) can be apple rangng from lamnar flow to fully turbulent mxng Applyng the MPC/LQG framework to these more complcate flows wll gve greater unerstanng of both the tme wnow neee for separaton an the relatve weghtngs of all the control parameters Ths, n turn, forms the bass for expanng the flow fel beyon the channel lmtatons, an applyng the metho to full oceanc/atmospherc flow fels VIII ACKNOWLEDGEMENTS The authors woul lke to thank Paul Beltz, Joseph Cessna, Chrstopher Colburn, an Dav Zhang for helpful scussons relate to ths work REFERENCES 1] Davs, R E (1991) Lagrangan Ocean Stues Annual Revew of Flu Mechancs, vol 3, pp ] Busnger, S, R Johnson & Talbot (6) Scentfc Insghts from Four Generatons of Lagrangan Balloons n Atmospherc Research Bulletn of the Amercan Meteorology Socety vol 87, no 11, pp ] Soeterboek, R (199) Prectve Control: a Unfe Approach Prentce Hall 4] Btmea, R R, M Gevers, & V Wertz (199) Aaptve Optmal Control: The Thnkng Mans GPC Prentce Hall 5] Bewley, T, P Mon & R Temam (1) DNS-base prectve control of turbulence: an optmal benchmark for feeback algorthms Journal of Flu Mechancs, vol 447, pp ] Fortmann, T & K Htz (1977) An Introucton to Lnear Control Systems Mercel Dekker, Inc 7] Gll P E & M W Leonar (3) Lmte-Memory Reuce-Hessan Methos for Large-Scale Unconstrane Optmzaton SIAM Journal on Optmzaton, vol 14, ssue, pp