Subway saions energy and air qualiy managemen wih sochasic opimizaion Trisan Rigau 1,2,4, Advisors: P. Carpenier 3, J.-Ph. Chancelier 2, M. De Lara 2 EFFICACITY 1 CERMICS, ENPC 2 UMA, ENSTA 3 LISIS, IFSTTAR 4 June 6, 2016 Saions Opimal Managemen June 6, 2016 1 / 30
Opimizaion for subway saions Paris urban railway ranspor sysem energy consumpion = 1 subway saions + 2 racion sysem 3 3 Subway saions presen a signicanly high pariculae maers concenraion We use opimizaion o harves unexploied energy ressources and improve air qualiy. Saions Opimal Managemen June 6, 2016 2 / 30
Ouline 1 Subway saions opimal managemen problem Energy Air qualiy Energy/Air managemen sysem 2 Two mehods o solve he problem We are looking for a policy Dynamic programming in he non Markovian case Model Predicive Conrol 3 Numerical resuls Random variables modeling Mehods Resuls Saions Opimal Managemen June 6, 2016 3 / 30
Ouline 1 Subway saions opimal managemen problem Energy Air qualiy Energy/Air managemen sysem 2 Two mehods o solve he problem We are looking for a policy Dynamic programming in he non Markovian case Model Predicive Conrol 3 Numerical resuls Random variables modeling Mehods Resuls Saions Opimal Managemen June 6, 2016 3 / 30
Energy Saions Opimal Managemen June 6, 2016 3 / 30
Subway saions ypical energy consumpion Saions Opimal Managemen June 6, 2016 4 / 30
Subway saions have unexploied energy ressources Saions Opimal Managemen June 6, 2016 5 / 30
Energy recovery requires a buer Saions Opimal Managemen June 6, 2016 6 / 30
Air qualiy Saions Opimal Managemen June 6, 2016 6 / 30
Subways arrivals generae pariculae maers Rails/brakes wear and resuspension increase PM10 concenraion Recovering energy improves air qualiy Saions Opimal Managemen June 6, 2016 7 / 30
Energy/Air managemen sysem Saions Opimal Managemen June 6, 2016 7 / 30
Subway saion microgrid concep Saions Opimal Managemen June 6, 2016 8 / 30
Objecive: We wan o minimize energy consumpion and paricles concenraion A parameer λ measures he relaive weighs of he 2 objecives: T Cos (E Supply Saion + E Supply Baery ) +λ CP In =0 }{{}}{{} Grid supply PM10 We should conrol he sysem every 5 seconds Saions Opimal Managemen June 6, 2016 9 / 30
We conrol he baery Saions Opimal Managemen June 6, 2016 10 / 30
Energy sysem equaions Sae of charge dynamics: Soc +1 = Soc 1 U B + ρ c (U B+ ρ } dc {{} Discharge + E Surplus Train ) } {{ } Charge We have consrains on he baery level o ensure good ageing: Soc Min Soc Soc Max We ensure he supply/demand balance a saion and baery nodes: U B+ + E Surplus Train }{{} Charge ESaion Demand }{{} Saion consumpion = E Supply Baery }{{} Grid supply = E Supply Saion }{{} Grid supply + ETrain Recovered }{{} E Train Available E Train Excess + U B }{{} Discharge Saions Opimal Managemen June 6, 2016 11 / 30
We conrol he saion venilaions Saions Opimal Managemen June 6, 2016 12 / 30
Paricles concenraion dynamics In he indoor air and on he oor CP In In +1 =C + ρr P + d V Venil S C P Floor }{{} Resuspension Ou (CP C In P ) } {{ } In/Ou exchange ρd V C P In + Q P }{{}}{{} V Deposiion Generaion CP Floor +1 =C Floor P + ρd S C P In }{{} Deposiion ρr V C P Floor }{{} Resuspension Saions Opimal Managemen June 6, 2016 13 / 30
We have many uncerainies Le W he random variables vecor of uncerainies a ime : Regeneraive braking : ETrain Available Saion consumpion : ESaion Demand Cos of elecriciy : Cos Paricles generaion : Q P Resuspension rae : ρ R Deposiion rae : ρ D Oudoor paricles concenraion : CP Ou Saions Opimal Managemen June 6, 2016 14 / 30
Summary of he equaions Sae of he sysem: X = Conrols: U = And he dynamics: U B U B+ d Venil, Soc C In P CP Floor X +1 = f (X, U, W +1 ) We add he non-anicipaiviy consrains: U σ(w 1,..., W ) Saions Opimal Managemen June 6, 2016 15 / 30
We se a sochasic opimal conrol problem s. ( min E T Cos (E Supply U U Saion + E Supply Baery ) + λcp ) } In Objecive =0 Soc +1 = Soc 1 U B + ρ c (U B+ ρ dc C In P +1 = C In + ρr P + d V Venil Ou (CP C P In ) S C P Floor ρd V C P In + Q P V C Floor P U B+ = C Floor +1 P + ρd S C In + E Surplus Train = E Supply ESaion Demand = E Supply Saion + UB Soc Min Soc Soc Max C In P 0 P ρr } + E Surplus Train ) Baery dynamics V C P Floor Baery + ETrain Recovered Paricles dynamics } } Supply/demand balance Consrains Saions Opimal Managemen June 6, 2016 16 / 30
Compac sochasic opimal conrol problem We obained a sochasic opimizaion problem consisen wih he general form of a ime addiive cos sochasic opimal conrol problem: min E X,U ( T 1 ) L (X, U, W +1 ) + K(X T ) =0 s.. X +1 = f (X, U, W +1 ) U σ(x 0, W 1,..., W ) U U Saions Opimal Managemen June 6, 2016 17 / 30
Ouline 1 Subway saions opimal managemen problem Energy Air qualiy Energy/Air managemen sysem 2 Two mehods o solve he problem We are looking for a policy Dynamic programming in he non Markovian case Model Predicive Conrol 3 Numerical resuls Random variables modeling Mehods Resuls Saions Opimal Managemen June 6, 2016 17 / 30
We are looking for a policy Saions Opimal Managemen June 6, 2016 17 / 30
Wha is a soluion? In he general case an opimal soluion is a funcion of pas uncerainies: U σ(x 0, W 1,..., W ) U = π (X 0, W 1,..., W ) This is an hisory-dependen policy In he Markovian case (noises ime independence) i is enough o limi he search o sae feedbacks: U = π (X ) Saions Opimal Managemen June 6, 2016 18 / 30
Backward inducion: Sochasic Dynamic Programming In he Markovian case we inroduce he value funcions: ( T 1 ) x X, V (x) = min E L (X, π (X ), W π +1) + K(X T ) Algorihm s. = X = x and dynamics Oine: We compue he value funcions by backward inducion using Bellman equaion, knowing he nal cos: ( ) V (x) = min E L (x, u, W +1 ) + V +1 (f (x, u, W +1 )) u U Online: We compue he conrol a ime using he equaion: ( ) u arg min E L (x, u, W +1 ) + V +1 (f (x, u, W +1 )) u U Saions Opimal Managemen June 6, 2016 19 / 30
Dynamic programming in he non Markovian case Saions Opimal Managemen June 6, 2016 19 / 30
Dynamic programming in he general case Bellman equaion does no hold in he non Markovian case. Le P be he probabiliy s. (W ) [ 1,T ] are ime independen bu keep he same marginal laws. Algorihm Oine: We produce value funcions wih Bellman equaion using his probabiliy measure: ( ) Ṽ (x) = min u U E P L (x, u, W +1 ) + Ṽ +1 (f (x, u, W +1 )) Online: We plug he compued value funcions as fuure coss a ime : ( ) u arg min L u U E P (x, u, W +1 ) + Ṽ +1 (f (x, u, W +1 )) Saions Opimal Managemen June 6, 2016 20 / 30
Remarks Wih P he probabiliy updaing W +1 marginal law aking ino accoun all he pas informaions: i, W i = w i. I is a ool o produce hisory-dependen conrols. If he (W ) 1..T +1 are independen he conrols are opimal and P = P Sochasic Dynamic Programming suers he well known "curse of dimensionaliy". Saions Opimal Managemen June 6, 2016 21 / 30
Model Predicive Conrol Saions Opimal Managemen June 6, 2016 21 / 30
Rollou algorihms To avoid value funcions compuaion we can plug a lookahead fuure cos for a given policy: ( ) u arg min E L (x, u, W +1 ) + J+1(f π (x, u, W +1 )) u U I gives he cos of conrolling he sysem in he fuure according o he given policy: ( T 1 x X +1, J+1(x) π =E s. =+1 ) L (X, π (X ), W +1) + K(X T ) X +1 = x, and he dynamics Saions Opimal Managemen June 6, 2016 22 / 30
Model Predicive Conrol Choosing π in he class of open loop policies minimizing he expeced fuure cos: i + 1, u i R n, x, π i (x) = u i u arg min u U min (u +1,...,u T 1 ) E ( L (x, u, W +1 ) + T 1 =+1 ) L (X, u, W +1) Wih E replacing noises by forecass, we obain a deerminisic problem. Algorihm Online: A every MPC sep, compue a forecas ( w +1,..., w T +1 ) using he observaions i, W i = w i. Then compue conrol u : u arg min u U min L (x, u, w +1 ) + (u +1,...,u T 1 ) T 1 =+1 L (x, u, w +1) MPC is ofen dened wih a rolling horizon. Saions Opimal Managemen June 6, 2016 23 / 30
Ouline 1 Subway saions opimal managemen problem Energy Air qualiy Energy/Air managemen sysem 2 Two mehods o solve he problem We are looking for a policy Dynamic programming in he non Markovian case Model Predicive Conrol 3 Numerical resuls Random variables modeling Mehods Resuls Saions Opimal Managemen June 6, 2016 23 / 30
Random variables modeling Saions Opimal Managemen June 6, 2016 23 / 30
Some random variables are aken deerminisic Regeneraive braking : ETrain Available Saion consumpion : ESaion Demand Cos of elecriciy : Cos Paricles generaion : Q P Resuspension rae : ρ R Deposiion rae : ρ D Oudoor paricles concenraion : CP Ou Saions Opimal Managemen June 6, 2016 24 / 30
Sochasic models We have muliple equiprobable scenarios: Braking energy and ouside PM10 concenraion every 5s We deduce he discree marginal laws from hese scenarios. Saions Opimal Managemen June 6, 2016 25 / 30
Deails on he mehods Sochasic Dynamic Programming: We compue value funcions every 5s. We can compue a conrol every 5s. The algorihm is coded in Julia. Model Predicive Conrol: The deerminisic problem is linearized leading o a MILP. I is solved every 15 min wih a 2 hours horizon. We use wo forecass sraegies: MPC1: Expecaion of each noise ignoring he noises dependence MPC2: Scenarios where he nex ouside PM10 concenraion is no oo far from he previous one Saions Opimal Managemen June 6, 2016 26 / 30
Resuls Saions Opimal Managemen June 6, 2016 26 / 30
Air qualiy simulaion Saions Opimal Managemen June 6, 2016 27 / 30
Energy recovery lower peaks Saions Opimal Managemen June 6, 2016 28 / 30
Energeic resuls Assessor: 50 scenarios of 24h wih ime sep = 5 sec Reference: Energy consumpion cos over a day wihou baery and venilaion conrol MPC1 SDP Oine comp. ime 0 12h Online comp. ime [10s,200s] [0s,1s] Economic savings -26% -31% On jus one assessmen scenario for he momen: MPC1: -22.3% MPC2: -22.7% SDP: -26.8% Saions Opimal Managemen June 6, 2016 29 / 30
Conclusion & Ongoing work Our sudy leads o he following conclusions: A baery and a proper venilaion conrol provide signican economic savings SDP provides slighly beer resuls han MPC bu requires more oine compuaion ime We are now focusing on: Using oher mehods ha handle more sae/conrol variables Taking ino accoun more uncerainy sources Calibraing air qualiy models for a more realisic concenraion dynamics behavior Ulimae goal: apply our mehods o laboraory and real size demonsraors Saions Opimal Managemen June 6, 2016 30 / 30