Way nding under Uncertainty in Continuous Time and Space by Dynamic Programming

Transcription

1 Way ndng under Uncerany n Connuous Tme and Space by Dynamc Programmng Dr. Serge P. Hoogendoorn (s.hoogendoorn@c.udelf.nl) Transporaon and Tra c Engneerng Secon Faculy of Cvl Engneerng and Geoscences - Delf Unversy of Technology PO Box GA Delf - The Neherlands 1 Inroducon Roue choce models generally descrbe how ndvduals choose beween a dscree number of roues beween her orgn and desnaon. Ths also when he hypohess of a ne number of avalable roues s no jus ed, e.g. pedesrans choosng her pah hrough a walkng facly, or opmal way ndng of Auomaed Guded Vehcles (AGV s) on non-dedcaed nfrasrucure (non-nfrasrucure bound). In hese cases, he subjecs can choose beween an n ne number of alernaves. Neverheless, n mos cases he roue choce problem s solved by choosng a ne number of roues hrough he facly, and consderng he resulng dscree problem ([1], [3], [4]). Clearly, such approxmaons may no be bes pracse. Ths paper dscusses subjecve ndvdual way ndng modellng n connuous me and space under unceran condons. Here, roues are descrbed by connuous pahs hrough he facly. The uncerany generally perans o he ra c condons expeced by he subjec. To descrbe he choce behavour, s assumed ha he pahs are chosen ha enal mnmal subjecve dsuly. Ths dsuly (or cos) can re ec among oher hngs roue ravel me, cos of geng oo close o obsacles and walls, number of sharp urns or rapd dreconal changes, expeced number of neracons wh oher subjecs, smulaon of he envronmen, ec. I urns ou ha he resulng problem can be solved by applcaon of he dynamc programmng prncple of Bellman [2], mplyng ha o solve he roue choce problem a second-order paral d erenal equaon needs o be solved. The paper descrbes boh numercal soluon approaches and applcaons of he approach o pedesran way ndng. 2 Problem formulaon Le A denoe he walkng area, and le x 2 A denoe he locaon of a subjec (e.g. a pedesran, or an AGV). To descrbe he roue choce behavour of he subjec, we hypohesse ha he uses an nernal model o esmae and predc he experenced roue coss. To hs end, he subjec wll esmae he curren poson a nsan, re eced by ^x. Moreover, he parcpan wll predc hs fuure posons x() for > 0 by usng an nernal predcon model dx = vd + ¾dw subjec o x() = ^x (1) where v = v( ) denoes velocy (.e. speed and drecon) of he subjec for >. In hs formulaon, w denoes a sandard Wener process, meanng ha for very small me perods [; + h), he ncrease w( + h) w() s a N(0;hI m )-dsrbued random varae, where I m denoes he m m deny marx; ¾ = ¾(x;v) s a 2 m marx, re ecng he way n whch he whe nose vecor w a ecs he locaon. Noe ha ¾ (w( + h) w()) s N(0;h¾¾ 0 )- dsrbued. The sochascy re ecs he uncerany n he expeced ra c condons and

2 he resulng e ecs on he subjec s knemacs. Ths uncerany sems from among oher hngs he lack of experence, and observably of prevalng ra c condons. Equaon (1) descrbes he predced pah x [;T) resulng from applyng he veloces v( ) durng he perod [;T). The ermnal poson x(t) s a random varae, whose dsrbuon depends on he appled veloces. Gven each velocy pah v [;T) and (esmaed) nal poson x() = ^x, we can deermne he expeced coss of applyng he conrol as follows Z T J(; ^x;v [;T) ) = E L( ;x( );v( ))d + Á(T;x(T)) where L denoes he so-called runnng coss and Á denoes he so-called ermnal coss. The runnng cos L( ;x( );v( ))d re ecs he coss ha are ncurred durng a very small me perod [ ; + d ), gven ha he subjec s locaed a x( ) and s applyng velocy v( ) o change he poson. Typcal examples are ravel me, coss ncurred by movng oo close o obsacles, and coss due movng a a ceran speed. The ermnal coss Á(T;x(T)) re ec he cos due o endng up a poson x(t) a he ermnal me T. These coss ypcally re ec he penaly ha may be ncurred when he subjec does no arrve a he desnaon areas D ½ A n me. Noe ha snce he roue coss are descrbed by means of an negral (2), he heory assumes ha he roue coss are addve (.e. he oal roue cos can be descrbed by he sum of he coss of s consuen subroues). We hypohesse ha he subjec ams o mnmse he subjecve expeced roue cos,.e. we am o deermne he opmal velocy v [;T) sasfyng v [;T) = arg mne Z T 3 Soluon approach L( ;x( );v( ))d + Á(T;x(T)) To solve he roue choce problem, le us de ne he so-called value funcon W(; ^x) by he expeced value of he coss upon applyng he opmal velocy v [;T) Z T W(; ^x) := E L( ;x ( );v ( ))d + Á(T;x (T)) subjec o (1). Now consder a very shor perod [; + h) of lengh h. We have and hus E Z +h L( ;x( );v( ))d = L(; ^x;v())h + O(h 2 ) (5) W (; ^x) = mn v [;+h) L(; ^x;v())h + E [W( + h;x( + h))] + O(h 2 ) ª (6) The random varable x( + h) can approxmae by x( + h) = ^x + hv() + ¾ p hw + O(h 3=2 ) (7) where w s a N(0;I m ) dsrbued random varable; W( + h;x( + h)) s also a random varable. Some sraghforward compuaons show ha E [W( + h;x( + h))] = W( + h; ^x + hv()) + h 2 X j (2) (3) (4) j j + O(h 3=2 ) ()

3 where = (x;v) = ¾(x;v)¾(x;v) 0. Subsuon () no (4), and subsequenly akng he lmh! 0 yelds he followng paral d erenal equaon < W(;x) = v : L(;x;v) + v + 1 X j W = (9) x j ; wh ermnal condons W(T;x(T)) = ½ Á x(t) 2 D J 0 elsewhere Á denoes he ermnal cos of arrvng a desnaon area D, and J 0 re ecs he penaly of no arrvng a eher of he desnaon areas D n me. The opmal velocy v sas es. 9 < v = arg mn : L(;x;v) + v + 1 X j W = (11) j ; Equaon (9) s he dynamc programmng equaon for he connuous sochasc dynamc user-opmal (CSDUO) pah choce problem. 4 Numercal soluon approach We can solve he dynamc programmng equaon (9) by dscresng he area A no small ± ±-cells, and consderng approxmae soluons on hs lace a xed me nsans k = h (.e. ± s he spaal sep sze, and h s he emporal sep sze). We can show ha he resulng problem s a Markov d uson process n wo dmensons wh neares-neghbor ransons ha are deermned by he sochasc d erenal equaons (1) (see [2]). Solvng hs (dscree) sochasc dynamc programmng problem s relaed o solvng equaon (9) by replacng he paral dervaves wh he approprae ne d erence quoens. Le e denoe he un vecor n he -h dmenson ( = 1; 2). Then, we denoe he forward ne d erence ( + x ) and backward ne d erences ( x ) by j (10) x W := ± 1 [W(;x ±e ) W(;x)] for = 1; 2 () For he second-order erms, we hen use he followng approxmaons 2 x W := ± 1 [W(;x + ±e ) 2W(;x) + W(;x ±e )]for = 1; 2 (13) x x j W : = 1 2 ± 2 [W(;x + (±e ±e j )) + 2W(;x) + W(;x (±e ±e j ))] (14) 1 2 ± 2 [W(;x + ±e ) + W(;x + ±e j ) + W(;x ±e ) + W(;x ±e j )] (15) In numercally approxmang (9), he followng numercal soluon approach s proposed [2]: W( h;x) = W(;x) hh(x; x W; 2 x W; x x j ) () where ( H(x; x W; 2 x W; x x j ) := mn L(;x;v) + X v + v + x W v x W + 1 X (x;v) 2 x 2 W + 1 X³ 9= + j 2 (x;v) + x x j W j (x;v) x x j W ; (17) wh b + := maxfb; 0g and b := mnfb; 0g. 6=j

4 5 Modellng pedesran behavour The prevous secons dscussed he CSDUO problem formulaon and numercal soluon approaches. Ths secon dscusses applcaon of he approach o modellng pedesran choce behavour. To hs end, we wll assume m = 2, ¾ = (x)i 2, where (x) s a scalar funcon of he poson x. An mporan facor deermnng pedesran roue choce are he presen obsacles. In hs example, obsacle l s descrbed by an area O l ½ A. Hoogendoorn and Bovy [5] hypohesse ha he runnng cos L can be expressed by a weghed sum,.e. L(x;v) = P c L (x;v). For llusraon purposes, we consder he componens L : 1. Travel me L 1 (x;v) = 1: 2. Dscomfor due o walkng oo close o obsacles L 2 = P l A le kx O lk=b l, where parameers A l and B l deermne he way n whch walkng oo close o obsacle l s valued (e.g. dependng on he surface of he obsacle); kx O l k denoes he mnmal Eucldean dsance from x o he area O l. 3. Energy consumpon L 3 = 1 2 hv;v. Usng equaon (11), can be shown easly ha he opmal velocy v equals v = 1 c 3 D x W(;x) (1) The paral dervave D x W(;x) can be nerpreed as he margnal cos of x. Equaon (1) shows ha he opmal velocy v s poned no he drecon e = v =kv k n whch he opmal coss W reduce mos rapdly. The opmal speed kv k depends on he rae kd x W(;x)k a whch he value funcon decreases n hs drecon. When hs rae s hgh, he opmal speed wll be equally hgh, and vce-versa. The dynamc programmng equaon (9) for he pedesran pah ndng W(;x) = c 1 + c 2 X l A l e kx O lk=b l 1 X 2c (x) 2 2 (19) Le us now llusrae he workngs of he approach by an example. The case consdered perans o Schphol Plaza, whch s a mul-purpose mul-modal ransfer saon. Fgure 1 depcs a snapsho of he mcroscopc smulaon model NOMAD developed a Delf Unversy of Technology. Le us consder a pedesran ha s locaed anywhere n Schphol Plaza and ams o walk o eher of he escalaors E6 or E7. To descrbe he roue choce behavour of he pedesran, he nfrasrucure was dvded no square cells of 0:25m 2, and he numercal soluon approach descrbed n secon 4 was appled. By dong so, we were able o sudy he e ec of uncerany on he pah choce behavour of he pedesran. For hs parcular example, we have assumed ha he level-of-uncerany s consan for all x, (x) = 0. Fgures 2-a and 2-b respecvely show he value funcon approxmaon for 0 = 0:01 and 0 = 0:25. The gures clearly show he d erences n W due o he varyng uncerany levels. Generally, uncerany causes smoohng of he value funcon. As a resul, he value funcon has a large value nearby obsacles and walls. Ths s caused by he fac ha a pedesran n no sure wheher he wll end up very near an obsacle n he near fuure, and hus ncur a very hgh cos. As a resul, we can observe ha he value funcon n very narrow passageways ends o ncrease very quckly.

5 V1 E1 E6 E2 E7 E3 E4 V2 E5 Fgure 1: Snapsho from he Schphol Plaza case usng he NOMAD mcrosmulaon model. Exs E1-E5 ndcae exs from Schphol Plaza; escalaors E6 and E7 ndcae exs o ran plaform. V1 and V2 depc he locaons of he newspaper vendors. The colors re ec pedesrans havng dsnc acvy schedules x2-axs (m) a) η = x1-axs (m) b) η = x1-axs (m) Fgure 2: Value funcons W(;x) for pedesrans leavng Schphol Plaza va escalaors for a) 0 = 0:01 and b) 0 = 0:25. Opmal pahs are perpendcular o so-value funcon curves.

6 Furhermore, we have skeched an opmal pah for each suaon o llusrae how pah choce can be e eced by hgh levels of uncerany. These pahs show ha when he uncerany s hgh, he pedesran wll be less nclned o use pahs ha raverse hrough narrow passageways, due o he hgher probably of havng o walk very near or even collde wh an obsacle (or oherwse experencng hgh delays). Le us nally remark ha n [5], he combned pah choce, desnaon / acvy area choce, and acvy schedulng problem s dscussed; Hoogendoorn and Bovy [5] also show how changng ra c condons can be ncluded n he modellng approach. 6 Conclusons and fuure research Ths paper descrbes an approach o model ndvdual roue choce behavour n connuous me and space under uncerany. Numercal soluon approaches were proposed o approxmae he dynamc programmng equaon. The approach s applcable o predc pahchoce behavour n nfrasrucure facles of realsc sze, as was shown by applcaon of he approach o pedesran behavour modellng n Schphol Plaza. Ths example was used o llusrae he e ec of uncerany on pedesran choce behavour. One of he man observaons s ha when uncerany ncreases, pedesrans are end o prefer roues gven hem wder berh. Ths can be explaned by observng ha when he condons become unceran, he lkelhood of experencng hgh coss on a narrow (.e. rsky) passageway ncreases. The paper focusses on roue choce behavour and uncerany peranng o ra c condons expeced by he pedesran. Oher ypes of uncerany, for nsance n he ermnal cos Á of arrvng a a ceran desnaon area (re ecng for nsance servce me, ec.) have no been consdered. Fuure research wll be dreced owards exendng he modellng approach o nclude oher ypes of uncerany. Acknowledgemen 1 Ths research s funded by he Socal Scence Research Councl (MaGW) of he Neherlands Organzaon for Scen c Research (NWO). References [1] DAAMEN, W., P.H.L. BOVY, AND S.P. HOOGENDOORN (01). Modellng Pedesrans n Transfer Saons. In: Pedesran and Evacuaon Dynamcs, Sprnger, [2] FLEMING, W.H., AND H.M. SONER (1993). Conrolled Markov Processes and Vscosy Soluons. Applcaons of Mahemacs 25. Sprner-Verlag. [3] GIPPS, P.G. (196) Smulaon of Pedesran Tra c n Buldngs. Schrfenrehe des Insus fuer Verkehrswesen 35, Unversy of Karlsruhe. [4] HAMACHER, H.W., AND S.A. TJANDRA (01). Mahemacal Modellng of Evacuaon Problems: A Sae of he Ar. In: Pedesran and Evacuaon Dynamcs, Sprnger, [5] HOOGENDOORN, S.P., AND P.H.L. BOVY (02). Pedesran Travel Behavor n Walkng Areas by Subjecve Uly Opmzaon. Transporaon Research Board Annual Meeng 02, Washngon; paper nr