A New Method for Marshaling of Freight Train with Generalization Capability Based on the Processing Time

Transcription

1 Proceedngs of the Internatonal MltConference of Engneers and Compter Scentsts 205 Vol I, A New Method for Marshalng of Freght Tran wth Generalzaton Capablty Based on the Processng Tme Yoch Hrashma Abstract Ths paper proposes a new marshalng method for assemblng an otgong tran from freght cars ncomng wth a random order. In the proposed method, marshalng plans based on the processng tme can be obtaned by a renforcement learnng system. Also, the ncomng freght cars are classfed nto several ``sb-tracks'' searchng better assgnment n order to redce the total processng tme. Then, the nmber of sb-tracks tlzed n the classfcaton s obtaned by learnng n order to yeld generalzaton capablty. In order to evalate the processng tme, the total transfer dstance of a locomotve and the total movement conts of freght cars are smltaneosly consdered. In the proposed method, each set of freght cars that have the same destnaton make a grop, and the desrable grop layot constttes the best otgong tran. Then, the order of movements of freght cars, the poston for each removed car, the layot of grops n a tran, the arrangement of cars n a grop and the nmber of cars to be moved are smltaneosly optmzed to acheve mnmzaton of the total processng tme for obtanng the desred arrangement of freght cars for an otbond tran. Index Terms Contaner Transfer Problem, Freght tran, Marshalng, Q-Learnng, Schedlng R I. INTRODUCTION alway transportaton has an mportant role n logstcs de to ts large transportaton capacty and small envronmental load. However, snce freght trans can transport goods only between ralway statons, modal shfts are reqred for area that has no ralway. Ths, the marshalng operaton at freght staton s reqred to ont several ral transports, or dfferent modes of transportaton ncldng ral. A freght tran s conssts of several freght cars, and each car has ts own destnaton. The tran drven by a locomotve travels several destnatons decoplng correspondng freght cars at each freght staton. In ntermodal transports ncldng ral, contaners carred nto the staton are loaded on freght cars and located at the freght yard n the arrvng order. The ntal layot of freght cars n the yard s determned by consderng both arrangement of ncomng tran and the arrvng order of the contaners carred by non-ral methods. Contaners carred nto the staton are loaded on freght cars and the ntal layot of freght cars s ths random. For Manscrpt receved Janary 8, 205; revsed Janary 0, 205. Y. Hrashma s wth Osaka Insttte of Technology, -79- Ktayama, Hrakata, Osaka, Japan (phone, fax: ; e-mal: yoch.hrashma@ot.ac.p. effcent shft n assemblng otgong tran, freght cars mst be rearranged before coplng to the freght tran. In general, the rearrangement process s condcted n a freght yard that conssts of a man-track and several sb-tracks. Freght cars ntally placed on sb-tracks are rearranged, and lned nto the man track. Ths seres of operaton s called marshalng, and several methods to solve the marshalng problem have been proposed []-[]. However, these methods do not consder the processng tme for each transfer movement of freght car that s moved by a locomotve. Moreover, n these methods, the marshalng s based on the layot generated by the classfcaton process that assgns each car nto a certan sb-track. Ths cases restrctons on the nmber of sb-tracks and the nmber of cars n a sb-track. On the other hands, n or research grop, several methods that can generate marshalng plans from the random layot have been proposed [4],[5]. Althogh these methods do not reqre pre-process to ease the marshalng process, they do not consder optmzaton of the ntal arrangement of the cars n sb-tracks based on the processng tme. In ths paper a new method for generatng marshalng plan of freght cars n a tran s proposed. In the proposed method, the ncomng freght cars are classfed nto several sb-tracks searchng better assgnment n order to redce the total processng tme. Then, classfcatons and marshalng plans based on the processng tme are obtaned by a renforcement learnng method [6]. Then, the nmber of sb-tracks sed n the classfcaton s selected to yeld generalzaton capablty n the learnng process. Smltaneosly, the desrable classfcaton of ncomng cars as well as, the optmal seqence of car-movements, the nmber of freght cars that can acheve the desred layot of otgong tran s obtaned by atonomos learnng. In the learnng process, a movement of a freght car conssts of 4 elements:. movng a locomotve to the car to be transferred, 2. coplng cars wth the locomotve,. transferrng cars to ther new poston by the locomotve, 4. decoplng the cars from the locomotve. The processng tmes for elements. and. are determned by the transfer dstance of the locomotve, the weght of the tran, and the performance of the locomotve. The total processng tmes for elements. and. are determned by the nmber of movements of freght cars. Ths, the transfer dstance of the locomotve and the nmber of movements of freght cars are smltaneosly consdered, and sed to evalate and mnmze the processng tme of marshalng for ISSN: (Prnt; ISSN: (Onlne

2 Proceedngs of the Internatonal MltConference of Engneers and Compter Scentsts 205 Vol I, obtanng the desred layot of freght cars for an otbond tran. The total processng tme of marshalng s consdered by sng a weghted cost of a transfer dstance of the locomotve and the nmber of movements of freght cars. Then, the order of movements of freght cars, the poston for each removed car, the arrangement of cars n a tran and the nmber of cars to be moved are smltaneosly optmzed to acheve mnmzaton of the total processng tme. In the learnng algorthm, the nmber of sb-tracks tlzed n the classfcaton s evalated. Then several layots generated by the classfcaton wth the selected sb-tracks are smltaneosly pdated. Ths featre yelds generalzaton capablty. In the realstc scale of problems, ndvdal pdate for reslts of classfcaton s not effectve becase the nmber of layots s hge. By selectng the nmber of sb-tracks that has the better evalaton, the learnng performance of the proposed method can be mproved. placed car s copled wth the adacent freght car. (a Classfcaton stage II. PROBLEM DESCRIPTION A. Freght Yard A freght yard s assmed to have man track and m sb-tracks. Defne k as the nmber of freght cars carred n and assgned to sb-tracks n a classfcaton stage. Then, they are moved to the man track by the desrable order based on ther destnaton n a marshalng stage. In the yard, a locomotve moves freght cars, and the movement of freght cars from sb-track to sb-track s called removal, and the car-movement from sb-track to man track s called rearrangement. For smplcty, the maxmm nmber of freght cars that each sb-track can have s assmed to be n, the th car s recognzed by an nqe symbol c, (=,, k. Fg. shows the otlne of freght yard n the case k=0, m=n=6. In the fgre, track Tm denotes the man track, and other tracks []-[6] are sb-tracks. The man track s lnked wth sb-tracks by a ont track, whch s sed for movng cars between sb-tracks, or for movng them from a sb-track to the man track. Fg. -(a depcts an example of classfcaton and Fg. -(b s an example of marshalng. In Fg. -(a, after cars c throgh c 2, and c 20 throgh c 26 are classfed nto sb-tracks [] [2] [], c 9 s placed on the sb-track [6]. Then, c 26 throgh c 0 carred by trcks are placed on sb-track [6] by the arrvng order. In Fg. -(b, freght cars are moved from sb-tracks, and lned n the man track by the descendng order, that s, rearrangement starts wth c 0 and fnshes wth c. When the locomotve L moves a certan car, other cars locatng between the locomotve and the car to be moved mst be removed to other sb-tracks. Ths operaton s called removal. Then, f k nm-(n- s satsfed for keepng adeqate space to condct removal process, every car can be rearranged to the man track. In each sb-track, postons of cars are defned by n rows. Every poston has nqe poston nmber represented by mn ntegers, and the poston nmber for cars at the man track s 0. Fg. 2 shows an example of poston ndex for k=0, m=n=6 and the layot of cars for Fg. -(b. In Fg. 2, the poston ``[a][]'' that s located at row ``[a]'' n the sb-track ``[]'' has the poston nmber. For nfed representaton of layot of cars n sb-tracks, the frst car s placed at the row ``[a]'' n every track, and a newly ISSN: (Prnt; ISSN: (Onlne [f] [e] [d] [c] [b] [a] (b Marshalng stage Fg. : Freght yard Poston Marshalng Fg. 2 Example of poston ndex yard and yard arrangement B. Desred layot n the man track In the man track, freght cars that have the same destnaton are placed at the neghborng postons. In ths case, removal operatons of these cars are not reqred at the destnaton regardless of arrangements of these cars. In order to consder ths featre n the desred layot n the man track, a grop s organzed by cars that have the same destnaton, and these cars can be placed at any postons n the grop. Then, makng a grop correspondng to each destnaton, the order of grops lned n the man track s predetermned by

3 Proceedngs of the Internatonal MltConference of Engneers and Compter Scentsts 205 Vol I, destnatons. Ths featre yelds several desrable layots n the man track. Fg : Grops and arrangements of freght cars Fg. depcts examples of desrable layots of cars and the desred layot of grops n the man track. In the fgre, freght cars c,, c 6 to the destnaton make grop, c 7,, c 8 to the destnaton2 make grop2, c 9,, c 25 to the destnaton make grop. Grops,2, are lned by ascendng order n the man track, whch make a desrable layot. Also, n the fgre, examples of layot n grop are n the dashed sqare. The layot of grops lned by the reverse order does not yeld addtonal removal actons at the destnaton of each grop. Ths, n the proposed method, the layot lned grops by the reverse order and the layot lned by ascendng order from both ends of the tran are regarded as desred layots. By defnng r as the nmber of grops, the total nmber of layots of grop s 2 r-. layot of cars and grops, the ntal layot of cars n sb-tracks, and the poston ndex n sb-tracks are depcted for m=n=4, k=9. c, c 2, c, c 4 are n grop, c 5, c 6, c 7, c 8 are n grop2, and grop mst be rearranged frst to the man track. In each grop, any layots of cars can be acceptable. In both cases, c 2 n step and c n step are appled the drect rearrangement. Also, n step 4, cars c, c 4, c 5 located adacent postons are copled wth each other and moved to the man track by a drect rearrangement operaton. In addton, at step 5 n case 2, cars n grop2 and grop are moved by a drect rearrangement, snce the postons of c 7, c 8, c 6, c 7 are satsfed the desred layot of grops n the man track. Whereas, at step 5, case ncldes 2 drect rearrangements separately for grop2 and grop. Fg 4: Grop layots n a ot gong tran Fg. 4 depcts examples of materal handlng operaton for extended layot of grops at the destnaton of grop. In the fgre, step shows the layot of the ncomng tran. In case (a, cars n grop are separated at the man track, and moved to a sb-track by the locomotve L at step 2. In cases (b,(c, cars n grop are carred n a sb-track, and grop s separated at the sb-track. In the cases, grop can be located wthot any removal actons for cars n each grop. Ths, these layots of grops are regarded as canddate for desred one n the learnng process of the proposed method. Fg. 5 shows an example of arrangement n sb-tracks exstng canddates for rearrangng cars that reqre no removal. At the top of fgre, from the left sde, a desred C. Marshalng process Fg 5: Examples of marshalng A marshalng process conssts of followng 7 operatons: ISSN: (Prnt; ISSN: (Onlne

4 Proceedngs of the Internatonal MltConference of Engneers and Compter Scentsts 205 Vol I, (I selectons: grop-layot n the man track, the nmber of sb-tracks sed n the classfcaton, (II classfcaton of the ncomng freght cars nto sb-tracks, (III rearrangement of freght cars to the man track, when they can be moved drectly, (IV selecton: a freght car to be rearranged nto the man track, (V selecton: removal destnatons of the cars n front of the car selected n (IV, (VI selecton: the nmber of cars to be moved n (V, (VII removal of the cars determned n (VI to the selected sb-track n (V. After operatons (I,(II are fnshed, (III-(VII are repeated ntl one of desrable layots s acheved n the man track, and a seres of operatons from the ntal one n (I to the fnal one n (III achevng the desrable layot s defned as a tral. Now, defne Go as the desred layot, m c as the nmber of sb-tracks sed n the classfcaton stage and h as the nmber of canddates of (Go, m c. Each canddate n operaton (I s represented by ( h, h=2 r- m c. In the operaton (II, a sb-track for each car s determned from the tal of the tran. The determned sb-track s defned as T C, and canddates of T C are defned as 2 (h+ 2 h+m. 2 are sb-tracks each of whch satsfes a part of the layot n Go. When there s no sch sb-track, T C s selected from m sb-tracks. Then, the nmber of grops classfed to T C s determned. Canddates are grops that satsfy a part of the layot n Go, and are defned as (h+m+ h+m+v, where v s the nmber of the canddates. In the operaton (IV, each grop has the predetermned poston n the man track. Then, the car to be rearranged s defned as c T, and canddates of c T s determned by the nmber of freght cars that have already rearranged to the man track and the grop layot n Go. n g s defned as the nmber of freght cars n grop to be rearranged, and 4 (h+m+v+ 4 h+m+v+n g as canddates of c T. In the operaton (V, the removal destnaton of cars located on the car to be rearranged s defned as T R. Then, defnng 5 (h+m+v+ n g + 5 h+m+v+ n g +m- as canddates of T R, excldng the sb-track that has the car to be removed, and the nmber of canddates s m-. In the operaton (VI, defnng n p as the nmber of removal cars reqred to rearrange c T, and defnng n q as the nmber of removal cars that can be located the sb-track selected n the operaton (V, the canddate nmbers of cars to be moved are determned by 6 ( 6 mn{n p,n q}, h+2m+v+ n g +m 6 h+2m+v+ n g +m +mn{n p, n q}. (III-(VII are repeated ntl all the cars are rearranged nto the man track. D. Transfer dstance of Locomotve When a locomotve transfers freght cars, the process of the nt transton s as follows: (E starts wthot freght cars, and reaches to the ont track, (E2 restarts n reverse drecton to the target car to be moved, (E onts them, (E4 plls ot them to the ont track, (E5 restarts n reverse drecton, and transfers them to the ndcated locaton, and (E6 dsonts them from the locomotve. Then, the transfer dstance of locomotve n (E, (E2, (E4 and (E5 s defned as D, D 2, D and D 4, respectvely, and the dstance of the nt transton D s calclated by D= D + D 2+ D +D 4. Also, defne the nt dstance of a movement for cars n each sb-track as D mnv, the length of ont track between adacent sb-tracks, or, sb-track and man track as D mnh. The locaton of the locomotve at the end of above process s the start locaton of the next movement process of the selected car. The ntal poston of the locomotve s located on the ont track nearest to the man track. Fg. 6 shows an example of transfer dstance. In the fgre, m=n=6, D mnv=d mnh=,k=8, (a s poston ndex, and (b depcts movements of locomotve and freght car. Also, the locomotve starts from poston 8, the target s located on the poston 8, the destnaton of the target s 4, and the nmber of cars to be moved s 2. Snce the locomotve moves wthot freght cars from 8 to 24, the transfer dstance s D+D2=2, D=5,D2=7, whereas t moves from 24 to 6 wth 2 freght cars, and the transfer dstance s D+D4= (D=7,D4=6. E. Processng tme for the nt transton In the process of the nt transton, the each tme for (E and (E6 s assmed to be the constant t E. The processng tmes for elements (E, (E2, (E4 and (E5 are determned by the transfer dstance of the locomotve D (=,2,,4, the weght of the freght cars W moved n the process, and the performance of the locomotve. Then, the tme each for (E, (E2, (E4 and (E5 s assmed to be obtaned by the fncton f(d, W derved consderng dynamcs of the locomotve, lmtaton of the velocty, and control rles. Ths, the processng tme for the nt transton 2 4 t U s calclated by t t f ( D,0 f ( D, W. U E The maxmm vale of t U s defned as t max and calclated by tmax te f ( kdmn v,0 f ( md mn h,0 ( f ( md n, W f ( kd, W mn h F. Drect rearrangement Fg 6: Transfer dstance mn v When there exsts a rearrangng car that has no car to be removed on t, ts rearrangement precedes any removals. In the case that several cars can be rearranged wthot a removal, rearrangements are repeated ntl all the canddates for rearrangement reqres at least one removal. If several canddates for rearrangement reqre no removal, the order of ISSN: (Prnt; ISSN: (Onlne

5 Proceedngs of the Internatonal MltConference of Engneers and Compter Scentsts 205 Vol I, selecton s random, becase any orders satsfy the desrable layot of grops n the man track. In ths case, the arrangement of cars n sb-tracks obtaned after rearrangements s nqe, so that the movement cont of cars has no correlaton wth rearrangement order of cars that reqre no removal. Ths operaton s called drect rearrangement. When a car n a certan sb-track can be rearrange drectly to the man track and when several cars located adacent postons n the same sb-track satsfy the layot of grop n the man track, they are copled and appled drect rearrangement. III. LEARNING ALGORITHM Defnng Q as an evalaton vale for (Go, m c, Q (Go, m c s pdated by the followng rle when one of desred layot s acheved n the man track: Q ( Go, m c l max Q ( Go, mc, ( Q ( Go, mc R (2 where l denotes the total movement conts reqred to acheve the desred layot, s learnng rate, s dscont factor calclated for each movement, R s reward that s gven only when one of desred layot s acheved n the man track. In order to yeld generalzaton, evalaton for each (Go, m c s condcted only n the classfcaton stage, and Go s sed alone for evalatons n marshalng stage. Defne s(t as the state at tme t, Tc as the sb-track selected as the destnaton for the removed car, p C as the nmber of classfed grops, q M as the movement conts of freght cars by drect rearrangement, and s as the state that follows s. In the classfcaton stage, Q 2,Q are defned as evalaton vales for (s, 2, (s 2, respectvely, where s =[s, Go], s 2=[s, T C]. Q 2(s,T C and Q (s 2, p C are pdated by followng rles: Q2 ( s, TC max Q ( s, Q ( V s2, pc ( Q ( s2, pc l R (all cars assgned V max Q2 ( s, 2 (otherwse 2 (4 In the rearrangement, defne q as the nmber of drect movements condcted seqentally, p M as the nmber of cars moved. Q 4,Q 5 and Q 6 are defned as evalaton vales for (s, 4,(s, 5,(s 4, 6 respectvely, where s =s, s 4=[s, c T], s 5=[s 4, T R]. Q 4(s,c T,Q 5(s 4, T R and Q 6(s 5,p M are pdated by followng rles: Q Q 4( T ( s,c max Q ( s, (5 5 5( 4 R s, T max Q ( s, (6 Tal 6 Fg. 7 Intal arrangement of ncomng tran Q ( s, p 6 5 M q ( Q6 ( s5, pm R V2, (7 V = max (, ( s a rearrangement 2 Q4 s' 4 4 (- Q6 ( s5, pm R V, V max Q5 ( s4', 5 ( s a removal 5 where s the learnng rate, R s the reward that s gven when one of desrable layot s acheved, and s the dscont factor that s sed to reflect the transfer dstance of locomotve and calclated by the followng eqaton: t max t t max U, (0,0. (8 Propagatng Q-vales by sng eqs.(2-(7, Q-vales are dsconted accordng to the transfer dstance of locomotve. In other words, by selectng the removal destnaton that has the largest Q-vale, the transfer dstance of locomotve can be redced. In the learnng stages, each, ( h+2m+v+ n g +m +mn{n p, n q} s selected by the soft-max acton selecton method [6] In the addressed problem, Q 2, Q, Q 4, Q 5, Q 6 become smaller when the nmber of dsconts becomes larger. Then, for complex problems, the dfference between probabltes n canddate selecton reman small at the ntal state and large at fnal state before achevng desred layot, even after repettve learnng. In ths case, obtaned evalaton does not contrbte to selectons n ntal stage of marshalng process, and search movements to redce the transfer dstance of locomotve s spoled n fnal stage. To conqer ths drawback, Q 2, Q, Q 4, Q 5, Q 6 are normalzed, and probablty P for selecton of each canddate s calclated by Q ( s, mn Q (, ~ s Q (, s max Q ( s, mn Q ( s, P( s, ( 2,,4,5,6 ~ exp( Q ( / P( ~, exp( Q ( / ~ exp( Q ( s, / ~, exp( Q ( s, / wherer s a thermo constant., Snce Q s pdated consderng the nmber of sb-tracks sed n the classfcaton, the ntal arrangement of cars n sb-tracks after operaton (II reflects the evalaton of m c. In the followng operatons, the evalaton of m c affects commonly to several arrangements n accordance wth selected m c. As a conseqent, a generalzaton capablty s yelded over the pdates of Q-vales derved from the ntal arrangement of cars n sb-tracks. In addton, snce pdate rles n operatons (IV,(V,(VI are ndependent from m c, correspondng Q-vales are sed commonly for all the m c. Head (8 (9 ISSN: (Prnt; ISSN: (Onlne

6 Proceedngs of the Internatonal MltConference of Engneers and Compter Scentsts 205 Vol I, IV. COMPUTER SIMULATIONS Compter smlatons are condcted for m=n=6,k=2 and learnng performances of followng methods are compared: (A Proposed method consderng the layot of grops wth generalzaton capablty, (B Proposed method wthot generalzaton capablty[5], (C Tranglar sortng method []. The ntal arrangement of ncomng tran s descrbed n Fg.7. The orgnal grops layot s grop, grop 2, grop, grop 4, grop 5, grop 6. Cars c,,c 7 are n grop, c 8,,c are n grop 2, c 4,,c 7 are n grop,c 8, c 9 are n grop 4, c 20 s n grop 5 and c 2 s n grop 6. Other parameters are set as =0.9, =0.2, =0.9, R=.0, =0.. (C accepts only the orgnal layot of grops, whereas other methods consder extended layot of grops. In (B, the evalatons n the classfcaton are condcted for Go alone, so that no generalzaton capablty s yelded. The locomotve and freght cars assmed to have the same length, and D mnv=d mnh=20m. The locomotve assmed to accelerate and decelerate the tran wth the constant force to be N and kg n weght. Also, all the freght cars have 00 the same weght kg. The velocty of the locomotve s lmted to no more than 0m/s. Then, the locomotve accelerates the tran ntl the velocty reaches 0m/s, keeps the velocty, and decelerates ntl the tran stops wthn the ndcated dstance D (=,2,,4. When the velocty does not reach 0m/s at the half way pont of D (=,2,,4, the locomotve starts to decelerate mmedately. The reslts are shown n Fg.8. In the fgre, horzontal axs expresses the nmber of trals and the vertcal axs expresses the mnmm processng tme fond n the past trals. Each reslt s averaged over 20 ndependent smlatons. In Fg.8, the learnng performance of method (A s better than that of (B, becase (A spreads the evalaton of the best soltons for the selected (Go,m c by the generalzaton capablty. Snce the grop layot for the otbond tran and classfcaton are fxed, (C s not effectve to redce the total processng tme as compared to (A,(B. Snce (A(B evalate the processng tme drectly, the plans generated by them are mch better than that by (C based on the total movement conts of freght cars. Snce the grop layot for the otbond tran and classfcaton are fxed, (C s not effectve to redce the total processng tme as compared to (A,(B. Snce (A(B evalate the processng tme drectly, the plans generated by them are mch better than that by (C based on the total movement conts of freght cars. Total processng tme for each method at the th tral are descrbed n Table for each method. V. CONCLUSION A new schedlng method has been proposed n order to rearrange and lne cars n the desrable order onto the man track consderng classfcatons for ncomng freght cars. x0 [Sec.] Fg 8 Learnng performances The learnng algorthm of the proposed method s derved based on the renforcement learnng that evalates the total processng tme of rearrangement. In order to redce the total processng tme of marshalng, the proposed method yelds generalzaton capablty for obtanng the nmber of sb-tracks sed n the classfcaton stage. In compter smlatons, the total processng tme of the marshalng plan derved by the proposed method has been redced by over 50% as compared to the conventonal tranglar sortng algorthm. In addton, the layot of freght cars n the classfcaton, the rearrange order of cars, the poston of each removal car, the nmber of cars to be removed, and the grop layot n the otgong tran has been obtaned smltaneosly so that the learnng performance of the proposed method has been mproved Table Reslts at th tral [Sec.] Best Average Worst (A (B (C REFERENCES x0 5 [] C. F. Daganzo, R. G. Dowlng, R. W. Hall: Ralroad classfcaton yard throghpt: The case of mltstage tranglar sortng, Transportaton Research, Part A, No.7, Vol. 2 (98, pp [2] U. Blasm, M. R. Bsseck, W. Hochstättler, C. Moll, H.-H. Scheel and T. Wnter: Schedlng Trams n the Mornng, Mathematcal Methods of Operatons Research Vol. 49, No. (2000, pp [] R. Jacob, P. Marton, J. Mae and M. Nnkesser: Mltstage Methods for Freght Tran Classfcaton, Proceedngs of 7th Workshop on Algorthmc Approaches for Transportaton Modelng, Optmzaton, and Systems (2007, pp [4] Y. Hrashma: An Intellgent Tran Marshalng Based on the Processng Tme Consderng Grop Layot of Freght Cars n IAENG Transactons on Electrcal Engneerng Volme, World Scentfc (20, pp [5] Y. Hrashma: A Renforcement Learnng System for Generatng Tran Marshalng Plan of Freght Cars Based on the Processng Tme Consderng Grop Layot, Proceedngs of The Internatonal MltConference of Engneers and Compter Scentsts 20, (20, pp.0-5. [6] R. Stton and A. Barto: Renforcement Learnng, MIT Press (999. ISSN: (Prnt; ISSN: (Onlne