Stochastic Perishable Inventory Control in a Service Facility System Maintaining Inventory for Service: Semi Markov Decision Problem

Transcription

1 Stochatic Perihable Inventory Control in a Service Facility Sytem Maintaining Inventory for Service: Semi Markov Deciion Problem R.Mugeh 1,S.Krihnakumar 2, and C.Elango 3 1 mugehrengawamy@gmail.com 2 krihmathew@gmail.com 3 chellaelango@gmail.com Abtract In thi article we addre the problem of optimally controlling the perihable inventory replenihment in a ervice facility ytem keeping inventory for ervice completion. We conider a finite ervice facility ytem having finite waiting pace with Poion arrival and exponentially ditributed ervice time and life time of item. For the given value of maximum inventory and waiting pace capacity, we determine the order quantitie at variou intance of time o that the long run expected cot rate i minimized. The problem i modeled a a emi Markov Deciion Problem. We prove the exitence of a tationary optimal policy and olve it by employing LP technique. Numerical example for different intance i provided to get inight into the ytem behavior. Keyword: Service facility, Replenihment control, Poion demand, Perihable item, Exponential ervice time, Markov Deciion Proce. 1. Introduction: Maintaining perihable inventory for ervice completion purpoe in ervice facilitie i a maor ytem phenomenon which need in depth tudy. Lat two decade, many reearcher in the field of operation and reource management contributed many reult. (Berman, O., Sapna, K.P.,Arivarigan, G., Elango, C.,Yadavalli, V., Arumugam, N., Krihnamoorthy, S., and Sivakumar, B). In mot of the tudie mentioned above the ytem i conidered a a Markov proce with finite or infinite tate pace. The expreion for tranition probability function and the infiniteimal generator matrix of the Markov proce are derived. The teady tate probability ditribution of the tate ha been found. Then by computing proper ytem performance meaure and impoing repective cot tructure, the cot analyi i done to get the optimal parameter of the ytem. 505

2 In all the above aid model, ytem performance meaure are computed for the uncontrolled ytem then a pecific cot tructure i impoed on the ytem, to get a optimum parameter value uing an optimization criteria. We believe that an integrated approach like Markov Deciion Proce model i mot appropriate to tudy ervice facility ytem (Queue-Inventory) and Maintenance ytem. Sapna, K.P., and Berman, O., [1] tudied one uch ytem under MDP tructure uing LPP method to control the ervice rate. So for in the literature only admiion control and ervice rate control problem are tudied under MDP regime. Hild Mohamed et. Al [6] analyzed a Markov deciion problem: Optimal control of erver in a ervice facility holding perihable inventory with impatient cutomer. The dynamic control of ervice rate with perihable/non-perihable inventory ytem give the ytem manager great flexibility in coping with the uncertainty of demand and deterioration of inventory. The ituation for MDP arie in ervice facility alo can be realized in a variety of other application beyond ervice facilitie. In manufacturing indutrie thee environment arie frequently. For example the problem arie in aembly time for Cathode ray tube (CRT) and liquid crytal diplay (LCD) and part of the aembly ob (CRT or LCD) require the ue of gla panel, which are motly upplied from outide ource. In thee indutrie the ob arriving to the worktation (facility) holding gla panel a inventory can be viewed a cutomer. The main contribution of thi paper i to precribe a control policy, which i optimal that yield minimum cot. The pecific optimal ordering policy for the inventory ytem i obtained by conidering the ordering cot at each tate of the ytem. The ordering rate are function of the number of cutomer and the inventory level. In thi article we conider a ervice facility ytem maintaining perihable inventory under MDP tructure. We get a continuou time MDP in which time between deciion epoch are exponentially ditributed. We can alo analyze the model by converting it to an equivalent more eay dicrete time proce uing uniformaization principle. Thi cale downed procee will behave well to apply long run expected total cot rate criteria to get optimal policy for finite horizon problem with finite tate and Randomized Markovian deciion. 506

3 Here, we ue only LPP method to optimize the expected total cot rate. Section 1 give a brief introduction and literature review of the problem. In ection 2, the model formulation i done with notation. Analyi part of the model i given in ection 3. Section 4 deal with long run Expected cot rate criteria to get the optimal vale of the ytem parameter. 2. Problem Formulation: Conider a ervice facility ytem with inventory maintained to atify the cutomer. Aume that the maximum capacity of inventory i S and a finite waiting pace N, there exit a forced balking when there are N cutomer waiting in the ytem. Cutomer arrive for ervice facility according to a Poion proce with parameter ( 0) and are erved according to a FCFS queue dicipline. One unit (item) from inventory i ued up to erve one cutomer. The item in tock are of perihable native with perihing rate θ. The ervice time follow an exponential ditribution with parameter ( 0). Whenever the inventory level reache to a prefixed level (0 < S),an order for Q=S- item placed and the lead time i exponential ditributed with parameter 0 repectively. The ize of the order i aduted at the time of replenihment o that immediately after replenihment the inventory level i S. Order deciion i made at each level below the reorder level. Let I(t) and X(t) denote the inventory level and the number of cutomer in the ytem at time t. Then, I t X t : t 0 i a two dimenional tochatic proce with tate pace, E E, 1 2 where E 1 = {0, 1, 2,,S} and E 2 = {0, 1, 2,,N}. Deciion Set: X(t) = r. The reordering deciion taken at each tate of the ytem (, r) E, where, I(t) = and Let A i (i =1, 2, 3) denote the et of poible action where, A 1 ={0}, A 2 = {0, 1}, A 3 = {2}, A=A 1 A 2 A 3. Suppoe Đ denote the cla of all tationary policie, then a policy f (equence of deciion) can be defined a a function f : E A, given by f (i, q) = {(k): (i, q) E i, k A i, i = 1, 2, 3} Let E 1 = {(i, q) E / f (i, q)= 0}. 507

4 E 2 = {(i, q) E / f (i, q)= 0 or 1}. E 3 = {(0, q) E / f (i, q)= 2}, 0 repreent no order, 1 mean reorder for Q = S- item and 2 mean compulory order for S item when inventory level i zero. Obective of the problem i to find the optimal reorder level o that the long run expected total cot rate i minimum. i i Notation and Aumption: 3. Analyi: E1 E2 E i the tate pace of thestochatic Proce I( t), X ( t) : t 0, 1. where E 0,1,2,,..., Sand E 0,1,2,,..., N A deciion et correponding to tate E. C a cot occured when action a i taken at tate ( i, q). 3. (i,q) p( iq, ) a the tranition probability from tate ( i, q) to tate (, r). when action a i taken at tate ( i, q) E. 5. Inventorylevel are reviwed at the time of ervice completion epoch. 6. Reordering policy i (, S): Q = S item ordered when the inventory level reache (prefixed level), where 0 S. 7. F- the cla of tationary policie. (,r) 4. Let R denote the tationary policy, which i randomized time invariant and Markovian Policy (MR). From our aumption it can be een that ( I t, X t ) : t 0 i denoted a the 508

5 R R controlled proce I t, X t : t 0 when policy R i adopted. Since the proce R R I t, X t : t 0 i a Markov Proce with finite tate pace E. The proce i completely Ergodic, if every tationary policy give raie to an irreducible Markov chain. It can be een that for every tationary policy, f f f F I,X R * exit, becaue the tate and action pace are finite. i completely Ergodic and alo the optimal tationary policy the et Let A r E1 repreent the et of all poible action for taken the ytem when it belong to {0,1}, 1 A {0}, 1 S, A A E1 {2}, 0 A randomized Markov deciion rule from the cla F i equivalent to the function f : E A given by Pd ( Ar), E1, where d t i the Markovian randomized deciion rule for t T. We denote the t et of deciion rule at time t by MR D t. If d t i the Markovian randomized deciion rule, the expected reward atifie the tranition probability relation. d (, ) pt, r i, q, dt i, q pt (,r) (i,q),a p ( ). t i q a aa r ( i, q), d ( i,q) r (i,q,a) p ( a). t t t d t ( i, q) aa For Markovian ( i, q) MR,d t depend on hitory analyi through the current tate of the proce E o that p Y t a Zt ht Pd ( h )( a) where Y t denote the action at time t and the t hitory proce Z t defined by Z 1 (w)= 1 and Z t (w)= { 1, 2, 3,, t } for 1 t N,N Randomized Markovian Policy t Order ize Q=S- Q+1=S-+1 Q+=S Probability p p -1 p 0 MR i the randomized Markovian policy. Under thi policy an action a A() i choen with probability a(),whenever the proce i in tate E. r 509

6 policy. Whenever () 0or1, the tationary randomized policy reduce to a familiar tationary a 3.1. Steady State Analyi: R R Let I t,x t : t 0 denote the proce.,x I t t : t 0 in which R i the policy adopted from our aumption made in the previou ection. The controlled proce {I R,X R } where R i the randomized Markovian policy in a Markov proce. Under the randomized policy, the expected long run total cot rate when policy i adopted i given by C hi c1w c2 a gb c p d. h -holding cot / unit item / unit time c 1 waiting cot / cutomer / unit time c 2 reordering cot / order g- balking cot / cutomer - ervice cot / cutomer I - mean inventory level w - mean waiting time in ytem a - reordering rate b - balking rate c - ervice completion rate d - expected perihing rate Our obective i to find an optimal policy For any fixed MR policy * for which MR and ( i, q),(, r) E, C * define P (, r,t) Pr I ( t),l ( t) r I (0) i,l (0) q iq C, ( i, q),(, r) E. for every MR policy in MR 510

7 Now P, r, t iq atifie the Kolmogorov forward differential equation P '(t) P(t)A, where A i an infiniteimal generator of the Markov proce {(I (t),x (t)) : t 0}. For each MR policy π, we get an irreducible Markov chain with the tate pace E and action pace A which are finite, P (, r) lim P iq, r; t t exit and i independent of initial tate condition. Thi implie the balance equation (5) (16) given below. Tranition in and out of a tate give a ytem of equation. Conider the typical tate (, r) that lie in the range +1 S-1; 1 r N 1. When (, r) lie in thi range, there i no order pending and hence tranition out of thi tate can be due to either by demand or a ervice completion. The correponding balance equation i given by equation (7). A ervice completion in tate (+1, r+1) will decreae both inventory level and number of cutomer by one unit, thu tranition made to tate (, r). When one cutomer arrive and enter the ytem (r < N) at tate (, r -1), the new tate i (, r). Conidering two different way of reaching tate (, r) and are reflected on the right hand ide of Eq. (7). When one item perih from the inventory (, r) give to (-1, r) with rate θ. Now the ytem of equation can be written in order a follow, i ( S ) P (S,0) p. P (, 0), (3) 0 ( S ) P (S, r) p. P (, r) P (S, r1), 0 1 r N 1, (4) ( S ) P (S, N) p. P (, N) P (S, N1), 0 (5) ( ) P (, 0) P ( 1,1), ( 1) P ( 1,0) 1 S 1, (6) ( ) P (, r) P ( 1, r 1) ( 1) P ( 1, r) P (, r1), 1 S 1;1 r N 1, (7) ( ) P(,N) P(,N1) (1) P π(+1,n-1), 1 S1, (8) 511

8 ( p ) P (, 0) ( 1) P (1, 0) P (1,1), 1, (9) ( p ) P (, r) ( 1) P (1, r) P ( 1, r1) P (, r1), 0 1 ;1 r N 1, (10) ( p ) P (, N) ( 1) P (1, N) P (, N1), 1, (11) ( p ) P (0,0) P (1,1) P (1, 0) (12) ( p ) P (0, r) P (1, r1) P (1, r) P (0, r1), 1 r N 1, ( p P (0, N) P (0, N1) P (1, N) (14) Together with the above et of equation, the total probability condition (, r) E P (, r) 1 (15) give teady tate probabilitie {P π (, r), (, r) E} uniquely. 3.2 Sytem Performance Meaure. The average inventory level in the ytem i given by S N I P 1 r0 Mean waiting time in the ytem i given by (,r). (16) N N m r 1 W P (,r) mp (,r). (17) p r1 0 k0 k m1 r1 The reorder rate i given by a N pp r0 0 (,r). (18) 512

9 The balking rate i given by S 3 P (, N). (19) 0 The ervice completion rate i given by c N S P r1 1 The expected perihable rate i given by (,r). (20) d S N 1 r0 P Now the long run expected cot rate i given by (,r). (21) N S N N S m S N c1 c1 m (,r) (,r) (,r) 2 (,r) 0 r0 r1 0 m1 r1 0 p r1 0 C h P r P P c r p P S S N S N g P (, N) P (,r) p P (,r) (22) 0 1 r1 1 r1 4. Linear Programming Problem: 4.1 Formulation of LPP In thi ection we propoe a LPP model within a MDP framework. Firt define the variable, D (, r, k) a a conditional probability definition D(, r, k) = Pr {deciion i k tate i (, r)} (23) Since 0 D (, r, k) 1, thi i compatible with the randomized time invariant Markovian policie. Here, the Semi Markovian deciion problem can be formulated a a linear programming problem. Hence 0 D (, r, k) 1 and D(,r,k) 1, 0 r N; 0 S. ka{0,1,2} y (, r, k) a follow. For the reformulation of the MDP a LPP, we define another variable 513

10 y(,r,k) D(,r,k)P (,r). (24) From the above definition of the tranition probabilitie P (,r) y(,r,k), (,r) E, k {0, 1, 2} A (25) ka Expreing P (, r) in term of y(,r,k), the expected total cot rate function (21) i given by Minimize S N S N C h y(, r,k) h. y(,0,0) c p. y(, r,k) 1 ka 1 r1 1 ka r1 0 N N S m S r c 2 m 2 y(, r,k) ka r1 0 ka m1 r1 0 pk c y(, r,k) S S N g y(, N,k) g y(0, N,0) y(, r,k) p Subect to the contraint, ka 1 ka 1 r1 S N ka 1 r1 (1) y(,r,k) 0, (,r) E, k A l, l 1,2, (2) 4.3 Lemma: 2 l1 (,r) El kal y(,r,k) 1, y(, r, k) (26) and the balance equation (3) (14) obtained by replacing P (,r) by y(,r,k). k ka policy. The optimal olution of the above Linear Programming Problem yield a determinitic Proof: From the equation y(,r,k) D(,r,k)P (,r) (27) 514

11 and P (, r) y(, r, k), (, r) E. (28) ka Since the deciion problem i completely ergodic every baic feaible olution to the above linear programming problem ha the property that for each (, r) E, y(, r, k) 0 for every ka. 5. Numerical illutration and Dicuion: In thi ytem we conider a problem to illutrate the method decribed in ection 4, through numerical example. We implemented TORA oftware to olve LPP by implex algorithm. We intuitively propoed a conecture that the reorderingrate(p ) to be employed depend only on the inventory level. Thi conecture can be proved for zero lead time and reorder i made at inventory level and the order quantity i aduted at the time of replenihment. Sapna, K. P., & Berman, O. [5] proved that the expected cot rate, S 1 c C h c k m p m 1 ( ) 2 (,k) 2 0 (1) k{0,1,2} where m, (1) N. 1 For 0 S, 1 r N N S N g p(, N) ( ) p(, r) p y(, r, k) 0 1 r1 ka 1 r1 p(,r) r1 P(,0). For 0 S, 1 1 p(,0). N S 1 Conider the MDP problem with the following parameter: 515

12 S = 3, = 2, N = 4, λ = 2, μ = 3, γ = 4, h = 0.1, c = 3; = 0, 1, 2, g = 5, p=2, β() = 2 Action(a)\prob. value p 2 p 1 p The optimum cot for the ytem i C = and Optimal policy i R*(0, 1, 2, 3) i (2, 0, 1, 0). N Y Y Y Y Y Y That i whenever the inventory level reache the reorder level (>0) the optimal deciion i to refill the inventory with Q=S- item. 6. Concluion and future reearch: Analyi of perihable inventory control at ervice facility i fairly recent ytem tudy. Mot of the previou work determined optimal ordering policie or ytem performance meaure. We approached the problem in a different way, given a ervice rate we determine the optimal ordering policy to be employed to minimize the long run expected cot rate. Thu the optimal inventory control in a perihable environment in the ervice facility i etablihed. In future we may extend thi model to perihable inventory ytem with dicrete time MDP. 7. Reference: [1] Berman, O. and Sapna, K.P., Optimal Control of Service for facilitie holding inventory, Computer and operation Reearch (2001), 28, [2] Berman, O. and Sapna, K.P., Inventory management at ervice facilitie for ytem with arbitrarily ditributed ervice time, Stochatic Model 16 (384), (2000), [3] Berman, O. Stochatic inventory policie for inventory management at ervice facilitie, Stochatic Model, 1999, 15, [4] Cinlar, C., Introduction to Stochatic Procee, Englewood Cliff, N. J., Prentice Hall,

13 [5] Elango, C., Inventory ytem at ervice facilitie, Ph. D Thei, (2002), Madurai Kamara Univerity, India. [6]Hilal Mohamed Al Hamadi, Sangeetha, N., and Sivakumar, B., Optimal control of ervice parameter for a perihable inventory ytem maintained in ervice facility with impatient cutomer. [7] Mine, H. and Oaki S., Markov Deciion Procee, American Elevier Publihing Company Inc, New York (1970). [8] Puterman, M.L., Markov Deciion Procee: Dicrete Stochatic Dynamic Programming, Wiley Intercience Publication Inc., (2005). [9] Henk C. Tim, A Firt Coure in Stochatic Model, John Wiley & Son Inc., (2003). 517