Inventory Control with Convex Costs
|
|
- Evelyn Moody
- 6 years ago
- Views:
Transcription
1 Inventory Control with Convex Costs Jian Yang and Gang Yu Department of Industrial and Manufacturing Engineering New Jersey Institute of Technology Newark, NJ Department of Management Science and Information Systems The University of Texas at Austin Austin, TX December 2000 Abstract For an infinite-horizon stochastic inventory control problem, we find that an optimal generalized base stock policy exists when the ordering and holdingbacklogging costs are both convex and an optimal generalized base stock policy exists when the ordering cost is further piecewise-linear. Keywords: Inventory Control, Markov Decision Process This author is supported, in part, by New Jersey Institute of Technology under Grant No
2 1 Introduction The inventory control problem is concerned with the control of order sizes with the purpose of meeting demands with the minimum total cost within a certain time frame. In most cases, costs in a certain period which affect the decisions include: ordering cost as a function of the order size in that period, inventory holding cost as a function of the inventory level carried over through that period, backlogging cost as a function of the negative inventory level carried through that period, and penalty incurred per unsatisfied demand in that period. In this paper, we study a stationary infinite-horizon stochastic inventory control problem with full backlogging under convex ordering and convex holding-backlogging costs. There is a tremendous amount of literature on stochastic and dynamic inventory control [1] [6]. When the ordering cost is convex and all other costs are linear, Karlin [5] showed the optimal policy to be a generalized base stock policy. That is, there exists a nonnegative function y(x) with 0 dy 1 such that, in any period, dx if the starting inventory level is x, order so that the inventory level is brought to max{x, y(x)}. This policy is better illustrated in terms of order size versus starting inventory level. In that term, the above policy states: There exists a nonnegative function y(x) with 1 dy dx 0 and lim x y(x) = 0 such that when the starting inventory level is x in any period, let the order size be y(x). Figures 1 and 2 depict the policy in terms of order up-to level versus starting inventory level and in terms of order size versus starting inventory level, respectively. *** Figure 1 is about here. *** *** Figure 2 is about here. *** For the single-period problem, when the ordering cost is further piecewise-linear, the optimal policy was shown to be a finite generalized base stock policy [4][6]. In terms of order size versus starting inventory level, the policy is as follows: If the ordering cost constitutes L pieces, then y(x) consists of 2L pieces in which pieces with rates 0 and -1 appear intermittently. In addition, ranges for the pieces with rate -1 are exactly the pieces between two consecutive kinks in the ordering cost function. Besides Karlin s work [4], Sobel [8] used a two-piece linear convex order cost function to model the proportional changes in the ordering cost brought by the order-level 2
3 deviation from a prescribed constant. With the assumption of full backlogging and convex holding-backlogging cost, Sobel concluded that the optimal policy is a twobase-stock generalized base stock policy. Figures 3 and 4 depict the finite generalized base stock policy in terms of order up-to level versus starting inventory level and in terms of order size versus starting inventory level. *** Figure 3 is about here. *** *** Figure 4 is about here. *** Although it might have been a folklore that the above results extend to the infinitehorizon cases under general convex and general piecewise-linear convex ordering costs, we have not seen a rigorous proof for it. In this paper, we attempt to fill this gap. Our approach differs from most work in stochastic inventory control in that we consider all quantities as discrete numbers. By taking limits and making ordinary smoothness assumptions, the results can be translated back into the continuous setting straightforwardly. Our major findings are as follows: When the stationary ordering cost is convex and the stationary holding-backlogging cost is convex, the optimal policy is a generalized base stock policy. When the stationary ordering cost is piecewise-linear convex and the stationary holding-backlogging cost is convex, the optimal policy is a finite generalized base stock policy. The rest of the paper is organized as follows: In Section 2, we introduce the problem formulation and the value iteration solution approach for the problem. In Section 3, we study the problem under general stationary convex ordering and convex holding-backlogging costs. In Section 4, we show the form of one optimal policy for the problem with the further assumption that the stationary ordering cost is piecewiselinear. We conclude the paper in Section 5. 2 Formulation and Convergence of Value Iteration Let us assume that the numbers of demands coming in various periods are i.i.d. nonnegative random variables with a random distribution {P d, d Z + }, such that P d 3
4 is the probability of d new orders coming in a period. Let α (0, 1) be the constant discount factor per period. Assume also that the stationary ordering cost function is a nonnegative convex function V ( ) with V (0) = 0. Finally, let nonnegative convex function C( ) with C(0) = 0 be the stationary holding-backlogging cost. That is, C(i) is the holding cost for carrying i items over one period when i > 0 and C(i) is the backlogging cost for carrying i unsatisfied items over one period when i < 0. The state in each period is the starting inventory level j. An ordering policy X {x(t), t Z + } with x(t) {x j (t), j Z} for any t Z + and x j (t) Z + for any j and t determines the size x j (t) of the order to be placed in any period t when the firm is in any state j. Under a given policy X, the system evolves as a Markov chain. The transition matrix at t, P t (x(t)), satisfies Pjj t (x P j+xj (t) j j(t)) = if j + x j(t) j 0 0 otherwise since it takes the demand of size j + x j (t) j in period t to change the inventory level from j in period t to j in period t + 1. Let zj be the minimum average discounted total cost incurred from period 0 to the infinite future when the state in the period is j. From Blackwell [2], we know that z = {z j, j Z} is a solution to the equations z j = min x j Z + {V (x j ) + P d [C(j + x j d) + αzj+x j d]} d=0 for all j s and there exists an optimal stationary policy x (t) = x {x j, j Z}. j, Define f(x, z) {f j (x j, z), j Z} to be a functional which satisfies that, for any Also, let Then, z satisfies and x satisfies f j (x j, z) = V (x j ) + P d [C(j + x j d) + αz j+xj d]. d=0 f (z) = min x Z + z = f (z ) = min x Z + f(x, z). f(x, z ) x = argmin x Z + f(x, z ). 4 (1)
5 According to Denardo [3], f ( ) is a contraction mapping. Thus, the mapping has a unique fixed point as our solution value z. And, we can use the value iteration method [7] to achieve the optimal solution z : 1) Pick an arbitrary z 0 ; 2) For every t 0, let z t+1 = f (z t ). Then, lim t z t = z. Our ensuing derivations hinge on the convergence of the value iteration method and that x solves the minimization problem for f(x, z ). 3 Under General Convex Ordering Costs The convexity of V ( ) and C( ) directly leads to the positive correlation between the starting inventory level and the inventory level right after ordering. That is, given x j, there is x j+1, such that j x j+1 j + x j, as in Theorem 1: Theorem 1 For any j, we must have some x j+1 which satisfies x j+1 x j 1. Proof: We need only to prove that for any x x j, f j+1 (x 1, z ) f j+1 (x j 1). Actually, for any such x, we have f j (x, z ) f j (x j, z ) due to the optimality of x j for f j (, z ). Furthermore, we have f j+1 (x 1, z ) f j (x, z ) = V (x 1) V (x) and f j+1 (x j 1, z ) f j (x j, z ) = V (x j 1) V (x j). But since V ( ) is convex, we get f j+1 (x 1, z ) f j+1 (x j 1, z ) f j (x, z ) f j (x j, z ) 0. The convexity of V ( ) and C( ) also leads to the convexity of zj j. That is, in our terms, z is convex in j. as a function of 5
6 Theorem 2 z is convex in j. Proof: We first prove the property that, if some z is convex in j, then for any j and nonnegative x 1 and x 2, there exist nonnegative x m and x M which satisfy G j (x 1, x 2, x m, x M, z) f j+1 (x 2, z) + f j 1 (x 1, z) f j (x m, z) f j (x M, z) 0. When x 2 x 1 + 1, we let x m be x 1 and x M be x 2. Otherwise, when x 2 x 1 + 2, if x 1 + x 2 is odd, we let x m be (x 1 + x 2 1)/2 and x M be (x 1 + x 2 + 1)/2; otherwise, we let both x m and x M be (x 1 + x 2 )/2. Note that, we always have x m + x M = x 1 + x 2, and when x 2 x 1 + 2, it is true that x x m x M x 2 1. Hence, when x 2 x 1 + 1, we have G j (x 1, x 2, x 1, x 2, z) = P d [C(j+x 2 d+1)+c(j+x 1 d 1) C(j+x 2 d) C(j+x 1 d) d=0 +α(z j+x2 d+1 + z j+x1 d 1 z j+x2 d z j+x1 d)]. The convexity of C( ) leads to the nonnegativity of the above term. When x 2 x 1 +2, we have G j (x 1, x 2, x m, x M, z) = V (x 2 ) + V (x 1 ) V (x M ) V (x m ) + P d [C(j + x 2 d + 1) + C(j + x 1 d 1) C(j + x M d) C(j + x m d) d=0 +α(z j x2 +d+1 + z j x1 +d 1 z j xm +d z j xm+d)]. The convexity of V ( ) and C( ) leads to the nonnegativity of the above term. Thus, the property is proved. Now, we proceed with induction. Let an arbitrary z 0 be convex in j. For any nonnegative t, let z t+1 = f (z t ). Suppose for a given nonnegative t, z t is convex in j. Now, we try to prove that z t+1 is convex in j as well. To this end, let x t be such that x t = argmin x Z + f(x, z t ). Then, from the convexity of z t in j and the property just proved, we know the existence of some nonnegative x m and x M which leads to f j+1 (x t j+1, z t ) + f j 1 (x t j 1, z t ) f j (x m, z t ) f j (x M, z t ) 0. 6
7 But, by definition, we have for any j and nonnegative x that Therefore, for any j, z t+1 j f j(x m, z t ) + f j (x M, z t ) 2 z t+1 j f j (x, z t ). f j 1(x t j 1, z t ) + f j+1 (x t j+1, z t ) 2 = zt+1 j 1 + zj+1 t+1. 2 Thus, z t+1 is convex in j. From the convergence of z t to z, we know that z is convex in j. Furthermore, the convexity of V ( ) and C( ) and the convexity of z in j allow some x j+1 to be no larger than x j. Theorem 3 For any j, given x j, we can find some x j+1 which is no larger than x j. Proof: We first show that, for any j, f j (x, z ) is a convex function of x. That is because for any positive x, we have f j (x + 1, z ) + f j (x 1, z ) 2f j (x, z ) = V (x + 1) + V (x 1) 2V (x) + P d [C(j + x d + 1) + C(j + x d 1) 2C(j + x d) d=0 +α(zj+x d+1 + zj+x d 1 2zj+x d)] 0 due to the convexity of V ( ) and C( ) and the convexity of z in j. Then, we prove the property that, for any nonnegative x, f j (x, z ) f j (x + 1, z ) implies f j+1 (x, z ) f j+1 (x + 1, z ). To this end, we have f j+1 (x + 1, z ) f j+1 (x, z ) f j (x + 1, z ) + f j (x, z ) = P d [C(j + x d + 2) + C(j + x d) 2C(j + x d + 1) d=0 +α(zj+x d+2 + zj+x d 2zj+x d+1)] 0 due to the convexity of C( ) and the convexity of z in j. Now, due to the optimality of x j for f j (, z ) and the convexity of f j (, z ), we have f j (x j, z ) f j (x j + 1, z ) f j (x j + 2, z ). 7
8 By the property just proved, we also have f j+1 (x j, z ) f j+1 (x j + 1, z ) f j+1 (x j + 2, z ). Therefore, for every j, we may always have some x j+1 which is no larger than x j. So, when the stationary ordering and holding-backlogging costs are convex, z is convex in j and there is an x satisfying x j+1 = x j 1 or x j for any j. Taking the limit to the continuous case, the policy is a generalized base stock policy. 4 Under Piecewise-Linear Convex Ordering Costs Here, we further assume that V ( ) is piecewise-linear whose kinks occur at X 0,..., X L with 0 X 0 < X 1 < < X L 1 < X L. For l = 1,..., L, the marginal ordering cost for order sizes between X l 1 and X l is assumed to be F l, with 0 F 1 < < F L. Therefore, for l = 1,..., L and X l 1 x X l 1, we have l 1 V (x) = F l x + (F l F l +1)X l. Figure 5 illustrates this piecewise-linear convex ordering cost function. l =1 *** Figure 5 is about here. *** We may perceive that, whenever x j becomes as small as X l 1 as j increases, it is possible to have x j+1 = x j 1. Theorem 4 confirms this. Theorem 4 For any j, l = 1,..., L, when X l x j X l 1, we can find some x j+1 to be x j 1. Proof: Since x j+1 may be either x j or x j 1, we need only to prove that f j+1 (x j 1, z ) f j+1 (x j, z ). For X l 1 x X l 1, it is true that V (x + 1) V (x) = F l. Hence, we have f j+1 (x j, z ) f j+1 (x j 1, z ) f j (x j + 1, z ) + f j (x j, z ) 8
9 = 2V (x j) V (x j 1) V (x j + 1) = 0. Since we also have f j (x j, z ) f j (x j + 1, z ), the theorem is proved. Note that, when X l x j X l and x j+1 = x j 1, we have zj+1 = zj F l. The above results lead to the existence of an optimal policy x of the following 2L-piece linear nonincreasing zigzag form: There is some K {K 0, K 1,..., K L 1 } with K 1,..., K L 1 all being nonnegative, such that x K 0 l 1 l j = =1 K l j for l = 1,..., L 1, and X l x j = 0 for j K 0, K 0 l 1 l =1 K l X l j K 0 l 1 l =1 K l X l 1 1 K 0 l l =1 K l X l j K 0 l 1 l =1 K l X l 1 (2) x j = K 0 L 1 l =1 K l j for j K 0 L 1 l =1 K l X L 1 1. The optimal solution value z is convex over j. In addition, for l = 1,..., L, K 0 l 1 l =1 K l X l j K 0 l 1 l =1 K l X l 1 1, we have z j+1 = z j F l. When taking the limit to the continuous case, the above policy is exactly the finite generalized base stock policy. 5 Conclusion In this paper, we have reached the forms of the optimal policies for the stochastic infinite-horizon inventory control problem with convex ordering and holdingbacklogging costs. Our work is an extension to the work by S. Karlin and M.J. Sobel on convex-cost stochastic inventory control problems. In the derivation, we have used more convenient representations for the generalized base stock policy and the finite generalized base stock policy. Moreover, our discrete-quantity approach appears to be efficient. 9
10 References [1] Arrow, K., S. Karlin, and H. Scarf (1958), Studies in the Mathematical Theory of Inventory and Production, Stanford University Press, Stanford, California. [2] Blackwell, D. (1962), Discrete Dynamic Programming, Annals of Mathematical Statistics, 35, pp [3] Denardo, E.V. (1967), Contraction Mappings in the Theory Underlying Dynamic Programming, SIAM Review, 9, pp [4] Karlin, S. (1958), One-Stage Inventory Models with Uncertainty, in K. Arrow, S. Karlin, and H. Scarf (Eds.), Studies in the Mathematical Theory of Inventory and Production, Stanford University Press, Stanford, California. [5] Karlin, S. (1958), Optimal Inventory Policy for the Arrow-Harris-Marschak Dynamic Model, in K. Arrow, S. Karlin, and H. Scarf (Eds.), Studies in the Mathematical Theory of Inventory and Production, Stanford University Press, Stanford, California. [6] Porteus, E.L. (1990), Stochastic Inventory Theory, in D.P. Heyman and M.J. Sobel (Eds.), Handbooks in Operations Research and Management Science, Volume 2: Stochastic Models, Elsevier Science Publishers B.V., North-Holland, pp [7] Puterman, M.L. (1990), Markov Decision Processes, in D.P. Heyman and M.J. Sobel (Eds.), Handbooks in Operations Research and Management Science, Volume 2: Stochastic Models, Elsevier Science Publishers B.V., North-Holland, pp [8] Sobel, M.J. (1970), Making Short-Run Changes in Production When the Employment Level is Fixed, Management Science, 18, pp
11 Order Up-To Level 0 Starting Inventory Level Figure 1: Generalized Base Stock Policy Representation I
12 Order Size 0 Starting Inventory Level Figure 2: Generalized Base Stock Policy Representation II
13 Order Up-To Level K 0 -K 1 -K 2 -X 2 0 K 0 -K 1 -X 2 K 0 -K 1 -X 1 K 0 -X 1 K 0 Starting Inventory Level Figure 3: Finite Generalized Base Stock Policy Representation I
14 Order Size X 2 X 1 0 K 0 -K 1 -K 2 -X 2 K 0 -K 1 -X 2 K 0 -K 1 -X 1 K 0 -X 1 K 0 Starting Inventory Level Figure 4: Finite Generalized Base Stock Policy Representation II
15 Ordering Cost F 2 X 2 +(F 1 -F 2 )X 1 F 1 X 1 0 X 1 X 2 Order Size Figure 5: Piecewise-Linear Convex Ordering Cost
ON THE STRUCTURE OF OPTIMAL ORDERING POLICIES FOR STOCHASTIC INVENTORY SYSTEMS WITH MINIMUM ORDER QUANTITY
Probability in the Engineering and Informational Sciences, 20, 2006, 257 270+ Printed in the U+S+A+ ON THE STRUCTURE OF OPTIMAL ORDERING POLICIES FOR STOCHASTIC INVENTORY SYSTEMS WITH MINIMUM ORDER QUANTITY
More informationKeywords: inventory control; finite horizon, infinite horizon; optimal policy, (s, S) policy.
Structure of Optimal Policies to Periodic-Review Inventory Models with Convex Costs and Backorders for all Values of Discount Factors arxiv:1609.03984v2 [math.oc] 28 May 2017 Eugene A. Feinberg, Yan Liang
More informationStochastic Models. Edited by D.P. Heyman Bellcore. MJ. Sobel State University of New York at Stony Brook
Stochastic Models Edited by D.P. Heyman Bellcore MJ. Sobel State University of New York at Stony Brook 1990 NORTH-HOLLAND AMSTERDAM NEW YORK OXFORD TOKYO Contents Preface CHARTER 1 Point Processes R.F.
More informationStructure of optimal policies to periodic-review inventory models with convex costs and backorders for all values of discount factors
DOI 10.1007/s10479-017-2548-6 AVI-ITZHAK-SOBEL:PROBABILITY Structure of optimal policies to periodic-review inventory models with convex costs and backorders for all values of discount factors Eugene A.
More informationSingle-stage Approximations for Optimal Policies in Serial Inventory Systems with Non-stationary Demand
Single-stage Approximations for Optimal Policies in Serial Inventory Systems with Non-stationary Demand Kevin H. Shang Fuqua School of Business, Duke University, Durham, North Carolina 27708, USA khshang@duke.edu
More informationOptimal ordering policies for periodic-review systems with replenishment cycles
European Journal of Operational Research 17 (26) 44 56 Production, Manufacturing and Logistics Optimal ordering policies for periodic-review systems with replenishment cycles Chi Chiang * Department of
More informationCoordinating Inventory Control and Pricing Strategies with Random Demand and Fixed Ordering Cost: The Finite Horizon Case
OPERATIONS RESEARCH Vol. 52, No. 6, November December 2004, pp. 887 896 issn 0030-364X eissn 1526-5463 04 5206 0887 informs doi 10.1287/opre.1040.0127 2004 INFORMS Coordinating Inventory Control Pricing
More informationOptimality Results in Inventory-Pricing Control: An Alternate Approach
Optimality Results in Inventory-Pricing Control: An Alternate Approach Woonghee Tim Huh, Columbia University Ganesh Janakiraman, New York University May 9, 2006 Abstract We study a stationary, single-stage
More informationA Proof of the EOQ Formula Using Quasi-Variational. Inequalities. March 19, Abstract
A Proof of the EOQ Formula Using Quasi-Variational Inequalities Dir Beyer y and Suresh P. Sethi z March, 8 Abstract In this paper, we use quasi-variational inequalities to provide a rigorous proof of the
More informationA Single-Unit Decomposition Approach to Multiechelon Inventory Systems
OPERATIONS RESEARCH Vol. 56, No. 5, September October 2008, pp. 1089 1103 issn 0030-364X eissn 1526-5463 08 5605 1089 informs doi 10.1287/opre.1080.0620 2008 INFORMS A Single-Unit Decomposition Approach
More informationStochastic Shortest Path Problems
Chapter 8 Stochastic Shortest Path Problems 1 In this chapter, we study a stochastic version of the shortest path problem of chapter 2, where only probabilities of transitions along different arcs can
More informationChapter 2 SOME ANALYTICAL TOOLS USED IN THE THESIS
Chapter 2 SOME ANALYTICAL TOOLS USED IN THE THESIS 63 2.1 Introduction In this chapter we describe the analytical tools used in this thesis. They are Markov Decision Processes(MDP), Markov Renewal process
More informationOptimal Control of Stochastic Inventory System with Multiple Types of Reverse Flows. Xiuli Chao University of Michigan Ann Arbor, MI 48109
Optimal Control of Stochastic Inventory System with Multiple Types of Reverse Flows Xiuli Chao University of Michigan Ann Arbor, MI 4809 NCKU Seminar August 4, 009 Joint work with S. Zhou and Z. Tao /44
More information1 Markov decision processes
2.997 Decision-Making in Large-Scale Systems February 4 MI, Spring 2004 Handout #1 Lecture Note 1 1 Markov decision processes In this class we will study discrete-time stochastic systems. We can describe
More information(s, S) Optimality in Joint Inventory-Pricing Control: An Alternate Approach
(s, S) Optimality in Joint Inventory-Pricing Control: An Alternate Approach Woonghee Tim Huh, Columbia University Ganesh Janakiraman, New York University May 10, 2004; April 30, 2005; May 15, 2006; October
More information(s, S) Optimality in Joint Inventory-Pricing Control: An Alternate Approach*
OPERATIONS RESEARCH Vol. 00, No. 0, Xxxxx 0000, pp. 000 000 issn 0030-364X eissn 1526-5463 00 0000 0001 INFORMS doi 10.1287/xxxx.0000.0000 c 0000 INFORMS (s, S) Optimality in Joint Inventory-Pricing Control:
More informationSingle-stage Approximations for Optimal Policies in Serial Inventory Systems with Non-stationary Demand
Single-stage Approximations for Optimal Policies in Serial Inventory Systems with Non-stationary Demand Kevin H. Shang Fuqua School of Business, Duke University, Durham, North Carolina 27708, USA khshang@duke.edu
More informationChapter 16 focused on decision making in the face of uncertainty about one future
9 C H A P T E R Markov Chains Chapter 6 focused on decision making in the face of uncertainty about one future event (learning the true state of nature). However, some decisions need to take into account
More informationTotal Expected Discounted Reward MDPs: Existence of Optimal Policies
Total Expected Discounted Reward MDPs: Existence of Optimal Policies Eugene A. Feinberg Department of Applied Mathematics and Statistics State University of New York at Stony Brook Stony Brook, NY 11794-3600
More informationOn the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of (s, S) Inventory Policies
On the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of (s, S) Inventory Policies Eugene A. Feinberg Department of Applied Mathematics and Statistics Stony Brook University
More informationOrdering Policies for Periodic-Review Inventory Systems with Quantity-Dependent Fixed Costs
OPERATIONS RESEARCH Vol. 60, No. 4, July August 2012, pp. 785 796 ISSN 0030-364X (print) ISSN 1526-5463 (online) http://dx.doi.org/10.1287/opre.1110.1033 2012 INFORMS Ordering Policies for Periodic-Review
More informationOptimal Control of an Inventory System with Joint Production and Pricing Decisions
Optimal Control of an Inventory System with Joint Production and Pricing Decisions Ping Cao, Jingui Xie Abstract In this study, we consider a stochastic inventory system in which the objective of the manufacturer
More informationA Hierarchy of Suboptimal Policies for the Multi-period, Multi-echelon, Robust Inventory Problem
A Hierarchy of Suboptimal Policies for the Multi-period, Multi-echelon, Robust Inventory Problem Dimitris J. Bertsimas Dan A. Iancu Pablo A. Parrilo Sloan School of Management and Operations Research Center,
More informationDynamic Inventory Models and Stochastic Programming*
M. N. El Agizy Dynamic Inventory Models and Stochastic Programming* Abstract: A wide class of single-product, dynamic inventory problems with convex cost functions and a finite horizon is investigated
More informationAre Base-stock Policies Optimal in Inventory Problems with Multiple Delivery Modes?
Are Base-stoc Policies Optimal in Inventory Problems with Multiple Delivery Modes? Qi Feng School of Management, the University of Texas at Dallas Richardson, TX 7508-0688, USA Guillermo Gallego Department
More information2001, Dennis Bricker Dept of Industrial Engineering The University of Iowa. DP: Producing 2 items page 1
Consider a production facility which can be devoted in each period to one of two products. For simplicity, we assume that the production rate is deterministic and that production is always at full capacity.
More informationMarkov decision processes and interval Markov chains: exploiting the connection
Markov decision processes and interval Markov chains: exploiting the connection Mingmei Teo Supervisors: Prof. Nigel Bean, Dr Joshua Ross University of Adelaide July 10, 2013 Intervals and interval arithmetic
More informationMarkov decision processes with threshold-based piecewise-linear optimal policies
1/31 Markov decision processes with threshold-based piecewise-linear optimal policies T. Erseghe, A. Zanella, C. Codemo Dept. of Information Engineering, University of Padova, Italy Padova, June 2, 213
More informationarxiv: v2 [math.oc] 25 Mar 2016
Optimal ordering policy for inventory systems with quantity-dependent setup costs Shuangchi He Dacheng Yao Hanqin Zhang arxiv:151.783v2 [math.oc] 25 Mar 216 March 18, 216 Abstract We consider a continuous-review
More informationMDP Preliminaries. Nan Jiang. February 10, 2019
MDP Preliminaries Nan Jiang February 10, 2019 1 Markov Decision Processes In reinforcement learning, the interactions between the agent and the environment are often described by a Markov Decision Process
More informationDiscrete-Time Markov Decision Processes
CHAPTER 6 Discrete-Time Markov Decision Processes 6.0 INTRODUCTION In the previous chapters we saw that in the analysis of many operational systems the concepts of a state of a system and a state transition
More informationTime is discrete and indexed by t =0; 1;:::;T,whereT<1. An individual is interested in maximizing an objective function given by. tu(x t ;a t ); (0.
Chapter 0 Discrete Time Dynamic Programming 0.1 The Finite Horizon Case Time is discrete and indexed by t =0; 1;:::;T,whereT
More informationA Customer-Item Decomposition Approach to Stochastic Inventory Systems with Correlation
A Customer-Item Decomposition Approach to Stochastic Inventory Systems with Correlation Yimin Yu Saif Benjaafar Program in Industrial and Systems Engineering University of Minnesota, Minneapolis, MN 55455
More informationFull terms and conditions of use:
This article was downloaded by: [148.251.232.83] On: 13 November 2018, At: 16:50 Publisher: Institute for Operations Research and the Management Sciences (INFORMS) INFORMS is located in Maryland, USA INFORMS
More informationAbstract Dynamic Programming
Abstract Dynamic Programming Dimitri P. Bertsekas Department of Electrical Engineering and Computer Science Massachusetts Institute of Technology Overview of the Research Monograph Abstract Dynamic Programming"
More informationSerial Supply Chains with Economies of Scale: Bounds and Approximations
OPERATIOS RESEARCH Vol. 55, o. 5, September October 2007, pp. 843 853 issn 0030-364X eissn 1526-5463 07 5505 0843 informs doi 10.1287/opre.1070.0406 2007 IFORMS Serial Supply Chains with Economies of Scale:
More informationA New Algorithm and a New Heuristic for Serial Supply Systems.
A New Algorithm and a New Heuristic for Serial Supply Systems. Guillermo Gallego Department of Industrial Engineering and Operations Research Columbia University Özalp Özer Department of Management Science
More informationInventory control with partial batch ordering
Inventory control with partial batch ordering Alp, O.; Huh, W.T.; Tan, T. Published: 01/01/2009 Document Version Publisher s PDF, also known as Version of Record (includes final page, issue and volume
More informationA monotonic property of the optimal admission control to an M/M/1 queue under periodic observations with average cost criterion
A monotonic property of the optimal admission control to an M/M/1 queue under periodic observations with average cost criterion Cao, Jianhua; Nyberg, Christian Published in: Seventeenth Nordic Teletraffic
More informationEconomics 2010c: Lectures 9-10 Bellman Equation in Continuous Time
Economics 2010c: Lectures 9-10 Bellman Equation in Continuous Time David Laibson 9/30/2014 Outline Lectures 9-10: 9.1 Continuous-time Bellman Equation 9.2 Application: Merton s Problem 9.3 Application:
More informationEconomic lot-sizing games
Economic lot-sizing games Wilco van den Heuvel a, Peter Borm b, Herbert Hamers b a Econometric Institute and Erasmus Research Institute of Management, Erasmus University Rotterdam, P.O. Box 1738, 3000
More informationLecture notes for Analysis of Algorithms : Markov decision processes
Lecture notes for Analysis of Algorithms : Markov decision processes Lecturer: Thomas Dueholm Hansen June 6, 013 Abstract We give an introduction to infinite-horizon Markov decision processes (MDPs) with
More information21 Markov Decision Processes
2 Markov Decision Processes Chapter 6 introduced Markov chains and their analysis. Most of the chapter was devoted to discrete time Markov chains, i.e., Markov chains that are observed only at discrete
More informationPurchase Contract Management with Demand Forecast Updates
Purchase Contract Management with Demand Forecast Updates Hongyan Huang Dept. of Systems Engineering and Engineering Management The Chinese University of Hong Kong, Shatin, Hong Kong and Suresh P. Sethi
More informationNew Markov Decision Process Formulations and Optimal Policy Structure for Assemble-to-Order and New Product Development Problems
New Markov Decision Process Formulations and Optimal Policy Structure for Assemble-to-Order and New Product Development Problems by EMRE NADAR Submitted to the Tepper School of Business in Partial Fulfillment
More informationSerial Inventory Systems with Markov-Modulated Demand: Solution Bounds, Asymptotic Analysis, and Insights
Serial Inventory Systems with Markov-Modulated Demand: Solution Bounds, Asymptotic Analysis, and Insights Li Chen Jing-Sheng Song Yue Zhang Fuqua School of Business, Duke University, Durham, NC 7708 li.chen@duke.edu
More informationA Dynamic model for requirements planning with application to supply chain optimization
This summary presentation is based on: Graves, Stephen, D.B. Kletter and W.B. Hetzel. "A Dynamic Model for Requirements Planning with Application to Supply Chain Optimization." Operations Research 46,
More informationComputational complexity estimates for value and policy iteration algorithms for total-cost and average-cost Markov decision processes
Computational complexity estimates for value and policy iteration algorithms for total-cost and average-cost Markov decision processes Jefferson Huang Dept. Applied Mathematics and Statistics Stony Brook
More informationON THE POLICY IMPROVEMENT ALGORITHM IN CONTINUOUS TIME
ON THE POLICY IMPROVEMENT ALGORITHM IN CONTINUOUS TIME SAUL D. JACKA AND ALEKSANDAR MIJATOVIĆ Abstract. We develop a general approach to the Policy Improvement Algorithm (PIA) for stochastic control problems
More informationStochastic Analysis of Bidding in Sequential Auctions and Related Problems.
s Case Stochastic Analysis of Bidding in Sequential Auctions and Related Problems S keya Rutgers Business School s Case 1 New auction models demand model Integrated auction- inventory model 2 Optimizing
More informationOptimal Control of Parallel Make-To-Stock Queues in an Assembly System
Optimal Control of Parallel Make-To-Stock Queues in an Assembly System Jihong Ou Heng-Qing Ye Weiyi Ning Department of Decision Sciences, NUS Business School, National University of Singapore, Singapore
More informationPercentile Threshold Policies for Inventory Problems with Partially Observed Markovian Demands
Percentile Threshold Policies for Inventory Problems with Partially Observed Markovian Demands Parisa Mansourifard Joint work with: Bhaskar Krishnamachari and Tara Javidi (UCSD) University of Southern
More informationA Duality-Based Relaxation and Decomposition Approach for Inventory Distribution Systems
A Duality-Based Relaxation and Decomposition Approach for Inventory Distribution Systems Sumit Kunnumkal, 1 Huseyin Topaloglu 2 1 Indian School of Business, Gachibowli, Hyderabad 500032, India 2 School
More informationWe consider the classic N -stage serial supply systems with linear costs and stationary
Newsvendor Bounds and Heuristic for Optimal Policies in Serial Supply Chains Kevin H. Shang Jing-Sheng Song Fuqua School of Business, Duke University, Durham, North Carolina 2778 Graduate School of Management,
More informationMake-to-Stock Production-Inventory Systems. and Lost Sales
Make-to-Stock Production-Inventory Systems with Compound Poisson emands, Constant Continuous Replenishment and Lost Sales By Junmin Shi A dissertation submitted to the Graduate School-Newark Rutgers, The
More informationMarkov Decision Processes and Dynamic Programming
Markov Decision Processes and Dynamic Programming A. LAZARIC (SequeL Team @INRIA-Lille) ENS Cachan - Master 2 MVA SequeL INRIA Lille MVA-RL Course How to model an RL problem The Markov Decision Process
More informationInformation Relaxation Bounds for Infinite Horizon Markov Decision Processes
Information Relaxation Bounds for Infinite Horizon Markov Decision Processes David B. Brown Fuqua School of Business Duke University dbbrown@duke.edu Martin B. Haugh Department of IE&OR Columbia University
More informationTime-dependent order and distribution policies in supply networks
Time-dependent order and distribution policies in supply networks S. Göttlich 1, M. Herty 2, and Ch. Ringhofer 3 1 Department of Mathematics, TU Kaiserslautern, Postfach 349, 67653 Kaiserslautern, Germany
More informationChapter 3. Dynamic Programming
Chapter 3. Dynamic Programming This chapter introduces basic ideas and methods of dynamic programming. 1 It sets out the basic elements of a recursive optimization problem, describes the functional equation
More informationUNCORRECTED PROOFS. P{X(t + s) = j X(t) = i, X(u) = x(u), 0 u < t} = P{X(t + s) = j X(t) = i}.
Cochran eorms934.tex V1 - May 25, 21 2:25 P.M. P. 1 UNIFORMIZATION IN MARKOV DECISION PROCESSES OGUZHAN ALAGOZ MEHMET U.S. AYVACI Department of Industrial and Systems Engineering, University of Wisconsin-Madison,
More informationValue Function Iteration
Value Function Iteration (Lectures on Solution Methods for Economists II) Jesús Fernández-Villaverde 1 and Pablo Guerrón 2 February 26, 2018 1 University of Pennsylvania 2 Boston College Theoretical Background
More informationZero-Inventory Conditions For a Two-Part-Type Make-to-Stock Production System
Zero-Inventory Conditions For a Two-Part-Type Make-to-Stock Production System MichaelH.Veatch Francis de Véricourt October 9, 2002 Abstract We consider the dynamic scheduling of a two-part-type make-tostock
More informationarxiv: v1 [math.oc] 5 Jan 2015
Optimal ordering policy for inventory systems with quantity-dependent setup costs Shuangchi He, Dacheng Yao, and Hanqin Zhang arxiv:151.783v1 [math.oc] 5 Jan 215 August 17, 218 Abstract We consider a continuous-review
More informationReflected Brownian Motion
Chapter 6 Reflected Brownian Motion Often we encounter Diffusions in regions with boundary. If the process can reach the boundary from the interior in finite time with positive probability we need to decide
More information1 Positive Ordering Costs
IEOR 4000: Production Management Professor Guillermo Gallego November 15, 2004 1 Positive Ordering Costs 1.1 (Q, r) Policies Up to now we have considered stochastic inventory models where decisions are
More informationStochastic (Random) Demand Inventory Models
Stochastic (Random) Demand Inventory Models George Liberopoulos 1 The Newsvendor model Assumptions/notation Single-period horizon Uncertain demand in the period: D (parts) assume continuous random variable
More informationCentral-limit approach to risk-aware Markov decision processes
Central-limit approach to risk-aware Markov decision processes Jia Yuan Yu Concordia University November 27, 2015 Joint work with Pengqian Yu and Huan Xu. Inventory Management 1 1 Look at current inventory
More informationOn the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of (s, S) Inventory Policies
On the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of (s, S) Inventory Policies Eugene A. Feinberg, 1 Mark E. Lewis 2 1 Department of Applied Mathematics and Statistics,
More informationStochastic Differential Equations.
Chapter 3 Stochastic Differential Equations. 3.1 Existence and Uniqueness. One of the ways of constructing a Diffusion process is to solve the stochastic differential equation dx(t) = σ(t, x(t)) dβ(t)
More informationIntroduction to Continuous-Time Dynamic Optimization: Optimal Control Theory
Econ 85/Chatterjee Introduction to Continuous-ime Dynamic Optimization: Optimal Control heory 1 States and Controls he concept of a state in mathematical modeling typically refers to a specification of
More information7.1 INTRODUCTION. In this era of extreme competition, each subsystem in different
7.1 INTRODUCTION In this era of extreme competition, each subsystem in different echelons of integrated model thrives to improve their operations, reduce costs and increase profitability. Currently, the
More informationMarkov Chains (Part 4)
Markov Chains (Part 4) Steady State Probabilities and First Passage Times Markov Chains - 1 Steady-State Probabilities Remember, for the inventory example we had (8) P &.286 =.286.286 %.286 For an irreducible
More informationDynamic stochastic game and macroeconomic equilibrium
Dynamic stochastic game and macroeconomic equilibrium Tianxiao Zheng SAIF 1. Introduction We have studied single agent problems. However, macro-economy consists of a large number of agents including individuals/households,
More informationBayesian Inference and the Symbolic Dynamics of Deterministic Chaos. Christopher C. Strelioff 1,2 Dr. James P. Crutchfield 2
How Random Bayesian Inference and the Symbolic Dynamics of Deterministic Chaos Christopher C. Strelioff 1,2 Dr. James P. Crutchfield 2 1 Center for Complex Systems Research and Department of Physics University
More informationMarkov decision processes
CS 2740 Knowledge representation Lecture 24 Markov decision processes Milos Hauskrecht milos@cs.pitt.edu 5329 Sennott Square Administrative announcements Final exam: Monday, December 8, 2008 In-class Only
More informationSerial Inventory Systems with Markov-Modulated Demand: Derivative Bounds, Asymptotic Analysis, and Insights
Serial Inventory Systems with Markov-Modulated Demand: Derivative Bounds, Asymptotic Analysis, and Insights Li Chen Samuel Curtis Johnson Graduate School of Management, Cornell University, Ithaca, NY 14853
More informationMarkov Decision Processes and Dynamic Programming
Markov Decision Processes and Dynamic Programming A. LAZARIC (SequeL Team @INRIA-Lille) Ecole Centrale - Option DAD SequeL INRIA Lille EC-RL Course In This Lecture A. LAZARIC Markov Decision Processes
More informationOptimal Backlogging Over an Infinite Horizon Under Time Varying Convex Production and Inventory Costs
Optimal Backlogging Over an Infinite Horizon Under Time Varying Convex Production and Inventory Costs Archis Ghate Robert L. Smith Industrial Engineering, University of Washington, Box 352650, Seattle,
More informationApproximate Dynamic Programming for High Dimensional Resource Allocation Problems
Approximate Dynamic Programming for High Dimensional Resource Allocation Problems Warren B. Powell Abraham George Belgacem Bouzaiene-Ayari Hugo P. Simao Department of Operations Research and Financial
More informationValue and Policy Iteration
Chapter 7 Value and Policy Iteration 1 For infinite horizon problems, we need to replace our basic computational tool, the DP algorithm, which we used to compute the optimal cost and policy for finite
More informationAsymptotically Optimal Inventory Control For Assemble-to-Order Systems
Asymptotically Optimal Inventory Control For Assemble-to-Order Systems Marty Reiman Columbia Univerisity joint work with Mustafa Dogru, Haohua Wan, and Qiong Wang May 16, 2018 Outline The Assemble-to-Order
More informationLecture 5 Linear Quadratic Stochastic Control
EE363 Winter 2008-09 Lecture 5 Linear Quadratic Stochastic Control linear-quadratic stochastic control problem solution via dynamic programming 5 1 Linear stochastic system linear dynamical system, over
More informationUNIVERSITY OF MANITOBA
Question Points Score INSTRUCTIONS TO STUDENTS: This is a 6 hour examination. No extra time will be given. No texts, notes, or other aids are permitted. There are no calculators, cellphones or electronic
More informationDYNAMIC LECTURE 5: DISCRETE TIME INTERTEMPORAL OPTIMIZATION
DYNAMIC LECTURE 5: DISCRETE TIME INTERTEMPORAL OPTIMIZATION UNIVERSITY OF MARYLAND: ECON 600. Alternative Methods of Discrete Time Intertemporal Optimization We will start by solving a discrete time intertemporal
More informationInfluence of product return lead-time on inventory control
Influence of product return lead-time on inventory control Mohamed Hichem Zerhouni, Jean-Philippe Gayon, Yannick Frein To cite this version: Mohamed Hichem Zerhouni, Jean-Philippe Gayon, Yannick Frein.
More informationOptimality Inequalities for Average Cost MDPs and their Inventory Control Applications
43rd IEEE Conference on Decision and Control December 14-17, 2004 Atlantis, Paradise Island, Bahamas FrA08.6 Optimality Inequalities for Average Cost MDPs and their Inventory Control Applications Eugene
More information7. Introduction to Numerical Dynamic Programming AGEC Summer 2012
This document was generated at 9:07 PM, 10/01/12 Copyright 2012 Richard T. Woodward 7. Introduction to Numerical Dynamic Programming AGEC 637 - Summer 2012 I. An introduction to Backwards induction Shively,
More informationTopic 6: Projected Dynamical Systems
Topic 6: Projected Dynamical Systems John F. Smith Memorial Professor and Director Virtual Center for Supernetworks Isenberg School of Management University of Massachusetts Amherst, Massachusetts 01003
More informationFinancial Optimization ISE 347/447. Lecture 21. Dr. Ted Ralphs
Financial Optimization ISE 347/447 Lecture 21 Dr. Ted Ralphs ISE 347/447 Lecture 21 1 Reading for This Lecture C&T Chapter 16 ISE 347/447 Lecture 21 2 Formalizing: Random Linear Optimization Consider the
More informationThis paper studies the optimization of the S T inventory policy, where T is the replenishment interval and S
MANUFACTURING & SERVICE OPERATIONS MANAGEMENT Vol. 14, No. 1, Winter 2012, pp. 42 49 ISSN 1523-4614 (print) ISSN 1526-5498 (online) http://dx.doi.org/10.1287/msom.1110.0353 2012 INFORMS Good and Bad News
More informationOn Parallel Machine Replacement Problems with General Replacement Cost Functions and Stochastic Deterioration
On Parallel Machine Replacement Problems with General Replacement Cost Functions and Stochastic Deterioration Suzanne Childress, 1 Pablo Durango-Cohen 2 1 Department of Industrial Engineering and Management
More informationTime-Varying Parameters
Kalman Filter and state-space models: time-varying parameter models; models with unobservable variables; basic tool: Kalman filter; implementation is task-specific. y t = x t β t + e t (1) β t = µ + Fβ
More informationPlanning and Acting in Partially Observable Stochastic Domains
Planning and Acting in Partially Observable Stochastic Domains Leslie Pack Kaelbling*, Michael L. Littman**, Anthony R. Cassandra*** *Computer Science Department, Brown University, Providence, RI, USA
More informationWIRELESS systems often operate in dynamic environments
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 61, NO. 9, NOVEMBER 2012 3931 Structure-Aware Stochastic Control for Transmission Scheduling Fangwen Fu and Mihaela van der Schaar, Fellow, IEEE Abstract
More informationThe Impact of Customer Impatience on Production Control
The Impact of Customer Impatience on Production Control Michael H. Veatch March 27, 27 Abstract Most analyses of make-to-stock production control assume that either all orders are eventually met (complete
More informationPiecewise linear approximations of the standard normal first order loss function and an application to stochastic inventory control
Piecewise linear approximations of the standard normal first order loss function and an application to stochastic inventory control Dr Roberto Rossi The University of Edinburgh Business School, The University
More informationDecision Sciences, Vol. 5, No. 1 (January 1974)
Decision Sciences, Vol. 5, No. 1 (January 1974) A PRESENT VALUE FORMULATION OF THE CLASSICAL EOQ PROBLEM* Robert R. Trippi, California State University-San Diego Donald E. Lewin, Joslyn Manufacturing and
More informationAbstrct. In this paper, we consider the problem of optimal flow control for a production system with one machine which is subject to failures and prod
Monotonicity of Optimal Flow Control for Failure Prone Production Systems 1 Jian-Qiang Hu 2 and Dong Xiang 3 Communicated By W.B. Gong 1 This work is partially supported by the national Science Foundation
More informationDynamic Programming with Hermite Interpolation
Dynamic Programming with Hermite Interpolation Yongyang Cai Hoover Institution, 424 Galvez Mall, Stanford University, Stanford, CA, 94305 Kenneth L. Judd Hoover Institution, 424 Galvez Mall, Stanford University,
More informationISM206 Lecture, May 12, 2005 Markov Chain
ISM206 Lecture, May 12, 2005 Markov Chain Instructor: Kevin Ross Scribe: Pritam Roy May 26, 2005 1 Outline of topics for the 10 AM lecture The topics are: Discrete Time Markov Chain Examples Chapman-Kolmogorov
More information