Dynamic Programming. Problem Sheets

Transcription

1 Dynamic Programming Department of Mathematics and Statistics Courses: Discrete Programming and Game Theory & Dynamic and Integer Programming and Game Theory Lecturer: Andreas Grothey, JCMB 6215, Problem Sheets 1. A industrial pump manufacturer has 1 pump in stock at the start of month 1 and has orders for d t pumps in months t = 1,..., 4. The company wishes to have 1 pump in stock at the start of month 5. Orders in a particular month may be met from stock or from that month s production. The cost of producing x pumps in a given month is r(x), and no more than 3 pumps can be produced in one month. The cost of having y pumps in stock at the start of a month is s(y), and there is no space for more than 2 pumps in stock at the start of a month. The company wishes to meet all its orders at minimum cost. Identify the states of the problem and write down the DP recurrence. Assume now that d 1 = 2, d 2 = 1, d 3 = 1, and d 4 = 1, that r(0) = 4, r(1) = 14, r(2) = 20 and r(3) = 24, and that s(0) = 0, s(1) = 4 and s(2) = 7. Draw a network whose shortest path corresponds to the best production schedule and find this by recursive fixing. 2. A new gas pipeline is to be build between points a and z. A survey of possible routes has produced cost estimates for the different possible legs of the routes. These are given in the following table. From a a To z z z Cost (a) Assume flow must be in the direction shown. Use recursive fixing to find the route which minimizes the cost. (b) (MSc only) Assume now flow can be in either direction in a pipe. Formulate this problem as a DP and solve it by an appropriate algorithm. (Try this after the extra MSc lectures on shortest path problems.) 3. Using Ex 1N in the lecture notes as an example, derive a DP recurrence for the quickest dipaths to a vertex z from other vertices. Do the DP calculation and find the tree of quickest paths to vertex z. 1

2 DIP/DPG, DP: Problem sheets 2 4. Formulate the problem of finding a shortest path between two vertices in a DAG as a linear programming problem. Repeat this for the longest path problem. 5. A gas field management company is planning production in a gas field over 4 years. The initial volume of gas is 8 units. As gas is extracted from the gas field its pressure drops and this limits the rate at which the gas can be extracted. A simple model of this is that the maximum extraction rate is proportional to the remaining gas in the field. Assume that in this field the maximum that can be extracted in a year is v/2, where v is the amount in the field at the start of the year. The company has negotiated long term contracts which guarantee to take as a much gas as the company wishes to sell at prices p i /unit in year i for i = 1,..., 4. Gas remaining in the field at the end of the 4th year will be sold off to another operator at p 5 /unit. The company wishes to maximise its profit. (Within the extraction limits any amount of gas can be extracted in a year i.e. extraction quantities are not discretized.) a) Formulate this problem as a linear programming problem. b) Identify the states of the problem and write down the DP recurrence for it. Show that the optimal policy is to extract in any year either as much or as little as possible. Assuming that p 1 = 12, p 2 = 15, p 3 = 20, p 4 = 11 and p 5 = 8, find the optimal solution by recursive fixing. 6. Assume that we are given a DAG with edge weights and that we are required to find a dipath from vertex 1 to vertex z whose highest edge weight is as small as possible. Formulate this problem as a DP. Use recursive fixing to find such a dipath from vertex 1 to z in the following digraph, where w i j is the weight of the diedge from vertex i to vertex j. i j z z z w i j [Note that objectives of the form in this example occur when we are trying to avoid a worst case. For example the weights might be the worst up-hill gradient on each leg of possible cycle routes between points 1 and z, and we want to find the route whose steepest gradient is a low as possible. Tutorial question 7 gives another example.] 7. Consider the problem of laying out words in a paragraph of text so that there are no large gaps between words. The approach used by TEX (the system which formatted this paragraph) is to minimize a measure of how much the

3 DIP/DPG, DP: Problem sheets 3 spaces between the words have to be varied from the ideal to fit the words in the lines. The author of TEX, Donald Knuth, suggests using DP to do this minimization. Consider the following simplified model of the problem. Assume that a paragraph is to contain n words and is to be W units wide and that both left and right margins have to be straight (so all lines except the last have to be exactly W wide, and the last has to be W wide.) Word i is w i units wide and this cannot be altered. Words are not to be split between lines. The minimum gap between words is 1 unit and the maximum is 4 units and gaps do not have to be an integer number of units. The goal is to layout the paragraph so that the largest gap between adjacent words (on the same line) anywhere in the paragraph is minimised. (a) Note that for a given sequence of words in one line the total of all the gaps between them is fixed. Within one line how should the total space between words be divided up so that the largest gap is as small as possible? (b) Formulate the problem as a DP. [Hint: for the problem of laying out the remainder of the paragraph you could take the state to be the number of the word which starts a line.] (c) (Optional) Assume the sentence below is to be the only sentence in a paragraph whose width is W = 28, and assume that w i = the number of letters in word i. Draw a DAG showing the possible state transitions and apply DP to find the layout whose largest gap is minimized. Mr Pusztai said he was given three days to reply to criticisms of his very disturbing findings. 8. The maintenance of a pump requires the jobs listed in the table below, which also give the jobs which must be completed before each job can start and the durations in hours of each job.

4 DIP/DPG, DP: Problem sheets 4 Job No. Job title Duration Predecessor jobs 1 Remove cover 2 2 Disconnect pipes 2 3 Assemble test rig Remove seals Remove damaged blades Replace damaged blades Analyse damaged blades Lubricate Test electrics Test running 3 3, 6, 8 11 Replace seals and cover 2 9, Reconnect pipes 2 9, 10 Draw a project network showing the above jobs as edges. (Hint: In addition to Start and Finish vertices you will need one vertex for each unique set of predecessors in the above table.) Find the earliest starting time, the latest finishing time and the slack for each job. 9. Knapsack problem: A production company has n electric submersible pumps (ESPs) in stock and wishes to allocate these to improve output from an oil field. The power for the ESPs is supplied by a electricity generator whose maximum output is W. ESP i requires power w i and increases operating profit by q i. The total increase in operating profit is the sum of the increases for each EPS. Identify the states of the system and formulate a DP recurrence for its solution. Assume n = 5, W = 8 and the values of q i and w i given in the table below. Use DP to find the optimal selection of pumps. Item i w i q i Knapsack problem with repeats: Assume that there are n types of EPS available and that any number of a particular type can be used. If r EPSs of type i are used the improvement in profit is rq i and the power required is rw i. The total power available is W and this and each w i are integer valued. Formulate the problem of maximizing profit as a DP using W + 1 states. Solve this problem using the data in Tut Ex 9. [Knapsack problems are much studied. The analogy is that objects from one of n types can be packed into a knapsack. The weight limit on a hike is W

5 DIP/DPG, DP: Problem sheets 5 and each object of type i brings an advantage q i during the hike and weighs w i. The problem is to pack the knapsack to maximize the advantage within the weight limit. Note that in the version of problem in this example we can have several items of the same type in the knapsack, whereas in Tut Ex 9 we can have only one of each type. Why are more states needed for the DP in that case?] 11. A gas field operator leases a field from the government and is required to pay the government a tax of c/unit of gas produced. It has sales contracts with the following structure. There is a price P t /unit for gas sold in day t for amounts up to a break point b t. If more than b t is sold in day t, then the extra gas beyond this break point sells at a price p t /unit. The field has a production limit of U/day. Write down an expression for the profit F t resulting from sale of x t units of gas in day t. (This should be a continuous piecewise linear function with at most one discontinuity of gradient. It should depend on P t, b t, p t, c and U.) Over a 7 day period the prices are as follows t p t b t P t The amount paid in tax per unit is c = 1 and the upper limit on production is 4 units/day. For each day find the optimal production. (It may help to sketch each curve.) Hence find the maximum profit over the week. Describe in general how the optimal gas production in a day depends on U, b t, P t, p t and c. What characterises the states of the problem? 12. The problem is as in the previous example except that we now assume that, in order to prevent damage to the gas reservoir, the operator is not permitted to change production by more than one unit per day. The production in day 0 is 2 units and the operator is not concerned about how the final production level might affect future profits. Identify the states of the problem and write down a DP recurrence for it. Assume that production is limited to integer values (0,1,2,3, or 4). Tabulate the profit for each day for each production level. Draw part of a network whose longest path will be the optimal solution to this problem. Use recursive fixing to calculate the optimal weekly production profile. Calculate the loss of profit which results from the imposition of the production change constraint.

6 DIP/DPG, DP: Problem sheets 6 Could any extra profit be obtained by allowing non integer production levels? 13. Apply Lagrangian relaxation to Ex 9 starting from a value of λ = 3.2 and find bounds on the optimal objective value of Ex Deterministic Hydro Scheduling: A hydro electric generator is fed from a reservoir whose maximum volume is Y. The maximum volume of water which can be passed through the generator per period is X. (More than this can be removed from the reservoir, but this water does not pass through the generator it is spilt.) In period t all electricity generated is sold for S t per unit. In period t a volume r t of water flows into the reservoir. Assume that the amount of electricity generated in period t is g(x t, y t ), where x t is the volume passed through the generator and y t is the volume in the reservoir at the start of the period. Assume also that within any period the rate of inflow of water, the rate of water passing through the turbine and the rate at which water spills are each constant. The reservoir level therefore changes linearly between the start and the end of the period. Assume that at the start of period 1 the volume of water in the reservoir is a know fixed amount ŷ, and that after T periods the remaining volume of water in the reservoir, y T+1, is valued at H(y T+1 ). (This is the total for all the remaining water.) The goal is to operate the generator to maximize the total income from the sale of electricity plus the final value of the water. a) Summarise the features of this inventory problem. b) Identify the states of the problem and write a DP recurrence for it. c) Assume that g(x, y) = xy. The planning horizon is 4 periods and the inflows in periods 1, 2, 3 and 4 are 1, 2, 3 and 1 respectively. Electricity can be sold for 12, 9, 7 and 8 per unit respectively in periods 1, 2, 3 and 4 respectively and the final water in the reservoir, y 5, has total value H(y 5 ) = 4y 2 5. Assume that Y = 3 and X = 2 and that the initial volume in the reservoir ŷ = 2 and that only integer amounts can pass through the generator or be spilt. Find the optimal operation policy by dynamic programming. Hints: You may find it useful for the formulations in b) to introduce variables w t which will give the volume of water spilt in periods t, i.e. the water leaving the reservoir which does not pass through the generator. In practice the operators of the reservoir can choose to spill water even when the reservoir is not full so the amount spilt can be treated as a variable in the problem and can be greater that the amount which is forced to spill. Usually it is not optimal to spill unless forced to do so.

7 DIP/DPG, DP: Problem sheets Consider the Deterministic Unit Commitment Problem given in the lectures, but now assume that when the generator is turned off it has to remain off for at least 3 periods. (Constraints of this type are commonly imposed to avoid different thermal expansions in different parts of the generator. Such differential expansions can cause generator failure.) Identify the states and write down a DP recurrence for the problem of maximising income from sales. 16. Consider again the pump production problem given in lectures, Ex. 4L. Assume now that at the start of a month the number of pumps which have to be delivered that month is known. For future months all that is known is that the probability of the demand being d is p d. If the demand for pumps in a given month is more than can be supplied, then the demand which is not met is supplied by a competitor and so the order is lost. The selling price of a pump is θ and any pumps in stock at the start of month 4 are valued at φ each. The company wishes to maximize its expected profit over months 1 to 3. Identify the states of the problem and write down the DP recurrence. Solve the problem for the case when there is one pump in stock at the start of month 1, the demand in month one is 2, p 1 = 0.6 and p 2 = 0.4, θ = 18 and φ = 12.

8 DIP/DPG, DP: Problem sheets Part of an off-shore oil field has already been explored and it is known that it contains 8 units. Exploration of the remainder of the field will take a year and an initial geological assessment is that the probability that this will contain a further 8 units is 0.3 and the probability of no extra oil is 0.7. The construction of an oil pipeline will take a year and its cost will depend on its capacity. The cost of a pipeline of capacity 1 or 3 units/year is 5 or 10. Thee pipeline building strategies are being evaluated: (1) Build nothing in year 1 and build one pipeline in year 2 once the size of the field is known; (2) Build one pipeline of capacity 1 in year 1 and then decide whether or not to build a second one in year 2; (3) Build one pipeline of capacity 3 in year 1 and decide whether or not to build a second one in year 2. The oil company has a contract which guarantees a price of 2/unit for all the oil produced from the field. Future income and costs are discounted at a rate of 20%/year. (e.g. income or cost x in year i has value or cost 0.8 i 1 x). An initial analysis shows that the best action in year 2 and the corresponding profit is as given in the following table: Build in Year 1 Extra oil Optimal build in year 2 Profit (Note that = i=2 0.8 i. Verify one of the other profit figures.) Draw a decision tree for the problem and find the pipe building strategy which maximizes expected profit. Find also the optimal strategy and profit if it were know at the start of year 1 that there was no extra oil in the unexplored section. Repeat this calculation on the assumption that it was know that this section did contain the extra 8 units. Hence find the EVPI for the problem. How much should the company be prepared to pay to get immediate information on the oil reserves in the field. 18. Rework the Stochastic Unit Commitment example given in the lectures now assuming that in each period the probability of a high price is Stochastic Hydro Scheduling: Assume that the problem is as in Ex 13 except that now the inflow in any period is uncertain and only becomes know immediately before the decision is made of how much to generate in that period. For periods beyond the current one, only the probabilities of inflows

9 DIP/DPG, DP: Problem sheets 9 are known. Assume that there are J possible inflows levels per period. In period t inflow level j is r tj and this occurs with probability p tj. a) Formulate a DP recurrence for the problem of maximising the expected profit. b) Assume now that the data is as before except for the inflows. Assume that J = 2, p tj = 1 2 for t = 1,..., 3 and j = 1, 2, and that r t1 = 0, 2, 2 and r t2 = 2, 4, 4 for t = 1, 2 and 3 respectively. [Note this is the same average inflow as in the deterministic version]. Find the maximum expected profit and the production strategy which achieves this. c) Note that the optimal expected value is lower that the optimal value for the expected value of the data (i.e. the problem solved in Ex 13). Comment briefly on whether or not you would expect this. 20. In a game show items with values 0, 100, , 400 and 500 are drawn with equal probabilities. The contestant can either accept the item and end the game or reject it, in which case a new draw is made (the rejected item is replaced so the probabilities stay the same in future draws). In total 5 draws can be made. Assume the contestant s goal is to maximise her expected winnings. Write down a DP recurrence for the problem and use this to solve the problem. (i.e. for each stage in the game and for each type of item drawn at that stage decide whether to accept it and end the game at that point or reject it and continue.) A problem of this sort is sometimes called the secretary problem or the marriage problem. In the marriage interpretation you have a series of relationships one after the other. It takes time in these relationships to discover the value of the potential marriage partner, and when this is discovered you have to decide whether to marry or look for a new relationship. The goal is to maximise the expected value of the person married before your shelf life expires. Other variants allow the desired marriage partner to reject the proposal! 21. A football team has two strategies, A (attack) and D (defend). The team chooses a strategy for each half and plays that strategy throughout the half. The team chooses the strategy for the first half before the first half starts, and chooses the strategy for the second half in the interval between halves (so the choice for the second half may depend on the score in the first half). In a game against a particular opponent assume that, if the team plays strategy S (= A or D) for an entire half, then the probability of its lead increasing by t in that half is p S (t). Note that t can be negative. (The opposing team has

10 DIP/DPG, DP: Problem sheets 10 a single strategy and the above probabilities assume this constant strategy is being played.) If the team wins it gets 3 points, if it draws it gets 1 point and if it loses it gets 0 points. Assume the team wants to maximize the expected number of points. Identify the states of the problem and write down the DP recurrence for it. 22. (a) A course of radiation therapy is to be delivered over T sessions. In each session one of K treatments is given. Treatment k (1 k K 1) delivers k 1, k or k + 1 units of radiation with probability p (d) k for d = 1, 0 or 1 respectively, and treatment k = 0 delivers no radiation (so can be described in the same way as above but with p ( 1) 0 = p (1) 0 = 0 and p (0) 0 = 1). The target total radiation dose is R units and the goal of the treatment is to minimize deviation from this target. (For example if R = 6 and the total delivered dose is 8, the deviation is 2, and if the total delivered dose is 5 the deviation is 1.) Assume that at the start of each session, before it is decided what treatment to give in that session, it is known what the total delivered dose up to that point has been. Identify the states of the problem and derive a DP recursion for the problem of minimizing the expected (or average ) deviation. What is the optimal treatment if at any stage the delivered dose r R? (There are machines being developed which will be able to measure the actual dose which has been delivered after the session that delivered it.) (b) In the real problem the body is split up into J different parts, and part j has a target dose of R j. Treatment k delivers a dose U (d) k,j to part j with probability p (d) k, for d D k. As before treatment k = 0 delivers no dose to any part. Identify the states of the problem and derive a DP recursion for the problem of minimizing the sum of the expected (or average ) deviations over all parts. (You may be able to write the recursion more neatly using vector notation, using r = (r 1, r 2,..., r J ) as the vector of delivered doses, and U (d) k = (U (d) k,1, U(d) k,2,..., U(d) k,j ). If you have difficulty with this notation, try the case first where there are just 2 parts, i.e. J = 2.) (c) Assume now that some parts of the body should have radiations above some target level (so only low deviations should be penalized) and some parts should have radiations below a target level (so only high deviations should be penalized). Explain how the problem in (b) can be modified to model this problem.

11 DIP/DPG, DP: Problem sheets One game in a quiz show consists of up to T questions. At the start of a game 1 is placed on the table, and each time a question is answered correctly the prize money is doubled. If any question is answered wrongly the game stops and the contestant looses everything. After each question is asked, the contestant can decide either to answer the question, or ask the audience for the answer or stop the game and take away all the money on the table. The contestant can only ask the audience for one answer during the game. Assume that once the contestant hears a particular question he or she either is certain of the answer or would have to guess at random among 4 alternative answers only one of which is correct. Assume that having watched a lot of games a contestant estimates that the probability of knowing (with certainty) the answer to question t is p t, and that the audience always gives the correct answer. Assume that the contestant wants to maximize his or her expected winnings. Identify the states of the problem and write down a DP recurrence for it. Could there be circumstances in which it is optimal for the contestant to guess the answer rather than stopping or asking the audience? 24. A mixture ABCDE of hydrocarbons is to be separated into its pure components A, B, C, D and E using a sequence of distillation columns. The components are listed in order of decreasing boiling point or relative volatility. A distillation column can split a mixture between any two components. The components on either side of the split are called the key components. (For example one distillation column can split ABC into A and BC (key components A and B), or AB and C (key components B and C), but not into AC and B. We assume that the separation is sharp, which means that no component appears in both outputs of a separator.) To a first approximation the cost of doing a split in a distillation column depends the separation coefficient of the key components and to the total amount being split. Assume a mixture of components i, i , k has to be split between components j and j +1. We shall assume the cost of doing the split is of the form cost = W ik S j, where S j is the separation coefficient for key components j and j + 1, and W ik = k j=i W j, and W j is the amount of components j in the mixture entering that distillation column. (The separation coefficient is large when the key components have similar relative volatilities). The amounts of components in the mixture and the separation coefficients are given in the table

12 DIP/DPG, DP: Problem sheets 12 Component A B C D E j W j S j Formulate a DP recurrence for this problem. Assuming that (as a result of the very high separation coefficient between C and D) in the optimal solution the split of C from D occurs when there are no other components present, solve the problem by DP.

13 DIP/DPG, DP: Problem sheets A product P = A 1 A 2 A 3 A 4 A 5 of 5 matrices A i, i = 1,..., 5, is to be calculated. The dimensions of the matrices are A 1 A 2 A 3 A 4 A Write down the number of multiplication needed to multiply a k m matrix and a m n matrix (using the normal method). Assume we wish to calculate P using as few multiplications in total as possible. Formulate a DP recurrence for this problem and then solve it.